Commit graph

2054 commits

Author SHA1 Message Date
MariusVuscanNx 4bded7e4c5
Enhanced Delete of variable persistence manager to allow drivers filtering with tags (#6558) 2025-04-04 11:13:54 +02:00
Raymond den Haan d4b0238501 Moved compression to separate feature 2025-04-03 15:40:17 +02:00
Sipke Schoorstra 7fadf51be6
Fix Output Persistence of Async Activities (#6542)
* Updated output handling of asynchronously run workflows to be the same as when run synchronously

* updated tests

* Refactor activity execution mapping and output persistence

Introduced `GetPersistableOutputAsync` in `IActivityExecutionMapper` to streamline output persistence logic. Refactored the handling of activity persistence properties, replacing repetitive code with reusable methods. Removed unused dependencies and redundant methods, optimizing code readability and maintainability.

* Remove docker-compose-datadog.yml from solution file.

The docker-compose-datadog.yml file is no longer included in the solution structure. This change cleans up unused references to ensure the solution remains consistent and up-to-date.

* Refactor workflow extensions and add new utilities

Split and reorganize workflow-related extension methods into `RunActivityExtensions` and `RunWorkflowExtensions` for better modularity. Removed deprecated methods from `ServiceProviderExtensions`. Updated tests and usages to reflect these changes.

---------

Co-authored-by: Bob Hauser <rhauser@kinaxis.com>
2025-04-01 07:53:00 +02:00
Sipke Schoorstra f7743a0fe6
Fix race condition in Distributed Workflow Runtime during same-workflow event publishing (#6538)
* Add asynchronous stimulus dispatching to workflow runtime

Introduced the BackgroundStimulusDispatcher and related components to enable asynchronous stimulus dispatching. Updated event publishing logic to support both synchronous and asynchronous delivery. These changes improve extensibility and allow better performance for handling workflow stimuli.

* Update event publishing to include isAsync parameter

Added a boolean parameter to differentiate between asynchronous and synchronous event publishing. Ensured correct value is passed based on workflow execution mode, improving clarity and functionality. Removed outdated TODO comment.

* Fix default value for Asynchronous and add activity type helper

Set the default value of the Asynchronous input property to `true` in `PublishEvent`. Additionally, utilize `ActivityTypeNameHelper` in `EventPublisher` to generate activity type names for improved consistency and clarity.

* Remove unused import from EventPublisher.cs

Eliminated the unnecessary import of Elsa.Workflows.Runtime.Requests. This cleanup helps reduce code clutter and improves maintainability.

* Set default value for Asynchronous property and update Payload.

Added a default value of `true` for the `Asynchronous` property to ensure consistency in event delivery behavior. Also, removed the unnecessary default value from the `Payload` property description for clarity.

* Simplify XML doc comment in IStimulusDispatcher interface

Refined the XML documentation for the `SendAsync` method by shortening the description and clarifying the statement. This improves readability and maintains the intent of the comment.

* Add MassTransit stimulus dispatching functionality

Introduced `DispatchStimulusRequestConsumer` and associated consumer definitions to enable stimulus dispatching. Added `MassTransitStimulusDispatcher` service and related options for configuring concurrency and endpoint behavior. Updated `MassTransitWorkflowDispatcherFeature` to register the stimulus dispatcher and allow enhanced dispatch configuration.

* Refactor stimulus dispatching to use a serialized message.

Introduced `DispatchStimulus` to encapsulate serialized requests. Updated dispatch logic to utilize `IPayloadSerializer` for serialization and deserialization, enabling message transport in a lightweight format.

* Remove Asynchronous input property from PublishEvent activity

The Asynchronous input property was removed as it was unnecessary, with a hardcoded `true` value now passed directly to the event publisher. This simplifies the code and ensures consistency in how events are published.

* Remove ProtoStringExtensions and consolidate string utilities.

Moved `EmptyIfNull` and `NullIfEmpty` methods to the existing `StringExtensions` class in `Elsa.Workflows.Core` for centralization. Updated references to use the consolidated extension methods. Deleted `ProtoStringExtensions` as it is now redundant.

* Add missing imports for Elsa.Extensions across mappers

Added `using Elsa.Extensions` to multiple mapper files in the runtime module. This ensures consistent access to shared extension methods, improving code clarity and reducing potential errors.

* Add missing Elsa.Extensions namespace to WorkflowInstance

The `using Elsa.Extensions` directive was added to ensure access to required extensions. This resolves potential issues with missing dependencies or functionality in the `WorkflowInstance` class.
2025-03-31 18:11:27 +02:00
Sipke Schoorstra 952cbb4edc
Add strict type checking for variable parsing (#6536)
* Add strict type checking for variable parsing

Introduced a `StrictMode` flag for variables, enforcing stricter type validation during parsing and conversions. Updated related methods and tests to reflect the stricter parsing behavior, ensuring consistent type compatibility and error handling.

* Refactor StrictMode handling and improve ObjectConverter logic

Moved StrictMode flag from Variable to ObjectConverter for better cohesion and updated related references. Additionally, removed unused parameters and simplified ObjectConverterOptions to streamline configuration and maintain consistency across the codebase.

* Refactor type conversion logic and update logging defaults

Improve type conversion reliability by introducing `TryConvertValue` and refining exception handling. Update JSON scenarios to use "Inherit" as the default for log persistence modes, ensuring consistency across configurations.

* Refactor exception handling in ObjectConverter.

Introduce a helper method `ReturnOrThrow` to streamline and centralize exception handling logic. This change also adds support for non-strict mode, allowing value fallback instead of throwing exceptions when enabled.

* Add numeric type checks and improve type conversion handling

Introduce `IsNumericType` extension method to identify numeric types. Enhance type conversion logic in `ObjectConverter` to handle numeric, boolean, and string types more accurately. Update variable deserialization workflows to handle conversion failures gracefully.

* Set a default comment for ObjectConverter.StrictMode assignment

Added a comment clarifying that StrictMode is set to its default value. This improves code readability and helps maintainers understand the intent.

* Fix formatting inconsistencies and improve code clarity

Removed redundant whitespace and adjusted formatting to align with coding standards. These changes enhance the readability and maintainability of the code without altering functionality.

* Fix typo in Program.cs variable comment

Corrected a minor issue in the comment for `identityTokenSection` to remove the unnecessary "Modify" text. This change improves code readability and removes potential confusion for developers.

* Simplify imports in ModifyVariableHandler.cs

Removed unused `Microsoft.Extensions.Options` and `Elsa.Workflows.Options` imports to clean up dependencies and improve maintainability. This change reduces clutter without affecting the existing functionality.
2025-03-31 18:11:14 +02:00
Sipke Schoorstra 8bf58c5c60
Add support for activity output testing (#6529)
* Add support for activity output testing

Removed obsolete test workflows and updated the If activity tests to align with simplified testing practices. Introduced `GetActivityOutput` extension for retrieving activity outputs from workflow results. Refactored related APIs to simplify execution flow and unified cancellation token handling in service methods.

* Remove unused Parlot.Fluent namespace in IfTests.cs

This commit removes an unused "using" directive for the Parlot.Fluent namespace in the IfTests.cs file. Cleaning up unused references helps maintain code clarity and reduces potential confusion.
2025-03-31 18:10:59 +02:00
Sipke Schoorstra ede5d5cad2
Fix null handling for cancellation tokens and string fields
Updated `_linkedTokenSource` initialization to allow null values and improved string field handling by applying `NullIfEmpty()` for `CorrelationId`, `TriggerActivityId`, and `ParentWorkflowInstanceId`. These changes ensure proper null-state management and eliminate potential discrepancies in downstream operations.
2025-03-28 20:56:47 +01:00
Sipke Schoorstra fdc57945e8
Refactor default parameter values to use null instead of default.
Replaced `default` with `null` for optional parameters across several classes to improve clarity and consistency. This change aligns with typical .NET conventions and ensures more predictable behavior when handling optional arguments.
2025-03-28 20:44:15 +01:00
Sipke Schoorstra 06b9907d85
Fix DbContext pooling (#6531)
* Enable DbContext pooling and refactor DI scopes

Added support for using DbContext pooling with a configurable option. Refactored to use scoped service providers for better isolation and lifecycle management. Updated persistence feature base to integrate sensitive data logging and warning configurations.

* Update package management and Elsa Studio version references

Centralized package versioning remains enabled, while unused package references and conditional groups for specific .NET targets are removed for simplification. Elsa Studio version is upgraded from 3.3.0-rc4 to 3.4.0-rc1.

* Simplify DbContextOptions setup logic

Removed conditional compilation and unused code to streamline the DbContextOptions configuration. This improves maintainability and ensures consistent behavior across frameworks. Sensitive data logging is no longer explicitly enabled here.
2025-03-26 16:05:40 +01:00
Sipke Schoorstra 9575cdeecf
Update Dapper Migrations (#6520)
* Update workflow to check for patch version 3.3.3

Revised the GitHub Actions workflow to detect the correct patch branch, updating from version 3.3.2 to 3.3.3. This ensures compatibility with the latest release process.

* Refactor WebhooksFeature to streamline configuration.

Simplified the configuration of sinks, sources, and HTTP clients by using direct actions instead of service-level options. Updated Apply method to finalize configurations and added notification handlers for better extensibility. Improved code readability and maintainability by consolidating logic and removing unnecessary dependencies.

* Update base_version in GitHub Actions workflow

Bump the base_version from 3.3.2 to 3.3.4 in the packages workflow file. This ensures compatibility and uses the latest base version for package management.

* Update release branch check to target patch 3.3.4

Modified the GitHub Actions workflow to check for the `patch/3.3.4` branch instead of `patch/3.3.3` during release events. This ensures compatibility with the latest patch branch in the release process.

* Handle conversion of enumerables to arrays in ObjectConverter (#6508)

* Handle conversion of enumerables to arrays in ObjectConverter

Added functionality to convert enumerables to arrays when the target type is an array. The implementation ensures each item is properly cast to the target array's element type and then added to the resulting array.

* Add unit test for converting object array to double array

This commit introduces a new test ensuring the proper conversion of an object array of doubles to a double array. It verifies both the validity of the conversion and that the result is not null.

* Update base version to 3.3.5 in workflow configuration

Bump the `base_version` from 3.3.4 to 3.3.5 in the GitHub Actions workflow file. Adjust the matching branch reference to correspond with the updated version.

* Add V3.4 migration with schema updates and new column

Introduces Migration V3_4 to modify WorkflowInstances by adding the IsExecuting column. Updates Entity Framework migration to dynamically handle schema changes across tables. Ensures improved compatibility and extendability within the database layer.

* Refactor Dapper migration execution and update database schema. (#6519)

* Refactor migration execution and update database schema.

Replaced RunMigrationsHostedService with RunMigrationsStartupTask using IStartupTask for improved lifecycle management. Updated DapperMigrationsFeature to use the new startup task. Added a new "SerializedProperties" column to ActivityExecutionRecords and ensured proper schema adjustments during migrations.

* Add TenantId column to BookmarkQueueItems table

This commit introduces a nullable TenantId column to the BookmarkQueueItems table. The change ensures better multi-tenancy support by allowing tenant-specific data segregation.
2025-03-20 12:11:05 +01:00
Matthew Vance aa989ce13d
Catch exceptions and log instead of crashing (#6477)
* Catch exceptions and log instead of crashing

Modified both WorkflowInstance and WorkflowDefinition state loading to prevent throwing exceptions if the state fails to load successfully.

Also modified the CleanupJob to catch exceptions thrown by cleaning up and logging as errors and continuing instead of crashing.

Fixes 6473

* Use LogError instead of LogWarning for deserialization failure.

Updated the log level to LogError when workflow definition state deserialization fails. This change ensures better visibility and prioritization of critical issues during state handling.

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2025-03-18 19:27:04 +01:00
Sipke Schoorstra 8e9d5e356e
Handle conversion of enumerables to arrays in ObjectConverter (#6508)
* Handle conversion of enumerables to arrays in ObjectConverter

Added functionality to convert enumerables to arrays when the target type is an array. The implementation ensures each item is properly cast to the target array's element type and then added to the resulting array.

* Add unit test for converting object array to double array

This commit introduces a new test ensuring the proper conversion of an object array of doubles to a double array. It verifies both the validity of the conversion and that the result is not null.
2025-03-18 16:09:49 +01:00
Sipke Schoorstra caa3f05d19
Merge branch 'main' into enh/otel 2025-03-17 15:21:36 +01:00
Sipke Schoorstra 9b0aa9d922
Remove redundant span tags for activity and workflow incidents
The tags "activity.incidents" and "workflow.incidents" were removed as they are unnecessary and provide no additional value. This change simplifies the middleware implementation while retaining essential telemetry data.
2025-03-17 15:05:24 +01:00
Sipke Schoorstra 2d485d253d
Remove redundant status setting in telemetry middleware.
The `SetStatus(ActivityStatusCode.Ok)` calls were unnecessary as the default behavior already sets the appropriate status. This cleanup simplifies the code and avoids redundant operations, improving maintainability.
2025-03-17 15:01:37 +01:00
Sipke Schoorstra 9822b9ca75
Refactor default value declarations for inputs and cleanup imports
Replaced `default!` with `null` for input property initializations to improve clarity and consistency. Removed an unused namespace import from `IErrorSpanHandler.cs` to clean up the codebase.
2025-03-17 10:56:24 +01:00
Sipke Schoorstra a835eb82ed
Remove message retry configuration in DispatchWorkflowRequestConsumer.
This ensures a workflow is not executed multiple times in case of an unhandled exception.
2025-03-17 10:14:05 +01:00
Sipke Schoorstra a04aaf69e8
Merge pull request #6503 from KnibbsyMan/feat/sql-injection-prevention
FEAT - Automatic SQL Expression Parameterization
2025-03-17 10:05:22 +01:00
Sipke Schoorstra 714d8b6f3d
Merge pull request #6501 from elsa-workflows/bug/faulting-executing
Refactor activity execution state handling with IDisposable
2025-03-17 08:40:46 +01:00
Sipke Schoorstra 735a513790
Merge branch 'main' into enh/otel 2025-03-16 22:36:25 +01:00
Sipke Schoorstra 80d7c1c256
Handle null values explicitly for varbinary columns
Added explicit handling for null values in varbinary columns by applying a `CAST(NULL AS varbinary(max))` operation. This ensures proper value insertion and avoids potential type errors. Cleaned up unnecessary comments and minor whitespace inconsistencies for improved readability.
2025-03-16 22:36:09 +01:00
Sipke Schoorstra dc7b95835d
Handle null values explicitly for varbinary columns
Added explicit handling for null values in varbinary columns by applying a `CAST(NULL AS varbinary(max))` operation. This ensures proper value insertion and avoids potential type errors. Cleaned up unnecessary comments and minor whitespace inconsistencies for improved readability.
2025-03-16 22:35:34 +01:00
Matt 95b67ea002 Updates error handling to allow null values to be passed to the query. Errors are now thrown for when no matching properties are found. 2025-03-16 16:42:32 +00:00
Matt 5f43966233 BaseSqlClient code tidy. 2025-03-16 00:59:53 +00:00
Matt 24da6ab288 Simplify SqlEvaluator implementation and tidy code. 2025-03-16 00:44:35 +00:00
Matt d3dc6f3597 Updated the SqlEvaluator to work with {{ }} rather than @ for expression. Supporting updates also added. 2025-03-15 22:06:08 +00:00
Sipke Schoorstra 83253b2fae
Add support for starting new trace contexts in workflows
Introduced `StartNewTrace` input to workflows and updated OpenTelemetry middleware to support initiating new trace contexts. This helps improve trace isolation and linking when desired, enhancing observability during workflow execution.
2025-03-15 21:14:10 +01:00
Sipke Schoorstra 5f74a83a77
Remove redundant status handling in tracing middleware
This commit eliminates unused status checks ("running" and "pending") from OpenTelemetry tracing middlewares. The changes simplify the codebase and improve maintainability by removing unnecessary logic.
2025-03-15 20:07:15 +01:00
Sipke Schoorstra fe65476bf2
Add WorkflowExecutionState to manage execution context state
Introduce `WorkflowExecutionState` to encapsulate execution state changes for workflows. This ensures proper handling of execution flags during the lifetime of the state object and resets them upon disposal.
2025-03-15 20:03:22 +01:00
Sipke Schoorstra ba08d97b38
Refactor activity execution state handling with IDisposable
Introduce `EnterExecution` method to manage activity execution state using a `using` block for cleaner and safer resource handling. Removed manual flags for tracking execution state, streamlining workflow logic and improving maintainability.
2025-03-15 20:01:02 +01:00
Matt d8b6eda93f Simplify SQL client implementations to reduce repetitive code. 2025-03-15 01:51:42 +00:00
Matt a4e6e75e55 Adds SQL injection prevention with the use of parameters in the SqlEvaluator. 2025-03-15 01:26:11 +00:00
Sipke Schoorstra 39932a6f09
Refactor OpenTelemetry tracing and improve span tagging.
Introduced new span tags for activities and workflows to enhance trace analysis. Optimized error handling in activity tracing by selecting the first applicable handler. Removed unused manual tracing configuration and commented out console exporters in server setup.
2025-03-14 23:55:02 +01:00
Sipke Schoorstra b624b2b4af
Refactor error span handling with prioritization and predicates
Introduce `Order` property and `CanHandle` method in `IErrorSpanHandler` to enable prioritized and conditional processing of error spans. Update handler implementations to utilize these properties and streamline logic. This enhances flexibility and improves the extensibility of error span handling.
2025-03-14 23:37:46 +01:00
Sipke Schoorstra e393cca031
Update error status and exception type handling
Enhanced error status to include detailed message when the workflow is faulted, providing better context in logs. Fixed the retrieval of exception type by replacing `GetType()` with `.Type` for improved accuracy in incident tagging.
2025-03-14 23:27:16 +01:00
Sipke Schoorstra fd31b5b605
Update OTEL fields and tags
Added and updated multiple package references, including OpenTelemetry, Datadog.Trace.Bundle, and various Microsoft.Extensions libraries. This ensures compatibility with the latest dependencies and introduces enhanced features for resilience and tracing.
2025-03-14 23:17:54 +01:00
Sipke Schoorstra 62da3c8618
Merge pull request #6491 from Green7/dapper-fix-delete-without-filter
Correct the issue with DeleteAsync not executing due to a premature p…
2025-03-14 23:07:26 +01:00
Sipke Schoorstra ac7428da0b
Merge pull request #6497 from truthz03/feature/6400
#6400 Add ClearProvider again to ActivityRegistry
2025-03-14 23:06:22 +01:00
Sipke Schoorstra d8208d2fe7
Merge pull request #6495 from elsa-workflows/enh/explicit-deterministic-variable-id
Refactor variable initialization for clarity and consistency
2025-03-14 23:05:35 +01:00
Sipke Schoorstra 146791f72a
Merge pull request #6475 from KnibbsyMan/feat/sql-enhancements
Enhancement - Additional SQL Activity Features
2025-03-14 23:04:50 +01:00
Sipke Schoorstra 094291b673
Merge branch 'main' into enh/workflowinstance-name 2025-03-14 23:03:33 +01:00
Thomas Trummer 90d8949661 #6400 Add ClearProvider again to ActivityRegistry 2025-03-14 10:13:02 +01:00
Thomas Trummer 996b831886 #6400 Add ClearProvider again to ActivityRegistry 2025-03-14 09:49:52 +01:00
Sipke Schoorstra 923e9d335d
Refactor variable initialization for clarity and consistency
Updated variable constructors across the codebase to use explicit names and initial values where applicable. Deprecated old constructor overloads and added new methods and overloads for better flexibility and readability. Minor cleanup includes replacing `default` keywords with `null` and streamlining code syntax.
2025-03-13 21:05:28 +01:00
Matt 2ff6df44c8
Merge branch 'elsa-workflows:main' into feat/sql-enhancements 2025-03-13 19:46:07 +00:00
Matt e43074755e
Merge branch 'elsa-workflows:main' into feat/datetime-picker 2025-03-13 19:21:39 +00:00
Matt 016600b838 Adds Support For New DateTimePicker UI Component And Re-Orders the InputUIHints. 2025-03-13 19:13:16 +00:00
Sipke Schoorstra e3e298984f
Refactor Variable class and add WithId method
Replaced `default` with `null` to improve readability and clarify initialization. Added a `WithId` method to allow assigning an ID to `Variable` instances for enhanced configurability. These changes improve usability and maintain consistency in the codebase.
2025-03-13 11:54:46 +01:00
Sipke Schoorstra 882ea8cf57
Refactor variable merging logic in ActivityExecutionContext
Replaced LINQ-based logic with a dictionary approach to merge variables. This ensures that variables are consistently merged by either name or ID when name is absent. It improves code clarity and handles edge cases more robustly.
2025-03-13 11:54:28 +01:00
Matt 819738d4d4
Merge branch 'elsa-workflows:main' into feat/sql-enhancements 2025-03-12 17:26:08 +00:00