Commit graph

6046 commits

Author SHA1 Message Date
Sipke Schoorstra 8bf58c5c60
Add support for activity output testing (#6529)
* Add support for activity output testing

Removed obsolete test workflows and updated the If activity tests to align with simplified testing practices. Introduced `GetActivityOutput` extension for retrieving activity outputs from workflow results. Refactored related APIs to simplify execution flow and unified cancellation token handling in service methods.

* Remove unused Parlot.Fluent namespace in IfTests.cs

This commit removes an unused "using" directive for the Parlot.Fluent namespace in the IfTests.cs file. Cleaning up unused references helps maintain code clarity and reduces potential confusion.
2025-03-31 18:10:59 +02:00
Sipke Schoorstra ede5d5cad2
Fix null handling for cancellation tokens and string fields
Updated `_linkedTokenSource` initialization to allow null values and improved string field handling by applying `NullIfEmpty()` for `CorrelationId`, `TriggerActivityId`, and `ParentWorkflowInstanceId`. These changes ensure proper null-state management and eliminate potential discrepancies in downstream operations.
2025-03-28 20:56:47 +01:00
Sipke Schoorstra fdc57945e8
Refactor default parameter values to use null instead of default.
Replaced `default` with `null` for optional parameters across several classes to improve clarity and consistency. This change aligns with typical .NET conventions and ensures more predictable behavior when handling optional arguments.
2025-03-28 20:44:15 +01:00
Sipke Schoorstra 0e6fdc7e0d Update package prefix logic for release candidate branches
Include release candidate (rc/*) branches in the condition to assign the "preview" package prefix. This ensures consistent versioning for branches intended for pre-production releases.
2025-03-26 16:23:17 +01:00
Sipke Schoorstra 06b9907d85
Fix DbContext pooling (#6531)
* Enable DbContext pooling and refactor DI scopes

Added support for using DbContext pooling with a configurable option. Refactored to use scoped service providers for better isolation and lifecycle management. Updated persistence feature base to integrate sensitive data logging and warning configurations.

* Update package management and Elsa Studio version references

Centralized package versioning remains enabled, while unused package references and conditional groups for specific .NET targets are removed for simplification. Elsa Studio version is upgraded from 3.3.0-rc4 to 3.4.0-rc1.

* Simplify DbContextOptions setup logic

Removed conditional compilation and unused code to streamline the DbContextOptions configuration. This improves maintainability and ensures consistent behavior across frameworks. Sensitive data logging is no longer explicitly enabled here.
2025-03-26 16:05:40 +01:00
Sipke Schoorstra 9575cdeecf
Update Dapper Migrations (#6520)
* Update workflow to check for patch version 3.3.3

Revised the GitHub Actions workflow to detect the correct patch branch, updating from version 3.3.2 to 3.3.3. This ensures compatibility with the latest release process.

* Refactor WebhooksFeature to streamline configuration.

Simplified the configuration of sinks, sources, and HTTP clients by using direct actions instead of service-level options. Updated Apply method to finalize configurations and added notification handlers for better extensibility. Improved code readability and maintainability by consolidating logic and removing unnecessary dependencies.

* Update base_version in GitHub Actions workflow

Bump the base_version from 3.3.2 to 3.3.4 in the packages workflow file. This ensures compatibility and uses the latest base version for package management.

* Update release branch check to target patch 3.3.4

Modified the GitHub Actions workflow to check for the `patch/3.3.4` branch instead of `patch/3.3.3` during release events. This ensures compatibility with the latest patch branch in the release process.

* Handle conversion of enumerables to arrays in ObjectConverter (#6508)

* Handle conversion of enumerables to arrays in ObjectConverter

Added functionality to convert enumerables to arrays when the target type is an array. The implementation ensures each item is properly cast to the target array's element type and then added to the resulting array.

* Add unit test for converting object array to double array

This commit introduces a new test ensuring the proper conversion of an object array of doubles to a double array. It verifies both the validity of the conversion and that the result is not null.

* Update base version to 3.3.5 in workflow configuration

Bump the `base_version` from 3.3.4 to 3.3.5 in the GitHub Actions workflow file. Adjust the matching branch reference to correspond with the updated version.

* Add V3.4 migration with schema updates and new column

Introduces Migration V3_4 to modify WorkflowInstances by adding the IsExecuting column. Updates Entity Framework migration to dynamically handle schema changes across tables. Ensures improved compatibility and extendability within the database layer.

* Refactor Dapper migration execution and update database schema. (#6519)

* Refactor migration execution and update database schema.

Replaced RunMigrationsHostedService with RunMigrationsStartupTask using IStartupTask for improved lifecycle management. Updated DapperMigrationsFeature to use the new startup task. Added a new "SerializedProperties" column to ActivityExecutionRecords and ensured proper schema adjustments during migrations.

* Add TenantId column to BookmarkQueueItems table

This commit introduces a nullable TenantId column to the BookmarkQueueItems table. The change ensures better multi-tenancy support by allowing tenant-specific data segregation.
2025-03-20 12:11:05 +01:00
Sipke Schoorstra ffe1a611d7
Remove unused datadog compose file and update branch reference.
The datadog compose file was removed from the solution as it is no longer needed. Additionally, the GitHub workflow was updated to reference the "rc/3.4.0" branch instead of "main" for the release process.
2025-03-18 19:41:46 +01:00
Sipke Schoorstra 2092b8aefe
Add 'rc/*' branch pattern to workflow triggers
This update includes the 'rc/*' branch pattern to the workflow triggers in the packages.yml file. It ensures workflows are triggered for release candidate branches, aligning with the project's branching strategy.
2025-03-18 19:30:16 +01:00
Matthew Vance aa989ce13d
Catch exceptions and log instead of crashing (#6477)
* Catch exceptions and log instead of crashing

Modified both WorkflowInstance and WorkflowDefinition state loading to prevent throwing exceptions if the state fails to load successfully.

Also modified the CleanupJob to catch exceptions thrown by cleaning up and logging as errors and continuing instead of crashing.

Fixes 6473

* Use LogError instead of LogWarning for deserialization failure.

Updated the log level to LogError when workflow definition state deserialization fails. This change ensures better visibility and prioritization of critical issues during state handling.

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2025-03-18 19:27:04 +01:00
Sipke Schoorstra 8e9d5e356e
Handle conversion of enumerables to arrays in ObjectConverter (#6508)
* Handle conversion of enumerables to arrays in ObjectConverter

Added functionality to convert enumerables to arrays when the target type is an array. The implementation ensures each item is properly cast to the target array's element type and then added to the resulting array.

* Add unit test for converting object array to double array

This commit introduces a new test ensuring the proper conversion of an object array of doubles to a double array. It verifies both the validity of the conversion and that the result is not null.
2025-03-18 16:09:49 +01:00
Sipke Schoorstra ebe32288d2
Merge pull request #6502 from elsa-workflows/enh/otel
Improve OTEL Spans and Tags for Workflow and Activity Execution
2025-03-17 15:23:13 +01:00
Sipke Schoorstra caa3f05d19
Merge branch 'main' into enh/otel 2025-03-17 15:21:36 +01:00
Sipke Schoorstra 9b0aa9d922
Remove redundant span tags for activity and workflow incidents
The tags "activity.incidents" and "workflow.incidents" were removed as they are unnecessary and provide no additional value. This change simplifies the middleware implementation while retaining essential telemetry data.
2025-03-17 15:05:24 +01:00
Sipke Schoorstra 2d485d253d
Remove redundant status setting in telemetry middleware.
The `SetStatus(ActivityStatusCode.Ok)` calls were unnecessary as the default behavior already sets the appropriate status. This cleanup simplifies the code and avoids redundant operations, improving maintainability.
2025-03-17 15:01:37 +01:00
Sipke Schoorstra af63b1b59a
Ignore docker compose for local testing 2025-03-17 13:42:57 +01:00
Sipke Schoorstra 9822b9ca75
Refactor default value declarations for inputs and cleanup imports
Replaced `default!` with `null` for input property initializations to improve clarity and consistency. Removed an unused namespace import from `IErrorSpanHandler.cs` to clean up the codebase.
2025-03-17 10:56:24 +01:00
Sipke Schoorstra 2b9792b7be
Merge pull request #6506 from elsa-workflows/bug/remove-mt-retry
Remove message retry configuration in DispatchWorkflowRequestConsumer
2025-03-17 10:18:33 +01:00
Sipke Schoorstra a835eb82ed
Remove message retry configuration in DispatchWorkflowRequestConsumer.
This ensures a workflow is not executed multiple times in case of an unhandled exception.
2025-03-17 10:14:05 +01:00
Sipke Schoorstra a04aaf69e8
Merge pull request #6503 from KnibbsyMan/feat/sql-injection-prevention
FEAT - Automatic SQL Expression Parameterization
2025-03-17 10:05:22 +01:00
Sipke Schoorstra 40a6d3ea8d
Merge pull request #6498 from truthz03/feature/AddJobRunnerWorkerCountSetting
Allow to configure JobRunner worker count
2025-03-17 10:04:14 +01:00
Thomas Trummer 84191f149d Use IOptions<MediatorOptions> to get workerCount for BackgroundCommandSenderHostedService, BackgroundEventPublisherHostedService and JobRunnerHostedService 2025-03-17 08:45:34 +01:00
Sipke Schoorstra 714d8b6f3d
Merge pull request #6501 from elsa-workflows/bug/faulting-executing
Refactor activity execution state handling with IDisposable
2025-03-17 08:40:46 +01:00
Sipke Schoorstra 735a513790
Merge branch 'main' into enh/otel 2025-03-16 22:36:25 +01:00
Sipke Schoorstra 80d7c1c256
Handle null values explicitly for varbinary columns
Added explicit handling for null values in varbinary columns by applying a `CAST(NULL AS varbinary(max))` operation. This ensures proper value insertion and avoids potential type errors. Cleaned up unnecessary comments and minor whitespace inconsistencies for improved readability.
2025-03-16 22:36:09 +01:00
Sipke Schoorstra dc7b95835d
Handle null values explicitly for varbinary columns
Added explicit handling for null values in varbinary columns by applying a `CAST(NULL AS varbinary(max))` operation. This ensures proper value insertion and avoids potential type errors. Cleaned up unnecessary comments and minor whitespace inconsistencies for improved readability.
2025-03-16 22:35:34 +01:00
Sipke Schoorstra 32da99e3ec
Add docker-compose file for Datadog and OpenTelemetry setup
Introduce a new docker-compose configuration to integrate Datadog and OpenTelemetry with services like PostgreSQL, RabbitMQ, Redis, and Elsa. Updated the Datadog API key in an existing compose file for security purposes. This setup enables detailed monitoring, tracing, and metrics collection for development environments.
2025-03-16 19:49:35 +01:00
Matt 95b67ea002 Updates error handling to allow null values to be passed to the query. Errors are now thrown for when no matching properties are found. 2025-03-16 16:42:32 +00:00
Matt 5f43966233 BaseSqlClient code tidy. 2025-03-16 00:59:53 +00:00
Matt 24da6ab288 Simplify SqlEvaluator implementation and tidy code. 2025-03-16 00:44:35 +00:00
Matt d3dc6f3597 Updated the SqlEvaluator to work with {{ }} rather than @ for expression. Supporting updates also added. 2025-03-15 22:06:08 +00:00
Sipke Schoorstra 83253b2fae
Add support for starting new trace contexts in workflows
Introduced `StartNewTrace` input to workflows and updated OpenTelemetry middleware to support initiating new trace contexts. This helps improve trace isolation and linking when desired, enhancing observability during workflow execution.
2025-03-15 21:14:10 +01:00
Sipke Schoorstra 5f74a83a77
Remove redundant status handling in tracing middleware
This commit eliminates unused status checks ("running" and "pending") from OpenTelemetry tracing middlewares. The changes simplify the codebase and improve maintainability by removing unnecessary logic.
2025-03-15 20:07:15 +01:00
Sipke Schoorstra fe65476bf2
Add WorkflowExecutionState to manage execution context state
Introduce `WorkflowExecutionState` to encapsulate execution state changes for workflows. This ensures proper handling of execution flags during the lifetime of the state object and resets them upon disposal.
2025-03-15 20:03:22 +01:00
Sipke Schoorstra ba08d97b38
Refactor activity execution state handling with IDisposable
Introduce `EnterExecution` method to manage activity execution state using a `using` block for cleaner and safer resource handling. Removed manual flags for tracking execution state, streamlining workflow logic and improving maintainability.
2025-03-15 20:01:02 +01:00
Matt d8b6eda93f Simplify SQL client implementations to reduce repetitive code. 2025-03-15 01:51:42 +00:00
Matt a4e6e75e55 Adds SQL injection prevention with the use of parameters in the SqlEvaluator. 2025-03-15 01:26:11 +00:00
Sipke Schoorstra 39932a6f09
Refactor OpenTelemetry tracing and improve span tagging.
Introduced new span tags for activities and workflows to enhance trace analysis. Optimized error handling in activity tracing by selecting the first applicable handler. Removed unused manual tracing configuration and commented out console exporters in server setup.
2025-03-14 23:55:02 +01:00
Sipke Schoorstra b624b2b4af
Refactor error span handling with prioritization and predicates
Introduce `Order` property and `CanHandle` method in `IErrorSpanHandler` to enable prioritized and conditional processing of error spans. Update handler implementations to utilize these properties and streamline logic. This enhances flexibility and improves the extensibility of error span handling.
2025-03-14 23:37:46 +01:00
Sipke Schoorstra e393cca031
Update error status and exception type handling
Enhanced error status to include detailed message when the workflow is faulted, providing better context in logs. Fixed the retrieval of exception type by replacing `GetType()` with `.Type` for improved accuracy in incident tagging.
2025-03-14 23:27:16 +01:00
Sipke Schoorstra fd31b5b605
Update OTEL fields and tags
Added and updated multiple package references, including OpenTelemetry, Datadog.Trace.Bundle, and various Microsoft.Extensions libraries. This ensures compatibility with the latest dependencies and introduces enhanced features for resilience and tracing.
2025-03-14 23:17:54 +01:00
Sipke Schoorstra 62da3c8618
Merge pull request #6491 from Green7/dapper-fix-delete-without-filter
Correct the issue with DeleteAsync not executing due to a premature p…
2025-03-14 23:07:26 +01:00
Sipke Schoorstra ac7428da0b
Merge pull request #6497 from truthz03/feature/6400
#6400 Add ClearProvider again to ActivityRegistry
2025-03-14 23:06:22 +01:00
Sipke Schoorstra d8208d2fe7
Merge pull request #6495 from elsa-workflows/enh/explicit-deterministic-variable-id
Refactor variable initialization for clarity and consistency
2025-03-14 23:05:35 +01:00
Sipke Schoorstra 146791f72a
Merge pull request #6475 from KnibbsyMan/feat/sql-enhancements
Enhancement - Additional SQL Activity Features
2025-03-14 23:04:50 +01:00
Sipke Schoorstra 85eec228cf
Merge pull request #6460 from elsa-workflows/enh/workflowinstance-name
Improved support for named workflow instances
2025-03-14 23:03:51 +01:00
Sipke Schoorstra 094291b673
Merge branch 'main' into enh/workflowinstance-name 2025-03-14 23:03:33 +01:00
Sipke Schoorstra 287eef1a27
Merge pull request #6494 from KnibbsyMan/feat/datetime-picker
DateTimePicker UI Hint Missing
2025-03-14 23:00:10 +01:00
Thomas Trummer 84be58fceb Allow to configure JobRunner worker count 2025-03-14 10:14:47 +01:00
Thomas Trummer 90d8949661 #6400 Add ClearProvider again to ActivityRegistry 2025-03-14 10:13:02 +01:00
Thomas Trummer 996b831886 #6400 Add ClearProvider again to ActivityRegistry 2025-03-14 09:49:52 +01:00