Commit graph

293 commits

Author SHA1 Message Date
Sverre Winkelmans bc70beff12
Add tenant headers support to BackgroundWorkflowCancellationDispatcher (#7040)
* Add tenant headers support to BackgroundWorkflowCancellationDispatcher

* Fix 'CreateHeaders' call
2026-01-16 09:07:06 +01:00
copilot-swe-agent[bot] f834b040f9 Add workflow dispatch notifications
- Created WorkflowDefinitionDispatching and WorkflowDefinitionDispatched notifications
- Created WorkflowInstanceDispatching and WorkflowInstanceDispatched notifications
- Updated BackgroundWorkflowDispatcher to emit notifications before and after dispatch
- Added integration tests to verify notifications are emitted correctly

Co-authored-by: KnibbsyMan <23156317+KnibbsyMan@users.noreply.github.com>
2025-12-20 22:30:43 +00:00
Sipke Schoorstra fdf3e385b1
Fix Race Condition Causing Duplicate Trigger Registration in Multi-Engine Environments + Add Concurrency Tests to Prevent Regression (#7131)
* Add component tests for concurrent trigger indexing to prevent duplicate trigger registration in multi-engine workflows (#7130)

* Pin `dotnet-ef` version in `.config/dotnet-tools.json` to address migration bug, update EF Core script modules, and enhance `DesignTimeDbContextFactoryBase` with improved option description.

* Add migrations for EF Core SQLite and MySQL to include unique indexing on triggers.

* Update ConcurrentTriggerIndexing test to verify unique constraint enforcement and throw `DbUpdateException` for duplicate triggers

* Update src/modules/Elsa.Persistence.EFCore/Modules/Runtime/TriggerStore.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Persistence.EFCore/Modules/Runtime/TriggerStore.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update EF Core script to include `Management` module and restore `Sqlite` provider in migration process

* Rename `SimpleHttpApiWorkflow.cs` to `HttpWorkflow.cs` in ConcurrentTriggerIndexing test.

* Remove 'locks' folder from project

Removed the 'locks' folder from the project structure.

* Clean up migration script by removing redundant column type changes

* Update EF Core Oracle migrations to change NVARCHAR2 fields to NCLOB for larger data storage

* Update EF Core Oracle migrations to use NCLOB for larger data storage

* Remove retry logic and logger dependency from EFCoreTriggerStore in TriggerStore implementation

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-05 09:17:14 +01:00
Sipke Schoorstra a1d4e541fc
Add Elsa Script DSL (#7076)
* Update packages.yml

* Update elsa-server-and-studio.yml

* Update elsa-server.yml

* Update elsa-studio.yml (#6715)

* Update ListWorkflowDefinitionsRequest.cs (#6761)

Remove unnecessary line breaks

* Correct namespace and import for `ConfigureEngineWithVariableTypes`.

* Resolves build issues, update package versions and restructure project references

- Updated multiple package versions in `Directory.Packages.props` for better dependency management, including `BenchmarkDotNet`, `FastEndpoints`, and `Microsoft.Extensions.Http.Resilience`.
- Minor version upgrade for `System.Formats.Asn1` in `_build.csproj`.
- Replaced project reference to `Elsa.csproj` with `Elsa.IO.Http.csproj` in `Elsa.ServerAndStudio.Web.csproj`, enhancing modularity.
- Added new using directive for `Elsa.IO.Http.Features` in `Program.cs` to support new HTTP functionalities.

* Remove unused project references from Elsa.sln

These changes indicate that the associated projects or dependencies are no longer needed or have been replaced by other components in the solution.

* Rename copilot-setup-steps.yml.yml to copilot-setup-steps.yml

* Update RawStringContent encoding in JsonContentFactory (#6786)

* Update RawStringContent encoding in JsonContentFactory

Modified the instantiation of `RawStringContent` to use a
new `UTF8Encoding` instance with `encoderShouldEmitUTF8Identifier`
set to `false`, affecting the handling of the UTF-8 byte order
mark (BOM) in serialized JSON content. Fixes a bug with content length being different than expected.

* Refactor JsonContentFactory to reuse UTF8Encoding

Introduced a private static readonly field `_utf8Encoding` in the `JsonContentFactory` class to improve code readability and performance. This change replaces the instantiation of `UTF8Encoding` in the `CreateHttpContent` method, allowing for the reuse of the same encoding instance.

---------

Co-authored-by: Max Brooks <Max@compyl.com>

* Enhance thread safety with ConcurrentDictionary usage (#6760)

* Enhance thread safety with ConcurrentDictionary usage

Replaced `IDictionary` with `ConcurrentDictionary` for
both `_scheduledTasks` and `_scheduledTaskKeys` to
improve thread safety in a multi-threaded environment.

Updated methods `RegisterScheduledTask`,
`RemoveScheduledTask`, and `RemoveScheduledTasks` to
utilize the `Remove` method of `ConcurrentDictionary`,
ensuring safe and efficient removal of scheduled tasks.

* Refactor task registration and removal logic

Updated `RegisterScheduledTask` to use `AddOrUpdate` for streamlined task management. This change simplifies the addition and updating of scheduled tasks by consolidating logic into a single operation. Introduced `RemoveScheduledTask` method to handle task removal by name, improving code organization and clarity.

* Improve task removal handling in LocalScheduler

Modified the `LocalScheduler` class to enhance the removal process of scheduled tasks from the `_scheduledTaskKeys` collection. The removal operation now captures the result in a variable and includes a conditional check to log a warning if the task was not found, improving error handling and debugging capabilities.

* Refactor task removal in LocalScheduler

Updated the removal process for scheduled tasks in `_scheduledTasks`.
The new implementation collects all corresponding keys and attempts to remove them individually, logging warnings for any failures. This enhances error handling and provides better debugging information.

---------

Co-authored-by: Max Brooks <Max@compyl.com>

* Add IAsyncEnumerable check to ItemSourceActivityExecutionContextExtensions.GetItemSource (#6897)

* Use FullName in WorkflowDictionary (#6923)

* Fixed ParentWorkflowInstanceId not being set (#7029)

Co-authored-by: Peter Klooster <peter.klooster@autotaalglas.nl>

* Remove unused solution projects and update package references

- Deleted several project references from `Elsa.sln` to clean up the solution.
- Updated `Directory.Packages.props` for consistency and alignment with the latest package versions.

* Simplify CI pipeline by removing `Test` step from `Compile+Test+Pack` process.

* Initial plan

* Add ElsaScript DSL module with parser and compiler

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add integration tests for ElsaScript DSL

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add comprehensive documentation for ElsaScript DSL

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Refactor workflow activity instantiation logic

- Removed `ActivityFactory` and its related interfaces and extensions.
- Introduced `ActivityActivator` for handling activity creation.
- Extended AST with support for comprehensive workflow structures:
  - Added nodes for flowcharts, if/else, loops, and variable declarations.
- Updated `IElsaScriptCompiler` to use asynchronous methods.
- Expanded `ElsaScriptParser` to simplify syntax for `UseNode` and argument parsing.
- Adjusted compiler and parser for compatibility with new workflow AST model.

* Refactor test method names for clarity and add new compiler and parser tests

- Updated method names in `CompilerTests` and `ParserTests` for better readability and description of test intent.
- Added tests for compiler and parser:
  - Support for workflows without the `workflow` keyword.

* Refactor `ElsaScriptParser` to improve statement parsing and introduce a tokenizer

- Added `TokenizeStatements` method to split source into statements for enhanced parsing accuracy.
- Updated logic to process statements instead of raw lines, reducing parsing complexity and improving reliability.
- Improved handling of workflow and statement parsing, including edge cases with braces, parentheses, and string literals.

* Introduce ElsaScript support for BlobStorage workflow provider

- Added the `Elsa.WorkflowProviders.BlobStorage.ElsaScript` module to enable ElsaScript-based workflow definitions for BlobStorage.
- Implemented `ElsaScriptBlobWorkflowFormatHandler` for parsing ElsaScript workflows stored in BlobStorage.
- Extended `ElsaScriptParser` to leverage Parlot for improved DSL parsing.
- Introduced `IBlobWorkflowFormatHandler` to centralize workflow format handling and parsing.
- Updated `Elsa.Server.Web` to reference the new module and include an ElsaScript "Hello World" example workflow.

* Refactor ElsaScript services, update logging, and improve workflow handling

- Changed `ElsaScriptCompiler` service registration from `Singleton` to `Scoped` for better dependency management.
- Enhanced the "Hello World" example workflow and added `CopyToOutputDirectory` configuration.
- Removed unused namespaces and adjusted references in multiple projects to improve maintainability.
- Updated logging levels in `appsettings.json` to reduce unnecessary debug output.
- Improved `PolymorphicObjectConverter` by removing redundant dependencies.
- Added missing references to enhance feature support and ensure compatibility.

* Refactor activity instantiation and improve argument handling in `ElsaScriptCompiler`

- Added support for positional arguments with constructor matching logic.
- Refactored `InstantiateActivityUsingConstructor` to enhance activity creation.
- Updated `ActivityDescriptor` and related types to include `ClrType` for streamlined activity resolution.
- Simplified `TypedActivityProvider` by annotating it with `[UsedImplicitly]`.
- Adjusted `ElsaScriptParser` to remove unnecessary options from string literal definitions.

* Add HTTP-enabled "Hello World" workflow and support for additional HTTP activity constructors

- Introduced a new ElsaScript example workflow `hello-world-http.elsa` with an HTTP endpoint and response.
- Enhanced `HttpEndpoint` and `WriteHttpResponse` activities with additional constructors for improved flexibility.
- Updated project to include the new workflow in the output directory.

* Enhance `ElsaScriptParser` with a custom parser to handle nested raw expressions for ElsaScript workflows

- Introduced `RawExpressionParser` to parse raw text after `=>` up to a matching closing parenthesis.
- Updated `elsaExpressionWithLang` and `elsaExpressionWithoutLang` to use `RawExpressionParser`.
- Trimmed whitespace in parsed expressions.
- Added integration and parser tests for complex workflows with variables and expressions.
- Updated example workflow `hello-world-http.elsa` to demonstrate expression usage.
- Added `Elsa.Http` module reference to enable HTTP-based activities.

* Update "Hello World" workflow to simplify naming and enhance response logic

- Renamed workflow from `HelloWorldHttpDsl2` to `HelloWorldHttpDsl`.
- Updated HTTP endpoint path to `/hello-world-dsl` for consistency.
- Improved response logic by utilizing `getMessage()` JavaScript function.

* Add support for `OriginalSource` in workflow materialization and enhance ElsaScript materializer

- Introduced `OriginalSource` property in `WorkflowDefinition` and `MaterializedWorkflow` for preserving original source representation (e.g., ElsaScript, JSON, YAML).
- Added `ElsaScriptWorkflowMaterializer` implementation to materialize workflows directly from ElsaScript source.
- Updated `DefaultWorkflowDefinitionStorePopulator` to determine `StringData` or `OriginalSource` based on materialized workflow format.
- Enhanced `WorkflowDefinitionMapper` to support symmetric round-tripping with `OriginalSource`.
- Registered `ElsaScriptWorkflowMaterializer` in `ElsaScriptFeature` for dependency injection.
- Updated `JsonBlobWorkflowFormatHandler` and added `OriginalSource` support for round-trip preservation.
- Simplified `ElsaScriptParser` by aligning variable and parser naming.

* Update V3_6 migrations for PostgreSQL, MySQL, and Oracle databases and associated designer files.

* Handle disposal and race conditions in `ScheduledCronTask`

- Added `_disposed` flag to prevent accessing disposed resources.
- Updated `_executionSemaphore` and `_scopeFactory` logic to safely handle `ObjectDisposedException`.
- Enhanced task scheduling and timer disposal with additional safeguards against race conditions.
- Modified tests to ensure proper disposal and logging behavior when handling edge cases.

* Add support for metadata in ElsaScript workflows and enhance parser and compiler functionality

- Introduced metadata syntax in ElsaScript workflows (e.g., `DisplayName`, `Description`, `Version`) to enable metadata-driven behavior.
- Enhanced `ElsaScriptCompiler` to process metadata and properly integrate it into `Workflow` objects.
- Updated `ElsaScriptParser` to parse program-level AST with support for multiple workflows and global use statements.
- Refactored tests to validate metadata parsing and ensure backward compatibility with existing workflows.
- Added new test cases to cover scenarios like metadata parsing, compilation, and multi-workflow programs.

* Add support for `foreach` loops in ElsaScript and remove `let` keyword

- Introduced `foreach` loop syntax in `ElsaScriptParser` and `ElsaScriptCompiler`, enabling iteration over collections with optional variable declaration.
- Updated `ForNode` and `ForEachNode` to include a `DeclaresVariable` flag for improved variable handling.
- Removed support for the `let` keyword in variable declarations, streamlining syntax to use `var` and `const` only.
- Enhanced `for` loop syntax to support optional `var` declaration and block or single-statement bodies.
- Refactored test cases to validate `foreach` and `for` loop enhancements and ensure backward compatibility.

* Simplify ElsaScript workflow syntax by removing redundant quotes in workflow identifiers and updating `for` loop syntax for clarity and consistency.

* Remove redundant quotes from workflow identifiers in integration tests

* Simplify Elsa scripts and improve error handling

- Removed redundant braces in workflow declarations for streamlined syntax.
- Enhanced logging in `JsonBlobWorkflowFormatHandler` and `ElsaScriptBlobWorkflowFormatHandler` to warn on parsing errors and provide context.
- Updated configuration to log errors for `Elsa.Workflows.ActivityRegistry`.
- Refined "Hello World" and "For Loop" workflows for clarity and added improved loop handling.

* Refine Elsa workflows and update compiler logic

- Simplified "Hello World" workflow by adding braces and improving consistency.
- Adjusted "For Loop" workflow to rename and clarify logic, including expression updates and variable handling.
- Fixed compiler mapping of `"cs"` to `"CSharp"` for better clarity.
- Enhanced "Hello World HTTP" workflow to correctly reference `variables.message` in expressions.

* Add flowchart support in ElsaScript parser, compiler, and integration tests

- Introduced `flowchart` syntax in `ElsaScriptParser` to support flowchart-based workflows.
- Updated `ElsaScriptCompiler` to compile `flowchart` nodes with labeled activities, connections, entry points, and variables.
- Added integration tests for parsing and compiling empty and simple flowcharts.
- Enhanced `FlowchartNode` and `LabeledActivityNode` for better representation of flowchart structures.
- Improved error handling and logging for invalid flowchart configurations.

* Add tests for compiling and parsing flowcharts with nodes, connections, and block nodes in ElsaScript

- Added integration tests for compiling and validating flowchart structures, including activities, connections, and entry points.
- Implemented parser tests for parsing flowcharts with node connections and block nodes.
- Updated project files to include new workflow examples for testing.

* Add Parlot package and update project file in integration tests

- Added `Parlot` package version `0.0.27` to `Directory.Packages.props`.
- Updated integration test project file to include a new `Include` directive for better targeting.

* Update Parlot package to version 1.5.2 in Directory.Packages.props

* Remove `elsa-server-and-studio.yml` workflow and update solution file

- Deleted `elsa-server-and-studio.yml` workflow as it's no longer needed.
- Updated `Elsa.sln` to remove reference to the deleted workflow.

* Remove `elsa-studio.yml` workflow and update solution and packages

- Deleted `elsa-studio.yml` workflow as it's no longer used.
- Updated `Elsa.sln` to remove reference to the deleted workflow.
- Changed `base_version` in `packages.yml` from `3.7.0` to `3.6.0`.

* Downgrade Docker image in `elsa-server.yml` workflow from `v3.7.0-preview` to `v3.6.0-preview`

* Update Docker image tag in `elsa-server.yml` workflow from `v3.6.0-preview` to `v3.6-preview`

* Add logging support to `LocalScheduler` and replace `Debug.WriteLine` with `ILogger`

* Remove unused `System.Collections.Generic` and `Elsa.Extensions` imports in `LocalScheduler`

- Cleaned up unnecessary using directives to improve code readability and maintainability.
- Minor whitespace adjustment for consistent formatting.

* Remove unnecessary whitespace in `LocalScheduler` for consistent formatting

* Improve exception handling in blob workflow format handlers

- Updated exception handling in `ElsaScriptBlobWorkflowFormatHandler` and `JsonBlobWorkflowFormatHandler` to gracefully catch and log all exceptions during workflow parsing.
- Adjusted comments to clarify behavior for invalid user-provided files, ensuring the workflow loading process is not disrupted.

* Refactor blob workflow format handlers to use `SupportedExtensions` for improved file filtering

- Added `SupportedExtensions` property to all blob format handlers to optimize blob storage browsing.
- Simplified `CanHandle` logic by removing extension checks, leveraging `SupportedExtensions` for initial filtering.
- Updated comments for clarity and consistency across handlers.

* Refactor `DefaultWorkflowDefinitionStorePopulator` to simplify `stringData` assignment logic and improve readability

* Remove outdated comment in `CompilerTests` about skipped tests

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor `ElsaScriptCompiler` to streamline type conversion logic, improve language mapping, and enhance asynchronous flowchart compilation

* [WIP] Update ParseError printing based on feedback (#7082)

* Initial plan

* Fix ParseError formatting to use Message and Position properties

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Replace `as` casts with direct casts in ParserTests for null safety (#7083)

* Initial plan

* Replace 'as' casts with direct casts in ParserTests for better null safety

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Fix Oracle column types for OriginalSource and other large text fields (#7079)

* Initial plan

* Fix Oracle OriginalSource and StringData column types to handle large data

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Refactor tests to replace type checks with `Assert.IsType` for improved clarity and type safety

* Initial plan (#7080)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Add `Parlot` package reference and update solution structure by removing and reorganizing projects and workflows.

* Set default expression language to "JavaScript" in `ElsaScriptCompiler`.

* Add integration test to verify default expression language resets between ElsaScript compilations

* Simplify UTF-8 encoding in JsonContentFactory (#7081)

* Initial plan

* Remove explicit UTF8Encoding in JsonContentFactory and use Encoding.UTF8

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Fix test to use Encoding.UTF8.GetByteCount for multi-byte character support

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: Ender <37611092+zengande@users.noreply.github.com>
Co-authored-by: Matt <knibbsy10@live.com>
Co-authored-by: Max Brooks <45081361+MaxBrooks114@users.noreply.github.com>
Co-authored-by: Max Brooks <Max@compyl.com>
Co-authored-by: FuJa0815 <30809803+FuJa0815@users.noreply.github.com>
Co-authored-by: Peter Klooster <crashkonijn@gmail.com>
Co-authored-by: Peter Klooster <peter.klooster@autotaalglas.nl>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-25 19:57:50 +01:00
Sipke Schoorstra aee09bb06f
Improves workflow deletion and task scheduling (#7088)
* Refactor workflow instance deletion to use `IWorkflowRuntime` for enhanced coordination and separation of concerns.

* Remove `EnumerableTypeConverter` and update related usages for serialization.

- Deleted the `EnumerableTypeConverter` class and its JSON serialization logic.
- Removed associated type descriptor attribute in `DefaultFormattersFeature`.
- Updated `ObjectFormatter` to handle collection serialization directly with JSON.

* Remove `EnumerableTypeConverter` tests and consolidate serialization logic into `ObjectFormatter`.

- Deleted `EnumerableTypeConverterTests` as the related functionality was removed.
- Added comprehensive tests in `ObjectFormatterTests` to handle serialization of collections and arrays with JSON.

* Add integration tests for `TriggerIndexer` to handle workflows with failing materialization

- Introduced comprehensive test scenarios verifying `DeleteTriggersAsync` behavior when workflows fail to load or partially succeed.
- Enhanced error handling in `TriggerIndexer` to skip failed workflows while ensuring remaining workflows are processed.

* Add exception handling in `TriggerIndexer.DeleteTriggersAsync` and integration tests

- Enhanced `DeleteTriggersAsync` with exception handling to skip failed workflows while processing others.
- Logged warnings for failed workflows without halting execution.
- Added comprehensive integration tests to verify behavior across success, failure, and mixed scenarios.
- Refactored tests for improved clarity, maintainability, and consistency.

* Add exception handling for `ResumeWorkflowTask` to skip deleted workflow instances

- Enhanced `ResumeWorkflowTask.ExecuteAsync` to handle `WorkflowInstanceNotFoundException` gracefully when a scheduled workflow instance is missing.
- Logged warnings for skipped executions to improve observability.
- Ensured remaining workflows and scheduled tasks are processed seamlessly without disruption.

* Add thread safety to `LocalScheduler` to prevent race conditions during concurrent scheduling

- Introduced a `lock` object to synchronize access to internal dictionaries.
- Resolved `IndexOutOfRangeException` caused by concurrent modifications during startup.
- Ensured thread-safe operations in `ScheduleAsync`, `ClearScheduleAsync`, and related methods.
- Improved reliability and stability of scheduling under concurrent workloads.

* Improve exception handling, thread safety, and workflow instance deletion

- Added exception handling in `TriggerIndexer.DeleteTriggersAsync` to skip failed workflows while continuing processing.
- Enhanced `ResumeWorkflowTask` to handle missing workflow instances gracefully and log warnings.
- Introduced thread synchronization in `LocalScheduler` with `lock` to prevent concurrent access issues.
- Implemented and refactored tests to ensure behavior consistency and improve maintainability.
- Added component tests for workflow deletion scenarios, covering running, completed, and non-existent workflows.

* Add component tests for workflow instance deletion and refactor bulk delete logic

- Added comprehensive component tests for workflow instance deletion scenarios (running, completed, bulk, and non-existent instances).
- Refactored `BulkDelete` API to use `IWorkflowInstanceManager` for proper cleanup of related records (execution logs, activity executions, bookmarks).

* Add integration tests and fakes for `TriggerIndexer` to verify behavior with failing and successful workflows

- Introduced `FailingMaterializer` and `WorkingMaterializer` for simulating failing and successful workflow materializations.
- Added `TriggerDeletionTestScenario`, `TriggerTestDataBuilder`, and related test data classes to define comprehensive test cases.
- Updated `DeleteTriggersAsync` tests with scenarios for materialization failures and mixed success.
- Improved test coverage and maintainability with reusable test data builders and utilities.
2025-11-21 20:58:38 +01:00
Sipke Schoorstra 2c0b3da5de
Addresses warnings and enforces null safety (#7051)
* Enhance null-safety annotations across modules and refactor for improved consistency:

- Added `null!` annotations to enforce non-nullability expectations.
- Updated workflows, tests, and runtime services to handle default null values reliably.
- Removed obsolete and unused APIs, simplifying interfaces and improving maintainability.
- Refactored methods and properties for clarity, thread-safety, and consistency.
- Adjusted test configurations for code coverage tracking and integration improvements.

* Refactor activity iteration in container serialization tests to simplify type casting.
2025-11-12 16:56:31 +01:00
Sipke Schoorstra 490c8a2c9e
Addresses warnings and enforces null safety (#7050)
* Simplify test setup by removing `Input` wrapper and fix trimming warnings with `UnconditionalSuppressMessage` annotations.

* Add trimming warning suppression to `ArgumentJsonConverter` with `UnconditionalSuppressMessage`

- Annotated `Read` method to suppress `IL2055` warning caused by dynamic type resolution.
- Provided justification reflecting dynamic type registration in the well-known type registry.

* Add root namespace to project file and suppress EF Core usage warning

- Added `RootNamespace` property to `Elsa.Persistence.EFCore.Common.csproj` for consistency.
- Annotated `DbSchemaAwareMigrationAssembly` with `SuppressMessage` to suppress EF1001 usage warning and provide justification.

* Suppress obsolete API warnings in workflow runtime projects

- Added `#pragma warning disable CS0618` to suppress obsolete member warnings in `ObsoleteWorkflowRuntime.cs`, `LocalWorkflowRuntime.Obsolete.cs`, and `DistributedWorkflowRuntime.Obsolete.cs`.
- Updated `Directory.Build.props` to globally suppress `CS0618` for backward compatibility during migration.

* Annotate properties with non-nullable default initializers across modules to enforce null safety.

* Add `Priority` and `Deprecated` properties to `StorageDriverDescriptor` model

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowRuntime.Obsolete.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime/Services/ObsoleteWorkflowRuntime.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime/Services/LocalWorkflowRuntime.Obsolete.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Clean up XML comments and improve consistency across modules.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-12 15:53:23 +01:00
Sipke Schoorstra 54af5ea714
Merge remote-tracking branch 'origin/patch/3.5.2' into develop/3.6.0 2025-11-11 17:01:22 +01:00
Sipke Schoorstra a74502d3ad
Fix trigger indexing logic and add tests for trigger persistence after workflow reload. (#6958)
* Fix outbound connection processing logic

Corrected `completedActivityExcecutedByBackwardConnection` to `completedActivityExecutedByBackwardConnection`. Improved flowgraph outbound connection handling by separating visitation and processing logic, ensuring skipped connections are propagated consistently.

* Add tests for decision implicit join workflows

Introduce new integration tests to verify workflows with implicit joins on both decision outcomes. Added corresponding workflow definitions and updated the test project to ensure compatibility. Refactored connection visit logic for better readability and maintainability.

* Refactor trigger indexing logic and add tests for trigger persistence after workflow reload.

Streamlined conditional trigger indexing in `DefaultWorkflowDefinitionStorePopulator`. Introduced a test to validate trigger persistence across reloads after publishing a new workflow version.
2025-10-10 09:21:11 +02:00
Sipke Schoorstra d0e7657998
Update nullable argument defaults and enhance ForEach tests with additional edge cases. (#6928)
* Update nullable argument defaults and enhance ForEach tests with additional edge cases.

* Refactor input evaluation and JavaScript evaluator for clarity and consistency.

* Update src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.InputEvaluation.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-27 17:55:40 +02:00
Sipke Schoorstra 003e88c4da
Update nullable argument defaults and enhance ForEach tests with additional edge cases. (#6928)
* Update nullable argument defaults and enhance ForEach tests with additional edge cases.

* Refactor input evaluation and JavaScript evaluator for clarity and consistency.

* Update src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.InputEvaluation.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-27 17:50:19 +02:00
Sipke Schoorstra f24b4394cf
Add WorkflowStateCommitted notification support and update state handling logic
- Introduced `WorkflowStateCommitted` notification to encapsulate workflow execution context, state, and instance details.
- Updated `DefaultCommitStateHandler` to publish `WorkflowStateCommitted` via `IMediator`.
- Adjusted `DispatchWorkflowExtensions` to use `WorkflowStateCommitted` for workflow completion.

Updates KubernetesClient and Microsoft packages (#6917)

* Remove Proto.Cluster.Kubernetes dependency due to vulnerability

- Temporarily removed `Proto.Cluster.Kubernetes` package and provider integration because of a vulnerability in its dependency (https://avd.aquasec.com/nvd/2025/cve-2025-9708).
- Adjusted related cluster provider and remote configuration logic.
- Updated `PortAttribute` default parameter for clarity.

* Revert "Remove Proto.Cluster.Kubernetes dependency due to vulnerability"

This reverts commit 0720d970968e4f7338825407258b34ddffb1d2a4.

* Add KubernetesClient package and update MicrosoftVersion to 9.0.9

- Added `KubernetesClient` package to the project dependencies.
- Updated `MicrosoftVersion` to `9.0.9` in `Directory.Packages.props`.
Update Polly packages

- Bump Polly and Polly.Extensions package versions to 8.6.3.

Update `Microsoft.AspNetCore.Authorization` to use `MicrosoftVersion` property

Ensure Docker images ship CA trust and add TLS smoke tests (#6918)

Remove TlsSmoke project and related solution references

- Deleted `TlsSmoke` project files (`Program.cs` and `TlsSmoke.csproj`).
- Removed `TlsSmoke` project reference from the solution file (`Elsa.sln`).

Add comprehensive Copilot coding agent instructions for repository onboarding (#6920)

* Initial plan

* Add comprehensive .github/copilot-instructions.md with validated build instructions

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Add ForEach tests, introduce asynchronous workflow runner and enhance workflow events. (#6926)

* Introduce asynchronous workflow runner and enhance workflow events.

- Added `AsyncWorkflowRunner` to enable asynchronous workflow execution and result tracking.
- Introduced new event arguments, such as `ActivityExecutedEventArgs` and `WorkflowStateCommittedEventArgs`.
- Expanded `WorkflowEvents` class to include `ActivityExecuted`, `ActivityExecutedLogUpdated`, and `WorkflowStateCommitted` events.
- Refactored event arguments into the `Elsa.Testing.Shared.EventArgs` namespace.
- Enhanced tests with `AsyncWorkflowRunner` and new event-driven workflow scenarios.

* Refactor event argument classes to unify namespace and simplify inheritance

* Add shared component DotSettings file to support namespace exclusions
Refactor `WaitAsync` call in `DispatchWorkflowsTests` to remove unnecessary generic type.
2025-09-25 20:58:21 +02:00
Sipke Schoorstra 6e6a66e3f5
Add WorkflowStateCommitted notification support and update state handling logic
- Introduced `WorkflowStateCommitted` notification to encapsulate workflow execution context, state, and instance details.
- Updated `DefaultCommitStateHandler` to publish `WorkflowStateCommitted` via `IMediator`.
- Adjusted `DispatchWorkflowExtensions` to use `WorkflowStateCommitted` for workflow completion.

Updates KubernetesClient and Microsoft packages (#6917)

* Remove Proto.Cluster.Kubernetes dependency due to vulnerability

- Temporarily removed `Proto.Cluster.Kubernetes` package and provider integration because of a vulnerability in its dependency (https://avd.aquasec.com/nvd/2025/cve-2025-9708).
- Adjusted related cluster provider and remote configuration logic.
- Updated `PortAttribute` default parameter for clarity.

* Revert "Remove Proto.Cluster.Kubernetes dependency due to vulnerability"

This reverts commit 0720d970968e4f7338825407258b34ddffb1d2a4.

* Add KubernetesClient package and update MicrosoftVersion to 9.0.9

- Added `KubernetesClient` package to the project dependencies.
- Updated `MicrosoftVersion` to `9.0.9` in `Directory.Packages.props`.
Update Polly packages

- Bump Polly and Polly.Extensions package versions to 8.6.3.

Update `Microsoft.AspNetCore.Authorization` to use `MicrosoftVersion` property

Ensure Docker images ship CA trust and add TLS smoke tests (#6918)


Remove TlsSmoke project and related solution references

- Deleted `TlsSmoke` project files (`Program.cs` and `TlsSmoke.csproj`).
- Removed `TlsSmoke` project reference from the solution file (`Elsa.sln`).

Add comprehensive Copilot coding agent instructions for repository onboarding (#6920)

* Initial plan

* Add comprehensive .github/copilot-instructions.md with validated build instructions

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Add ForEach tests, introduce asynchronous workflow runner and enhance workflow events. (#6926)

* Introduce asynchronous workflow runner and enhance workflow events.

- Added `AsyncWorkflowRunner` to enable asynchronous workflow execution and result tracking.
- Introduced new event arguments, such as `ActivityExecutedEventArgs` and `WorkflowStateCommittedEventArgs`.
- Expanded `WorkflowEvents` class to include `ActivityExecuted`, `ActivityExecutedLogUpdated`, and `WorkflowStateCommitted` events.
- Refactored event arguments into the `Elsa.Testing.Shared.EventArgs` namespace.
- Enhanced tests with `AsyncWorkflowRunner` and new event-driven workflow scenarios.

* Refactor event argument classes to unify namespace and simplify inheritance

* Add shared component DotSettings file to support namespace exclusions
Refactor `WaitAsync` call in `DispatchWorkflowsTests` to remove unnecessary generic type.
2025-09-25 20:54:38 +02:00
Sipke Schoorstra 525f21bcc8
Merge remote-tracking branch 'origin/patch/3.5.1' into develop/3.6.0 2025-09-15 18:46:48 +02:00
Sipke Schoorstra b613ff6b61
Addresses workflow identity reload conflict (#6909)
* Refactor `IndexTriggersAsync` to use `WorkflowDefinition` and update null assignment for serialization logic.

* Change default parameter value from `default` to `null` in `UseFluentStorageProvider` method signature.

* Add in-memory workflows provider and materializer for integration tests

Introduced `InMemoryWorkflowsProvider` and `InMemoryWorkflowMaterializer` to support integration testing scenarios for workflow definition population. Enhanced workflow handling with fluent method `WithId` for `WorkflowBuilder`. Updated event publishing and dependency injection logic.

* Remove extraneous whitespace in DefaultWorkflowDefinitionStorePopulator.

* Refine test class documentation for `WorkflowDefinitionStorePopulation` scenario.
2025-09-15 11:56:01 +02:00
Sipke Schoorstra 72ad243ed4
Merge 3.5.1 into 3.6.0 (#6907)
* Introduce `IWorkflowResumer` and `ActivityInputEvaluatorContext`, refactor endpoint handling, extend logging, and improve bookmark queue processing.

* Remove deprecated WorkflowContexts module and optimize project.

Deleted the Elsa.Studio.WorkflowContexts module and references from solution files. Corrected minor errors in remaining code and updated project configurations to align with the new structure.```

* Update GitHub workflows to track `develop/3.6.0` branch instead of `patch/3.5.1`.

* Fix inconsistent formatting in `InputDescriptor` constructor and properties.

* Add XML documentation for `DictionaryValueEvaluator` in `UIHints/Dictionary` module

* Refactor `DictionaryValueEvaluator` to improve readability and simplify dictionary evaluation logic.
2025-09-13 13:05:52 +02:00
Sipke Schoorstra c019cfa3da
- Fix persistence issue by converting ChangeTrackingDictionary to a standard dictionary in BackgroundActivityInvoker.
- Adjust lock acquisition timeout in `appsettings.json` for improved distributed locking behavior.
- Remove unused Oracle setup and volume configurations from `docker-compose.yml`.
2025-09-12 19:58:19 +02:00
Sipke Schoorstra c549f49dfb
Fix race condition when sending same stimuli (#6895)
* Introduce `WorkflowResumer` service and deprecate `BookmarkResumer`.

- Adds `IWorkflowResumer` and its implementation for workflow resumption.
- Marks `BookmarkResumer` and related interfaces as obsolete.
- Refactors dependent services to use `WorkflowResumer`.
- Enhances `ResumeBookmarkRequest` to include `ActivityInstanceId`.
- Updates logging and queue handling logic to align with the new resumption approach.

* Update lock key prefix in `WorkflowResumer` for consistency with service naming.

* Add exception handling for distributed lock acquisition in `WorkflowResumer`

- Wrap distributed lock logic with `try-catch` to handle `TimeoutException`.
- Improve error message when lock acquisition fails due to timeout.
- Preserve existing workflow resumption behavior and logging.

* Update src/modules/Elsa.Workflows.Runtime/Filters/BookmarkFilter.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Optimize `BookmarkFilter` hashing logic for improved performance and readability.

* Merge remote-tracking branch 'origin/enh/locked-bookmark-resumption-2' into enh/locked-bookmark-resumption-2

* Remove unused variable and redundant line breaks for cleaner code.

* Clean up logging configuration by removing unused debug log levels.

* Update src/modules/Elsa.Workflows.Runtime/Services/WorkflowResumer.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Handle collections in `BookmarkFilter` hashing to ensure determinism and improve compatibility.

* Refactor `BookmarkFilter` hashing logic for clarity and consistency.

* Improve `TimeoutException` handling with a more descriptive message in `WorkflowResumer`.

* Update src/modules/Elsa.Workflows.Runtime/Filters/BookmarkFilter.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Simplify `BookmarkFilter` by utilizing `using` directives and refining type references.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-11 22:30:25 +02:00
Sipke Schoorstra 82c3ceeb08
Refactor workflow services and update project references
Fixed code indentation in WorkflowStateExtractor, adjusted output handling in LocalWorkflowClient, and replaced project references in Elsa.Api and Elsa.Http. Reactivated JavaScript engine configuration in Elsa.Server.Web for enhanced script execution capabilities.
2025-08-01 22:44:15 +02:00
Sipke Schoorstra 30139dedc6
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 2025-08-01 22:09:48 +02:00
Sipke Schoorstra 207356cf5b
Add Bookmarks property to workflow state mapping in LocalWorkflowClient 2025-07-30 13:11:02 +02:00
Matt cf6a17e5b3
Merge pull request #6789 from n84ck/develop/3.5.0
Update BackgroundStimulusDispatcher.cs
2025-07-28 23:34:23 +01:00
Sipke Schoorstra 9b1a76d047
Extend ActivityExecutionRecordSnapshot and update DefaultActivityExecutionMapper to include additional activity execution details. 2025-07-24 21:38:18 +02:00
Sipke Schoorstra 61179bcb53
Remove redundant activity metadata properties from ActivityExecutionRecordSnapshot and streamline mapping logic
Deleted unused metadata properties to simplify `ActivityExecutionRecordSnapshot`. Updated `GetOrMapCapturedActivityExecutionRecordAsync` to maintain serialized snapshots when mapping, ensuring consistency in activity execution records.
2025-07-24 19:47:26 +02:00
Sipke Schoorstra 931d242f40
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 2025-07-18 21:29:09 +02:00
Sipke Schoorstra ed14a1e577
Refactor activity execution record serialization with snapshots (#6807)
- Introduced `ActivityExecutionRecordSnapshot` for encapsulated serialized data.
- Updated `DefaultActivityExecutionMapper` to build serialized snapshots.
- Adjusted `ActivityExecutionLogStore` to persist pre-serialized snapshots.
- Streamlined package version management with `MicrosoftVersion` property.
2025-07-18 14:14:19 +02:00
Sipke Schoorstra b0fd60330a
Simplify activity execution record retrieval with async mapping extension method 2025-07-16 20:28:57 +02:00
Sipke Schoorstra adbea90ccd
Introduce activity execution record capturing and serialization improvements (#6800)
* Introduce activity execution record capturing and serialization improvements

- Added middleware for capturing activity execution records during workflow execution.
- Introduced async mapping in `DefaultActivityExecutionMapper` with additional serialization support.
- Enhanced `ActivityExecutionRecord` with new serialized properties for efficient storage.
- Updated extensions to include `UseActivityExecutionLogCapturing`.
- Simplified logging persistence by leveraging pre-serialized values in `ActivityExecutionLogStore`.

* Update src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime/Middleware/Activities/CaptureActivityExecutionRecordMiddleware.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-16 18:05:32 +02:00
Sipke Schoorstra cfb48ffcbf
Refactor database extensions and support migrations for V3.6 (#6788)
* Refactor database extensions and support migrations for V3.6

Remove `DatabaseFacadeExtensions` and introduce `IWorkflowReferenceQuery` with its default implementation. Implement database schema updates for PostgreSQL, MySQL, and Oracle to enhance compatibility with the V3.6 data structure.

* Remove commented-out code and standardize null default assignment in `IWorkflowDefinitionStore` interface
2025-07-14 09:48:22 +02:00
n84ck f11b7d5a21
Update BackgroundStimulusDispatcher.cs
Include tenant headers during command dispatch.
2025-07-12 18:40:52 +02:00
Sipke Schoorstra 2bf9ecc709
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 2025-06-13 19:09:08 +02:00
Sipke Schoorstra 0811a4042d
Merge remote-tracking branch 'origin/patch/3.4.1' into develop/3.5.0 2025-06-13 19:04:29 +02:00
Sipke Schoorstra c694a18c13
Enhances Mediator with Tenant Context Propagation (#6738)
* Update package versions in Directory.Packages.props

Upgraded multiple package dependencies to latest versions, ensuring compatibility, security, and access to the newest features.

* Refactor mediator pipeline to support tenant context propagation

- Introduced `TenantPropagatingMiddleware` to handle tenant context propagation during command execution.
- Added `SetupMediatorPipelines` hosted service for configuring mediator pipelines.
- Enhanced `CommandPipeline` and builder to allow middleware insertion, removal, and reordering.
- Updated `CommandContext` and related components to support headers for tenant context handling.
- Improved logging and refactored `BackgroundWorkflowDispatcher` to include tenant headers during command dispatch.

* Fix typos in XML documentation and improve middleware extension clarity

- Corrected duplicated slashes in XML doc comments in `ICommandSender.cs`.
- Refined phrasing in `MiddlewareExtensions.cs` to clarify method parameters and improve readability.

* Update src/common/Elsa.Mediator/Middleware/Command/Components/CommandLoggingMiddleware.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-13 14:26:51 +02:00
Sipke Schoorstra 3aae05317d
Refactor TriggerStore methods and update configurations.
Refactored `TriggerStore` to implement pagination and ordering for `FindManyAsync` methods with support for tenant-agnostic filtering. Modified app settings to change database provider to SQL Server and adjusted logging levels to reduce verbosity. Fixed workflow cancellation service to better handle child instances tasks.
2025-06-05 09:03:37 +02:00
Sipke Schoorstra 59bfcf0ba5
Add distributed workflow runtime implementation.
Introduced `DistributedWorkflowRuntime` to support distributed workflow execution with locking mechanisms. Added new module `Elsa.Workflows.Runtime.Distributed` with key services, features, and client implementations for handling distributed bookmarks and workflow clients. Updated integration and component tests to use the new distributed runtime where relevant.
2025-06-03 09:58:20 +02:00
Sipke Schoorstra 8fb797eaeb
Remove Elsa.Expressions.Dsl module and related DSL features
This commit completely removes the Elsa.Expressions.Dsl module, including all associated features, services, models, and dependencies. The DSL functionality, including its custom activities and integration options, has been fully eliminated from the project.
2025-06-02 21:52:10 +02:00
Sipke Schoorstra ddda9639ca
Merge remote-tracking branch 'origin/develop/3.5.0' 2025-06-01 18:19:32 +02:00
Sipke Schoorstra e09f096ef9
Adds activity execution metadata support (#6699)
* Add logging to DefaultActivityExecutionMapper constructor

Introduced an ILogger dependency to DefaultActivityExecutionMapper and added a debug log statement in GetPersistableDictionary. This aids in tracking log persistence mode for improved debugging and state visibility.

* Simplify activity execution log mapping logic.

Replaced asynchronous mapping with synchronous mapping to simplify the logic flow. This change reduces task overhead and improves code clarity while maintaining functionality.

* Update activity execution mapping and comment out unused method

Replaced direct dictionary usage with a cloned dictionary to ensure data integrity in `DefaultActivityExecutionMapper`. Commented out an unused method in `ActivityExecutionExtensions` to suppress its execution for now.

* Add logging to ActivityExecutionLogStore for property tracking

Integrate ILogger to track and log details of activity execution records, specifically focusing on properties and their serialization. This enhancement improves debugging and provides better insights into the execution flow.

* Introduce Metadata field for activity execution handling

Replaces the use of Properties with Metadata across activity execution models and services for storing lightweight, persistent data. Updated serialization, database schema, and relevant APIs to support this change while ensuring backward compatibility. Adjusted logging and extension methods for Metadata integration.

* Reset V3.5 Runtime Migrations

* Add EF Core migrations for MySQL and SQL Server schema updates

Introduced migrations to support schema changes for MySQL and SQL Server. Changes include new columns for bookmarks and activity execution records, updates to existing columns, and creation of additional indexes. These updates aim to enhance database structure and query performance.

* Add support for metadata in workflow execution context

Introduce a `Metadata` property to `ActivityExecutionContextState` to enhance workflow state management. Updated `WorkflowStateExtractor` to handle metadata merging and preservation. Added an alias for `RetryAttemptRecordList` in `ExpressionOptions` for improved type handling.

* Remove logger dependency from DefaultActivityExecutionMapper

Eliminated the ILogger dependency and related logging calls from DefaultActivityExecutionMapper to simplify the class. This reduces unnecessary coupling and streamlines the activity execution mapping process.
2025-05-30 15:37:57 +02:00
Sipke Schoorstra 7b75f0c89f
Implement retry attempt capturing (#6674)
* Update default initializations and input parameters to `null`

Replaced `default!` with explicit `null` for input parameters and properties throughout various classes. Adjusted constructors' default values for consistency and readability. This change ensures better clarity and alignment with nullable reference types.

* Add Polly-based resilience integration for retry tracking

Introduce Polly diagnostics to log retry events in the execution context. Updated resilience strategy interfaces and implementations to support Polly's context and retry event tracking.

* Refactor resilience and retry handling, add flaky endpoint.

Removed custom Polly-based diagnostic listeners and observers in favor of a transient status code utility class. Introduced a mock "flaky" endpoint for testing failure scenarios and updated configuration for resilience strategies. Minor namespace fixes

* Add retry attempt recording to resilience feature

Introduce `IRetryAttemptRecorder` and its implementations to enable recording of retry attempts during activity execution. Updated `ResilientActivityInvoker` to persist retry attempts and modified `ResilienceFeature` to support configurable retry attempt recorders.

* Add retry attempt tracking and retrieval functionality

Introduced mechanisms to track and fetch retry attempts, including new interfaces, reader implementations, API endpoints, and related models. These enhancements improve resilience tracking and data access for activity execution across workflows.

* Add GetOutcome method to RetryAttempt model

Introduce a GetOutcome method to encapsulate logic for determining the retry attempt's outcome. It prioritizes the Result, falls back to the Exception message, or defaults to "Unknown" if neither is available. This improves clarity and reusability of the outcome evaluation.

* Add scoped registration for _retryAttemptReader

This change ensures that _retryAttemptReader is registered in the DI container as a scoped service.

* Refactor retry mechanism to support detailed retry metadata

Introduced a `CollectRetryDetails` method to `IResilientActivity` for enhanced retry data collection. Updated `RetryAttemptRecord` to include a `Details` dictionary for capturing metadata, replacing previous `Result` and `Exception` fields. These changes simplify the retry recording process and improve extensibility for tracking retry details across activities.

* Add support for capturing background activity properties

Introduced functionality to capture and persist background activity properties during workflow execution. This includes defining a key for properties, capturing them in middleware, and storing them in the workflow execution context. These changes ensure properties are handled consistently alongside other activity data.

* Add support for storing and propagating activity execution properties

Introduced a `Properties` dictionary to track additional metadata in activity execution records and stats, enabling richer diagnostics and tracing. Refactored resilience logic to improve retry handling and propagate retry-related flags in workflows. Enhanced database queries to map serialized properties for execution summaries.

* Add retry propagation for background activity execution

Introduced a mechanism to propagate the retry-attempted flag across activity execution contexts. Added a new notification `BackgroundActivityExecutionCompleted` and updated related middleware to send this notification. Enhanced resilience features to handle and propagate retry state effectively.

* Refactor default parameters and values to use 'null'.

Replaced 'default' with 'null' for optional parameters and values in `AddExecutionLogEntry`, improving clarity and ensuring semantic consistency with nullable types. No functional changes were introduced.

* Refactor flaky endpoint and enhance resilience support.

Replaced the "Flaky" endpoint with a more robust "SimulateResponseEndpoint" under a new module. Introduced a status code lookup utility and improved resilience strategies with configurable backoff types. Updated serialization to support enum conversions and enhanced caching behavior for response simulation.

* Update activity execution models with nullable properties

Replaced `default!` initializations with `null!` to ensure correct handling of nullable string properties in `ActivityExecutionRecord`. Added a new `Properties` dictionary to `ActivityExecutionRecordSummary` to store additional activity execution data. This enhances model flexibility and data extensibility.

* Add support for recording resilience strategy in context

Introduced a new method to store resilience strategy details in the activity execution context for enhanced diagnostics. Updated `ResilientActivityInvoker` to serialize and set the resilience strategy using this method, leveraging `JsonSerializer`.

* Remove redundant PropertyNamingPolicy assignment

The PropertyNamingPolicy was set to the default value (CamelCase), making the assignment unnecessary. This change simplifies the code while maintaining existing functionality.

* Set JSON property naming policy to camelCase

Updated JSON serialization settings to use camelCase naming for property names. This improves consistency with standard JSON naming conventions and ensures compatibility with camelCase-based APIs.

* Remove unused Endpoints folder reference from project file

The Endpoints folder reference in the project file was unnecessary and has been removed. This cleanup helps maintain a tidy and accurate project structure.

* Remove unused RetryAttemptFilter and add Polly packages

Removed the obsolete RetryAttemptFilter class as it was no longer in use. Added Polly and Polly.Extensions packages to the project to support resilience and fault-handling strategies. This update aligns with keeping dependencies relevant and reducing unused artifacts.

* Add resilience integration test for FlowSendHttpRequest (#6692)

* Refactor and fix resilience test cases for clarity and accuracy

Simplified imports, adjusted code structure, and corrected attempt indexing logic in resilience tests. These changes improve readability, maintainability, and ensure accurate validation of retry attempts in test scenarios.
2025-05-26 11:47:09 +02:00
Sipke Schoorstra 6d414a79db
Merge remote-tracking branch 'origin/develop/3.5.0' 2025-05-21 09:55:35 +02:00
Sipke Schoorstra 1c2326ed80
feat(runtime): optional workflow output (#6659) 2025-05-20 11:57:29 +02:00
Sipke Schoorstra 2dcd852833
Enhances workflow runtime resilience and signaling (#6643)
* Refactor BookmarkQueueSignaler to use Channel for signaling.

Replaced TaskCompletionSource with a bounded Channel to improve concurrency control and simplify the code. This change ensures better handling of multiple producers while maintaining a single reader model.

* Refactor BookmarkQueueWorker to improve resilience and clarity

Replaced Debouncer with Throttler for rate limiting and added error handling to log exceptions and ensure the worker loop continues safely while allowing proper shutdown on cancellation.

* Handle missing workflow instance during bookmark resumption

Add exception handling for `WorkflowInstanceNotFoundException` to handle cases where the workflow instance does not exist. Log a debug message and return `ResumeBookmarkResult.NotFound()` when such instances are encountered. This ensures better error management and logging for bookmark resumption.

* Refactor default initializations and rename completion methods.

Replaced `default!` with `null!` for input properties to improve clarity and consistency. Renamed methods to better reflect their purpose, changing `CheckIfCompletedAsync` to `AttemptToCompleteAsync`. These changes enhance code readability and maintainability.

* Refactor to use specific exceptions for workflow errors

Replaced generic `InvalidOperationException` with `WorkflowInstanceNotFoundException` and `WorkflowGraphNotFoundException` for improved error context. This enhances clarity and enables more precise error handling.

* Change default value of WorkflowInstanceId to null

Updated the property `WorkflowInstanceId` to use `null!` instead of `default!` to better align with nullable reference type semantics. This ensures clarity and consistency in the codebase regarding expected default values.

* Add handling for WorkflowInstanceSaved in SignalBookmarkQueueWorker

Updated the SignalBookmarkQueueWorker to implement INotificationHandler for WorkflowInstanceSaved. This ensures that workflow instance save events now trigger the bookmark queue worker, improving event handling consistency.

* Update comment to clarify bookmark and workflow instance check

Expanded the comment to explain that the queue item is stored not only when a bookmark is missing but also when the associated workflow instance is not yet in the database. This improves clarity for future maintainers regarding queuing conditions.
2025-05-13 13:51:47 +02:00
Sipke Schoorstra 5d312a637c
Merge remote-tracking branch 'origin/rc/3.4.0' into develop/3.5.0 2025-04-28 19:06:20 +02:00
Sipke Schoorstra c8ed08cc85
Remove obsolete properties and refactor activity evaluation (#6603)
* Remove obsolete properties and refactor activity evaluation

Refactored activity input and log persistence property evaluation using improved notification handlers. Removed redundant `ActivityState` property and associated serialization logic, ensuring payloads are only serialized when necessary. All changes streamline workflow processing and enhance maintainability.

* Refactor mediator call to inline cancellation token.

Replaced the separate variable for the cancellation token with an inline reference for clarity and reduced redundancy. This simplifies the code without altering functionality.
2025-04-18 16:21:25 +02:00
Sipke Schoorstra 476656ccce
Implement internal state activity persistence and logging mechanisms (#6601)
* Implement internal state activity persistence and logging mechanisms

Updated property handling to support nullable dictionaries and improved persistable states. Adjusted serialization logic to handle optional fields more robustly, ensuring better compatibility with log persistence mappings and internal state evaluations.

* Replace default! with null! for string properties

Updated string properties in various records to use null! instead of default! for consistency and clarity. Additionally, adjusted methods to check collection existence before serialization and streamlined object initializations with simplified syntax where possible.

* Fix nullable types in DeserializeActivityState method

Updated the method's return type and JSON deserialization to properly handle nullable values. This ensures better alignment with the method's behavior and avoids potential null reference issues.

* Fix null reference issues in InputOutputLoggingTests

Replaced forced null dereferences with safe navigation checks to prevent potential null reference exceptions. This ensures more robust and error-free test execution for activity state validations.
2025-04-18 14:29:18 +02:00
Sipke Schoorstra b88d12cdd8
Fixed Log Persistence Mode Evaluation For Activity Execution (#6595)
* Refactor log persistence mode evaluation for activity execution

Updated functionality to evaluate and filter log persistence modes during activity execution. Added middleware, interfaces, and supporting methods to manage log configuration, ensuring only persistable inputs and outputs are retained in execution mapping. Updated pipeline to include the new middleware.

* Refactor log persistence mapping and evaluation logic

Updated log persistence evaluation to use separate input/output maps, ensuring a more modular and maintainable structure. Adjusted property key handling and introduced a dedicated helper method for evaluating persistence properties. Added middleware for log persistence mode evaluation to the workflow execution pipeline.

* Refactor log persistence logic and restructure namespaces.

Centralizes log persistence logic under a dedicated `LogPersistence` namespace. Simplifies interfaces and methods to streamline functionality, ensuring clarity and consistency in log persistence evaluations.

* Refactor log persistence methods for encapsulation.

Converted multiple public methods to private to enhance encapsulation and adherence to the principle of least privilege. Introduced a helper method `ResolveFinalLogPersistenceMode` to improve code clarity and maintainability.

* Refactor log persistence with improved type safety and structure

Updated methods and properties related to log persistence to remove nullable types, enhance clarity, and ensure type safety. Simplified configurations and refactored logic for evaluating persistence modes, reducing redundancy and improving maintainability.

* Add XML documentation to IActivityPropertyLogPersistenceEvaluator

This commit introduces XML comments to provide clarity on the purpose and functionality of the interface and its methods, aiding developers in understanding their usage and behavior during workflow execution.

* Refine log persistence logic in activity execution mapping.

Ensure that log persistence modes are correctly handled by adding explicit checks for `LogPersistenceMode.Inherit`. This prevents potential ambiguities and ensures accurate property mapping during workflow execution.

* Refactor state handling and log persistence evaluation.

Updated method signatures for stricter type consistency and improved readability. Introduced additional resolution step in log persistence to handle legacy configurations more effectively. This enhances code maintainability and alignment with expected behavior.

* Update src/modules/Elsa.Workflows.Core/Extensions/ActivityExtensions.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-17 18:03:06 +02:00
Sipke Schoorstra 2eec542f50
Merge remote-tracking branch 'origin/rc/3.4.0' 2025-04-16 08:41:04 +02:00
Sipke Schoorstra e5c397d4fb
Refactor workflow instance creation and execution logic. (#6586)
Updated `CreateAndRunInstanceAsync` to separate instance creation and execution with locking to handle nested workflow scenarios. Made `RunInstanceAsync` public to facilitate reuse in the distributed workflow client.
2025-04-14 11:57:14 +02:00
Sipke Schoorstra 4266421a4c
Refactor Fault Propagation and Simplify Bookmark Management (#6545)
* Refactor bookmark management and add new features

Streamlined bookmark handling by eliminating temporary storage in `ActivityExecutionContext` and directly managing bookmarks in `WorkflowExecutionContext`. Documented architectural decisions using ADRs.

* Regenerate EF Core migrations

* Refactor fault tracking to use AggregatedFaultCount property.

Replaces FaultCount with AggregatedFaultCount across the codebase to improve clarity and consistency in fault tracking. Updates related methods, properties, and data mappings to align with the new terminology. Fixes initialization issue with nullable inputs in Fault class.

* Add migration to track fault counts in runtime (V3.5)

This migration adds a new column, "AggregatedFaultCount," to the "ActivityExecutionRecords" table. The column is an integer, non-nullable, with a default value of 0, and enables tracking aggregated fault occurrences. The migration also includes a rollback to remove this column if needed.

* Fix typo in ADR 0004 regarding bookmark management convention

Corrected a spelling mistake in the ADR documentation by changing "determins" to "determines." This ensures clarity and maintains the professionalism of the document. No functional changes were made.

* Refine fault propagation logic for child-parent activities

Replace automatic fault transitions of parent activities with an aggregate fault count for descendant activities. This avoids premature state changes while still indicating child activity faults, improving workflow resilience and accuracy.

* Remove signal-driven fault propagation ADR and renumber bookmarks ADR

The ADR for signal-driven fault propagation was deleted, and the direct bookmark management ADR was renamed and renumbered accordingly. Related references in the table of contents, graph, and solution file were updated to reflect these changes.

* Refactor DeleteBookmarks to improve readability.

Reformatted the BookmarkFilter initialization for better clarity and maintainability. This change ensures the code is more aligned with modern C# conventions and improves overall readability. No behavior or functionality has been altered.

* Refactor naming for "AggregatedFaultCount" to "AggregateFaultCount"

Standardized the terminology across the codebase and migrations by renaming all references of "AggregatedFaultCount" to "AggregateFaultCount" for improved consistency and readability. Updated relevant logic, models, migrations, and database contexts accordingly.
2025-04-10 15:01:49 +02:00
Sipke Schoorstra e2288d0b34
Fix infinitely waiting Alterations Workflow (#6561)
* Refactor TenantId string handling and nullability checks.

Moved `StringExtensions` to a common module for reuse. Updated tenant-related logic to utilize null-safe string extensions, enhancing consistency and simplifying nullability handling across the codebase.

* Set StrictMode to false by default in ObjectConverter

Modified the default value of StrictMode to `false` to enable the original flexible behavior. Developers can opt into strict mode by explicitly setting it to `true`. This change aims to enhance backward compatibility and minimize unexpected strict conversions.

* Add tenant ID retrieval to ElsaDbContextBase constructor

Retrieve the current tenant ID if available using ITenantAccessor and assign it to the TenantId property. This ensures proper handling of multi-tenancy scenarios in the database context initialization.

* Disable DbContext pooling, manual OTEL instrumentation, and strict mode.

DbContext pooling is turned off to prevent potential issues with shared context instances. Manual OpenTelemetry instrumentation is disabled to rely on automatic instrumentation instead. Strict mode is also disabled to allow more flexibility in object conversion.

* Fix infinitely waiting Alterations Workflow

Replaced workflow dispatch logic with BookmarkQueue and StimulusHasher for triggering workflows. This fixes the issue where the Alterations workflow would signal completion while a later step awaits a completion bookmark. The Bookmark Queue now handles this.
2025-04-05 10:37:05 +02:00