Commit graph

108 commits

Author SHA1 Message Date
Sipke Schoorstra faebea76a0
Adds Publish Event Activity Tests (#7093)
* Refactor workflow instance deletion to use `IWorkflowRuntime` for enhanced coordination and separation of concerns.

* Remove `EnumerableTypeConverter` and update related usages for serialization.

- Deleted the `EnumerableTypeConverter` class and its JSON serialization logic.
- Removed associated type descriptor attribute in `DefaultFormattersFeature`.
- Updated `ObjectFormatter` to handle collection serialization directly with JSON.

* Remove `EnumerableTypeConverter` tests and consolidate serialization logic into `ObjectFormatter`.

- Deleted `EnumerableTypeConverterTests` as the related functionality was removed.
- Added comprehensive tests in `ObjectFormatterTests` to handle serialization of collections and arrays with JSON.

* Add integration tests for `TriggerIndexer` to handle workflows with failing materialization

- Introduced comprehensive test scenarios verifying `DeleteTriggersAsync` behavior when workflows fail to load or partially succeed.
- Enhanced error handling in `TriggerIndexer` to skip failed workflows while ensuring remaining workflows are processed.

* Add exception handling in `TriggerIndexer.DeleteTriggersAsync` and integration tests

- Enhanced `DeleteTriggersAsync` with exception handling to skip failed workflows while processing others.
- Logged warnings for failed workflows without halting execution.
- Added comprehensive integration tests to verify behavior across success, failure, and mixed scenarios.
- Refactored tests for improved clarity, maintainability, and consistency.

* Add exception handling for `ResumeWorkflowTask` to skip deleted workflow instances

- Enhanced `ResumeWorkflowTask.ExecuteAsync` to handle `WorkflowInstanceNotFoundException` gracefully when a scheduled workflow instance is missing.
- Logged warnings for skipped executions to improve observability.
- Ensured remaining workflows and scheduled tasks are processed seamlessly without disruption.

* Add thread safety to `LocalScheduler` to prevent race conditions during concurrent scheduling

- Introduced a `lock` object to synchronize access to internal dictionaries.
- Resolved `IndexOutOfRangeException` caused by concurrent modifications during startup.
- Ensured thread-safe operations in `ScheduleAsync`, `ClearScheduleAsync`, and related methods.
- Improved reliability and stability of scheduling under concurrent workloads.

* Improve exception handling, thread safety, and workflow instance deletion

- Added exception handling in `TriggerIndexer.DeleteTriggersAsync` to skip failed workflows while continuing processing.
- Enhanced `ResumeWorkflowTask` to handle missing workflow instances gracefully and log warnings.
- Introduced thread synchronization in `LocalScheduler` with `lock` to prevent concurrent access issues.
- Implemented and refactored tests to ensure behavior consistency and improve maintainability.
- Added component tests for workflow deletion scenarios, covering running, completed, and non-existent workflows.

* Add component tests for workflow instance deletion and refactor bulk delete logic

- Added comprehensive component tests for workflow instance deletion scenarios (running, completed, bulk, and non-existent instances).
- Refactored `BulkDelete` API to use `IWorkflowInstanceManager` for proper cleanup of related records (execution logs, activity executions, bookmarks).

* Add integration tests and fakes for `TriggerIndexer` to verify behavior with failing and successful workflows

- Introduced `FailingMaterializer` and `WorkingMaterializer` for simulating failing and successful workflow materializations.
- Added `TriggerDeletionTestScenario`, `TriggerTestDataBuilder`, and related test data classes to define comprehensive test cases.
- Updated `DeleteTriggersAsync` tests with scenarios for materialization failures and mixed success.
- Improved test coverage and maintainability with reusable test data builders and utilities.

* Refactor `ActivityExecutionContextExtensions` to use instance methods for improved readability and encapsulation

* Refactor extension methods to use instance methods for improved encapsulation and readability in core workflow modules

* Add component tests for event-based workflows and update usages of `Event` activity

- Added `BlockingEventWorkflow` and `TriggerEventWorkflow` for testing event-based workflow scenarios.
- Added `EventTests` to verify workflow behavior with event publishing and triggering.
- Refactored existing integration tests to use `Runtime.Activities.Event` for consistency.

* Add unit tests for `EventBase` functionality

- Introduced `EventBaseTests` to validate core `EventBase` logic, including bookmark creation, event stimulus handling, and callback invocation.
- Added tests for scenarios involving event payloads, trigger indexing, and result output determination.
- Verified behavior consistency with various event names and callback executions.

* Add tests and workflows to validate event publishing and consumption

- Introduced `ConsumerWorkflow`, `PublishGlobalEventWorkflow`, and `PublishAndConsumeEventWorkflow` to test global and local event publishing scenarios.
- Added component tests (`PublishEventTests`) to verify event propagation and workflow triggering mechanisms.
- Implemented unit tests for `PublishEvent` with various parameters (event name, payload, correlation ID).

* Remove unused `using` directives in event-related component tests and workflows

* Refactor `PublishEventTests` and `EventBaseTests` to improve test coverage, simplify test logic, and consolidate duplicate code.

* Add `NullIfWhiteSpace` extension method and update `PublishEvent` logic to use it in correlation ID handling

- Refactored `PublishEventTests` to account for cases where correlation ID is whitespace.
- Improved test coverage for `PublishEvent` activity with additional inline test cases.

* Refactor `PublishEventTests` to verify payload transmission and enhance `ConsumerWorkflow` to capture and validate event payloads.

* Refactor `PublishEventTests` to add timeout mechanism for workflow instance retrieval; enhance `ConsumerWorkflow` to declare output variable for payload validation.

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/Activities/Primitives/Event/PublishEventTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove `EventBaseTests` and `CancelInboundAncestorsAsync` for cleanup and redundant logic removal.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-25 19:06:16 +01:00
Sipke Schoorstra aee09bb06f
Improves workflow deletion and task scheduling (#7088)
* Refactor workflow instance deletion to use `IWorkflowRuntime` for enhanced coordination and separation of concerns.

* Remove `EnumerableTypeConverter` and update related usages for serialization.

- Deleted the `EnumerableTypeConverter` class and its JSON serialization logic.
- Removed associated type descriptor attribute in `DefaultFormattersFeature`.
- Updated `ObjectFormatter` to handle collection serialization directly with JSON.

* Remove `EnumerableTypeConverter` tests and consolidate serialization logic into `ObjectFormatter`.

- Deleted `EnumerableTypeConverterTests` as the related functionality was removed.
- Added comprehensive tests in `ObjectFormatterTests` to handle serialization of collections and arrays with JSON.

* Add integration tests for `TriggerIndexer` to handle workflows with failing materialization

- Introduced comprehensive test scenarios verifying `DeleteTriggersAsync` behavior when workflows fail to load or partially succeed.
- Enhanced error handling in `TriggerIndexer` to skip failed workflows while ensuring remaining workflows are processed.

* Add exception handling in `TriggerIndexer.DeleteTriggersAsync` and integration tests

- Enhanced `DeleteTriggersAsync` with exception handling to skip failed workflows while processing others.
- Logged warnings for failed workflows without halting execution.
- Added comprehensive integration tests to verify behavior across success, failure, and mixed scenarios.
- Refactored tests for improved clarity, maintainability, and consistency.

* Add exception handling for `ResumeWorkflowTask` to skip deleted workflow instances

- Enhanced `ResumeWorkflowTask.ExecuteAsync` to handle `WorkflowInstanceNotFoundException` gracefully when a scheduled workflow instance is missing.
- Logged warnings for skipped executions to improve observability.
- Ensured remaining workflows and scheduled tasks are processed seamlessly without disruption.

* Add thread safety to `LocalScheduler` to prevent race conditions during concurrent scheduling

- Introduced a `lock` object to synchronize access to internal dictionaries.
- Resolved `IndexOutOfRangeException` caused by concurrent modifications during startup.
- Ensured thread-safe operations in `ScheduleAsync`, `ClearScheduleAsync`, and related methods.
- Improved reliability and stability of scheduling under concurrent workloads.

* Improve exception handling, thread safety, and workflow instance deletion

- Added exception handling in `TriggerIndexer.DeleteTriggersAsync` to skip failed workflows while continuing processing.
- Enhanced `ResumeWorkflowTask` to handle missing workflow instances gracefully and log warnings.
- Introduced thread synchronization in `LocalScheduler` with `lock` to prevent concurrent access issues.
- Implemented and refactored tests to ensure behavior consistency and improve maintainability.
- Added component tests for workflow deletion scenarios, covering running, completed, and non-existent workflows.

* Add component tests for workflow instance deletion and refactor bulk delete logic

- Added comprehensive component tests for workflow instance deletion scenarios (running, completed, bulk, and non-existent instances).
- Refactored `BulkDelete` API to use `IWorkflowInstanceManager` for proper cleanup of related records (execution logs, activity executions, bookmarks).

* Add integration tests and fakes for `TriggerIndexer` to verify behavior with failing and successful workflows

- Introduced `FailingMaterializer` and `WorkingMaterializer` for simulating failing and successful workflow materializations.
- Added `TriggerDeletionTestScenario`, `TriggerTestDataBuilder`, and related test data classes to define comprehensive test cases.
- Updated `DeleteTriggersAsync` tests with scenarios for materialization failures and mixed success.
- Improved test coverage and maintainability with reusable test data builders and utilities.
2025-11-21 20:58:38 +01:00
Sipke Schoorstra 490c8a2c9e
Addresses warnings and enforces null safety (#7050)
* Simplify test setup by removing `Input` wrapper and fix trimming warnings with `UnconditionalSuppressMessage` annotations.

* Add trimming warning suppression to `ArgumentJsonConverter` with `UnconditionalSuppressMessage`

- Annotated `Read` method to suppress `IL2055` warning caused by dynamic type resolution.
- Provided justification reflecting dynamic type registration in the well-known type registry.

* Add root namespace to project file and suppress EF Core usage warning

- Added `RootNamespace` property to `Elsa.Persistence.EFCore.Common.csproj` for consistency.
- Annotated `DbSchemaAwareMigrationAssembly` with `SuppressMessage` to suppress EF1001 usage warning and provide justification.

* Suppress obsolete API warnings in workflow runtime projects

- Added `#pragma warning disable CS0618` to suppress obsolete member warnings in `ObsoleteWorkflowRuntime.cs`, `LocalWorkflowRuntime.Obsolete.cs`, and `DistributedWorkflowRuntime.Obsolete.cs`.
- Updated `Directory.Build.props` to globally suppress `CS0618` for backward compatibility during migration.

* Annotate properties with non-nullable default initializers across modules to enforce null safety.

* Add `Priority` and `Deprecated` properties to `StorageDriverDescriptor` model

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowRuntime.Obsolete.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime/Services/ObsoleteWorkflowRuntime.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime/Services/LocalWorkflowRuntime.Obsolete.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Clean up XML comments and improve consistency across modules.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-12 15:53:23 +01:00
FuJa0815 c017082f01
Added IEnumerableTypeConverter (#7020)
* Added IEnumerableTypeConverter

* fixed bug and added tests

* Add resource disposal mechanism in ScheduledCronTask

- Introduced `_disposed` flag to prevent execution after disposal.
- Updated disposal logic to ensure proper release of resources.
- Adjusted unit tests to verify new disposal behavior and prevent unintended timer actions.

* Enhance scheduling tasks with edge case handling and disposal improvements

- Added `_disposed` flag to `ScheduledRecurringTask` and `ScheduledSpecificInstantTask` to prevent execution after disposal.
- Adjusted timer setup logic to handle zero/negative delays with a minimum delay of 1ms.
- Updated disposal logic to ensure proper resource cleanup even during timer callbacks.
- Introduced extensive unit tests for edge cases such as small, zero, or negative delay scenarios and proper disposal behavior.

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/VariablesArray/Activities/RemoveTopElementStep.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Common/Serialization/IEnumerableTypeConverter.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Common/Serialization/IEnumerableTypeConverter.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add integration tests for EnumerableTypeConverter and update solution file

- Introduced `Elsa.Common.IntegrationTests` project for testing serialization behavior in `EnumerableTypeConverter`.
- Added tests to verify proper handling of strings, byte arrays, and collections during JSON serialization.
- Registered `EnumerableTypeConverter` in `DefaultFormattersFeature`.
- Renamed `IEnumerableTypeConverter` to `EnumerableTypeConverter` for consistency.
- Updated solution file to include the new integration test project.

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-12 11:07:33 +01:00
Sipke Schoorstra 72ad243ed4
Merge 3.5.1 into 3.6.0 (#6907)
* Introduce `IWorkflowResumer` and `ActivityInputEvaluatorContext`, refactor endpoint handling, extend logging, and improve bookmark queue processing.

* Remove deprecated WorkflowContexts module and optimize project.

Deleted the Elsa.Studio.WorkflowContexts module and references from solution files. Corrected minor errors in remaining code and updated project configurations to align with the new structure.```

* Update GitHub workflows to track `develop/3.6.0` branch instead of `patch/3.5.1`.

* Fix inconsistent formatting in `InputDescriptor` constructor and properties.

* Add XML documentation for `DictionaryValueEvaluator` in `UIHints/Dictionary` module

* Refactor `DictionaryValueEvaluator` to improve readability and simplify dictionary evaluation logic.
2025-09-13 13:05:52 +02:00
Sipke Schoorstra bc3543fcb6
Merge 3.5.0 2025-08-06 20:40:27 +02:00
Sipke Schoorstra 133e8dfc63
Introduce TenantDeleted event to handle tenant cleanup (#6843)
* Introduce `TenantDeleted` event to handle tenant cleanup

Added a `TenantDeleted` event to differentiate between tenant deactivation and deletion. Updated event handlers and services to support unregistering resources only during tenant deletion, ensuring clearer separation of responsibilities. Included ADR documentation for the new event.

* Remove unnecessary blank line in tenant deactivation logic
2025-08-06 08:43:39 +02:00
Sipke Schoorstra 931d242f40
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 2025-07-18 21:29:09 +02:00
Sipke Schoorstra ed14a1e577
Refactor activity execution record serialization with snapshots (#6807)
- Introduced `ActivityExecutionRecordSnapshot` for encapsulated serialized data.
- Updated `DefaultActivityExecutionMapper` to build serialized snapshots.
- Adjusted `ActivityExecutionLogStore` to persist pre-serialized snapshots.
- Streamlined package version management with `MicrosoftVersion` property.
2025-07-18 14:14:19 +02:00
Sipke Schoorstra ed49f21895
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 2025-07-04 20:35:53 +02:00
lukhipolito-nexxbiz c4451c4f40
Feat/6732 zip archive activities (#6751)
* Io module and content strategies

* Restructuring the folders and projects

* Create zip activity + shared content resolver

* Fixing build on github.com

* Update src/modules/Elsa.IO/Services/ContentResolver.cs

explicit nullable field

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* QoL improvements

* More QoL improvements + removing some logic duplications

* Update src/modules/Elsa.IO.Compression/Activities/CreateZipArchive.cs

avoiding duplicate extensions

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Assigning file path correctly to the variable

* More QoL changes, new Elsa.IO.Http module, more modular content strategy

* Addresing QoL review comments + Correct extension handling

---------

Co-authored-by: lucas.hipolito <lukhipolito@yahoo.com.br>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-30 16:56:51 +02:00
Sipke Schoorstra 1f2353cad8
Remove obsolete modules and associated code
Eliminated the `Elsa.Labels`, `Elsa.Environments`, and `Elsa.OpenTelemetry` modules along with their handlers, contracts, models, and related functionality. This cleanup improves maintainability and aligns the codebase with recent architectural changes.
2025-06-02 20:05:10 +02:00
Sipke Schoorstra 7b75f0c89f
Implement retry attempt capturing (#6674)
* Update default initializations and input parameters to `null`

Replaced `default!` with explicit `null` for input parameters and properties throughout various classes. Adjusted constructors' default values for consistency and readability. This change ensures better clarity and alignment with nullable reference types.

* Add Polly-based resilience integration for retry tracking

Introduce Polly diagnostics to log retry events in the execution context. Updated resilience strategy interfaces and implementations to support Polly's context and retry event tracking.

* Refactor resilience and retry handling, add flaky endpoint.

Removed custom Polly-based diagnostic listeners and observers in favor of a transient status code utility class. Introduced a mock "flaky" endpoint for testing failure scenarios and updated configuration for resilience strategies. Minor namespace fixes

* Add retry attempt recording to resilience feature

Introduce `IRetryAttemptRecorder` and its implementations to enable recording of retry attempts during activity execution. Updated `ResilientActivityInvoker` to persist retry attempts and modified `ResilienceFeature` to support configurable retry attempt recorders.

* Add retry attempt tracking and retrieval functionality

Introduced mechanisms to track and fetch retry attempts, including new interfaces, reader implementations, API endpoints, and related models. These enhancements improve resilience tracking and data access for activity execution across workflows.

* Add GetOutcome method to RetryAttempt model

Introduce a GetOutcome method to encapsulate logic for determining the retry attempt's outcome. It prioritizes the Result, falls back to the Exception message, or defaults to "Unknown" if neither is available. This improves clarity and reusability of the outcome evaluation.

* Add scoped registration for _retryAttemptReader

This change ensures that _retryAttemptReader is registered in the DI container as a scoped service.

* Refactor retry mechanism to support detailed retry metadata

Introduced a `CollectRetryDetails` method to `IResilientActivity` for enhanced retry data collection. Updated `RetryAttemptRecord` to include a `Details` dictionary for capturing metadata, replacing previous `Result` and `Exception` fields. These changes simplify the retry recording process and improve extensibility for tracking retry details across activities.

* Add support for capturing background activity properties

Introduced functionality to capture and persist background activity properties during workflow execution. This includes defining a key for properties, capturing them in middleware, and storing them in the workflow execution context. These changes ensure properties are handled consistently alongside other activity data.

* Add support for storing and propagating activity execution properties

Introduced a `Properties` dictionary to track additional metadata in activity execution records and stats, enabling richer diagnostics and tracing. Refactored resilience logic to improve retry handling and propagate retry-related flags in workflows. Enhanced database queries to map serialized properties for execution summaries.

* Add retry propagation for background activity execution

Introduced a mechanism to propagate the retry-attempted flag across activity execution contexts. Added a new notification `BackgroundActivityExecutionCompleted` and updated related middleware to send this notification. Enhanced resilience features to handle and propagate retry state effectively.

* Refactor default parameters and values to use 'null'.

Replaced 'default' with 'null' for optional parameters and values in `AddExecutionLogEntry`, improving clarity and ensuring semantic consistency with nullable types. No functional changes were introduced.

* Refactor flaky endpoint and enhance resilience support.

Replaced the "Flaky" endpoint with a more robust "SimulateResponseEndpoint" under a new module. Introduced a status code lookup utility and improved resilience strategies with configurable backoff types. Updated serialization to support enum conversions and enhanced caching behavior for response simulation.

* Update activity execution models with nullable properties

Replaced `default!` initializations with `null!` to ensure correct handling of nullable string properties in `ActivityExecutionRecord`. Added a new `Properties` dictionary to `ActivityExecutionRecordSummary` to store additional activity execution data. This enhances model flexibility and data extensibility.

* Add support for recording resilience strategy in context

Introduced a new method to store resilience strategy details in the activity execution context for enhanced diagnostics. Updated `ResilientActivityInvoker` to serialize and set the resilience strategy using this method, leveraging `JsonSerializer`.

* Remove redundant PropertyNamingPolicy assignment

The PropertyNamingPolicy was set to the default value (CamelCase), making the assignment unnecessary. This change simplifies the code while maintaining existing functionality.

* Set JSON property naming policy to camelCase

Updated JSON serialization settings to use camelCase naming for property names. This improves consistency with standard JSON naming conventions and ensures compatibility with camelCase-based APIs.

* Remove unused Endpoints folder reference from project file

The Endpoints folder reference in the project file was unnecessary and has been removed. This cleanup helps maintain a tidy and accurate project structure.

* Remove unused RetryAttemptFilter and add Polly packages

Removed the obsolete RetryAttemptFilter class as it was no longer in use. Added Polly and Polly.Extensions packages to the project to support resilience and fault-handling strategies. This update aligns with keeping dependencies relevant and reducing unused artifacts.

* Add resilience integration test for FlowSendHttpRequest (#6692)

* Refactor and fix resilience test cases for clarity and accuracy

Simplified imports, adjusted code structure, and corrected attempt indexing logic in resilience tests. These changes improve readability, maintainability, and ensure accurate validation of retry attempts in test scenarios.
2025-05-26 11:47:09 +02:00
Sipke Schoorstra 9aa239719d
Add IResilienceStrategy Abstraction with Category Matching and Expression-Based Configuration (#6637)
* Add resilience module with core interfaces and services

Introduced a new `Elsa.Resilience` module and its core components to support resilient services and activities. This includes resilience strategies, providers, and attributes, along with integration into the existing HTTP module for enhanced fault tolerance. Added solution and project references for proper dependency management.

* Add resilience strategy framework with HTTP strategy support

Introduced a resilience strategy architecture, including a configurable `HttpResilienceStrategy` with retry capabilities, strategy serialization, and integration with existing modules. Enhanced ResilienceFeature to support registration of strategy types and updated application configuration to enable resilience strategies. This change ensures more robust and fault-tolerant HTTP request handling.

* Add JSON serialization support for resilience configuration

Introduced `ConfigurationExtensions` to enable JSON serialization of configuration sections. Updated resilience strategies to utilize the new extension methods and adjusted JSON serialization logic to support polymorphism with `$type` discriminator. Minor modifications were made to support deserialization and property mutability.

* Add resilience strategy support to workflows and API clients

Introduced resilience strategy configuration, serialization, and execution support across workflows and API clients. Added new APIs, models, and services to enhance fault tolerance capabilities for activities and HTTP interactions.

* Refactor resilience services for improved modularity.

Replaced `IResilienceService` with new modular interfaces (`IResilienceStrategyCatalog`, `IResilienceStrategyConfigEvaluator`, `IResilientActivityInvoker`) and corresponding implementations. Enhanced maintainability by simplifying components and responsibilities, ensuring better separation of concerns.

* Rename methods in ResilienceStrategyCatalog for clarity

Updated method names in `ResilienceStrategyCatalog` and its interfaces for better readability and alignment with naming conventions. Replaced `GetAllStrategiesAsync` with `ListAsync` and `GetStrategyAsync` with `GetAsync` across the codebase.

* Refactor resilience handling in HTTP activities.

Replaced `ResilienceCategory` property with `ResilienceCategoryAttribute` for a cleaner implementation. Updated `IResilientActivity` to simplify its interface and adjusted related modifications accordingly. Introduced `IResilientActivityInvoker` to enhance resilience strategy execution.

* Add support for additional resilience and scripting features

Extended resilience strategy handling with serialization support, added `HttpResilienceStrategy` type in JavaScript handler, and refined object conversion logic for interfaces. Minor adjustments to `Expression` class properties for consistency.

* Remove `UseResilience` call from Program.cs

This call was redundant and no longer necessary for the application. Its removal simplifies the code and ensures only required middleware is used.

* Remove commented-out JSON converter code in serializer setup

Cleaned up unused and commented-out converter initialization code in `ResilienceStrategySerializer`. This improves readability and removes unnecessary clutter from the file.

* Fix typo in method names from 'Resiliency' to 'Resilience'

Renamed methods to maintain consistency in naming conventions across the codebase. This change ensures clarity and alignment with established terminology.

* Refactor namespace for ConfigurationResilienceStrategySource

Updated the namespace of ConfigurationResilienceStrategySource to "StrategySources" for better alignment with naming conventions and structure. Removed an unused namespace reference in ResilienceFeature for cleanup.

* Mark EnableResiliency as obsolete in SendHttpRequestBase.

The EnableResiliency property is now marked with the [Obsolete] attribute. Developers are encouraged to use the common Resilience Strategy setting instead for managing HTTP request resiliency. This change ensures better consistency and alignment with the broader resilience strategy.

* Restrict ResilienceCategoryAttribute to class targets only

Removed support for using ResilienceCategoryAttribute on properties. This change enforces a stricter and more focused usage of the attribute, ensuring it applies only to class-level declarations.

* Add documentation for IResilientActivityInvoker interface

Include summaries and parameter descriptions for the `InvokeAsync` method. This improves code clarity and helps developers understand the functionality and usage of the resilient activity invocation process.

* Fix logical operator precedence in type comparison check

Parentheses were added to ensure correct evaluation of conditions when checking type compatibility. This prevents potential logical errors when determining the target type in object conversions.

* Add support for resilience source identification

Introduce the `ResilienceSourceNameAttribute` to allow naming of resilience sources. Updated `ResilienceStrategyCatalog` to utilize the attribute for prefixing strategy IDs, improving source identification and traceability. Applied the attribute to `ConfigurationResilienceStrategySource` as an example.

* Revert "Add support for resilience source identification"

This reverts commit 19b4e7121d6330b5de4f692b78da4c1e4a2d1f67.

* Reapply "Add support for resilience source identification"

This reverts commit 8bcba9d040c4eb247077aec6d90dc02817adcbd5.

* Revert "Reapply "Add support for resilience source identification""

This reverts commit ee04d35e7930956c752dda3ed150ca34a535e66c.
2025-05-12 10:09:04 +02:00
Sipke Schoorstra 2eec542f50
Merge remote-tracking branch 'origin/rc/3.4.0' 2025-04-16 08:41:04 +02:00
Sipke Schoorstra e2288d0b34
Fix infinitely waiting Alterations Workflow (#6561)
* Refactor TenantId string handling and nullability checks.

Moved `StringExtensions` to a common module for reuse. Updated tenant-related logic to utilize null-safe string extensions, enhancing consistency and simplifying nullability handling across the codebase.

* Set StrictMode to false by default in ObjectConverter

Modified the default value of StrictMode to `false` to enable the original flexible behavior. Developers can opt into strict mode by explicitly setting it to `true`. This change aims to enhance backward compatibility and minimize unexpected strict conversions.

* Add tenant ID retrieval to ElsaDbContextBase constructor

Retrieve the current tenant ID if available using ITenantAccessor and assign it to the TenantId property. This ensures proper handling of multi-tenancy scenarios in the database context initialization.

* Disable DbContext pooling, manual OTEL instrumentation, and strict mode.

DbContext pooling is turned off to prevent potential issues with shared context instances. Manual OpenTelemetry instrumentation is disabled to rely on automatic instrumentation instead. Strict mode is also disabled to allow more flexibility in object conversion.

* Fix infinitely waiting Alterations Workflow

Replaced workflow dispatch logic with BookmarkQueue and StimulusHasher for triggering workflows. This fixes the issue where the Alterations workflow would signal completion while a later step awaits a completion bookmark. The Bookmark Queue now handles this.
2025-04-05 10:37:05 +02:00
Raymond den Haan d4b0238501 Moved compression to separate feature 2025-04-03 15:40:17 +02:00
Sipke Schoorstra fb1feb5ac4
Merge branch 'rc/3.4.0' 2025-04-01 10:23:04 +02:00
Sipke Schoorstra 952cbb4edc
Add strict type checking for variable parsing (#6536)
* Add strict type checking for variable parsing

Introduced a `StrictMode` flag for variables, enforcing stricter type validation during parsing and conversions. Updated related methods and tests to reflect the stricter parsing behavior, ensuring consistent type compatibility and error handling.

* Refactor StrictMode handling and improve ObjectConverter logic

Moved StrictMode flag from Variable to ObjectConverter for better cohesion and updated related references. Additionally, removed unused parameters and simplified ObjectConverterOptions to streamline configuration and maintain consistency across the codebase.

* Refactor type conversion logic and update logging defaults

Improve type conversion reliability by introducing `TryConvertValue` and refining exception handling. Update JSON scenarios to use "Inherit" as the default for log persistence modes, ensuring consistency across configurations.

* Refactor exception handling in ObjectConverter.

Introduce a helper method `ReturnOrThrow` to streamline and centralize exception handling logic. This change also adds support for non-strict mode, allowing value fallback instead of throwing exceptions when enabled.

* Add numeric type checks and improve type conversion handling

Introduce `IsNumericType` extension method to identify numeric types. Enhance type conversion logic in `ObjectConverter` to handle numeric, boolean, and string types more accurately. Update variable deserialization workflows to handle conversion failures gracefully.

* Set a default comment for ObjectConverter.StrictMode assignment

Added a comment clarifying that StrictMode is set to its default value. This improves code readability and helps maintainers understand the intent.

* Fix formatting inconsistencies and improve code clarity

Removed redundant whitespace and adjusted formatting to align with coding standards. These changes enhance the readability and maintainability of the code without altering functionality.

* Fix typo in Program.cs variable comment

Corrected a minor issue in the comment for `identityTokenSection` to remove the unnecessary "Modify" text. This change improves code readability and removes potential confusion for developers.

* Simplify imports in ModifyVariableHandler.cs

Removed unused `Microsoft.Extensions.Options` and `Elsa.Workflows.Options` imports to clean up dependencies and improve maintainability. This change reduces clutter without affecting the existing functionality.
2025-03-31 18:11:14 +02:00
Sipke Schoorstra 51dff1a061
Support for Writing Custom Trigger Activities Using Existing Trigger Infrastructure (#6527)
* Refactor bookmark naming to use "Name" instead of "ActivityTypeName".

Replaces usages of "ActivityTypeName" with "Name" across relevant classes, filters, and database mappings for clarity and consistency. Maintains backward compatibility where necessary and updates corresponding indices, filters, and methods for proper functionality.

* Add migration for V3_5 with schema updates for EF Core

This migration modifies the `Triggers` and `Bookmarks` tables by adding the `Name` column, updating its nullable state, and creating corresponding indexes. Changes apply to both SQLite and MySQL contexts, ensuring compatibility across databases.

* Refactor bookmark filtering and tenant events handling.

Unified the bookmark filtering logic with overloads that accept multiple names, improving flexibility and reusability. Simplified object initializations in tenant events to enhance code readability and reduce verbosity. Added task continuation logic to background task execution for better task sequencing and error handling.

* Refactor bookmark creation to use target-typed `new` expressions.

Replaces explicit `CreateBookmarkArgs` instantiations with concise target-typed `new` expressions for improved readability and reduced redundancy. This does not alter functionality but simplifies the code structure.

* Make callback parameters optional and adjust OTEL settings

Updated methods to allow optional callbacks for improved flexibility. Refactored Delay activity to reuse helper methods. Adjusted OTEL instrumentation settings to enable console exporter and disable manual instrumentation.

* Refactor stimulus handling and streamline event workflows.

Introduces `WaitForEvent` and `GetEventInput` extensions to improve ActivityExecutionContext usability. Replaces generic filter methods with targeted single-name filtering, simplifying scheduling logic. Updates stimulus names for consistency and deprecates obsolete methods to enhance clarity and maintainability.

* Refactor `Event` activity handling and event stimulus logic.

Replaces inline event stimulus creation with a new `GetEventStimulus` helper method for cleaner code. Simplifies event execution handling by removing redundant logic in `ExecuteAsync`. Extends `WaitForEvent` to handle workflow triggers more efficiently.

* Refactor and enhance Timer and Delay execution logic

Introduced `TimerBase` for shared timer functionality and refactored `Timer` to extend it. Improved method names for clarity, replacing `ResumeIn`/`ResumeAt` with `DelayFor`/`DelayUntil`. Enhanced flexibility in bookmark handling and activity execution context extensions.

* Add custom activities and refactor HTTP stimulus handling

Introduce new custom activities (CustomDelay, CustomEvent, CustomHttpEndpoint, CustomTimer) to enhance workflow functionality. Refactor HTTP stimulus handling by replacing activity type names with a new centralized HttpStimulusNames constant, improving consistency and maintainability. Additionally, streamline HTTP endpoint logic with new helper extensions and simplify related services to reduce redundancy.

* Add HttpEndpointBase abstraction to simplify HTTP endpoints

Introduce a new `HttpEndpointBase` class to centralize common logic for HTTP endpoint activities. Refactored `CustomHttpEndpoint` to inherit from this new base class, reducing redundancy and improving maintainability.

* Refactor events framework with base class for event activities

Introduce `EventBase` to streamline implementations of event-driven activities. Updated `CustomEvent` to inherit from `EventBase`, reducing duplicate logic and improving maintainability. Removed unnecessary dependencies in `CustomTimer`.

* Refactor bookmark creation to use target-typed `new` expressions.

Replaces explicit `CreateBookmarkArgs` instantiations with concise target-typed `new` expressions for improved readability and reduced redundancy. This does not alter functionality but simplifies the code structure.

Make callback parameters optional and adjust OTEL settings

Updated methods to allow optional callbacks for improved flexibility. Refactored Delay activity to reuse helper methods. Adjusted OTEL instrumentation settings to enable console exporter and disable manual instrumentation.

Refactor stimulus handling and streamline event workflows.

Introduces `WaitForEvent` and `GetEventInput` extensions to improve ActivityExecutionContext usability. Replaces generic filter methods with targeted single-name filtering, simplifying scheduling logic. Updates stimulus names for consistency and deprecates obsolete methods to enhance clarity and maintainability.

Refactor `Event` activity handling and event stimulus logic.

Replaces inline event stimulus creation with a new `GetEventStimulus` helper method for cleaner code. Simplifies event execution handling by removing redundant logic in `ExecuteAsync`. Extends `WaitForEvent` to handle workflow triggers more efficiently.

Refactor and enhance Timer and Delay execution logic

Introduced `TimerBase` for shared timer functionality and refactored `Timer` to extend it. Improved method names for clarity, replacing `ResumeIn`/`ResumeAt` with `DelayFor`/`DelayUntil`. Enhanced flexibility in bookmark handling and activity execution context extensions.

Add custom activities and refactor HTTP stimulus handling

Introduce new custom activities (CustomDelay, CustomEvent, CustomHttpEndpoint, CustomTimer) to enhance workflow functionality. Refactor HTTP stimulus handling by replacing activity type names with a new centralized HttpStimulusNames constant, improving consistency and maintainability. Additionally, streamline HTTP endpoint logic with new helper extensions and simplify related services to reduce redundancy.

Add HttpEndpointBase abstraction to simplify HTTP endpoints

Introduce a new `HttpEndpointBase` class to centralize common logic for HTTP endpoint activities. Refactored `CustomHttpEndpoint` to inherit from this new base class, reducing redundancy and improving maintainability.

Refactor events framework with base class for event activities

Introduce `EventBase` to streamline implementations of event-driven activities. Updated `CustomEvent` to inherit from `EventBase`, reducing duplicate logic and improving maintainability. Removed unnecessary dependencies in `CustomTimer`.

* Move HttpEndpointOptions model to its own file

The HttpEndpointOptions class was moved from an extension file to its own dedicated file for better organization and modularity. This model defines HTTP endpoint properties such as path, methods, authorization, policies, request timeout, and size limit. The change improves code clarity and structure.

* Fix unnecessary whitespace in Timer.cs

Removed an extra whitespace line in the Timer.cs file to maintain code formatting consistency. No functional changes were made to the code.

* Remove extraneous whitespace in IStimulusSender.cs file

Eliminate unnecessary blank line in the IStimulusSender interface for improved code cleanliness. This change enhances readability and aligns with coding standards.
2025-03-21 23:16:56 +01:00
Sipke Schoorstra 5ae23ce7e5
Add Workflow Heartbeat Middleware and update methods
Introduce `WorkflowHeartbeatMiddleware` to monitor workflow liveness and update timestamps during execution. Extend WorkflowInstanceStore implementations to support updating the `UpdatedAt` timestamp across various data stores. Enhance Dapper query builder and workflow execution pipeline with related functionality.
2025-02-22 16:05:36 +01:00
Sipke Schoorstra 7b5aa6073d Add support for content item-backed workflow definitions
Introduced a `ContentItemWorkflowDefinitionStore` and related mappers, extensions, and index updates to manage workflow definitions as content items. Refactored the system to replace existing handlers with a store-based approach and optimized querying and mapping workflows.
2025-02-13 23:39:56 +01:00
Sipke Schoorstra 7a758f8850 Fix multitenancy support in Hangfire services and jobs
Refactored Hangfire-related services and jobs to include tenant context management using ITenantAccessor and ITenantFinder. Updated job constructors and method signatures to support tenant-specific execution. Improved exception-throwing syntax for better readability in BackgroundActivityInvoker.
2025-02-06 16:39:38 +01:00
Sipke Schoorstra 2bb3af1e5d Fix Oracle migrations 2025-02-01 00:38:38 +01:00
Sipke Schoorstra 0b5f580b0d Reorder tenant services initialization for proper execution.
Ensure `RunStartupTasks` is added before background and recurring tasks as order is critical for correct startup and tenant activation behavior. This prevents potential misalignment in task execution during tenant lifecycle events.

Fixes #6269
2025-01-15 18:40:38 +01:00
Sipke Schoorstra 6c0f7a48e7 Refactor recurring tasks scheduling logic.
Removed `ConfigureRecurringTasksScheduleStartupTask` and moved its functionality into `RecurringTaskScheduleManager`. Simplified recurring task configuration and streamlined dependencies, improving maintainability. Added retention policies to `Elsa.Server.Web`.
2025-01-02 11:11:42 +01:00
Sipke Schoorstra 9828737ad0 Add validation for null factory types and missing type strings
Ensure `WorkerManager` and `TypeTypeConverter` handle invalid inputs gracefully by throwing exceptions. This prevents potential runtime issues caused by null factory types or unresolved type strings.
2025-01-01 18:18:02 +01:00
Sipke Schoorstra 76d685744d Update target frameworks and package versions to latest
Dropped .NET 6 support and upgraded projects to .NET 8 and .NET 9 frameworks. Updated several package dependencies to their latest stable or preview versions to ensure compatibility and leverage improvements.
2024-12-22 10:00:56 +01:00
Sipke Schoorstra a9c8ad64f1 Add concurrency locks to DefaultTenantService operations
Introduced SemaphoreSlim for initialization and refresh methods to ensure thread safety in DefaultTenantService. Improved tenant unregistration to handle scope cleanup only when mappings exist. These changes enhance reliability and prevent race conditions during tenant operations.
2024-12-12 11:20:45 +01:00
Sipke Schoorstra 692937d7d7
Enhance Multitenancy with Runtime Tenant Management and Task Handling (#6173)
* Work in progress: Add DefaultTenantService for tenant management

Introduce `DefaultTenantService` and its corresponding interface `ITenantService` to manage tenant operations such as finding, getting, and listing tenants. Update `MultitenantBackgroundService` to utilize `DefaultTenantService` for handling tenant lifecycle events. This enhancement standardizes tenant operations and improves the maintainability of the multitenancy feature.

* WIP

* Add multitenancy event handlers and task interfaces

Implemented new interfaces IBackgroundTaskStarter and ITaskExecutor to manage task lifecycle events efficiently. Introduced new classes such as RunBackgroundTasks, RunStartupTasks, and StartRecurringTasks for handling tenant activation and deactivation events. Modified TaskExecutor to implement these interfaces and adjusted tenant registration logic to invoke these new handlers.

* Refactor multitenancy and task management services.

Remove background and recurring task runners, and integrate tenant activation and deactivation into the multitenancy feature. Enable multitenancy in the server application and create a new service for tenant activation and deactivation. This refactor simplifies the management of tenant-specific tasks and enhances the modularity of the platform.

* Refactor background service to use startup tasks

Replaced hosted service implementation with startup tasks for executing multi-tenant tasks and EF Core migrations. Introduced `PriorityAttribute` to manage task execution order, ensuring migrations run before other services that require database access. This simplifies tenant activation with an ordered task execution and removes redundant classes.

* Refactor MultitenancyFeature service registrations

Reorganized service registrations for better clarity and maintainability. Changed the registration of some services to use factory delegates for retrieving existing services to ensure correct dependencies. This refactor improves the flexibility of the tenant lifecycle event handling.

* Update V3_3 migration files

* Add tenant management endpoints and enhance tenant handling

Implemented tenant management endpoints including Add, Get, List, and Update. Enhanced tenant handling by introducing configuration and store-based providers, and improved error logging for tenant updates. Adjusted various internal functionalities to better support multitenancy features through different persistence providers.

* Implement tenant deletion endpoint and refactor migration setup.

Introduce a new API endpoint to handle tenant deletions while providing appropriate responses based on successful or unsuccessful attempts. Refactor migration handling by replacing startup tasks with hosted services across various modules to streamline the migration execution process.

* Add and integrate ConfigurationJsonConverter

Introduce a `ConfigurationJsonConverter` to handle JSON serialization and deserialization of `IConfiguration` objects. This change centralizes configuration serialization logic, leading to cleaner and more maintainable code. Updated various parts of the codebase to use the new serialization utility, ensuring a consistent approach throughout the application.

* Refactor JSON conversion and update tenant endpoint.

Removed unused workflow references and streamlined JSON handling in `ConfigurationJsonConverter`. Simplified tenant ID handling by removing `IIdentityGenerator` and setting a default value for `UpdatedTenant.Id`.

* Add logging for cancelled recurring tasks

Integrated ILogger to log information when a recurring task is canceled due to an OperationCanceledException. This change enhances troubleshooting by providing clearer insights into task cancellations and their underlying reasons, improving maintainability and observability of the task execution process.

* Disable multitenancy support and adjust default Tenant ID.

Multitenancy is now disabled by setting 'useMultitenancy' to false in the configuration. Additionally, the default Tenant's ID has been changed from null to an empty string to prevent potential null reference issues.

* Remove MultitenantHostedService abstraction file

The MultitenantHostedService.cs file was removed as it is no longer necessary. Its responsibilities have likely been refactored or integrated into another service, indicating a simplification or restructuring of the multitenancy handling in the codebase.

* Rename PriorityAttribute to OrderAttribute for clarity.

This change improves the clarity of the code by renaming PriorityAttribute to OrderAttribute, reflecting its actual purpose. All occurrences of the attribute in the codebase have been updated accordingly to maintain consistency. This makes the intent of the code more understandable for future maintenance and development.

* Fix message key retrieval in ProduceMessage activity

Update the ProduceMessage activity to use GetOrDefault for retrieving the message key. This change ensures that a null key is used if no explicit key is provided or if the key is empty or whitespace, preventing potential errors during message production.

* Refactor multitenancy and scheduling services.

Removed DefaultTenantContextInitializer interface and class, refactored tenant activation/deactivation to use try-catch logging, and updated tenant context handling to use IDisposable for context push. New activities and workflows added in Elsa.Server.Web, and scheduling services enhanced to schedule jobs with explicit job keys and groups. Also, adjusted configurations to enable multitenancy, providing improved maintainability and flexibility.

* Remove Example1 activities and disable multitenancy

Deleted Example1Activity, Example1Workflow, and FirstActivity classes to clean up unused code and simplify the codebase. Disabled multitenancy by setting useMultitenancy to false, likely to streamline configuration and resource utilization.

* Fix and normalize URL path concatenation.

Ensure that the base URLs in both base path providers consistently end with a forward slash. This normalization prevents potential issues with endpoint routing and path concatenation, improving overall URL construction robustness.
2024-12-03 15:07:55 +01:00
FunShow 5ac63f1485
fix JavaScript BigInt mapping to BigInteger serialization incorrect (#6164)
* fix: fix JavaScript BigInt mapping to BigInteger serialization incorrect

* test: add test for BigIntegerJsonConverter

* test: add test for BigIntegerJsonConverter

---------

Co-authored-by: funshow.liu <funshow.liu@didatravel.com>
2024-12-03 15:07:40 +01:00
Sipke Schoorstra 6022df165c
Kafka: Update ProduceMessage activity with support for specifying a Key (#6166)
* Add Key to Kafka ProduceMessage activity

Deleted unnecessary Consumer and Producer workflow classes and the OrderReceived message class to clean up code. Refactored Kafka producer interface and implementation to include message keys for improved message handling. Updated configuration to enable Kafka and removed unused service registrations.

* Add Kafka factory classes and type alias registry

Introduce GenericConsumerFactory and GenericProducerFactory for handling Kafka consumer and producer creation. Implement a TypeAliasRegistry to manage type aliases, enabling cleaner configuration through aliases. Update the OrderReceived message class and ensure better integration with the server web program via these new components.

* Handle empty topics and predicates in Kafka worker.

Ensure the Kafka consumer unsubscribes when no topics are available to subscribe to. Additionally, add a check to handle empty string values for predicates, allowing workflow triggers to proceed in this scenario.

* Disable Kafka usage in Elsa Server Web configuration

Kafka has been disabled in the current configuration by setting the useKafka constant to false. This change might be intended to switch to a different messaging system or to simplify the current setup by removing unnecessary services. Ensure that any dependencies on Kafka are handled elsewhere in the application.
2024-11-29 19:49:31 +01:00
Sipke Schoorstra 09a5c79211
Fix Tenant ID propagation for MassTransit (#6144)
* Refactor tenant middleware and consumer implementations

Update `TenantConsumeMiddleware` to utilize `ITenantFinder` and `ITenantScopeFactory` for tenant context management. Refactor `WorkflowMessageConsumer` to use constructor parameters directly for improved dependency injection. Additionally, replace private scope variable in `TenantScope` with a public property to enhance code clarity.

* Refactor to replace IWorkflowDispatcher with IStimulusSender

Updated WorkflowMessageConsumer to utilize IStimulusSender for handling message-triggered activities. Simplified dependencies and method calls to align with the new interface, improving the code's clarity and maintainability.
2024-11-24 23:28:38 +01:00
Sipke Schoorstra b534b42a60
Update Kafka Module: Add Support for Configuring Consumer and Producer Factories (#6139)
* Enable Kafka Worker Factory and Refactor Worker Implementation

Introduce a flexible worker factory mechanism allowing custom worker creation with DefaultWorkerFactory as the initial implementation. Enhance Worker class to be generic, remove manual consumer configuration, and streamline message processing logic, improving code maintainability and extensibility.

* Refactor Kafka configuration properties

Renamed configuration properties in Consumer and Producer entities. Updated references in the codebase to use the new `Config` property instead of `ConsumerConfig` and `BootstrapServers`. Adjusted appsettings.json to match the new configuration schema.

* Add Kafka producer and consumer implementation

Implemented classes and interfaces to handle Kafka producers and consumers, including `ProducerProxy`, `ConsumerProxy`, and related context classes and factories. Refactored existing code to utilize these new implementations, replacing worker terminology with consumer and addressing context-specific fields.

* Remove redundant code in DefaultConsumerFactory and SendMessage

Removed commented-out unused return statement in DefaultConsumerFactory. Also eliminated explicit producer.Dispose() call in SendMessage, as the 'using' statement already handles resource cleanup.

* Add ExpandoObject producer and consumer factories

Replaced DefaultSerializers with new JsonSerializer and JsonDeserializer classes. Introduced ExpandoObjectProducerFactory and ExpandoObjectConsumerFactory to handle dynamic types. Updated workflow and configuration to use the new factories.

* Refactor bookmark processing and manage worker subscriptions

Refactored bookmark processing logic to utilize extension methods. Optimized worker subscriptions by centralizing topic subscription management and added logging for subscribed topics. This improves maintainability and clarity of the codebase.

* Refactor worker creation to use ActivatorUtilities

Updated WorkerManager to instantiate workers using ActivatorUtilities for better dependency injection support. This enhances code readability and maintains consistency with the service provider approach used throughout the codebase.
2024-11-22 20:07:14 +01:00
AlmightyLks 153397e63c
Fix all broken summary xml docs (#6123)
* Add proper summary tags back

* Apply suggestions from code review

Co-authored-by: Robin Sue <robinsue@live.de>

* 69 more files fixed

---------

Co-authored-by: Robin Sue <robinsue@live.de>
2024-11-18 13:41:27 +01:00
Sipke Schoorstra d3d171f59f Refactor retention module to use recurring tasks
Replaced the CleanupHostedService with CleanupRecurringTask for better scheduling and simpler configuration. Converted several classes to use constructor-based dependency injection and streamlined some type assignments. Removed unneeded files and updated existing code to follow more concise conventions. This enhances maintainability and readability.
2024-10-29 08:36:31 +01:00
Sipke Schoorstra 939fb95a97
Add multitenancy support for background tasks (#6059)
* Remove initial migrations

Deleted obsolete initial migration files from multiple databases: MySQL, SQL Server, SQLite, and PostgreSQL. This cleanup helps maintain a streamlined and updated migration history.

* Add Document base class and create tenant-specific indices

Introduced a new abstract `Document` base class to unify common properties. Implemented tenant-specific unique indices across multiple collections by including `TenantId` alongside `Id` to ensure uniqueness within tenant scopes.

* Remove outdated migration files

Deleted various migration files under MySql, PostgreSql, Sqlite, and SqlServer directories. This cleanup removes unnecessary schema definitions and helps to streamline the codebase.

* Refactor workflow identity assignment logic

Streamline workflow identity handling to ensure consistent assignment of Id, DefinitionId, and TenantId values. This change integrates tenant prefix and version suffix cleanly, enhancing clarity and maintainability.

* Enable multitenancy support

Added configuration for a new tenant (tenant-1) in appsettings.json and enabled multitenancy feature in Program.cs. This change allows the application to support multiple tenants, with specific configurations for each.

* Refactor route table update to run as startup task

Replaced `UpdateRouteTableHostedService` with `UpdateRouteTableStartupTask` to ensure route table updates are executed during application startup instead of as a hosted service. Updated configuration in `HttpFeature` and adjusted trigger validation logic in `ValidateWorkflowRequestHandler`.

* Add recurring task scheduling and single-node task support.

Introduce `IntervalExpressionType`, recurring task scheduling classes, and `SingleNodeTaskAttribute`. Update `RecurringTasksRunner` to handle schedules and add single-node task logic to `StartupTasksRunner`. Ensure proper namespace changes and configure sample recurring tasks.

* Refactor recurring tasks scheduling system

Replaced existing scheduling classes with a more modular and granular approach. Introduced new classes and interfaces like `ISchedule`, `CronSchedule`, `IntervalSchedule`, and `RecurringTaskScheduleManager`. Updated related methods and code to comply with the new design.

* Refactor background task management

Removed `ExpiredSecretsHostedService` and refactored it into a recurring task. Introduced `TaskExecutor` for shared task execution logic. Updated and renamed feature classes to better represent their purpose, improving task scheduling and execution management.

* Add BackgroundTask abstract class to Elsa.Common module

This new abstract class implements the IBackgroundTask interface with default methods for executing, starting, and stopping tasks asynchronously. It provides a basic framework for background task management in the Elsa.Common module.

* Switch to CreateAsyncScope in DefaultTenantScopeFactory

Updated the CreateScope method to use CreateAsyncScope instead of CreateScope. This change improves asynchronous handling of service scopes within the DefaultTenantScopeFactory class.

* Add tenant handling and move StartWorkers background task

Introduce ITenantAccessor in Worker class for multitenancy support. Rename and relocate StartWorkers service to BackgroundTask, ensuring smoother workflow initialization. Also, update the configuration to support Azure Service Bus connection string.

* Add tenant support and refactor ProtoActor client

Integrated ITenantAccessor in ProtoActorWorkflowClient class to handle multi-tenancy. Refactored methods in the client to support custom headers and added async disposable pattern in various services for proper resource management. Additionally, enabled Azure Service Bus and updated related documentation.

* Add support for custom headers in ProtoActor grain methods

Introduced a T4 template to generate grain methods with custom headers, enabling the use of tenant ID in requests. Updated `ProtoActorWorkflowClient` to employ these methods, removing redundant code and directly utilizing the client for various workflow operations.

* Add tenant middleware to MassTransit configurations

Introduced multitenancy middleware for MassTransit message handling. Added new message type `OrderReceived` and updated RabbitMQ setup in Elsa Server. Applied middleware to configure tenant data on send, publish, and consume operations.

* Add new product workflow and streamline ID handling

Introduced a new `RequestResponseWorkflow` for handling product requests. Simplified ID handling in `WorkflowBuilder` and `ClrWorkflowsProvider` by defaulting to empty strings and adding a version prefix. Enhanced `HttpWorkflowsMiddleware` to correctly parse full request paths.

* Remove redundant files and update configuration

Deleted unused files `Product.cs` and `RequestResponseWorkflow.cs` to clean up the codebase. Updated `Program.cs` configuration: switched MassTransitBroker to Memory and disabled multitenancy.

* Remove MultitenantRecurringTaskService and update AzureServiceBus

Removed `MultitenantRecurringTaskService` and adjusted related code for Azure Service Bus to work without it. This includes removal of tenant accessor dependency from `Worker` and cleanup of service configuration flags in `Program.cs`.

* Increase signal wait timeout to 10000 milliseconds.

Extended the default timeout for signal awaiting methods from 8000 to 10000 milliseconds. This change ensures more flexible and resilient waiting periods, reducing timeout occurrences in scenarios with longer processing times.

* Refactor scheduling service to be a background task

Renamed `CreateSchedulesHostedService` to `CreateSchedulesBackgroundTask` and refactored it to inherit from `BackgroundTask` instead of `BackgroundService`. Simplified the constructor by injecting the required dependencies directly, eliminating the need for a scoped factory.

* Refactor workflow version suffix formatting

Changed the version suffix format from `:v{version}` to `v{version}` and adjusted the ID concatenation accordingly. This improves consistency and readability of workflow IDs.

* Enable multitenancy support in Quartz scheduler

Added `TenantJobListener` to inject tenant context into jobs. Modified `QuartzWorkflowScheduler` to incorporate tenant IDs into job data maps and adjusted the configuration to acknowledge multitenancy settings.

* Remove ConfigureSchedulerHostedService and TenantJobListener

Consolidated tenant resolution logic into JobExecutionExtensions class. Updated ResumeWorkflowJob and RunWorkflowJob to use the new extension method for tenant retrieval. This simplifies the QuartzSchedulerFeature setup by removing the hosted service configuration.

* Refactor HTTP feature and update route table task

Move 'UpdateRouteTableStartupTask' from 'HostedServices' to 'Tasks' and update dependency injection configurations accordingly. Simplify 'DefaultRouteTableUpdater' by removing unnecessary options and tenant-agnostic settings from filters.

* Disable multitenancy in Program.cs

The useMultitenancy flag has been changed from true to false. This update affects the Elsa.Server.Web application configuration.

* Simplify variable usage in HttpWorkflowsMiddleware

Replaced 'fullPath' variable with 'path' to streamline code. This change enhances readability by reducing redundancy and ensures consistency in variable naming throughout the method.

* Enable multitenancy and refactor tenant handling logic

Enable multitenancy in the application and refactor tenant handling logic to use ITenantFinder and ITenantContextInitializer interfaces. Added header constants, updated middleware to use these interfaces, and moved extension methods to the appropriate namespace.

* Add input validation to user registration form

Implemented checks to ensure all required fields are filled and that input data adheres to format requirements. This change reduces errors and enhances form reliability.

* Remove unused import from TenantPrefixHttpEndpointRoutesProvider

This change cleans up the code by removing an unnecessary import statement. It improves code readability and reduces clutter, making future maintenance easier. The functionality remains unchanged.

* Rename filter scope to "tenantPublish" in Probe method

Updated the Probe method in TenantPublishMiddleware.cs to use "tenantPublish" instead of "tenantSend" for better clarity. Ensures consistency with the method's context and aligns with naming conventions.

* Refactor: Remove extraneous whitespace

Eliminate unnecessary whitespace in ProtoActorWorkflowClient.cs for cleaner code. This change helps maintain consistent formatting and improves readability.

* Refactor DefaultRegistriesPopulator for cleaner initialization

Converted constructor to use read-only fields directly, removing unnecessary instance variables. This change simplifies the code by reducing redundancy and making the constructor cleaner.
2024-10-28 19:38:24 +01:00
Sipke Schoorstra bdaae64e72 Add base path providers and standardize tenant accessor property
Introduce DefaultHttpEndpointBasePathProvider and TenantPrefixHttpEndpointBasePathProvider to manage HTTP endpoint base paths. Update ITenantAccessor property from CurrentTenant to Tenant for consistency across the codebase.
2024-10-17 19:47:55 +02:00
Sipke Schoorstra 99ab5e4ba1 Refactor StopAsync method to return completed task
Modified the StopAsync method to directly return a completed task instead of running through tenants asynchronously. This change simplifies the shutdown process and removes redundant tenant processing during service stop.
2024-10-15 23:47:17 +02:00
Sipke Schoorstra 225ad49ea8
Improved multitenancy support for HTTP workflows with per-tenant DbContext (#6032)
* Refactor: Update namespaces and add TenantExtensions

Updated namespaces throughout the project to improve clarity and consistency by moving from 'Common' to appropriate modules. Added TenantExtensions class to simplify fetching connection strings for tenants.

* Implement multitenant DB connection strings

Redesign tenant-specific classes to support multitenancy more effectively. Introduce `MultitenantBackgroundService` and `MultitenantHostedService` for handling tasks per tenant.

* Refactor constructors and remove redundant code

Simplified the constructor parameters for `MultitenantBackgroundService` and `List` class. Removed the unused parameter in `MultitenantBackgroundService` and redundant folder inclusion in the project file. Updated the method calls to use direct parameters in `List` class.

* Remove unused import in ActivityDescriptors Endpoint

The Elsa.Common.Multitenancy import was removed as it is unused in the List/Endpoint.cs file. Removing unused imports helps to improve code readability and maintainability. This change does not affect functionality.
2024-10-15 23:16:50 +02:00
Sipke Schoorstra 7e7a899bbf
Implement multitenant HTTP routing (#6031)
* Add tenant awareness to bookmark handling and route resolution

Added tenant ID support across various components, including bookmark updates, route resolution, and middleware processing. This ensures that bookmark and route operations can now appropriately handle tenant-specific data, improving the system's multitenancy capabilities.

* Add Multitenant HTTP Routing feature to Tenants module

Introduced a new MultitenantHttpRoutingFeature class to the Elsa.Tenants.AspNetCore module, enhancing the tenant resolution capabilities. Moved RoutePrefixTenantResolver from Elsa.Http to Elsa.Tenants.AspNetCore and updated relevant project references and namespaces accordingly. This refactor improves modularity and separation of concerns between HTTP and tenancy features.

* Refactor route handling and tenant configuration

Removed redundant `RouteTableExtensions` and replaced with new route providers and updaters, enhancing flexibility and modularity. Introduced tenant-specific HTTP endpoint configurations for better customization and configuration management.

* Rename HttpEndpointBookmarkStimulus to HttpEndpointBookmarkPayload

Refactor various classes and methods to reflect the renaming from `HttpEndpointBookmarkStimulus` to `HttpEndpointBookmarkPayload`. Add and configure new extension methods for tenant route handling, update the route provider to support multi-tenancy, and adjust the tenants provider to bind configuration properly.

* Add HeaderTenantResolver and refactor Http namespace.

Introduce HeaderTenantResolver to resolve tenants via HTTP headers. Refactor multiple classes and interfaces to move from the Elsa.Http.Models namespace directly into Elsa.Http for clarity and consistency.

* Add Host-based tenant resolution

Implemented a HostTenantResolver to resolve tenants based on the request's host and updated tenant configurations with host information. Modified the tenant resolver pipeline and added the new host resolver to the service registrations.

* Add tenant-aware caching and accessor support

Enhanced caching by incorporating tenant identifiers into cache keys for more granular cache management. Introduced ITenantAccessor dependencies in various services to retrieve the current tenant information. This ensures that cache entries are correctly isolated per tenant.

* Reorder tenant resolvers for pipeline setup.

Reordered the tenant resolvers in the pipeline to prioritize HostTenantResolver before RoutePrefixTenantResolver. This ensures that tenant resolution is correctly aligned with host-based resolving before checking the route prefix.

* Remove unused imports

This commit eliminates redundant `using` directives across multiple files to streamline the codebase. This cleanup helps improve code readability and maintainability by removing unnecessary dependencies.
2024-10-14 21:27:11 +02:00
Sipke Schoorstra a5cc3fc9e8
Refactor Tenant Resolution to Use Async Local Storage for Operation-wide Access (#6022)
* Remove obsolete tenant-related classes and add ASP.NET Core middleware

Refactored tenant resolution by removing obsolete interfaces and classes, such as `IAmbientTenantAccessor` and `ITenantResolutionStrategy`. Introduced new ASP.NET Core middleware for tenant resolution, encapsulated in the new `Elsa.Tenants.AspNetCore` project. Updated related usage in various parts of the application to align with these changes.

* Remove HttpContextTenantResolver.

Removed HttpContextTenantResolver from the multitenancy pipeline and related service registrations. This simplifies the tenant resolution by relying on remaining resolvers like ClaimsTenantResolver and RoutePrefixTenantResolver.

* Add Elsa solution definition file

This commit introduces the main solution file, Elsa.slnx, defining the folder structure, projects, and configuration for the Elsa repository. This includes folders for Docker, documentation, pipelines, samples, scripts, source code, and tests.

* Refactor DefaultAccessTokenIssuer for clarity and efficiency

Refactored the DefaultAccessTokenIssuer class by simplifying its constructor and utilizing scoped variables for token options. Improved token creation logic by adding a dedicated method to configure token options, enhancing code readability and maintainability.

* Remove Elsa.slnx solution file

No dotnet build support yet.

* Refactor tenant resolver service registrations

Updated the service registrations to use interfaces for DefaultTenantResolver and DefaultTenantResolverPipelineInvoker. This improves the code's flexibility, making it easier to replace or extend these implementations in the future.

* Add multitenancy support and tenant scope management

Introduced ITenantScopeFactory and related implementations for tenant scope management across the application. Enhanced the HTTP workflows middleware to handle tenants and updated relevant configurations and extension methods to support tenant resolution.

* Remove unnecessary folder inclusion

The <Folder> tag for "Modules\Modules\" was redundant and has been removed to clean up the project file. This change will not affect the existing functionality or project structure.

* Rename Create to CreateScope and improve authorization.

Updated the method name from Create to CreateScope for better clarity in the TenantScopeFactory. Fixed a logical error in the authorization process, ensuring proper status code setting for unauthorized requests, and refactored token expiration calculation for clarity.

* Add tenant agnostic filters and remove tenant setup

This commit introduces tenant agnostic filters in AutoUpdateTests to ensure workflows can trigger regardless of tenant. Additionally, it removes tenant configuration from WorkflowServer setup as it is no longer required for the current tests.
2024-10-12 12:08:09 +02:00
Sipke Schoorstra e1d0cc2e27 Add DefaultTenantAccessor and ITenantAccessor interface
Introduce ITenantAccessor interface and its implementation, DefaultTenantAccessor, for tenant management in the multitenancy module. This setup will allow future enhancements for tenant-specific functionality.
2024-10-06 14:49:02 +02:00
Sipke Schoorstra dd812645c7 Refactor imports to reduce use of Elsa.Common.Contracts
Consolidate imports by replacing Elsa.Common.Contracts with Elsa.Common and Elsa.Common.Multitenancy. This update streamlines import statements across various modules, improving code readability and maintainability.
2024-10-05 18:30:45 +02:00
Marko Lahma 12f947cf9b
Upgrade Quartz and some warning fixes (#5949)
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-09-16 01:55:24 -07:00
Sipke Schoorstra 329a79be2e
Secrets API (#5967)
* Add initial implementation for Elsa Secrets modules

This commit introduces new projects for managing secrets within the Elsa framework: `Elsa.Secrets.Api`, `Elsa.Secrets.Core`, and `Elsa.Secrets.Management`. These projects include essential interfaces, models, entities, and endpoints to handle secret storage, retrieval, and management. Specific features include API endpoints for listing secrets, models for secret filtering, and interfaces for encryption key handling.

* Add weavers

* Refactor encryption handling in secrets management

Implemented a new architecture for handling encryption keys and algorithms within the secrets management system. Replaced old encryption key entities and related interfaces with a more modular and extensible approach. Added new services and models to improve encryption and decryption processes, enhancing maintainability and scalability.

* Add IEncryptor interface for encryption functionality

Introduced the IEncryptor interface to standardize encryption operations within the Elsa.Secrets.Management module. This interface includes the EncryptAsync method to handle encryption using a specified key ID and value.

* Add dependency on SecretsFeature and configure secrets provider

Integrate the SecretsFeature dependency and configure a secrets provider within the SecretsManagementFeature class. This adds the StoreSecretProvider to the service collection and ensures the secrets provider is correctly set up. Also, rename method from WithSecretsProvider to UseSecretsProvider for clarity.

* Add EF Core and SQLite support for Secrets module

Introduced Entity Framework Core and SQLite support for the Secrets module, including migration files, EF Core configurations, context factory, and store implementation. Added necessary extensions and configuration code to integrate with the existing API and features. Included updates to the main web application to utilize the new persistence providers.

* Update Microsoft.SemanticKernel to version 1.18.2

Upgrade Microsoft.SemanticKernel package to the latest version to ensure compatibility and new features. Remove unused Elsa.Agents.Persistence using directive from Program.cs for code cleanliness.

* Update workflows and add Agents module

Changed workflow branch targets from `main` to `feature/secrets`. Updated Docker image tags and added the `Agents` module to the Elsa Studio WebAssembly project.

* Enable agent activities in workflow configuration

This change introduces the `.UseAgentActivities()` method in the workflow configuration, enhancing the workflow capabilities. By doing so, it ensures that agent activities are appropriately integrated and available for use in the application.

* Add EF Core migrations for MySQL and SQL Server

Added Entity Framework Core migrations and related configurations to support MySQL and SQL Server for the Agents Persistence module. These changes include new migration files, context factories, and project configurations.

* Fix migration assembly reference and update method syntax

Changed the migration assembly reference in SqlServerProvidersExtensions. Updated method syntax in WorkflowManagementFeature to use array shorthand format.

* Add secret management functionalities

Introduced secret management services with CRUD operations, notifications, and bulk actions. Added unique name generation and validation for secrets, and implemented corresponding API endpoints.

* Enhance Secret Management Feature

Added Elsa.Extensions import and updated MemorySecretStore registration to use the AddMemoryStore method with Secret. This improves code modularity and adheres to the updated registration method conventions.

* Remove encryption services and update migration

Removed multiple files related to encryption services and their dependencies, including encryption algorithms and key providers. Also updated a migration script to reflect schema changes, removing specific columns and constraints.

* Implement versioning and retrieval for secrets management

Added "IsLatest" flag and cloning mechanism for secrets to support versioning. Introduced a new API endpoint for fetching decrypted secret input models. Refactored encryption and decryption logic to handle empty values gracefully.

* Add secret management functionalities

Introduced services and interfaces for secret name generation, validation, and updating. Updated secret handling to include expiration metadata. Refactored methods in ISecretManager to streamline secret creation and update processes.

* Remove DisableSyntaxSelection class and references

Deleted the DisableSyntaxSelection class and its references from various files. This includes removing its registration as a Scoped service and associated usage in the `RunJavaScript` activity.

* Add new migration for secrets and update DefaultSecretManager

Re-created migration files for V3_3 to include the ExpiresIn column. Updated DefaultSecretManager to utilize identityGenerator for generating Id and SecretId, and added additional fields like CreatedAt, UpdatedAt, and IsLatest.
2024-09-15 17:12:13 -07:00
Sipke Schoorstra b2b29a7f4f Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-06 22:47:05 +02:00
Sipke Schoorstra d2fc7fb005
Rework Workflow Context Feature (#5861)
* Refactor workflow context extension structure

Removed unused WorkflowContextWorkflowDefinitionExtensions and renamed several extension and middleware classes for clarity and consistency. Simplified context provider type handling using JsonArray in custom properties.

* Refactor context provider and update email workflow

Refactored `CustomerWorkflowContextProvider` to use constructor parameter directly, removing redundant fields and constructors. Updated comments and list syntax in `CustomerCommunicationsWorkflow`. Improved README.md documentation for workflow contexts and context providers.

* Adjust logging levels and improve workflow logic

Modified `appsettings.json` to reduce the verbosity of logging by adjusting log levels. Updated `CustomerCommunicationsWorkflow` to include a timer activity and simplified the workflow context parameter setting.

* Remove PropertyBag and related functionalities.

Deleted the PropertyBag class and extension methods, and removed all instances where PropertyBag was utilized across various modules. This simplifies the workflow configuration and management by relying solely on CustomProperties for storing metadata.

* Add workflow context management extensions and optimizations

Introduces `WorkflowExecutionContextExtensions` for loading and saving workflow contexts more efficiently. Replaces `JsonArray` usage with `List<object>` for context provider types, simplifying the type management in the workflow execution context. Streamlines `WorkflowContextActivityExecutionMiddleware` by removing redundant service scope creation and enhancing readability.

* Remove redundant timer in CustomerCommunicationsWorkflow

The timer activity that was set to not start the workflow has been removed to streamline the workflow process. Updated the README to include instructions on how to start the CustomerCommunicationsWorkflow using the REST API or Elsa Studio.

* Refactor middleware to streamline context management

Removed dependency on `IServiceScopeFactory` and refactored middleware to directly load and save workflow contexts. Simplified workflow context provider invocation, enhancing code readability and maintainability.

* Remove redundant PropertyBag property

The PropertyBag property was commented out and is no longer needed in the WorkflowDefinition class. This change removes the commented-out code to clean up the class definition.

* Add JsonSerializerOptions for deserialization

Introduce JsonSerializerOptions with camelCase policy and case insensitivity for property names. This ensures consistent deserialization of custom properties in activity workflow context settings.
2024-08-06 22:36:24 +02:00
Sipke Schoorstra 19d1da12ec Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-24 22:33:36 +02:00
raymonddenhaan c413f210a9
Add activity execution log sink (#5833)
* Add activity execution log sink

Introduced an interface `IActivityExecutionLogSink` for storing workflow execution log records and implemented it as `StoreActivityExecutionLogSink`. This centralizes and streamlines the handling of persisting execution logs into the workflow's runtime, thus making the process more modular and maintainable. The `PersistActivityExecutionLogMiddleware` has also been refactored to use this sink, replacing the direct usage of the workflow execution log store.

* Refactor logging interfaces and implementations

Replaced specific logging interfaces and implementations with generic ones. The individual interfaces for workflow and activity execution logging have been replaced with a single ILogRecordExtractor and ILogRecordSink interface. Specific implementations have been adjusted to use these new interfaces. This allows for greater flexibility and reuse of logging code.

* Introduce ILogRecordStore interface

The `ILogRecordStore` interface has been introduced to provide a centralized place for handling log records. Both `IActivityExecutionStore` and `IWorkflowExecutionLogStore` have been updated to inherit from this new interface. As a result of this change, the `SaveManyAsync` methods in these two interfaces have been removed to avoid redundancy.
2024-07-24 22:30:14 +02:00