49 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sipke Schoorstra
|
b09a564812
|
Fix Multitenancy Support and Normalize Tenant ID Handling (#7217)
* Enable multitenancy support and normalize tenant ID handling. - Activate multitenancy in `Program.cs`. - Introduce `NormalizeTenantId` method for consistent tenant ID usage. - Update tenant-related classes and features to support normalization logic. * Add ADR for adopting empty string as the default tenant ID - Standardized the tenant ID for the default tenant to use an empty string (`""`) instead of `null`. - Documented the rationale and migration considerations in ADR 0007. - Updated ADR table of contents and graph for new entry. * Apply suggestion from @sfmskywalker * Update doc/adr/graph.dot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Normalize spacing and improve readability in `Program.cs`. Fix multitenancy condition formatting. * Fix ADR numbering and update TOC * Add ADRs for flowchart execution model, tenant deletion event, merge modes, and default tenant ID - Introduced ADR 0005: Token-centric flowchart execution model for improved loop and join handling. - Added ADR 0006: Tenant Deleted event for distinct handling of tenant removal. - Documented ADR 0007: Explicit merge modes for flowchart joins, improving reliability and configurability. - Included ADR 0008: Standardization of empty string as the default tenant ID for consistency and clarity. * Add unit tests for tenant ID normalization and multitenancy pipeline invoker - Added comprehensive unit tests for tenant ID normalization to ensure consistent handling of null, empty, and valid IDs. - Introduced tests for the multitenancy pipeline invoker covering various tenant resolution scenarios. - Updated solution to include new unit testing projects for `Elsa.Tenants` and `Elsa.Common`. * Update unit tests for `ActivityConstructionResult` - Refactor test parameterization to verify `HasExceptions` property more explicitly. - Simplify exception creation logic in helper methods. - Improve test assertions by combining act and assert phases where applicable. * Enable configuration-based multitenancy with tenant-specific settings - Introduced a configuration-based tenant provider to streamline tenant initialization and customization. - Added tenant ID handling filters to ensure tenant ID is applied and filtered automatically. - Deprecated the `CommonPersistenceFeature` in favor of modular persistence feature extension. * Update database indexes to include `TenantId` for multitenancy support - Added `TenantId` to unique constraints on `Triggers` table across all EFCore providers. - Adjusted index names to reflect the updated constraints. - Updated trigger configuration to ensure uniqueness includes `TenantId`. * Add tenant filtering to `DefaultWorkflowDefinitionStorePopulator` - Introduced `ITenantAccessor` to support tenant-specific filtering of workflow definitions. - Updated logic to skip workflows not matching the current tenant. * Update doc/adr/toc.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove `CommonPersistenceFeature` as it has been deprecated * Add tenant-specific filtering to workflow import logic in `DefaultWorkflowDefinitionStorePopulator` * Replace hardcoded tenant ID with `Tenant.DefaultTenantId` in integration tests * Update database indexes and migration logic to support `TenantId` for multitenancy - Added `TenantId` to unique constraints on the `Triggers` table and updated index names. - Included logic to drop outdated indexes without `TenantId` during migration. - Adjusted tests to account for `TenantId` in workflow identity and indexing scenarios. * Remove `TenantId` from workflow identity construction in concurrent trigger indexing tests * Introduce `SelectiveMockLockProvider` for precise lock mocking in tests - Added `SelectiveMockLockProvider` to allow targeted lock mocking without affecting unrelated background operations. - Updated test services to use `SelectiveMockLockProvider` in place of `TestDistributedLockProvider`. - Refactored `DistributedLockResilienceTests` to support selective mocking for deterministic and reliable assertions. * Update Elsa.sln Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` for consistent filtering * Refactor `TenantResolverResult` to support explicit resolved/unresolved state handling - Updated `TenantResolverResult` to include an explicit `_isResolved` property. - Adjusted `ResolveTenantId()` and `IsResolved` logic for improved clarity and robustness. - Simplified tenant resolution invocation in `TenantResolverBase`. - Removed redundant normalization in `DefaultTenantResolverPipelineInvoker`. * Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` and `ClrWorkflowsProvider`. * Refactor `DefaultWorkflowDefinitionStorePopulatorTests`: streamline object initializations and add tenant-specific test coverage for `PopulateStoreAsync`. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Sipke Schoorstra
|
fa04e1ebcd
|
Adds activity host registration support (#7172)
* Add support for activity host registration across workflows Introduced new APIs and updates to enable registering custom activity hosts in the workflow management system. This includes modifications to attributes, service registrations, and extensions to streamline integration for advanced activity hosting scenarios. * Remove unused `using` directives across Workflow Management module * Add support for host method activity registration and description Introduce new APIs to enable activity registration from public async methods (Task/Task<T>) on CLR types. Includes `HostMethodActivitiesOptions`, `HostMethodActivity`, `HostMethodActivityProvider`, and `HostMethodActivityDescriber` for dynamic activity generation and execution. * Refactor host method activity execution and cleanup. Reworked `HostMethodActivity` to support resumable workflows, improved parameter handling with pluggable value providers, and removed obsolete `AgentExecutionContext`. Enhanced method resolution, async handling, and input/output descriptor logic for better flexibility and maintainability. * Refactor `Bookmark` model to use mutable properties and update XML documentation. * Refactor `BookmarkExecutionContextExtensions` to improve structure, add `GenerateBookmarkTriggerToken` method, and enhance maintainability. * Add extensibility for host method parameter binding with pluggable value providers Introduced `IHostMethodParameterValueProvider` interface for custom parameter resolution, along with `DefaultHostMethodParameterValueProvider`, `DelegateHostMethodParameterValueProvider`, and `FromServicesAttribute` for flexible binding options. Enhances host method activity execution by supporting DI resolution and workflow input handling. * Refactor nullable usage and improve bookmark management logic Updated null assignment for consistency across files and refined logic for detecting and handling newly added bookmarks. Adjusted method signatures and parameters in the DecoratedStoryWriterAgent class for more explicit input handling. These changes enhance code readability, maintainability, and robustness. * Update src/modules/Elsa.Workflows.Management/Activities/CodeFirst/HostMethodActivityProvider.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Ensure `CallbackMethodName` is set and skip bookmarks with empty values * Update src/modules/Elsa.Workflows.Management/Features/WorkflowManagementFeature.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/Elsa.Workflows.Management/Contracts/IHostMethodActivityDescriber.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/Elsa.Workflows.Core/Attributes/InputAttribute.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Refactor `CodeFirst` namespace to `HostMethod` for improved clarity and align with updated activity execution logic. Enhance DI-based parameter resolution and update XML documentation for `HostMethodActivitiesOptions`. * Add `Penguin` activity host with sample activity methods and register in Elsa pipeline * Add `TestHostMethod` activities and corresponding component tests. Register `TestHostMethod` as an activity host in the workflow server. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Sipke Schoorstra
|
a1d4e541fc
|
Add Elsa Script DSL (#7076)
* Update packages.yml * Update elsa-server-and-studio.yml * Update elsa-server.yml * Update elsa-studio.yml (#6715) * Update ListWorkflowDefinitionsRequest.cs (#6761) Remove unnecessary line breaks * Correct namespace and import for `ConfigureEngineWithVariableTypes`. * Resolves build issues, update package versions and restructure project references - Updated multiple package versions in `Directory.Packages.props` for better dependency management, including `BenchmarkDotNet`, `FastEndpoints`, and `Microsoft.Extensions.Http.Resilience`. - Minor version upgrade for `System.Formats.Asn1` in `_build.csproj`. - Replaced project reference to `Elsa.csproj` with `Elsa.IO.Http.csproj` in `Elsa.ServerAndStudio.Web.csproj`, enhancing modularity. - Added new using directive for `Elsa.IO.Http.Features` in `Program.cs` to support new HTTP functionalities. * Remove unused project references from Elsa.sln These changes indicate that the associated projects or dependencies are no longer needed or have been replaced by other components in the solution. * Rename copilot-setup-steps.yml.yml to copilot-setup-steps.yml * Update RawStringContent encoding in JsonContentFactory (#6786) * Update RawStringContent encoding in JsonContentFactory Modified the instantiation of `RawStringContent` to use a new `UTF8Encoding` instance with `encoderShouldEmitUTF8Identifier` set to `false`, affecting the handling of the UTF-8 byte order mark (BOM) in serialized JSON content. Fixes a bug with content length being different than expected. * Refactor JsonContentFactory to reuse UTF8Encoding Introduced a private static readonly field `_utf8Encoding` in the `JsonContentFactory` class to improve code readability and performance. This change replaces the instantiation of `UTF8Encoding` in the `CreateHttpContent` method, allowing for the reuse of the same encoding instance. --------- Co-authored-by: Max Brooks <Max@compyl.com> * Enhance thread safety with ConcurrentDictionary usage (#6760) * Enhance thread safety with ConcurrentDictionary usage Replaced `IDictionary` with `ConcurrentDictionary` for both `_scheduledTasks` and `_scheduledTaskKeys` to improve thread safety in a multi-threaded environment. Updated methods `RegisterScheduledTask`, `RemoveScheduledTask`, and `RemoveScheduledTasks` to utilize the `Remove` method of `ConcurrentDictionary`, ensuring safe and efficient removal of scheduled tasks. * Refactor task registration and removal logic Updated `RegisterScheduledTask` to use `AddOrUpdate` for streamlined task management. This change simplifies the addition and updating of scheduled tasks by consolidating logic into a single operation. Introduced `RemoveScheduledTask` method to handle task removal by name, improving code organization and clarity. * Improve task removal handling in LocalScheduler Modified the `LocalScheduler` class to enhance the removal process of scheduled tasks from the `_scheduledTaskKeys` collection. The removal operation now captures the result in a variable and includes a conditional check to log a warning if the task was not found, improving error handling and debugging capabilities. * Refactor task removal in LocalScheduler Updated the removal process for scheduled tasks in `_scheduledTasks`. The new implementation collects all corresponding keys and attempts to remove them individually, logging warnings for any failures. This enhances error handling and provides better debugging information. --------- Co-authored-by: Max Brooks <Max@compyl.com> * Add IAsyncEnumerable check to ItemSourceActivityExecutionContextExtensions.GetItemSource (#6897) * Use FullName in WorkflowDictionary (#6923) * Fixed ParentWorkflowInstanceId not being set (#7029) Co-authored-by: Peter Klooster <peter.klooster@autotaalglas.nl> * Remove unused solution projects and update package references - Deleted several project references from `Elsa.sln` to clean up the solution. - Updated `Directory.Packages.props` for consistency and alignment with the latest package versions. * Simplify CI pipeline by removing `Test` step from `Compile+Test+Pack` process. * Initial plan * Add ElsaScript DSL module with parser and compiler Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Add integration tests for ElsaScript DSL Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Add comprehensive documentation for ElsaScript DSL Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Refactor workflow activity instantiation logic - Removed `ActivityFactory` and its related interfaces and extensions. - Introduced `ActivityActivator` for handling activity creation. - Extended AST with support for comprehensive workflow structures: - Added nodes for flowcharts, if/else, loops, and variable declarations. - Updated `IElsaScriptCompiler` to use asynchronous methods. - Expanded `ElsaScriptParser` to simplify syntax for `UseNode` and argument parsing. - Adjusted compiler and parser for compatibility with new workflow AST model. * Refactor test method names for clarity and add new compiler and parser tests - Updated method names in `CompilerTests` and `ParserTests` for better readability and description of test intent. - Added tests for compiler and parser: - Support for workflows without the `workflow` keyword. * Refactor `ElsaScriptParser` to improve statement parsing and introduce a tokenizer - Added `TokenizeStatements` method to split source into statements for enhanced parsing accuracy. - Updated logic to process statements instead of raw lines, reducing parsing complexity and improving reliability. - Improved handling of workflow and statement parsing, including edge cases with braces, parentheses, and string literals. * Introduce ElsaScript support for BlobStorage workflow provider - Added the `Elsa.WorkflowProviders.BlobStorage.ElsaScript` module to enable ElsaScript-based workflow definitions for BlobStorage. - Implemented `ElsaScriptBlobWorkflowFormatHandler` for parsing ElsaScript workflows stored in BlobStorage. - Extended `ElsaScriptParser` to leverage Parlot for improved DSL parsing. - Introduced `IBlobWorkflowFormatHandler` to centralize workflow format handling and parsing. - Updated `Elsa.Server.Web` to reference the new module and include an ElsaScript "Hello World" example workflow. * Refactor ElsaScript services, update logging, and improve workflow handling - Changed `ElsaScriptCompiler` service registration from `Singleton` to `Scoped` for better dependency management. - Enhanced the "Hello World" example workflow and added `CopyToOutputDirectory` configuration. - Removed unused namespaces and adjusted references in multiple projects to improve maintainability. - Updated logging levels in `appsettings.json` to reduce unnecessary debug output. - Improved `PolymorphicObjectConverter` by removing redundant dependencies. - Added missing references to enhance feature support and ensure compatibility. * Refactor activity instantiation and improve argument handling in `ElsaScriptCompiler` - Added support for positional arguments with constructor matching logic. - Refactored `InstantiateActivityUsingConstructor` to enhance activity creation. - Updated `ActivityDescriptor` and related types to include `ClrType` for streamlined activity resolution. - Simplified `TypedActivityProvider` by annotating it with `[UsedImplicitly]`. - Adjusted `ElsaScriptParser` to remove unnecessary options from string literal definitions. * Add HTTP-enabled "Hello World" workflow and support for additional HTTP activity constructors - Introduced a new ElsaScript example workflow `hello-world-http.elsa` with an HTTP endpoint and response. - Enhanced `HttpEndpoint` and `WriteHttpResponse` activities with additional constructors for improved flexibility. - Updated project to include the new workflow in the output directory. * Enhance `ElsaScriptParser` with a custom parser to handle nested raw expressions for ElsaScript workflows - Introduced `RawExpressionParser` to parse raw text after `=>` up to a matching closing parenthesis. - Updated `elsaExpressionWithLang` and `elsaExpressionWithoutLang` to use `RawExpressionParser`. - Trimmed whitespace in parsed expressions. - Added integration and parser tests for complex workflows with variables and expressions. - Updated example workflow `hello-world-http.elsa` to demonstrate expression usage. - Added `Elsa.Http` module reference to enable HTTP-based activities. * Update "Hello World" workflow to simplify naming and enhance response logic - Renamed workflow from `HelloWorldHttpDsl2` to `HelloWorldHttpDsl`. - Updated HTTP endpoint path to `/hello-world-dsl` for consistency. - Improved response logic by utilizing `getMessage()` JavaScript function. * Add support for `OriginalSource` in workflow materialization and enhance ElsaScript materializer - Introduced `OriginalSource` property in `WorkflowDefinition` and `MaterializedWorkflow` for preserving original source representation (e.g., ElsaScript, JSON, YAML). - Added `ElsaScriptWorkflowMaterializer` implementation to materialize workflows directly from ElsaScript source. - Updated `DefaultWorkflowDefinitionStorePopulator` to determine `StringData` or `OriginalSource` based on materialized workflow format. - Enhanced `WorkflowDefinitionMapper` to support symmetric round-tripping with `OriginalSource`. - Registered `ElsaScriptWorkflowMaterializer` in `ElsaScriptFeature` for dependency injection. - Updated `JsonBlobWorkflowFormatHandler` and added `OriginalSource` support for round-trip preservation. - Simplified `ElsaScriptParser` by aligning variable and parser naming. * Update V3_6 migrations for PostgreSQL, MySQL, and Oracle databases and associated designer files. * Handle disposal and race conditions in `ScheduledCronTask` - Added `_disposed` flag to prevent accessing disposed resources. - Updated `_executionSemaphore` and `_scopeFactory` logic to safely handle `ObjectDisposedException`. - Enhanced task scheduling and timer disposal with additional safeguards against race conditions. - Modified tests to ensure proper disposal and logging behavior when handling edge cases. * Add support for metadata in ElsaScript workflows and enhance parser and compiler functionality - Introduced metadata syntax in ElsaScript workflows (e.g., `DisplayName`, `Description`, `Version`) to enable metadata-driven behavior. - Enhanced `ElsaScriptCompiler` to process metadata and properly integrate it into `Workflow` objects. - Updated `ElsaScriptParser` to parse program-level AST with support for multiple workflows and global use statements. - Refactored tests to validate metadata parsing and ensure backward compatibility with existing workflows. - Added new test cases to cover scenarios like metadata parsing, compilation, and multi-workflow programs. * Add support for `foreach` loops in ElsaScript and remove `let` keyword - Introduced `foreach` loop syntax in `ElsaScriptParser` and `ElsaScriptCompiler`, enabling iteration over collections with optional variable declaration. - Updated `ForNode` and `ForEachNode` to include a `DeclaresVariable` flag for improved variable handling. - Removed support for the `let` keyword in variable declarations, streamlining syntax to use `var` and `const` only. - Enhanced `for` loop syntax to support optional `var` declaration and block or single-statement bodies. - Refactored test cases to validate `foreach` and `for` loop enhancements and ensure backward compatibility. * Simplify ElsaScript workflow syntax by removing redundant quotes in workflow identifiers and updating `for` loop syntax for clarity and consistency. * Remove redundant quotes from workflow identifiers in integration tests * Simplify Elsa scripts and improve error handling - Removed redundant braces in workflow declarations for streamlined syntax. - Enhanced logging in `JsonBlobWorkflowFormatHandler` and `ElsaScriptBlobWorkflowFormatHandler` to warn on parsing errors and provide context. - Updated configuration to log errors for `Elsa.Workflows.ActivityRegistry`. - Refined "Hello World" and "For Loop" workflows for clarity and added improved loop handling. * Refine Elsa workflows and update compiler logic - Simplified "Hello World" workflow by adding braces and improving consistency. - Adjusted "For Loop" workflow to rename and clarify logic, including expression updates and variable handling. - Fixed compiler mapping of `"cs"` to `"CSharp"` for better clarity. - Enhanced "Hello World HTTP" workflow to correctly reference `variables.message` in expressions. * Add flowchart support in ElsaScript parser, compiler, and integration tests - Introduced `flowchart` syntax in `ElsaScriptParser` to support flowchart-based workflows. - Updated `ElsaScriptCompiler` to compile `flowchart` nodes with labeled activities, connections, entry points, and variables. - Added integration tests for parsing and compiling empty and simple flowcharts. - Enhanced `FlowchartNode` and `LabeledActivityNode` for better representation of flowchart structures. - Improved error handling and logging for invalid flowchart configurations. * Add tests for compiling and parsing flowcharts with nodes, connections, and block nodes in ElsaScript - Added integration tests for compiling and validating flowchart structures, including activities, connections, and entry points. - Implemented parser tests for parsing flowcharts with node connections and block nodes. - Updated project files to include new workflow examples for testing. * Add Parlot package and update project file in integration tests - Added `Parlot` package version `0.0.27` to `Directory.Packages.props`. - Updated integration test project file to include a new `Include` directive for better targeting. * Update Parlot package to version 1.5.2 in Directory.Packages.props * Remove `elsa-server-and-studio.yml` workflow and update solution file - Deleted `elsa-server-and-studio.yml` workflow as it's no longer needed. - Updated `Elsa.sln` to remove reference to the deleted workflow. * Remove `elsa-studio.yml` workflow and update solution and packages - Deleted `elsa-studio.yml` workflow as it's no longer used. - Updated `Elsa.sln` to remove reference to the deleted workflow. - Changed `base_version` in `packages.yml` from `3.7.0` to `3.6.0`. * Downgrade Docker image in `elsa-server.yml` workflow from `v3.7.0-preview` to `v3.6.0-preview` * Update Docker image tag in `elsa-server.yml` workflow from `v3.6.0-preview` to `v3.6-preview` * Add logging support to `LocalScheduler` and replace `Debug.WriteLine` with `ILogger` * Remove unused `System.Collections.Generic` and `Elsa.Extensions` imports in `LocalScheduler` - Cleaned up unnecessary using directives to improve code readability and maintainability. - Minor whitespace adjustment for consistent formatting. * Remove unnecessary whitespace in `LocalScheduler` for consistent formatting * Improve exception handling in blob workflow format handlers - Updated exception handling in `ElsaScriptBlobWorkflowFormatHandler` and `JsonBlobWorkflowFormatHandler` to gracefully catch and log all exceptions during workflow parsing. - Adjusted comments to clarify behavior for invalid user-provided files, ensuring the workflow loading process is not disrupted. * Refactor blob workflow format handlers to use `SupportedExtensions` for improved file filtering - Added `SupportedExtensions` property to all blob format handlers to optimize blob storage browsing. - Simplified `CanHandle` logic by removing extension checks, leveraging `SupportedExtensions` for initial filtering. - Updated comments for clarity and consistency across handlers. * Refactor `DefaultWorkflowDefinitionStorePopulator` to simplify `stringData` assignment logic and improve readability * Remove outdated comment in `CompilerTests` about skipped tests * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Refactor `ElsaScriptCompiler` to streamline type conversion logic, improve language mapping, and enhance asynchronous flowchart compilation * [WIP] Update ParseError printing based on feedback (#7082) * Initial plan * Fix ParseError formatting to use Message and Position properties Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Replace `as` casts with direct casts in ParserTests for null safety (#7083) * Initial plan * Replace 'as' casts with direct casts in ParserTests for better null safety Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Fix Oracle column types for OriginalSource and other large text fields (#7079) * Initial plan * Fix Oracle OriginalSource and StringData column types to handle large data Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Refactor tests to replace type checks with `Assert.IsType` for improved clarity and type safety * Initial plan (#7080) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Add `Parlot` package reference and update solution structure by removing and reorganizing projects and workflows. * Set default expression language to "JavaScript" in `ElsaScriptCompiler`. * Add integration test to verify default expression language resets between ElsaScript compilations * Simplify UTF-8 encoding in JsonContentFactory (#7081) * Initial plan * Remove explicit UTF8Encoding in JsonContentFactory and use Encoding.UTF8 Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Fix test to use Encoding.UTF8.GetByteCount for multi-byte character support Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: Ender <37611092+zengande@users.noreply.github.com> Co-authored-by: Matt <knibbsy10@live.com> Co-authored-by: Max Brooks <45081361+MaxBrooks114@users.noreply.github.com> Co-authored-by: Max Brooks <Max@compyl.com> Co-authored-by: FuJa0815 <30809803+FuJa0815@users.noreply.github.com> Co-authored-by: Peter Klooster <crashkonijn@gmail.com> Co-authored-by: Peter Klooster <peter.klooster@autotaalglas.nl> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Sipke Schoorstra
|
32b04ded28
|
Enables distributed runtime for workflows
Configures the workflow runtime to use a distributed implementation, enhancing scalability and resilience. Adds debug logging for Elsa. |
||
|
Sipke Schoorstra
|
bc3543fcb6
|
Merge 3.5.0 | ||
|
Sipke Schoorstra
|
adad649fa5
|
Update tenant HTTP prefix in appsettings.json for consistency | ||
|
Sipke Schoorstra
|
2bf9ecc709
|
Merge remote-tracking branch 'origin/develop/3.5.0' into develop/3.6.0 | ||
|
Sipke Schoorstra
|
0811a4042d
|
Merge remote-tracking branch 'origin/patch/3.4.1' into develop/3.5.0 | ||
|
Sipke Schoorstra
|
c694a18c13
|
Enhances Mediator with Tenant Context Propagation (#6738)
* Update package versions in Directory.Packages.props Upgraded multiple package dependencies to latest versions, ensuring compatibility, security, and access to the newest features. * Refactor mediator pipeline to support tenant context propagation - Introduced `TenantPropagatingMiddleware` to handle tenant context propagation during command execution. - Added `SetupMediatorPipelines` hosted service for configuring mediator pipelines. - Enhanced `CommandPipeline` and builder to allow middleware insertion, removal, and reordering. - Updated `CommandContext` and related components to support headers for tenant context handling. - Improved logging and refactored `BackgroundWorkflowDispatcher` to include tenant headers during command dispatch. * Fix typos in XML documentation and improve middleware extension clarity - Corrected duplicated slashes in XML doc comments in `ICommandSender.cs`. - Refined phrasing in `MiddlewareExtensions.cs` to clarify method parameters and improve readability. * Update src/common/Elsa.Mediator/Middleware/Command/Components/CommandLoggingMiddleware.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Sipke Schoorstra
|
935d6987d5
|
Commits workflow state during alteration (#6736)
* Reduce default logging verbosity in appsettings.json * Commit workflow state during alteration execution Added `ICommitStateHandler` dependency and implemented workflow state commitment in `DefaultAlterationRunner` to ensure state persistence during alteration execution. |
||
|
Sipke Schoorstra
|
ad49a29017
|
Merge remote-tracking branch 'origin/develop/3.5.0' | ||
|
Sipke Schoorstra
|
c0bc5302cf
|
Configures SQLite as default database provider
Configures the application to use SQLite as the default database provider. Removes the logger dependency from the EFCoreActivityExecutionStore, simplifying the constructor. |
||
|
Sipke Schoorstra
|
3aae05317d
|
Refactor TriggerStore methods and update configurations.
Refactored `TriggerStore` to implement pagination and ordering for `FindManyAsync` methods with support for tenant-agnostic filtering. Modified app settings to change database provider to SQL Server and adjusted logging levels to reduce verbosity. Fixed workflow cancellation service to better handle child instances tasks. |
||
|
Sipke Schoorstra
|
1f2353cad8
|
Remove obsolete modules and associated code
Eliminated the `Elsa.Labels`, `Elsa.Environments`, and `Elsa.OpenTelemetry` modules along with their handlers, contracts, models, and related functionality. This cleanup improves maintainability and aligns the codebase with recent architectural changes. |
||
|
Sipke Schoorstra
|
e09f096ef9
|
Adds activity execution metadata support (#6699)
* Add logging to DefaultActivityExecutionMapper constructor Introduced an ILogger dependency to DefaultActivityExecutionMapper and added a debug log statement in GetPersistableDictionary. This aids in tracking log persistence mode for improved debugging and state visibility. * Simplify activity execution log mapping logic. Replaced asynchronous mapping with synchronous mapping to simplify the logic flow. This change reduces task overhead and improves code clarity while maintaining functionality. * Update activity execution mapping and comment out unused method Replaced direct dictionary usage with a cloned dictionary to ensure data integrity in `DefaultActivityExecutionMapper`. Commented out an unused method in `ActivityExecutionExtensions` to suppress its execution for now. * Add logging to ActivityExecutionLogStore for property tracking Integrate ILogger to track and log details of activity execution records, specifically focusing on properties and their serialization. This enhancement improves debugging and provides better insights into the execution flow. * Introduce Metadata field for activity execution handling Replaces the use of Properties with Metadata across activity execution models and services for storing lightweight, persistent data. Updated serialization, database schema, and relevant APIs to support this change while ensuring backward compatibility. Adjusted logging and extension methods for Metadata integration. * Reset V3.5 Runtime Migrations * Add EF Core migrations for MySQL and SQL Server schema updates Introduced migrations to support schema changes for MySQL and SQL Server. Changes include new columns for bookmarks and activity execution records, updates to existing columns, and creation of additional indexes. These updates aim to enhance database structure and query performance. * Add support for metadata in workflow execution context Introduce a `Metadata` property to `ActivityExecutionContextState` to enhance workflow state management. Updated `WorkflowStateExtractor` to handle metadata merging and preservation. Added an alias for `RetryAttemptRecordList` in `ExpressionOptions` for improved type handling. * Remove logger dependency from DefaultActivityExecutionMapper Eliminated the ILogger dependency and related logging calls from DefaultActivityExecutionMapper to simplify the class. This reduces unnecessary coupling and streamlines the activity execution mapping process. |
||
|
Sipke Schoorstra
|
7b75f0c89f
|
Implement retry attempt capturing (#6674)
* Update default initializations and input parameters to `null` Replaced `default!` with explicit `null` for input parameters and properties throughout various classes. Adjusted constructors' default values for consistency and readability. This change ensures better clarity and alignment with nullable reference types. * Add Polly-based resilience integration for retry tracking Introduce Polly diagnostics to log retry events in the execution context. Updated resilience strategy interfaces and implementations to support Polly's context and retry event tracking. * Refactor resilience and retry handling, add flaky endpoint. Removed custom Polly-based diagnostic listeners and observers in favor of a transient status code utility class. Introduced a mock "flaky" endpoint for testing failure scenarios and updated configuration for resilience strategies. Minor namespace fixes * Add retry attempt recording to resilience feature Introduce `IRetryAttemptRecorder` and its implementations to enable recording of retry attempts during activity execution. Updated `ResilientActivityInvoker` to persist retry attempts and modified `ResilienceFeature` to support configurable retry attempt recorders. * Add retry attempt tracking and retrieval functionality Introduced mechanisms to track and fetch retry attempts, including new interfaces, reader implementations, API endpoints, and related models. These enhancements improve resilience tracking and data access for activity execution across workflows. * Add GetOutcome method to RetryAttempt model Introduce a GetOutcome method to encapsulate logic for determining the retry attempt's outcome. It prioritizes the Result, falls back to the Exception message, or defaults to "Unknown" if neither is available. This improves clarity and reusability of the outcome evaluation. * Add scoped registration for _retryAttemptReader This change ensures that _retryAttemptReader is registered in the DI container as a scoped service. * Refactor retry mechanism to support detailed retry metadata Introduced a `CollectRetryDetails` method to `IResilientActivity` for enhanced retry data collection. Updated `RetryAttemptRecord` to include a `Details` dictionary for capturing metadata, replacing previous `Result` and `Exception` fields. These changes simplify the retry recording process and improve extensibility for tracking retry details across activities. * Add support for capturing background activity properties Introduced functionality to capture and persist background activity properties during workflow execution. This includes defining a key for properties, capturing them in middleware, and storing them in the workflow execution context. These changes ensure properties are handled consistently alongside other activity data. * Add support for storing and propagating activity execution properties Introduced a `Properties` dictionary to track additional metadata in activity execution records and stats, enabling richer diagnostics and tracing. Refactored resilience logic to improve retry handling and propagate retry-related flags in workflows. Enhanced database queries to map serialized properties for execution summaries. * Add retry propagation for background activity execution Introduced a mechanism to propagate the retry-attempted flag across activity execution contexts. Added a new notification `BackgroundActivityExecutionCompleted` and updated related middleware to send this notification. Enhanced resilience features to handle and propagate retry state effectively. * Refactor default parameters and values to use 'null'. Replaced 'default' with 'null' for optional parameters and values in `AddExecutionLogEntry`, improving clarity and ensuring semantic consistency with nullable types. No functional changes were introduced. * Refactor flaky endpoint and enhance resilience support. Replaced the "Flaky" endpoint with a more robust "SimulateResponseEndpoint" under a new module. Introduced a status code lookup utility and improved resilience strategies with configurable backoff types. Updated serialization to support enum conversions and enhanced caching behavior for response simulation. * Update activity execution models with nullable properties Replaced `default!` initializations with `null!` to ensure correct handling of nullable string properties in `ActivityExecutionRecord`. Added a new `Properties` dictionary to `ActivityExecutionRecordSummary` to store additional activity execution data. This enhances model flexibility and data extensibility. * Add support for recording resilience strategy in context Introduced a new method to store resilience strategy details in the activity execution context for enhanced diagnostics. Updated `ResilientActivityInvoker` to serialize and set the resilience strategy using this method, leveraging `JsonSerializer`. * Remove redundant PropertyNamingPolicy assignment The PropertyNamingPolicy was set to the default value (CamelCase), making the assignment unnecessary. This change simplifies the code while maintaining existing functionality. * Set JSON property naming policy to camelCase Updated JSON serialization settings to use camelCase naming for property names. This improves consistency with standard JSON naming conventions and ensures compatibility with camelCase-based APIs. * Remove unused Endpoints folder reference from project file The Endpoints folder reference in the project file was unnecessary and has been removed. This cleanup helps maintain a tidy and accurate project structure. * Remove unused RetryAttemptFilter and add Polly packages Removed the obsolete RetryAttemptFilter class as it was no longer in use. Added Polly and Polly.Extensions packages to the project to support resilience and fault-handling strategies. This update aligns with keeping dependencies relevant and reducing unused artifacts. * Add resilience integration test for FlowSendHttpRequest (#6692) * Refactor and fix resilience test cases for clarity and accuracy Simplified imports, adjusted code structure, and corrected attempt indexing logic in resilience tests. These changes improve readability, maintainability, and ensure accurate validation of retry attempts in test scenarios. |
||
|
Sipke Schoorstra
|
9aa239719d
|
Add IResilienceStrategy Abstraction with Category Matching and Expression-Based Configuration (#6637)
* Add resilience module with core interfaces and services Introduced a new `Elsa.Resilience` module and its core components to support resilient services and activities. This includes resilience strategies, providers, and attributes, along with integration into the existing HTTP module for enhanced fault tolerance. Added solution and project references for proper dependency management. * Add resilience strategy framework with HTTP strategy support Introduced a resilience strategy architecture, including a configurable `HttpResilienceStrategy` with retry capabilities, strategy serialization, and integration with existing modules. Enhanced ResilienceFeature to support registration of strategy types and updated application configuration to enable resilience strategies. This change ensures more robust and fault-tolerant HTTP request handling. * Add JSON serialization support for resilience configuration Introduced `ConfigurationExtensions` to enable JSON serialization of configuration sections. Updated resilience strategies to utilize the new extension methods and adjusted JSON serialization logic to support polymorphism with `$type` discriminator. Minor modifications were made to support deserialization and property mutability. * Add resilience strategy support to workflows and API clients Introduced resilience strategy configuration, serialization, and execution support across workflows and API clients. Added new APIs, models, and services to enhance fault tolerance capabilities for activities and HTTP interactions. * Refactor resilience services for improved modularity. Replaced `IResilienceService` with new modular interfaces (`IResilienceStrategyCatalog`, `IResilienceStrategyConfigEvaluator`, `IResilientActivityInvoker`) and corresponding implementations. Enhanced maintainability by simplifying components and responsibilities, ensuring better separation of concerns. * Rename methods in ResilienceStrategyCatalog for clarity Updated method names in `ResilienceStrategyCatalog` and its interfaces for better readability and alignment with naming conventions. Replaced `GetAllStrategiesAsync` with `ListAsync` and `GetStrategyAsync` with `GetAsync` across the codebase. * Refactor resilience handling in HTTP activities. Replaced `ResilienceCategory` property with `ResilienceCategoryAttribute` for a cleaner implementation. Updated `IResilientActivity` to simplify its interface and adjusted related modifications accordingly. Introduced `IResilientActivityInvoker` to enhance resilience strategy execution. * Add support for additional resilience and scripting features Extended resilience strategy handling with serialization support, added `HttpResilienceStrategy` type in JavaScript handler, and refined object conversion logic for interfaces. Minor adjustments to `Expression` class properties for consistency. * Remove `UseResilience` call from Program.cs This call was redundant and no longer necessary for the application. Its removal simplifies the code and ensures only required middleware is used. * Remove commented-out JSON converter code in serializer setup Cleaned up unused and commented-out converter initialization code in `ResilienceStrategySerializer`. This improves readability and removes unnecessary clutter from the file. * Fix typo in method names from 'Resiliency' to 'Resilience' Renamed methods to maintain consistency in naming conventions across the codebase. This change ensures clarity and alignment with established terminology. * Refactor namespace for ConfigurationResilienceStrategySource Updated the namespace of ConfigurationResilienceStrategySource to "StrategySources" for better alignment with naming conventions and structure. Removed an unused namespace reference in ResilienceFeature for cleanup. * Mark EnableResiliency as obsolete in SendHttpRequestBase. The EnableResiliency property is now marked with the [Obsolete] attribute. Developers are encouraged to use the common Resilience Strategy setting instead for managing HTTP request resiliency. This change ensures better consistency and alignment with the broader resilience strategy. * Restrict ResilienceCategoryAttribute to class targets only Removed support for using ResilienceCategoryAttribute on properties. This change enforces a stricter and more focused usage of the attribute, ensuring it applies only to class-level declarations. * Add documentation for IResilientActivityInvoker interface Include summaries and parameter descriptions for the `InvokeAsync` method. This improves code clarity and helps developers understand the functionality and usage of the resilient activity invocation process. * Fix logical operator precedence in type comparison check Parentheses were added to ensure correct evaluation of conditions when checking type compatibility. This prevents potential logical errors when determining the target type in object conversions. * Add support for resilience source identification Introduce the `ResilienceSourceNameAttribute` to allow naming of resilience sources. Updated `ResilienceStrategyCatalog` to utilize the attribute for prefixing strategy IDs, improving source identification and traceability. Applied the attribute to `ConfigurationResilienceStrategySource` as an example. * Revert "Add support for resilience source identification" This reverts commit 19b4e7121d6330b5de4f692b78da4c1e4a2d1f67. * Reapply "Add support for resilience source identification" This reverts commit 8bcba9d040c4eb247077aec6d90dc02817adcbd5. * Revert "Reapply "Add support for resilience source identification"" This reverts commit ee04d35e7930956c752dda3ed150ca34a535e66c. |
||
|
Sipke Schoorstra
|
fd31b5b605
|
Update OTEL fields and tags
Added and updated multiple package references, including OpenTelemetry, Datadog.Trace.Bundle, and various Microsoft.Extensions libraries. This ensures compatibility with the latest dependencies and introduces enhanced features for resilience and tracing. |
||
|
Sipke Schoorstra
|
ff60c4c461
|
Add SQL Server connection string to appsettings.json
This change introduces a connection string for SQL Server to the configuration file. It enables the application to connect to a SQL Server database for data persistence and retrieval. |
||
|
Sipke Schoorstra
|
5be82dd9ab
|
Add support for Citus and YugabyteDB integration
Introduced Docker Compose configurations for Citus and YugabyteDB clusters and updated connection strings in `appsettings.json`. Modified `Program.cs` to enable Entity Framework Core support for both databases and updated the `SqlDatabaseProvider` enum accordingly. Adjusted the Docker Compose setup and bumped `Yarp.ReverseProxy` package version. |
||
|
Sipke Schoorstra
|
13d7e91e0d
|
Remove agent-related configurations and dependencies.
This commit removes the "Agents" configuration section, related API keys, services, and persistence logic across the codebase. Unused agent-related NuGet packages and code references were also eliminated to simplify the project and focus on core functionality. |
||
|
Sipke Schoorstra
|
42ab1a8944
|
Merge branch 'main' into feat/4832 | ||
|
Sipke Schoorstra
|
33d291cfcc
|
Add webhook configuration and support to Elsa Server
Integrated webhook sinks via appsettings.json for handling "Run Task" events. Enabled webhook functionality in the application pipeline and added the necessary project reference for Elsa.Webhooks. This enhances event-driven capabilities and improves extensibility. |
||
|
Sipke Schoorstra
|
3bbb057378
|
Set WorkflowHeartbeatMiddleware logging level to Debug
This change updates the logging configuration to include Debug-level logs for WorkflowHeartbeatMiddleware. It helps in better monitoring and debugging of workflow heartbeat operations. |
||
|
Sipke Schoorstra
|
0a21b5201a
|
Add workflow restart functionality for handling interruptions
Introduced a mechanism to identify and restart interrupted workflows. This includes a new `IWorkflowRestarter` contract, its default implementation, and a recurring task for handling restarts. Additionally, updated configurations and added extensions to improve workflow instance filtering and liveness tracking. |
||
|
Sipke Schoorstra
|
6c0d1ea66c |
Refactor database setup and EFCore provider configurations.
Revised database initialization scripts to better handle Postgres and Oracle environments, introduced schema-specific configurations for Oracle EFCore, and cleaned up obsolete or redundant entity model setup. Streamlined project structure by relocating and renaming files for SQLite and Oracle EFCore setups, improving maintainability and readability. |
||
|
Sipke Schoorstra
|
006e0bbbb4 |
Ignore EF Core pending model changes warnings and update Oracle DSN.
Added configuration to suppress EF Core PendingModelChangesWarning for better compatibility with newer .NET versions. Updated Oracle connection string to use "localhost" for consistency and clarity in appsettings.json. |
||
|
Sipke Schoorstra
|
26e2b1fb56 |
Add Oracle database support to the project
This commit introduces Oracle database integration by adding necessary configurations, entity mappings, and Docker Compose setup. Oracle-specific mappings ensure compatibility with NCLOB for large data handling. The changes also include updates to appsettings, enum for SqlDatabaseProvider, and project references to support Oracle. |
||
|
Sipke Schoorstra
|
48c453d153 |
Enable customizable Hangfire job storage and deprecate obsolete APIs.
Added support for configuring Hangfire job storage per database provider, including PostgreSql, SQLite, and SQL Server. Introduced new methods for flexible Hangfire setup, while marking older APIs and storage configuration extensions as obsolete. Refactored related configurations for streamlined and centralized job scheduling logic. |
||
|
Sipke Schoorstra
|
c5bf6fd0e3 |
Switch database provider to SQLite
Updated the `DatabaseProvider` setting in `appsettings.json` from MySQL to SQLite. This change likely simplifies development or testing by using a lightweight, file-based database. |
||
|
Sipke Schoorstra
|
021795f53a |
Enable MySQL primitive collections support and update configs
Add support for EF Core primitive collections in MySQL by specifying `EnablePrimitiveCollectionsSupport()` in the DbContext configuration. Updated the Docker Compose file to rename the MySQL volume and adjusted appsettings.json to include a MySQL connection string and set MySQL as the default database provider. |
||
|
Sipke Schoorstra
|
4651b7a46d
|
Add schema registry support to Kafka module (#6190)
This commit introduces a schema registry functionality by adding interfaces and classes to manage schema registry definitions in the Elsa.Kafka module. It updates KafkaOptions to include schema registries and modifies classes to support schema registry configurations for producers and consumers. Additionally, it updates package references to include necessary dependencies for schema registry support. |
||
|
Sipke Schoorstra
|
2b63c2beb5 | Add Kafka consumers + producers to reference project | ||
|
Sipke Schoorstra
|
692937d7d7
|
Enhance Multitenancy with Runtime Tenant Management and Task Handling (#6173)
* Work in progress: Add DefaultTenantService for tenant management Introduce `DefaultTenantService` and its corresponding interface `ITenantService` to manage tenant operations such as finding, getting, and listing tenants. Update `MultitenantBackgroundService` to utilize `DefaultTenantService` for handling tenant lifecycle events. This enhancement standardizes tenant operations and improves the maintainability of the multitenancy feature. * WIP * Add multitenancy event handlers and task interfaces Implemented new interfaces IBackgroundTaskStarter and ITaskExecutor to manage task lifecycle events efficiently. Introduced new classes such as RunBackgroundTasks, RunStartupTasks, and StartRecurringTasks for handling tenant activation and deactivation events. Modified TaskExecutor to implement these interfaces and adjusted tenant registration logic to invoke these new handlers. * Refactor multitenancy and task management services. Remove background and recurring task runners, and integrate tenant activation and deactivation into the multitenancy feature. Enable multitenancy in the server application and create a new service for tenant activation and deactivation. This refactor simplifies the management of tenant-specific tasks and enhances the modularity of the platform. * Refactor background service to use startup tasks Replaced hosted service implementation with startup tasks for executing multi-tenant tasks and EF Core migrations. Introduced `PriorityAttribute` to manage task execution order, ensuring migrations run before other services that require database access. This simplifies tenant activation with an ordered task execution and removes redundant classes. * Refactor MultitenancyFeature service registrations Reorganized service registrations for better clarity and maintainability. Changed the registration of some services to use factory delegates for retrieving existing services to ensure correct dependencies. This refactor improves the flexibility of the tenant lifecycle event handling. * Update V3_3 migration files * Add tenant management endpoints and enhance tenant handling Implemented tenant management endpoints including Add, Get, List, and Update. Enhanced tenant handling by introducing configuration and store-based providers, and improved error logging for tenant updates. Adjusted various internal functionalities to better support multitenancy features through different persistence providers. * Implement tenant deletion endpoint and refactor migration setup. Introduce a new API endpoint to handle tenant deletions while providing appropriate responses based on successful or unsuccessful attempts. Refactor migration handling by replacing startup tasks with hosted services across various modules to streamline the migration execution process. * Add and integrate ConfigurationJsonConverter Introduce a `ConfigurationJsonConverter` to handle JSON serialization and deserialization of `IConfiguration` objects. This change centralizes configuration serialization logic, leading to cleaner and more maintainable code. Updated various parts of the codebase to use the new serialization utility, ensuring a consistent approach throughout the application. * Refactor JSON conversion and update tenant endpoint. Removed unused workflow references and streamlined JSON handling in `ConfigurationJsonConverter`. Simplified tenant ID handling by removing `IIdentityGenerator` and setting a default value for `UpdatedTenant.Id`. * Add logging for cancelled recurring tasks Integrated ILogger to log information when a recurring task is canceled due to an OperationCanceledException. This change enhances troubleshooting by providing clearer insights into task cancellations and their underlying reasons, improving maintainability and observability of the task execution process. * Disable multitenancy support and adjust default Tenant ID. Multitenancy is now disabled by setting 'useMultitenancy' to false in the configuration. Additionally, the default Tenant's ID has been changed from null to an empty string to prevent potential null reference issues. * Remove MultitenantHostedService abstraction file The MultitenantHostedService.cs file was removed as it is no longer necessary. Its responsibilities have likely been refactored or integrated into another service, indicating a simplification or restructuring of the multitenancy handling in the codebase. * Rename PriorityAttribute to OrderAttribute for clarity. This change improves the clarity of the code by renaming PriorityAttribute to OrderAttribute, reflecting its actual purpose. All occurrences of the attribute in the codebase have been updated accordingly to maintain consistency. This makes the intent of the code more understandable for future maintenance and development. * Fix message key retrieval in ProduceMessage activity Update the ProduceMessage activity to use GetOrDefault for retrieving the message key. This change ensures that a null key is used if no explicit key is provided or if the key is empty or whitespace, preventing potential errors during message production. * Refactor multitenancy and scheduling services. Removed DefaultTenantContextInitializer interface and class, refactored tenant activation/deactivation to use try-catch logging, and updated tenant context handling to use IDisposable for context push. New activities and workflows added in Elsa.Server.Web, and scheduling services enhanced to schedule jobs with explicit job keys and groups. Also, adjusted configurations to enable multitenancy, providing improved maintainability and flexibility. * Remove Example1 activities and disable multitenancy Deleted Example1Activity, Example1Workflow, and FirstActivity classes to clean up unused code and simplify the codebase. Disabled multitenancy by setting useMultitenancy to false, likely to streamline configuration and resource utilization. * Fix and normalize URL path concatenation. Ensure that the base URLs in both base path providers consistently end with a forward slash. This normalization prevents potential issues with endpoint routing and path concatenation, improving overall URL construction robustness. |
||
|
Sipke Schoorstra
|
844af43f39 |
Refactor database provider configuration logic
Moved SQL Database Provider initialization from a constant to configuration-based dynamic parsing. Enhanced logging configuration for specific components and removed unused tenant and Kafka configurations in appsettings.json for clarity and efficiency. |
||
|
Sipke Schoorstra
|
6022df165c
|
Kafka: Update ProduceMessage activity with support for specifying a Key (#6166)
* Add Key to Kafka ProduceMessage activity Deleted unnecessary Consumer and Producer workflow classes and the OrderReceived message class to clean up code. Refactored Kafka producer interface and implementation to include message keys for improved message handling. Updated configuration to enable Kafka and removed unused service registrations. * Add Kafka factory classes and type alias registry Introduce GenericConsumerFactory and GenericProducerFactory for handling Kafka consumer and producer creation. Implement a TypeAliasRegistry to manage type aliases, enabling cleaner configuration through aliases. Update the OrderReceived message class and ensure better integration with the server web program via these new components. * Handle empty topics and predicates in Kafka worker. Ensure the Kafka consumer unsubscribes when no topics are available to subscribe to. Additionally, add a check to handle empty string values for predicates, allowing workflow triggers to proceed in this scenario. * Disable Kafka usage in Elsa Server Web configuration Kafka has been disabled in the current configuration by setting the useKafka constant to false. This change might be intended to switch to a different messaging system or to simplify the current setup by removing unnecessary services. Ensure that any dependencies on Kafka are handled elsewhere in the application. |
||
|
Sipke Schoorstra
|
b534b42a60
|
Update Kafka Module: Add Support for Configuring Consumer and Producer Factories (#6139)
* Enable Kafka Worker Factory and Refactor Worker Implementation Introduce a flexible worker factory mechanism allowing custom worker creation with DefaultWorkerFactory as the initial implementation. Enhance Worker class to be generic, remove manual consumer configuration, and streamline message processing logic, improving code maintainability and extensibility. * Refactor Kafka configuration properties Renamed configuration properties in Consumer and Producer entities. Updated references in the codebase to use the new `Config` property instead of `ConsumerConfig` and `BootstrapServers`. Adjusted appsettings.json to match the new configuration schema. * Add Kafka producer and consumer implementation Implemented classes and interfaces to handle Kafka producers and consumers, including `ProducerProxy`, `ConsumerProxy`, and related context classes and factories. Refactored existing code to utilize these new implementations, replacing worker terminology with consumer and addressing context-specific fields. * Remove redundant code in DefaultConsumerFactory and SendMessage Removed commented-out unused return statement in DefaultConsumerFactory. Also eliminated explicit producer.Dispose() call in SendMessage, as the 'using' statement already handles resource cleanup. * Add ExpandoObject producer and consumer factories Replaced DefaultSerializers with new JsonSerializer and JsonDeserializer classes. Introduced ExpandoObjectProducerFactory and ExpandoObjectConsumerFactory to handle dynamic types. Updated workflow and configuration to use the new factories. * Refactor bookmark processing and manage worker subscriptions Refactored bookmark processing logic to utilize extension methods. Optimized worker subscriptions by centralizing topic subscription management and added logging for subscribed topics. This improves maintainability and clarity of the codebase. * Refactor worker creation to use ActivatorUtilities Updated WorkerManager to instantiate workers using ActivatorUtilities for better dependency injection support. This enhances code readability and maintains consistency with the service provider approach used throughout the codebase. |
||
|
Sipke Schoorstra
|
f53e024d25
|
Add Elsa.Kafka Module for Kafka Integration with Message Sending and Receiving Activities (#6108)
* Add Kafka module with integration and example setup Introduced the Kafka module providing consumer integration and activities into the project. This includes new classes for consumer handling, configuration, and activities. An example setup using Docker Compose is also added to facilitate development and testing. * Refactor KafkaTransportMessage to inline Timestamp namespace Simplify the namespace usage for the Timestamp type within the KafkaTransportMessage record. This change eliminates the need for a separate using directive for Timestamp, enhancing code readability and maintainability. * Add support for handling Kafka transport messages This commit introduces the capability to handle and trigger workflows based on Kafka transport messages. It adds a new handler, notifications, and updates the message stimulus to include correlating fields. Additionally, the Kafka consumers are now managed more modularly with updated startup tasks and mediator integration. * Enable Kafka integration and fix Kafka options naming Added support for Kafka integration in Elsa.Server.Web by setting up Kafka configurations in appsettings.json and updating Program.cs. Also, renamed `ConsumerConfigs` to `ConsumerDefinitions` in Kafka options for clarity. * Add consumer definition enumeration and dropdown support Introduced `IConsumerDefinitionEnumerator` for managing consumer definitions across providers and implemented in `ConsumerDefinitionEnumerator` class. Enhanced `KafkaFeature` to register these services and updated the `MessageReceived` activity to use a dropdown UI hint for consuming definitions. Improved `StartConsumersTask` by refactoring consumer definition retrieval logic. * Add SendMessage activity and refine Kafka messaging Introduce a new SendMessage activity for Kafka, enabling message publishing to specific topics. Refine KafkaTransportMessage model by removing headers and timestamp fields. Adjust the StimulusSender logic to streamline the bookmark queuing process and fix key-value pairing in dropdown options. Update appsettings for corrected Kafka bootstrap server and topic configurations. * Add producer and topic management support Introduced interfaces and implementations for managing producer and topic definitions along with their respective enumerators and list providers. Updated `SendMessage` activity to include producer selection and refactored consumer definition providers for better consistency. * Refactor Kafka configuration property names Renamed Kafka configuration properties for better consistency and readability across the codebase. Updated property names from `ProducerDefinitions` to `Producers`, `ConsumerDefinitions` to `Consumers`, and `TopicDefinitions` to `Topics`. Added missing input attribute in `SendMessage.cs` and registered additional handlers in `KafkaFeature.cs`. * Add custom serializers for Kafka message handling Introduced `DefaultSerializers` class for custom serialization and deserialization of Kafka messages. Updated `MessageReceived` and `SendMessage` activities to use these custom serializers, and modified `KafkaOptions` to include them. * Fix ExpandoObject serialization method parameter Changed the serialization type from `ExpandoObject` to the actual type of the object to ensure proper serialization. This ensures that derived types are correctly handled during the serialization process. * Add Producer and Consumer workflows for Kafka Introduced two new workflows: `ProducerWorkflow` and `ConsumerWorkflow` for handling Kafka messages. Updated `DefaultSerializers` to use camelCase property naming and modified `appsettings.json` to include `topic-2` and format entries. * Add JSON serialization to log output in ConsumerWorkflow This change enhances the log output by serializing messages to JSON format before writing them. The addition of System.Text.Json ensures that the message content is presented in a structured and standardized format in logs. * Add correlation strategies and update Kafka features Implemented HeaderCorrelationStrategy and NullCorrelationStrategy, and updated KafkaFeature to support customizable correlation strategies. Added correlation ID handling to Kafka transport messages and updated config and handlers accordingly. * Add tenant accessor to ConsumerDefinitionWorkflowContextProvider Integrated ITenantAccessor to the provider to support tenant-specific context loading. Updated the constructor and LoadAsync method to retrieve the tenant information and use it for context-specific operations. * Switch to MySQL and disable Kafka This commit changes the SQL database provider from SQLite to MySQL and disables Kafka use. It also includes necessary adjustments such as adding MySQL handling in configuration and connection setups, updating `docker-compose` to include MySQL services, and referencing MySQL projects in the `.csproj` file. * Add UI property handlers to multiple features This commit introduces various UI property handlers across several features such as Python, JavaScript, CSharp, and Workflow features to enhance user interface property handling. It also updates the property UI handler resolution logic to better manage cases where providers are not available. Furthermore, adjustments were made in the server configuration to switch database providers and enable Kafka. * Refactor property UI handler retrieval logic Modified the logic to fetch property UI handlers by preloading them into a list and then filtering. This change improves readability and potentially performance by reducing repetitive service provider calls. * Incremental work on Kafka workers and predicate evaluation * Merge BookmarkInvoker with BookmarkResumer * Register IWorkerManager * Change lifetime scope of WorkerManager to Singleton * **Introduce topic subscription handling for Kafka workers** Added `IWorkerTopicSubscriber` interface and its implementation for managing topic subscriptions. Enhanced workers to bind triggers and bookmarks dynamically based on existing data. Updated several classes and methods to support topic-based subscriptions and headers. * Refactor trigger matching logic. Extract trigger matching conditions into `IsMatchAsync` method for reuse. This enhances code maintainability and readability by reducing redundancy. The new `GetTopic` helper method isolates the topic retrieval logic. * Add Name property to MassTransitActivityTypeProvider This commit inserts the Name property in the returned object within the MassTransitActivityTypeProvider class. It ensures that the typeName is included, providing a clearer definition of the activity type. * Add handling for deleted bookmarks and refactor bookmark removal Added a new event handler for `BookmarksDeleted` to ensure removed bookmarks are processed correctly. Refactored the bookmark removal logic into a helper method to reduce code duplication and streamline the workflow. * Switch to asynchronous bookmark queue processing Refactored the `TriggerWorkflows` handler to use `IBookmarkQueue` instead of directly invoking the `IBookmarkResumer`. This change aims to improve scalability by queueing bookmark resumption requests, enabling better load distribution and async processing. Added necessary helpers and configuration options to support this functionality. * Remove unused IBookmarkResumer dependency Simplify the constructor by removing the unused IBookmarkResumer dependency. This cleanup reduces potential confusion and improves code maintainability without impacting functionality. * Add support for local message processing Introduced an `IsLocal` property to `MessageReceivedStimulus` for determining if the message event is local to a specific workflow instance. Updated `BookmarkBinding` and related handler methods to utilize `CorrelationId` for local event matching. Removed unused `CorrelatingFields` from `MessageReceived` activity. * Add nullability checks to IWorker retrieval methods Updated `GetWorker` methods to return nullable `IWorker` to handle cases where a worker might not exist. Modified code to include null checks and conditional operations to prevent potential null reference exceptions when accessing worker methods. * Add filtering based on activity type name for triggers and bookmarks This commit introduces filtering for triggers and bookmarks based on the `MessageReceived` activity type name. It also adds an option to mark messages as local in the `SendMessage` activity, where local messages are delivered to the current workflow instance only. These changes help enhance the management and targeted delivery of messages within the workflow framework. * Add new Kafka topics and clean up producers config New topics "topic-3" and "topic-4" were added to the Kafka settings. Unused topic references were removed from the producers configuration to simplify and improve clarity. * Add predicate to KafkaConsumerActivity in ConsumerWorkflow Introduced a predicate to the KafkaConsumerActivity using JavaScript expressions to filter messages based on OrderId. This ensures only relevant messages are processed in the workflow. |
||
|
Sipke Schoorstra
|
939fb95a97
|
Add multitenancy support for background tasks (#6059)
* Remove initial migrations
Deleted obsolete initial migration files from multiple databases: MySQL, SQL Server, SQLite, and PostgreSQL. This cleanup helps maintain a streamlined and updated migration history.
* Add Document base class and create tenant-specific indices
Introduced a new abstract `Document` base class to unify common properties. Implemented tenant-specific unique indices across multiple collections by including `TenantId` alongside `Id` to ensure uniqueness within tenant scopes.
* Remove outdated migration files
Deleted various migration files under MySql, PostgreSql, Sqlite, and SqlServer directories. This cleanup removes unnecessary schema definitions and helps to streamline the codebase.
* Refactor workflow identity assignment logic
Streamline workflow identity handling to ensure consistent assignment of Id, DefinitionId, and TenantId values. This change integrates tenant prefix and version suffix cleanly, enhancing clarity and maintainability.
* Enable multitenancy support
Added configuration for a new tenant (tenant-1) in appsettings.json and enabled multitenancy feature in Program.cs. This change allows the application to support multiple tenants, with specific configurations for each.
* Refactor route table update to run as startup task
Replaced `UpdateRouteTableHostedService` with `UpdateRouteTableStartupTask` to ensure route table updates are executed during application startup instead of as a hosted service. Updated configuration in `HttpFeature` and adjusted trigger validation logic in `ValidateWorkflowRequestHandler`.
* Add recurring task scheduling and single-node task support.
Introduce `IntervalExpressionType`, recurring task scheduling classes, and `SingleNodeTaskAttribute`. Update `RecurringTasksRunner` to handle schedules and add single-node task logic to `StartupTasksRunner`. Ensure proper namespace changes and configure sample recurring tasks.
* Refactor recurring tasks scheduling system
Replaced existing scheduling classes with a more modular and granular approach. Introduced new classes and interfaces like `ISchedule`, `CronSchedule`, `IntervalSchedule`, and `RecurringTaskScheduleManager`. Updated related methods and code to comply with the new design.
* Refactor background task management
Removed `ExpiredSecretsHostedService` and refactored it into a recurring task. Introduced `TaskExecutor` for shared task execution logic. Updated and renamed feature classes to better represent their purpose, improving task scheduling and execution management.
* Add BackgroundTask abstract class to Elsa.Common module
This new abstract class implements the IBackgroundTask interface with default methods for executing, starting, and stopping tasks asynchronously. It provides a basic framework for background task management in the Elsa.Common module.
* Switch to CreateAsyncScope in DefaultTenantScopeFactory
Updated the CreateScope method to use CreateAsyncScope instead of CreateScope. This change improves asynchronous handling of service scopes within the DefaultTenantScopeFactory class.
* Add tenant handling and move StartWorkers background task
Introduce ITenantAccessor in Worker class for multitenancy support. Rename and relocate StartWorkers service to BackgroundTask, ensuring smoother workflow initialization. Also, update the configuration to support Azure Service Bus connection string.
* Add tenant support and refactor ProtoActor client
Integrated ITenantAccessor in ProtoActorWorkflowClient class to handle multi-tenancy. Refactored methods in the client to support custom headers and added async disposable pattern in various services for proper resource management. Additionally, enabled Azure Service Bus and updated related documentation.
* Add support for custom headers in ProtoActor grain methods
Introduced a T4 template to generate grain methods with custom headers, enabling the use of tenant ID in requests. Updated `ProtoActorWorkflowClient` to employ these methods, removing redundant code and directly utilizing the client for various workflow operations.
* Add tenant middleware to MassTransit configurations
Introduced multitenancy middleware for MassTransit message handling. Added new message type `OrderReceived` and updated RabbitMQ setup in Elsa Server. Applied middleware to configure tenant data on send, publish, and consume operations.
* Add new product workflow and streamline ID handling
Introduced a new `RequestResponseWorkflow` for handling product requests. Simplified ID handling in `WorkflowBuilder` and `ClrWorkflowsProvider` by defaulting to empty strings and adding a version prefix. Enhanced `HttpWorkflowsMiddleware` to correctly parse full request paths.
* Remove redundant files and update configuration
Deleted unused files `Product.cs` and `RequestResponseWorkflow.cs` to clean up the codebase. Updated `Program.cs` configuration: switched MassTransitBroker to Memory and disabled multitenancy.
* Remove MultitenantRecurringTaskService and update AzureServiceBus
Removed `MultitenantRecurringTaskService` and adjusted related code for Azure Service Bus to work without it. This includes removal of tenant accessor dependency from `Worker` and cleanup of service configuration flags in `Program.cs`.
* Increase signal wait timeout to 10000 milliseconds.
Extended the default timeout for signal awaiting methods from 8000 to 10000 milliseconds. This change ensures more flexible and resilient waiting periods, reducing timeout occurrences in scenarios with longer processing times.
* Refactor scheduling service to be a background task
Renamed `CreateSchedulesHostedService` to `CreateSchedulesBackgroundTask` and refactored it to inherit from `BackgroundTask` instead of `BackgroundService`. Simplified the constructor by injecting the required dependencies directly, eliminating the need for a scoped factory.
* Refactor workflow version suffix formatting
Changed the version suffix format from `:v{version}` to `v{version}` and adjusted the ID concatenation accordingly. This improves consistency and readability of workflow IDs.
* Enable multitenancy support in Quartz scheduler
Added `TenantJobListener` to inject tenant context into jobs. Modified `QuartzWorkflowScheduler` to incorporate tenant IDs into job data maps and adjusted the configuration to acknowledge multitenancy settings.
* Remove ConfigureSchedulerHostedService and TenantJobListener
Consolidated tenant resolution logic into JobExecutionExtensions class. Updated ResumeWorkflowJob and RunWorkflowJob to use the new extension method for tenant retrieval. This simplifies the QuartzSchedulerFeature setup by removing the hosted service configuration.
* Refactor HTTP feature and update route table task
Move 'UpdateRouteTableStartupTask' from 'HostedServices' to 'Tasks' and update dependency injection configurations accordingly. Simplify 'DefaultRouteTableUpdater' by removing unnecessary options and tenant-agnostic settings from filters.
* Disable multitenancy in Program.cs
The useMultitenancy flag has been changed from true to false. This update affects the Elsa.Server.Web application configuration.
* Simplify variable usage in HttpWorkflowsMiddleware
Replaced 'fullPath' variable with 'path' to streamline code. This change enhances readability by reducing redundancy and ensures consistency in variable naming throughout the method.
* Enable multitenancy and refactor tenant handling logic
Enable multitenancy in the application and refactor tenant handling logic to use ITenantFinder and ITenantContextInitializer interfaces. Added header constants, updated middleware to use these interfaces, and moved extension methods to the appropriate namespace.
* Add input validation to user registration form
Implemented checks to ensure all required fields are filled and that input data adheres to format requirements. This change reduces errors and enhances form reliability.
* Remove unused import from TenantPrefixHttpEndpointRoutesProvider
This change cleans up the code by removing an unnecessary import statement. It improves code readability and reduces clutter, making future maintenance easier. The functionality remains unchanged.
* Rename filter scope to "tenantPublish" in Probe method
Updated the Probe method in TenantPublishMiddleware.cs to use "tenantPublish" instead of "tenantSend" for better clarity. Ensures consistency with the method's context and aligns with naming conventions.
* Refactor: Remove extraneous whitespace
Eliminate unnecessary whitespace in ProtoActorWorkflowClient.cs for cleaner code. This change helps maintain consistent formatting and improves readability.
* Refactor DefaultRegistriesPopulator for cleaner initialization
Converted constructor to use read-only fields directly, removing unnecessary instance variables. This change simplifies the code by reducing redundancy and making the constructor cleaner.
|
||
|
Sipke Schoorstra
|
ca78f73e0a
|
Introduce Log Persistence Strategy (#6057)
* Implement log persistence strategy management Added interfaces, services, and strategies for log persistence. Introduced new endpoint to list available log persistence strategies. Updated configurations and dependency injections accordingly. * Refactor log record methods to asynchronous Updated methods for extracting and persisting log records to be asynchronous, enhancing performance and scalability. This change includes modifying interfaces and implementations for better async support in workflow execution logging. * Remove commented code * Support nullable values in ActivityState dictionaries Update ActivityState to support nullable values by changing type to 'IDictionary<string, object?>'. Enhanced DefaultActivityExecutionMapper to handle multiple persistence strategies for logging inputs and outputs. * Rename ShouldPersistAsync to GetPersistenceModeAsync Refactor method names for log persistence strategies to improve readability and consistency. Added summary comments for clarification and removed redundant configurations from appsettings.json. Added implicit uses and updated namespaces for better maintainability. * Refactor activity payload and output retrieval logic Extract payload and output retrieval into `GetPayload` and `GetOutputs` methods respectively. This modularizes the code for better readability and maintainability, and allows for potential reusability of these methods in other parts of the codebase. * Add new project reference and update PostgreSQL provider usage Added a project reference to Elsa.Agents.Persistence.EntityFrameworkCore.PostgreSql in the test project file. Also modified the WorkflowServer setup to specify the assembly in the PostgreSQL provider configuration. * Add agent persistence to WorkflowServer Integrated agent support and persistence using PostgreSQL in WorkflowServer. This includes adding necessary project references and configuring agents in the workflow server setup. |
||
|
Sipke Schoorstra
|
cdea979cc5 |
Enable agent support and remove unused tenant configurations
Turned on agent support by setting `useAgents` to true in `Program.cs`. Also, cleaned up the `appsettings.json` file by removing configurations for tenants 'tenant-1' and 'tenant-2' which are no longer needed. |
||
|
Sipke Schoorstra
|
225ad49ea8
|
Improved multitenancy support for HTTP workflows with per-tenant DbContext (#6032)
* Refactor: Update namespaces and add TenantExtensions Updated namespaces throughout the project to improve clarity and consistency by moving from 'Common' to appropriate modules. Added TenantExtensions class to simplify fetching connection strings for tenants. * Implement multitenant DB connection strings Redesign tenant-specific classes to support multitenancy more effectively. Introduce `MultitenantBackgroundService` and `MultitenantHostedService` for handling tasks per tenant. * Refactor constructors and remove redundant code Simplified the constructor parameters for `MultitenantBackgroundService` and `List` class. Removed the unused parameter in `MultitenantBackgroundService` and redundant folder inclusion in the project file. Updated the method calls to use direct parameters in `List` class. * Remove unused import in ActivityDescriptors Endpoint The Elsa.Common.Multitenancy import was removed as it is unused in the List/Endpoint.cs file. Removing unused imports helps to improve code readability and maintainability. This change does not affect functionality. |
||
|
Sipke Schoorstra
|
7e7a899bbf
|
Implement multitenant HTTP routing (#6031)
* Add tenant awareness to bookmark handling and route resolution Added tenant ID support across various components, including bookmark updates, route resolution, and middleware processing. This ensures that bookmark and route operations can now appropriately handle tenant-specific data, improving the system's multitenancy capabilities. * Add Multitenant HTTP Routing feature to Tenants module Introduced a new MultitenantHttpRoutingFeature class to the Elsa.Tenants.AspNetCore module, enhancing the tenant resolution capabilities. Moved RoutePrefixTenantResolver from Elsa.Http to Elsa.Tenants.AspNetCore and updated relevant project references and namespaces accordingly. This refactor improves modularity and separation of concerns between HTTP and tenancy features. * Refactor route handling and tenant configuration Removed redundant `RouteTableExtensions` and replaced with new route providers and updaters, enhancing flexibility and modularity. Introduced tenant-specific HTTP endpoint configurations for better customization and configuration management. * Rename HttpEndpointBookmarkStimulus to HttpEndpointBookmarkPayload Refactor various classes and methods to reflect the renaming from `HttpEndpointBookmarkStimulus` to `HttpEndpointBookmarkPayload`. Add and configure new extension methods for tenant route handling, update the route provider to support multi-tenancy, and adjust the tenants provider to bind configuration properly. * Add HeaderTenantResolver and refactor Http namespace. Introduce HeaderTenantResolver to resolve tenants via HTTP headers. Refactor multiple classes and interfaces to move from the Elsa.Http.Models namespace directly into Elsa.Http for clarity and consistency. * Add Host-based tenant resolution Implemented a HostTenantResolver to resolve tenants based on the request's host and updated tenant configurations with host information. Modified the tenant resolver pipeline and added the new host resolver to the service registrations. * Add tenant-aware caching and accessor support Enhanced caching by incorporating tenant identifiers into cache keys for more granular cache management. Introduced ITenantAccessor dependencies in various services to retrieve the current tenant information. This ensures that cache entries are correctly isolated per tenant. * Reorder tenant resolvers for pipeline setup. Reordered the tenant resolvers in the pipeline to prioritize HostTenantResolver before RoutePrefixTenantResolver. This ensures that tenant resolution is correctly aligned with host-based resolving before checking the route prefix. * Remove unused imports This commit eliminates redundant `using` directives across multiple files to streamline the codebase. This cleanup helps improve code readability and maintainability by removing unnecessary dependencies. |
||
|
Sipke Schoorstra
|
a5cc3fc9e8
|
Refactor Tenant Resolution to Use Async Local Storage for Operation-wide Access (#6022)
* Remove obsolete tenant-related classes and add ASP.NET Core middleware Refactored tenant resolution by removing obsolete interfaces and classes, such as `IAmbientTenantAccessor` and `ITenantResolutionStrategy`. Introduced new ASP.NET Core middleware for tenant resolution, encapsulated in the new `Elsa.Tenants.AspNetCore` project. Updated related usage in various parts of the application to align with these changes. * Remove HttpContextTenantResolver. Removed HttpContextTenantResolver from the multitenancy pipeline and related service registrations. This simplifies the tenant resolution by relying on remaining resolvers like ClaimsTenantResolver and RoutePrefixTenantResolver. * Add Elsa solution definition file This commit introduces the main solution file, Elsa.slnx, defining the folder structure, projects, and configuration for the Elsa repository. This includes folders for Docker, documentation, pipelines, samples, scripts, source code, and tests. * Refactor DefaultAccessTokenIssuer for clarity and efficiency Refactored the DefaultAccessTokenIssuer class by simplifying its constructor and utilizing scoped variables for token options. Improved token creation logic by adding a dedicated method to configure token options, enhancing code readability and maintainability. * Remove Elsa.slnx solution file No dotnet build support yet. * Refactor tenant resolver service registrations Updated the service registrations to use interfaces for DefaultTenantResolver and DefaultTenantResolverPipelineInvoker. This improves the code's flexibility, making it easier to replace or extend these implementations in the future. * Add multitenancy support and tenant scope management Introduced ITenantScopeFactory and related implementations for tenant scope management across the application. Enhanced the HTTP workflows middleware to handle tenants and updated relevant configurations and extension methods to support tenant resolution. * Remove unnecessary folder inclusion The <Folder> tag for "Modules\Modules\" was redundant and has been removed to clean up the project file. This change will not affect the existing functionality or project structure. * Rename Create to CreateScope and improve authorization. Updated the method name from Create to CreateScope for better clarity in the TenantScopeFactory. Fixed a logical error in the authorization process, ensuring proper status code setting for unauthorized requests, and refactored token expiration calculation for clarity. * Add tenant agnostic filters and remove tenant setup This commit introduces tenant agnostic filters in AutoUpdateTests to ensure workflows can trigger regardless of tenant. Additionally, it removes tenant configuration from WorkflowServer setup as it is no longer required for the current tests. |
||
|
Sipke Schoorstra
|
bbedd61138
|
Implement Activity State Filtering and JavaScript Integration (#5993)
* Add secret scripting integration for JavaScript Introduced a new `Elsa.Secrets.Scripting` module that provides secret management capabilities within JavaScript workflows. This includes configuring the Jint engine to use workflow variables, adding new type and variable definition providers, and integrating with existing secret management features. * Refactor secret name extraction to a separate method Moved the logic for extracting secret names from the main method to a dedicated private method `GetSecretNamesFromExpression`. This improves code readability and maintains the single responsibility principle by delegating secret name extraction to its own method. * Add input evaluation, sensitive input handling, and middleware refactor Introduced methods for evaluating activity input properties and handling inputs marked as sensitive. Refactored `ExecutionLogMiddleware` constructor for consistency. Enhanced `SendHttpRequestBase` to mark authorization inputs as potentially containing secrets. Removed obsolete entries and adjusted persistence logic for clarity. * Refactor IActivityStateProtector interface Remove unused using directives and unnecessary comments. Simplify the definition of the `ProtectedActivityStateContext` record. * Add activity state filtering mechanism Introduce an abstract filter base class, context, and result models to enable filtering of activity state. Implement a default filter manager to run these filters and apply a specific filter for obfuscating HTTP request headers. Update necessary dependencies and extension methods to integrate the new filtering functionality. * Add expired secrets management Implemented services to manage expired secrets by periodically checking and updating their status. Introduced a new hosted service to perform the sweep and configurable options for the sweep interval. Updated related classes and configurations accordingly. * Update SweepInterval in appsettings.json Changed the Secrets Management SweepInterval from 30 seconds to 4 hours. This adjustment aims to reduce the frequency of sweep operations and improve overall system performance. * Update comment to reflect configuring engine with secrets The comment was changed to better describe the handler's function, specifying that it configures the Jint engine with secrets instead of workflow variables. This clarifies the purpose and usage of the handler in the context of the code. * Remove unused inputDescriptors variable This commit removes the inputDescriptors variable, which was declared but never used in DefaultActivityExecutionMapper.cs. This helps in cleaning up the code and potentially reducing memory usage. Ensuring that all declared variables are utilized can improve code readability and maintainability. |
||
|
Sipke Schoorstra
|
d6c14d9878
|
Simplify Workflow Variables with JS (#5946)
* Add variable support and engine configuration for JavaScript Implemented handling of workflow variables in JavaScript expressions, including new handlers, notifications, and variable definitions. Enhanced type definition services and providers to include variable definitions, updated dependency injections, and applied modifications for improved backend API configuration. * Add ObjectConverterHelper for JS object conversion Implemented ObjectConverterHelper to convert .NET objects to JavaScript objects in EvaluateJavaScript context. Updated ConfigureEngineWithVariables handler to process and convert variables using the new helper utility. * Add Customer and Order models and update Program.cs Created new Customer and Order model classes in the Models namespace. Updated Program.cs to include and alias these models for use in the application. * Add two new activities and integration test Introduced `Activity1` and `Activity2` under `src/apps/Elsa.Server.Web/Activities`. Additionally, created a new integration test `VariablesInteropTests` to validate JavaScript variable modifications within workflows. * Refactor to use IBookmarkQueue instead of IBookmarkResumer Replaced IBookmarkResumer with IBookmarkQueue in various classes for enqueueing bookmark queue items. Added logging for better traceability and included additional helper imports for activity type name generation. * Add correlationId tag to OpenTelemetry tracing This change adds a correlationId tag to the tracing for workflow executions if the context contains a correlationId. This enhancement improves traceability and correlation across distributed systems. * Set Correlation ID header in MassTransit messages Added logic to set the "X-Correlation-ID" header in MassTransit messages if the CorrelationId is present. This ensures that the messages can be correlated properly across different parts of the system. * Reduce logging verbosity in appsettings.json Removed detailed debug logs for various Elsa workflows and middleware components from the appsettings.json. This change aims to streamline the log outputs, focusing on warnings and critical information to improve readability and debug efficiency. * Add OpenTelemetry.Api package version 1.9.0 Include OpenTelemetry.Api to list of package versions in Directory.Packages.props. This addition aims to enhance application monitoring and observability. * Add JavaScript variable handling integration test Introduced integration tests for JavaScript activities to verify they can access and modify native variables. Added classes for data setup, test execution, and workflow definition with corresponding NUnit tests. * Remove unused activities and models Deleted several unused activity classes, models, and middleware to simplify the codebase. This cleanup helps reduce code complexity and improves maintainability. Updated Program.cs to reflect these deletions. * Remove correlation ID header setting from dispatch Simplified the workflow dispatching process by removing the redundant setting of the X-Correlation-ID header in two places. This change should improve code readability and maintainability. * Format code block consistently Corrected the indentation of the code block for better readability and consistency. This ensures all properties in the 'DispatchWorkflowInstance' initialization are properly aligned. No functional changes were made in this commit. * Remove VariablesInteropTests.cs from integration tests Deleted the VariablesInteropTests.cs file which contained a single test method testing JavaScript-to-JSON serialization. This cleanup removes unnecessary test code from the repository. |
||
|
Sipke Schoorstra
|
223536c90a |
Remove unused using statements and configure agents.
Removed several unused `using` statements within multiple files to clean up the codebase. Additionally, enhanced the configuration for the `Agent` module in `appsettings.json` and enabled agent-related features in `Program.cs`. |
||
|
Sipke Schoorstra
|
631036f404
|
Proto.Actor implementation for ChangeTokenSignalPublisher (#5817)
* **Refactor ProtoActor modules and integrate new core module** Removed obsolete proto actor-related files and introduced a new core module under `Elsa.ProtoActor.Core` to centralize ProtoActor functionalities. Updated services and extensions to align with the new core structure, focusing on efficient persistence and actor system configurations. * Add Proto.Actor-based distributed caching module Introduces a new module `Elsa.Caching.Distributed.ProtoActor` for Proto.Actor-based distributed caching, including configuration extensions, proto files, and required services. Refactors some existing Proto.Actor-related features and updates Dockerfile and example projects to use the new module. * Refactor ProtoActor cache handling and virtual actor setup. Reorganize the distributed caching by introducing LocalCacheVirtualActorProvider and StartLocalCacheActor. Update WorkflowInstanceVirtualActorProvider for better cluster kind handling. Adjust namespaces in Protobuf definitions for consistency. * Refactor LocalCacheImpl to use IChangeTokenSignalInvoker Replace IChangeTokenSignaler with IChangeTokenSignalInvoker to align with updated dependency contract. Adjust method call to use InvokeAsync for triggering token signals with cancellation support. * Refactor ConfigureClusterConfig and config mutation Change ConfigureClusterConfig from Action to Func for better flexibility. Update clusterConfig and remoteConfig to support reassignment from configuration methods. * Add ProtoActor support for distributed caching Introduced ProtoActor as a new distributed caching transport option. Updated the configuration and workflow runtime settings to utilize ProtoActor. Added necessary project reference for Elsa.Caching.Distributed.ProtoActor in the .csproj file. * Add LocalNodeStrategy and integrate it in actor provider Introduced `LocalNodeStrategy` to handle member placement on the current node. Integrated the new strategy in `LocalCacheVirtualActorProvider`, ensuring it uses `LocalNodeStrategy` for member management. * Prevent duplicate member additions based on ID. Updated the member checking logic to include member IDs. In addition, this change improves the robustness of the member management in `LocalNodeStrategy.cs`. * Refactor virtual actor configuration into a separate method Moved virtual actor setup logic from `ProtoActorFeature` to a new `AddVirtualActors` method to improve code readability and reusability. Updated related files to maintain consistency and enhance documentation clarity. * Refactor LocalCache to use PubSub for change token signals Replaced direct event stream usage with PubSub in LocalCache implementation. Updated service and hosted service to support PubSub subscription and publishing. Removed obsolete Start and Stop RPC methods from LocalCache service definition. * Add UsedImplicitly attribute to notification handler This change introduces the [UsedImplicitly] attribute to the DistributedWorkflowDefinitionNotificationsHandler class. The attribute is intended to prevent any accidental removal by static analysis tools, ensuring the class remains available for dynamic usage scenarios. * Refactor caching and signal handling mechanisms Replaced `TriggerChangeTokenSignalConsumer` with `ChangeTokenSignalInvoker` and added new decorators for change token handling. Renamed namespaces and file paths for better consistency and clarity. Updated test files to align with these changes. * Remove unnecessary interface dependencies from services Eliminated the ISignalManager and related interfaces to streamline dependency management. Updated services and test components to use concrete implementations directly, reducing complexity and improving maintainability. * Remove Shared.proto and associated imports Deleted the Shared.proto file and removed related import statements across multiple files. This cleanup also involved modifying the proto actor provider and project file to exclude references to Shared.proto. * Simplify namespaces in component test helpers Consolidated several namespaces into 'Elsa.Workflows.ComponentTests.Helpers' to reduce redundancy and improve maintainability. Removed unnecessary using directives in multiple test files for cleaner and more readable code. * Refactor imports in component tests Consolidated various helper imports in component tests by removing redundant specific references and utilizing general 'Elsa.Workflows.ComponentTests.Helpers'. This change simplifies the dependency management and ensures cleaner and more maintainable code. * Remove redundant state persistence calls Eliminated multiple calls to PersistStateAsync in WorkflowInstanceImpl.cs as they were unnecessary given that the WorkflowRunner already invokes the commit handler. This change simplifies the workflow execution and cancellation logic by avoiding redundant state persistence operations. * Add workflowInstanceId to response mapping Updated methods to include workflowInstanceId in response mapping functions for consistency and clarity. Additionally, fixed project reference paths and added error handling for missing workflow variables in tests. * Remove unnecessary variable existence check Removed a redundant check for the existence of the "Workflow1:variable-1" key in the variables dictionary. This streamlines the test and relies on the assumption that the key exists as expected without explicit validation. * Add Kubernetes deployment and service configurations Introduced a Deployment and Service configuration for the Kubernetes cluster. Updated Dockerfiles and build script to align with port 8080 configuration and renamed images for consistency. Updated solution file to include new deployment files. * Add Kubernetes cluster integration Introduced Kubernetes cluster provider for Proto.Actor and configured the application to use it if running in a Kubernetes environment. Added necessary RBAC roles, role bindings, and service accounts to support Kubernetes integration. Updated deployment configuration and package references to include Proto.Cluster.Kubernetes. * Refactor deployment configurations and add service support. Reorganized deployment YAML files into designated subdirectories for elsa-server, postgres, plant-uml, and trace-lens. Introduced new configuration maps, service accounts, roles, and service bindings. Updated .NET environment variables and solution structure to reflect these changes. * Update service configurations and environment variables Renamed and split services in trace-lens to isolate the OTEL collector. Updated environment variables in elsa-server to enhance instrumentation, connection strings, and profiling settings. Adjusted OTEL exporter endpoint to match the new service naming. * Increase deployment replicas to 3 Updated the 'replicas' field in the deployment configuration to enhance the system's availability and load balancing. This change ensures that three instances of 'elsa-server' will be running simultaneously. * Rename LocalCacheImpl to LocalCache and add logging Renamed `LocalCacheImpl` class to `LocalCache` to better reflect its purpose. Added a logging statement in `OnReceive` method to log incoming `ProtoTriggerChangeTokenSignal` messages. These changes improve code readability and debugging. * Disable OTEL console exporters and set session affinity Disabled console exporters for logs, metrics, and traces in the OTEL configuration to reduce unnecessary console output. Additionally, set session affinity to 'None' in the elsa-server service configuration for load balancing. * Rename WorkflowInstanceImpl to WorkflowInstance Updated the class name from WorkflowInstanceImpl to WorkflowInstance for clarity and simplicity. Adjusted all relevant references and instances in the codebase to match the new class name. * Remove ActivityIncidentStateMapper and Update ProtoBuf Mappings Removed the unused ActivityIncidentStateMapper class to streamline the codebase. Updated all related ProtoBuf mappings and imports to ensure consistency and remove redundancy across the project. * Remove duplicate actor spawn verification timeout setting The code had a redundant setting for actor spawn verification timeout, which was specified twice. This commit removes the duplicate line to ensure cleaner and more maintainable configuration. * Remove unused imports from Program.cs Eliminated unnecessary imports for ActivityExecution, WorkflowExecution, and k8s libraries. This cleanup helps reduce the code footprint and may improve compile time. * Remove debug logging from LocalCache actor The `Console.WriteLine` statement was removed from the `OnReceive` method in `LocalCache.cs`. This change eliminates unnecessary console output during the token signal handling, improving performance and reducing log clutter. |
||
|
Sipke Schoorstra
|
283824fcfd
|
Enable Proto Actor Tracing for TraceLens (#5800)
* Add database initialization script and update dependencies Added a script to initialize the 'tracelens' database and modified the Docker setup to include this script. Refactored and improved the ProtoActorFeature class, added OpenTelemetry dependencies, and updated project settings. * Enable OpenTelemetry integration for Proto.Actor Added OpenTelemetry environment configuration details to the README and included the Proto.OpenTelemetry package in the project file. Updated the ProtoActorFeature to apply tracing with OpenTelemetry to WorkflowInstanceActor. * Refactor VariablePersistenceManager to use primary constructor This refactor simplifies the VariablePersistenceManager by moving the storageDriverManager initialization into the primary constructor. It removes the redundant field and constructor, aligning with the concise nature of modern C# syntax, and ensures consistency in accessing the storageDriverManager throughout the class. * Remove unused Open Telemetry code from Program.cs The code for configuring Open Telemetry was commented out but not removed, cluttering the file. This commit cleans up Program.cs by deleting these unused lines, maintaining a cleaner and more readable codebase. * Add metrics and tracing configurations for ProtoActorFeature Introduced methods to enable metrics and tracing in ProtoActorFeature. Removed redundant properties and updated the workflow runtime to utilize the new configurations. * Add Directory.Build.props for shared project settings Introduce Directory.Build.props to centralize common project settings and dependencies. Consolidate target framework, language version, and package references to reduce duplication. Remove redundant property definitions from Elsa.Server.Web.csproj. * Move apps from bundles to apps folder and Elsa module to modules folder |
Renamed from src/bundles/Elsa.Server.Web/appsettings.json (Browse further)