Commit graph

63 commits

Author SHA1 Message Date
Sipke Schoorstra 21e982c2c6
Add shell reload API endpoints and client support (#7353)
* feat: add specification and quality checklist for Shell Reload API endpoints

* feat: implement Shell Reload API endpoints and associated documentation

* feat: enhance Shell Reload API documentation and add tasks for implementation phases

* feat: implement Shell Reload API features with endpoints, interface contracts, models, and component tests

* feat: update Shell Reload API responses and tests to reflect changes in error handling and response structure

* Fix shell reload follow-up review issues (#7354)

* Initial plan

* Address shell reload review feedback

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Dispose shell reload semaphore

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Harden shell reload follow-up fixes

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Api/Endpoints/Shells/Reload/Endpoint.cs

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'Missed opportunity to use Where'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Api/Endpoints/Shells/Reload/Endpoint.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-09 08:37:11 +01:00
Sipke Schoorstra 081e4ccac0
Track workflow execution in AppComponentTest to prevent premature disposal. 2026-02-21 15:57:46 +01:00
Sipke Schoorstra a8a3fbf552
Introduce TestBookmarkQueueWorker to eliminate throttling in component tests and ensure timely completion of workflows. Enhance disposal logic to prevent TaskCanceledException by waiting for workflows to complete. 2026-02-03 10:51:25 +01:00
Sipke Schoorstra 7bc9035f5e
Implement null TenantId for tenant-agnostic entities (#7226)
* Add ADR for Null Tenant ID, implement tenant-agnostic logic

Introduce ADR-0009 to document the use of `null` for tenant-agnostic entities, enhancing multitenancy handling. Update multitenancy features across the codebase, including EF Core query filters and ActivityRegistry, to handle null as a tenant ID, ensuring tenant-agnostic entities are accessible across all tenants.

* Add multitenancy support in `ActivityTestFixture` by registering `ITenantAccessor`.

* Update src/modules/Elsa.Common/Multitenancy/Implementations/DefaultTenantAccessor.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove unused `using Elsa.Common.Multitenancy;` from WorkflowDefinitionActivityDescriptorFactory (#7230)

* Initial plan

* Remove unused using Elsa.Common.Multitenancy statement

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Optimize ActivityRegistry.Find to prefer tenant-specific descriptors without performance regression (#7227)

* Initial plan

* Optimize Find(string type) to prefer tenant-specific descriptors with single-pass iteration

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Apply review feedback: combine if statements and add comprehensive unit tests

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Refactor tests for DRYness using theories and helper methods

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Clean up extra whitespace in test file

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Fix default tenant data visibility leak by removing NullIfEmpty conversion (#7229)

* Initial plan

* Remove NullIfEmpty conversion to align with ADR-0008 and ADR-0009

- Updated ElsaDbContextBase to use empty string for default tenant
- Updated ApplyTenantId to stop converting empty string to null
- Updated TenantAwareDbContextFactory to preserve empty string for default tenant
- Updated Store.cs to preserve empty string for default tenant
- This ensures: null = tenant-agnostic (visible to all), "" = default tenant

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add database migration to convert null TenantId to empty string for SqlServer

- Added Management migration to convert null to "" for WorkflowDefinitions and WorkflowInstances
- Added Runtime migration to convert null to "" for all runtime entities
- This ensures existing default tenant data is properly migrated per ADR-0008
- Note: Similar migrations needed for PostgreSql, MySql, Sqlite, and Oracle providers

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Clarify tenant handling logic in `ElsaDbContextBase` with new ADR references.

* Add tenant ID convention analysis documents and quick reference guide

* Implement tenant-agnostic functionality across modules

- Introduce `AgnosticTenantId` constant to manage tenant-agnostic entities.
- Modify entity handling logic to respect tenant-agnostic designations.
- Adjust workflow processing to include tenant-agnostic workflows.
- Update caching and activity descriptor logic to accommodate the `AgnosticTenantId`.

* Refactor tenant management and registry logic in `ActivityRegistry` for improved clarity and separation of tenant-specific and tenant-agnostic activity descriptors. Remove `TestTenantResolver` and update workflow definition handling for tenant support.

* Refactor `ActivityRegistry`: prioritize tenant-specific descriptors over tenant-agnostic and simplify descriptor retrieval logic.

* Improve async handling in `CommandHandlerInvokerMiddleware` to await tasks without blocking

* Update ADR to use asterisk as sentinel value for tenant-agnostic entities

Replace the previous convention of using `null` for tenant-agnostic entities with an asterisk (`"*"`) for improved clarity and system architecture. Updated ADR documentation, TOC, and dependency graph accordingly.

* Remove migration `ConvertNullTenantIdToEmptyString` and its associated designer file to clean up the codebase.

* Refactor `ActivityRegistry`: streamline activity descriptor removal logic and simplify tenant ID checks.

* Update Elsa.sln

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Simplify `RefreshDescriptorsAsync` by removing unnecessary local variable `currentTenantId`.

* Remove unused `currentTenantId` variable from `ActivityRegistry`.

* Add detailed semantic flow and key points to ADR 0009

Document the tenant ID flow from entity creation to query, emphasizing normalization and tenant-agnostic workflows. Update semantic flow diagrams and provide testing considerations for preserving `"*"` values in multi-tenant scenarios.

* Remove outdated Tenant ID Analysis and associated documents

* Add security-by-default design for tenant-agnostic entities in ADR

Enhance Architecture Decision Record to detail explicit requirements for tenant-agnostic database entities, highlighting differences between in-memory activity descriptors and persistent entities. Emphasize importance of setting `TenantId = "*"` to prevent accidental data leakage.

* Normalize tenant ID grouping in `ActivityRegistry` to unify null and agnostic IDs, reducing redundant processing.

* Refactor `SignalManager`: improve timeout handling and streamline signal task cancellation.

* Update src/modules/Elsa.Workflows.Core/Models/TenantRegistryData.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Core/Services/ActivityRegistry.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor tests to use `Tenant.AgnosticTenantId` instead of `null` for tenant-agnostic descriptors.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Enhance logging in recurring tasks: add error handling and logger support to prevent crashes in scheduled timers.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
2026-02-02 10:59:01 +01:00
Sipke Schoorstra b09a564812
Fix Multitenancy Support and Normalize Tenant ID Handling (#7217)
* Enable multitenancy support and normalize tenant ID handling.

- Activate multitenancy in `Program.cs`.
- Introduce `NormalizeTenantId` method for consistent tenant ID usage.
- Update tenant-related classes and features to support normalization logic.

* Add ADR for adopting empty string as the default tenant ID

- Standardized the tenant ID for the default tenant to use an empty string (`""`) instead of `null`.
- Documented the rationale and migration considerations in ADR 0007.
- Updated ADR table of contents and graph for new entry.

* Apply suggestion from @sfmskywalker

* Update doc/adr/graph.dot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Normalize spacing and improve readability in `Program.cs`. Fix multitenancy condition formatting.

* Fix ADR numbering and update TOC

* Add ADRs for flowchart execution model, tenant deletion event, merge modes, and default tenant ID

- Introduced ADR 0005: Token-centric flowchart execution model for improved loop and join handling.
- Added ADR 0006: Tenant Deleted event for distinct handling of tenant removal.
- Documented ADR 0007: Explicit merge modes for flowchart joins, improving reliability and configurability.
- Included ADR 0008: Standardization of empty string as the default tenant ID for consistency and clarity.

* Add unit tests for tenant ID normalization and multitenancy pipeline invoker

- Added comprehensive unit tests for tenant ID normalization to ensure consistent handling of null, empty, and valid IDs.
- Introduced tests for the multitenancy pipeline invoker covering various tenant resolution scenarios.
- Updated solution to include new unit testing projects for `Elsa.Tenants` and `Elsa.Common`.

* Update unit tests for `ActivityConstructionResult`

- Refactor test parameterization to verify `HasExceptions` property more explicitly.
- Simplify exception creation logic in helper methods.
- Improve test assertions by combining act and assert phases where applicable.

* Enable configuration-based multitenancy with tenant-specific settings

- Introduced a configuration-based tenant provider to streamline tenant initialization and customization.
- Added tenant ID handling filters to ensure tenant ID is applied and filtered automatically.
- Deprecated the `CommonPersistenceFeature` in favor of modular persistence feature extension.

* Update database indexes to include `TenantId` for multitenancy support

- Added `TenantId` to unique constraints on `Triggers` table across all EFCore providers.
- Adjusted index names to reflect the updated constraints.
- Updated trigger configuration to ensure uniqueness includes `TenantId`.

* Add tenant filtering to `DefaultWorkflowDefinitionStorePopulator`

- Introduced `ITenantAccessor` to support tenant-specific filtering of workflow definitions.
- Updated logic to skip workflows not matching the current tenant.

* Update doc/adr/toc.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove `CommonPersistenceFeature` as it has been deprecated

* Add tenant-specific filtering to workflow import logic in `DefaultWorkflowDefinitionStorePopulator`

* Replace hardcoded tenant ID with `Tenant.DefaultTenantId` in integration tests

* Update database indexes and migration logic to support `TenantId` for multitenancy

- Added `TenantId` to unique constraints on the `Triggers` table and updated index names.
- Included logic to drop outdated indexes without `TenantId` during migration.
- Adjusted tests to account for `TenantId` in workflow identity and indexing scenarios.

* Remove `TenantId` from workflow identity construction in concurrent trigger indexing tests

* Introduce `SelectiveMockLockProvider` for precise lock mocking in tests

- Added `SelectiveMockLockProvider` to allow targeted lock mocking without affecting unrelated background operations.
- Updated test services to use `SelectiveMockLockProvider` in place of `TestDistributedLockProvider`.
- Refactored `DistributedLockResilienceTests` to support selective mocking for deterministic and reliable assertions.

* Update Elsa.sln

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` for consistent filtering

* Refactor `TenantResolverResult` to support explicit resolved/unresolved state handling

- Updated `TenantResolverResult` to include an explicit `_isResolved` property.
- Adjusted `ResolveTenantId()` and `IsResolved` logic for improved clarity and robustness.
- Simplified tenant resolution invocation in `TenantResolverBase`.
- Removed redundant normalization in `DefaultTenantResolverPipelineInvoker`.

* Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` and `ClrWorkflowsProvider`.

* Refactor `DefaultWorkflowDefinitionStorePopulatorTests`: streamline object initializations and add tenant-specific test coverage for `PopulateStoreAsync`.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-30 19:54:13 +01:00
Sipke Schoorstra 8f6ce7eb8c
Merge remote-tracking branch 'origin/main' into release/3.6.0 2025-12-29 19:57:50 +01:00
Sipke Schoorstra ca268c16ad
Enhances distributed lock handling with resilience (#7161)
* Add retry mechanism for distributed locks with transient error handling and logging

- Introduced Polly-based retry pipeline for distributed lock acquisition in `DistributedWorkflowClient` to handle transient errors such as network issues or database connection failures.
- Added detailed logging for retry attempts and lock release errors.
- Updated project dependencies to include Polly.

* Refactor transient exception handling to shared resilience module.

Migrated transient exception detection logic from scheduling module to a new shared resilience module. Updated services, jobs, and features to utilize the centralized `ITransientExceptionDetectionService`. This change improves maintainability and promotes reusability across modules.

* Add unit tests for transient exception detection and resilience strategy evaluation.

- Introduced comprehensive unit tests for `DefaultTransientExceptionDetector`, `ResilienceStrategyCatalog`, `ResilienceStrategyConfigEvaluator`, and `TransientExceptionDetectionService`.
- Added helper classes and test data factories to facilitate reusable test patterns for resilience modules.
- Updated solution to include `Elsa.Resilience.Core.UnitTests` project.

* Add component tests for distributed lock resilience

- Introduced new tests to verify retry behavior during transient lock acquisition and release failures.
- Added `TestDistributedLockProvider` and related mocks for simulating transient failures.
- Updated `WorkflowServer` test services to support the new distributed lock test scenarios.

* Refactor distributed lock resilience tests

- Consolidated test logic: streamlined test providers, injected services, and reusable test patterns.
- Simplified `TestDistributedLockProvider` implementation with enhanced initialization and failure simulation.
- Reorganized tests for transient acquisition/release failures to use parameterized `Theory` for improved maintainability.

* Refactor transient exception handling: rename interfaces and classes for consistency, update references across codebase, and improve code readability.

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Simplify WorkflowServer setup and DistributedLockResilienceTests by replacing IDistributedLockProvider with TestDistributedLockProvider.

* Remove unused `using` directives in unit tests to improve code cleanliness.

* Remove `TransientExceptionTypes` helper and inline its usage in tests for improved maintainability.

* Add descriptive `DisplayName` attributes to unit tests for improved test clarity.

* Fix redundant exception checking in TransientExceptionDetector (#7162)

* Initial plan

* Fix redundant exception checking in TransientExceptionDetector

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Extract MaxRetryAttempts constant in DistributedLockResilienceTests (#7164)

* Initial plan

* Extract MaxRetryAttempts constant to eliminate hardcoded magic number

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Make TestDistributedLockProvider thread-safe with Interlocked operations (#7163)

* Initial plan

* Make TestDistributedLockProvider thread-safe using Interlocked operations

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/DistributedLockResilience/DistributedLockResilienceTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Resilience.Core/Services/DefaultTransientExceptionStrategy.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Include `CancellationToken` in distributed lock handling methods for improved cancellation support.

* Initial plan (#7168)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Cache detector list in TransientExceptionDetector to avoid repeated allocations (#7167)

* Initial plan

* Cache detector list in field to avoid repeated allocations

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Use IReadOnlyList instead of List for better intent expression

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Fix TestDistributedLockProvider registration to properly decorate IDistributedLockProvider (#7166)

* Initial plan

* Fix TestDistributedLockProvider registration to use Decorate pattern and fix variable reference bug

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add runtime check for TestDistributedLockProvider registration

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Refactor `DistributedWorkflowClient` to simplify `Lazy<ResiliencePipeline>` initialization.

* Add integration tests for DistributedWorkflowClient lock resilience (#7165)

* Initial plan

* Fix compilation error: use correct parameter name transientExceptionDetector

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add integration tests for DistributedWorkflowClient lock resilience

- Add SimpleWorkflow for testing distributed lock scenarios
- Add tests exercising RunInstanceAsync with transient lock failures
- Verify retry logic works correctly with actual workflow execution
- Test both acquisition and release failure scenarios
- Decorate IDistributedLockProvider to use TestDistributedLockProvider

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Address code review feedback

- Add explanatory comment for TestDistributedLockProvider cast
- Remove unnecessary blank line for consistent formatting

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* Simplify transient exception strategy by refactoring message pattern matching logic.

* Enhance distributed lock mock to support per-lock failure configuration and improve resilience tests.

* Refactor `TestDistributedLockProvider` to streamline failure handling logic and improve code clarity.

* Remove unused methods and redundant test case from `DistributedLockResilienceTests`.

* Refactor `DistributedLockResilienceTests` to simplify workflow client creation, consolidate assertion logic, and remove redundant test cases.

* Format `ResilienceStrategyCatalogTests` by removing redundant line breaks in test setup.

* Refactor `TransientExceptionDetectorTests` to simplify test setup, consolidate test cases, and remove redundant logic.

* Handle `InvalidOperationException` in `XunitLogger` to suppress logging errors during inactive tests.

* Update workflows to use .NET 10 and adjust resilience tests project configuration.

* Refactor distributed runtime feature and integrations to improve resilience handling, configure services fluently, and add cancellation safeguards in background services.

* Update `base_version` to `3.7.0` in GitHub workflow configuration.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
2025-12-29 19:53:48 +01:00
Sipke Schoorstra fa04e1ebcd
Adds activity host registration support (#7172)
* Add support for activity host registration across workflows

Introduced new APIs and updates to enable registering custom activity hosts in the workflow management system. This includes modifications to attributes, service registrations, and extensions to streamline integration for advanced activity hosting scenarios.

* Remove unused `using` directives across Workflow Management module

* Add support for host method activity registration and description

Introduce new APIs to enable activity registration from public async methods (Task/Task<T>) on CLR types. Includes `HostMethodActivitiesOptions`, `HostMethodActivity`, `HostMethodActivityProvider`, and `HostMethodActivityDescriber` for dynamic activity generation and execution.

* Refactor host method activity execution and cleanup.

Reworked `HostMethodActivity` to support resumable workflows, improved parameter handling with pluggable value providers, and removed obsolete `AgentExecutionContext`. Enhanced method resolution, async handling, and input/output descriptor logic for better flexibility and maintainability.

* Refactor `Bookmark` model to use mutable properties and update XML documentation.

* Refactor `BookmarkExecutionContextExtensions` to improve structure, add `GenerateBookmarkTriggerToken` method, and enhance maintainability.

* Add extensibility for host method parameter binding with pluggable value providers

Introduced `IHostMethodParameterValueProvider` interface for custom parameter resolution, along with `DefaultHostMethodParameterValueProvider`, `DelegateHostMethodParameterValueProvider`, and `FromServicesAttribute` for flexible binding options. Enhances host method activity execution by supporting DI resolution and workflow input handling.

* Refactor nullable usage and improve bookmark management logic

Updated null assignment for consistency across files and refined logic for detecting and handling newly added bookmarks. Adjusted method signatures and parameters in the DecoratedStoryWriterAgent class for more explicit input handling. These changes enhance code readability, maintainability, and robustness.

* Update src/modules/Elsa.Workflows.Management/Activities/CodeFirst/HostMethodActivityProvider.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Ensure `CallbackMethodName` is set and skip bookmarks with empty values

* Update src/modules/Elsa.Workflows.Management/Features/WorkflowManagementFeature.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Management/Contracts/IHostMethodActivityDescriber.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Core/Attributes/InputAttribute.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor `CodeFirst` namespace to `HostMethod` for improved clarity and align with updated activity execution logic. Enhance DI-based parameter resolution and update XML documentation for `HostMethodActivitiesOptions`.

* Add `Penguin` activity host with sample activity methods and register in Elsa pipeline

* Add `TestHostMethod` activities and corresponding component tests. Register `TestHostMethod` as an activity host in the workflow server.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-27 20:52:52 +01:00
Sipke Schoorstra dd6bfbb6b0
Comment out SQL Server image configuration in component test infrastructure. 2025-12-11 10:05:39 +01:00
Sipke Schoorstra 14189494b2
Switch component tests database configuration from PostgreSQL to SQL Server. 2025-12-11 09:58:45 +01:00
Sipke Schoorstra 183934c1af
Revert component tests database configuration from SQL Server to PostgreSQL. 2025-12-11 09:54:16 +01:00
Sipke Schoorstra 1107b48200
Switch component tests from PostgreSQL to SQL Server and update related configurations. 2025-12-11 09:45:37 +01:00
Sipke Schoorstra c79275fcca
Restore Scope.Dispose in AppComponentTest to ensure proper resource cleanup during test execution. 2025-12-10 11:25:29 +01:00
Sipke Schoorstra 8730f0cdeb
Restore Scope.Dispose in AppComponentTest and update WorkflowServer DB connection string with pool size configuration 2025-12-10 11:25:17 +01:00
Sipke Schoorstra da88712e75
Increase PostgreSQL test container max connections to 100 for improved parallelism in component tests 2025-12-10 11:10:45 +01:00
Sipke Schoorstra bcb613cf86
Remove redundant database configuration and simplify connection string in WorkflowServer. 2025-12-10 11:01:52 +01:00
Sipke Schoorstra 56089c95f1
Update workflow to dump Docker logs on failure and improve PostgreSQL container configuration
- Add Docker logs capturing step in GitHub Actions workflow for better debugging.
- Update PostgreSQL test container to use `postgres:16-alpine` with enhanced configuration options.
- Set database connection `Max Pool Size` to 20 in component tests.
2025-12-10 10:55:52 +01:00
Sipke Schoorstra 2c0b3da5de
Addresses warnings and enforces null safety (#7051)
* Enhance null-safety annotations across modules and refactor for improved consistency:

- Added `null!` annotations to enforce non-nullability expectations.
- Updated workflows, tests, and runtime services to handle default null values reliably.
- Removed obsolete and unused APIs, simplifying interfaces and improving maintainability.
- Refactored methods and properties for clarity, thread-safety, and consistency.
- Adjusted test configurations for code coverage tracking and integration improvements.

* Refactor activity iteration in container serialization tests to simplify type casting.
2025-11-12 16:56:31 +01:00
Sipke Schoorstra a3278ccf97
Add BulkDispatchWorkflows tests and improve activity coverage (#7026)
* Refactor BulkDispatchWorkflows tests and improve activity coverage

- Migrate `BulkDispatchWorkflows` tests to a new structure under `Scenarios/BulkDispatch`.
- Introduce comprehensive test cases for bulk dispatch scenarios, including empty items, correlation IDs, child ports, dictionary items, and invalid definitions.
- Add multiple reusable workflows such as `BulkChildWorkflow`, `BulkDispatchAndWaitWorkflow`, and `BulkDispatchFireAndForgetWorkflow` for enhanced test coverage.
- Update namespaces and project files to reflect the restructuring and new workflow definitions.
- Enhance functionality for `IWorkflowInstanceVariableManager` by adding methods for variable retrieval from workflow states and instances.

* Add additional assertions for WriteLine activity tests and update workflow builder in BulkDispatch tests

- Enhanced `BulkDispatchWorkflowsTests` by validating written texts for accuracy.
- Refactored `BulkDispatchFireAndForgetWorkflow` to update activity and namespace references.

* Add `SlowBulkChildWorkflow` and update `BulkDispatchFireAndForgetWorkflow` test structure

- Introduced `SlowBulkChildWorkflow` with a delay and logging functionality for bulk dispatch testing.
- Modified `BulkDispatchFireAndForgetWorkflow` to use `SlowBulkChildWorkflow` and updated test items.
- Enhanced `BulkDispatchWorkflowsTests` with child workflow completion validation and timing assertions.

* Refactor BulkDispatch tests by removing dictionary item workflow and streamlining child workflow handling

- Removed `BulkDispatchWithDictionaryItemsWorkflow` and associated test cases.
- Updated `BulkChildWorkflow` to use input binding.
- Streamlined child workflow event handling in `BulkDispatchWorkflowsTests` using a reusable helper method.
- Enhanced `CorrelationIdFunction` support with JavaScript expression handling.

* Move unit test workflows to `Scenarios/Activities` and update namespace references

- Relocated all test workflows from `Scenarios/BulkDispatch` and `Scenarios/ExecuteWorkflows` to `Scenarios/Activities`.
- Updated namespaces and imports to reflect the new folder structure.

* Refactor BulkDispatch and FlowJoin test workflows:

- Moved `BulkDispatch` workflows to `Scenarios/Activities/BulkDispatchWorkflows`.
- Renamed `FlowJoins` to `FlowJoin` and updated namespaces to match.
- Adjusted imports and references to align with new folder structure and updated class definitions.

* Remove unused folder reference from `Elsa.Workflows.ComponentTests.csproj`.

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/Activities/BulkDispatchWorkflows/BulkDispatchWorkflowsTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-04 15:44:44 +01:00
Sipke Schoorstra ff1f0833ad
Restore EF Core persistence layer (#6931)
* Restore EF Core persistence layer

Originally, we had moved EF Core to the extensions repo, but in practice this turned out to be rather impractical when debugging elsa-core, given that we would lose all of the state (workflow definitions, instances, etc.) across application restarts.

* Integrate PostgreSQL support into EF Core persistence and enhance BlobStorage workflows provider setup.
2025-09-27 20:39:18 +02:00
Sipke Schoorstra 918caaa09a
Removed unused ForEach activity namespace from WorkflowServer. 2025-09-25 21:01:28 +02:00
Sipke Schoorstra f24b4394cf
Add WorkflowStateCommitted notification support and update state handling logic
- Introduced `WorkflowStateCommitted` notification to encapsulate workflow execution context, state, and instance details.
- Updated `DefaultCommitStateHandler` to publish `WorkflowStateCommitted` via `IMediator`.
- Adjusted `DispatchWorkflowExtensions` to use `WorkflowStateCommitted` for workflow completion.

Updates KubernetesClient and Microsoft packages (#6917)

* Remove Proto.Cluster.Kubernetes dependency due to vulnerability

- Temporarily removed `Proto.Cluster.Kubernetes` package and provider integration because of a vulnerability in its dependency (https://avd.aquasec.com/nvd/2025/cve-2025-9708).
- Adjusted related cluster provider and remote configuration logic.
- Updated `PortAttribute` default parameter for clarity.

* Revert "Remove Proto.Cluster.Kubernetes dependency due to vulnerability"

This reverts commit 0720d970968e4f7338825407258b34ddffb1d2a4.

* Add KubernetesClient package and update MicrosoftVersion to 9.0.9

- Added `KubernetesClient` package to the project dependencies.
- Updated `MicrosoftVersion` to `9.0.9` in `Directory.Packages.props`.
Update Polly packages

- Bump Polly and Polly.Extensions package versions to 8.6.3.

Update `Microsoft.AspNetCore.Authorization` to use `MicrosoftVersion` property

Ensure Docker images ship CA trust and add TLS smoke tests (#6918)

Remove TlsSmoke project and related solution references

- Deleted `TlsSmoke` project files (`Program.cs` and `TlsSmoke.csproj`).
- Removed `TlsSmoke` project reference from the solution file (`Elsa.sln`).

Add comprehensive Copilot coding agent instructions for repository onboarding (#6920)

* Initial plan

* Add comprehensive .github/copilot-instructions.md with validated build instructions

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Add ForEach tests, introduce asynchronous workflow runner and enhance workflow events. (#6926)

* Introduce asynchronous workflow runner and enhance workflow events.

- Added `AsyncWorkflowRunner` to enable asynchronous workflow execution and result tracking.
- Introduced new event arguments, such as `ActivityExecutedEventArgs` and `WorkflowStateCommittedEventArgs`.
- Expanded `WorkflowEvents` class to include `ActivityExecuted`, `ActivityExecutedLogUpdated`, and `WorkflowStateCommitted` events.
- Refactored event arguments into the `Elsa.Testing.Shared.EventArgs` namespace.
- Enhanced tests with `AsyncWorkflowRunner` and new event-driven workflow scenarios.

* Refactor event argument classes to unify namespace and simplify inheritance

* Add shared component DotSettings file to support namespace exclusions
Refactor `WaitAsync` call in `DispatchWorkflowsTests` to remove unnecessary generic type.
2025-09-25 20:58:21 +02:00
Sipke Schoorstra 72ad243ed4
Merge 3.5.1 into 3.6.0 (#6907)
* Introduce `IWorkflowResumer` and `ActivityInputEvaluatorContext`, refactor endpoint handling, extend logging, and improve bookmark queue processing.

* Remove deprecated WorkflowContexts module and optimize project.

Deleted the Elsa.Studio.WorkflowContexts module and references from solution files. Corrected minor errors in remaining code and updated project configurations to align with the new structure.```

* Update GitHub workflows to track `develop/3.6.0` branch instead of `patch/3.5.1`.

* Fix inconsistent formatting in `InputDescriptor` constructor and properties.

* Add XML documentation for `DictionaryValueEvaluator` in `UIHints/Dictionary` module

* Refactor `DictionaryValueEvaluator` to improve readability and simplify dictionary evaluation logic.
2025-09-13 13:05:52 +02:00
Sipke Schoorstra 59bfcf0ba5
Add distributed workflow runtime implementation.
Introduced `DistributedWorkflowRuntime` to support distributed workflow execution with locking mechanisms. Added new module `Elsa.Workflows.Runtime.Distributed` with key services, features, and client implementations for handling distributed bookmarks and workflow clients. Updated integration and component tests to use the new distributed runtime where relevant.
2025-06-03 09:58:20 +02:00
Sipke Schoorstra 8fb797eaeb
Remove Elsa.Expressions.Dsl module and related DSL features
This commit completely removes the Elsa.Expressions.Dsl module, including all associated features, services, models, and dependencies. The DSL functionality, including its custom activities and integration options, has been fully eliminated from the project.
2025-06-02 21:52:10 +02:00
Sipke Schoorstra 1f2353cad8
Remove obsolete modules and associated code
Eliminated the `Elsa.Labels`, `Elsa.Environments`, and `Elsa.OpenTelemetry` modules along with their handlers, contracts, models, and related functionality. This cleanup improves maintainability and aligns the codebase with recent architectural changes.
2025-06-02 20:05:10 +02:00
Matt 6b88698200 Merge 'Main' in and resolves conflicts. 2025-05-22 00:46:15 +01:00
Matt 10348c5ea3 Update all dependencies to new projects. 2025-05-21 00:20:13 +01:00
Sipke Schoorstra 787b3ed8eb
Refactor default field initializations to use null instead of default!.
This update ensures consistency and improves clarity in specifying nullable types and default values. Replaced `default!` with `null` in various activity classes and properties to better align with modern C# nullable reference type practices.
2025-05-20 20:26:57 +02:00
Sipke Schoorstra 13d7e91e0d
Remove agent-related configurations and dependencies.
This commit removes the "Agents" configuration section, related API keys, services, and persistence logic across the codebase. Unused agent-related NuGet packages and code references were also eliminated to simplify the project and focus on core functionality.
2025-03-02 16:08:51 +01:00
Sipke Schoorstra cb6a7dfc11 Update RabbitMQ image to version 4-management
Upgraded RabbitMQ from version 3-management to 4-management across tests and the Docker setup. This ensures compatibility with the latest features and improvements in RabbitMQ while maintaining consistency across environments.
2024-12-14 11:44:52 +01:00
Sipke Schoorstra 4793c6eac5 Refactor ProtoActor and workflow testing logic
Reorganized ProtoActor configuration to be conditionally applied based on runtime settings. Updated test workflows to simplify signal handling and improved test structure by introducing scoped dependencies and removing unused event subscriptions.
2024-12-14 09:59:29 +01:00
Sipke Schoorstra 1ed6695872
Add ExecuteWorkflow activity (#6134)
* Add ExecuteWorkflow activity and enable multitenancy

Introduced the ExecuteWorkflow activity to create and execute workflow instances, and updated runtime settings to use ProtoActor for distributed caching transport. Enabled multitenancy for the Elsa Server Web application.

* Namespace renames (#6135)

* Create ExecuteWorkflows tests and workflows

Add unit tests for executing workflows, including MainWorkflow and SubroutineWorkflow. Also, rename ExecutedWorkflowResult to ExecuteWorkflowResult in the runtime module for consistency.

* Add output definitions to WorkflowBuilder

Refactored the `WithInput` method for clarity and added multiple `WithOutput` methods to support different ways of defining workflow outputs. These changes enhance the flexibility and readability of the workflow configuration process by providing a consistent API for input and output definitions.

* Refactor namespaces in component tests

Updated namespaces from Helpers to specific contexts like Fixtures, Abstractions, Consumers, Decorators, etc., to improve code organization and readability. This change affects multiple files across different test scenarios and modules.

* Disable RabbitMQ and multitenancy, rename test class

Commented out RabbitMQ usage in WorkflowServer.cs to focus on other transports. Changed multitenancy flag to false in Program.cs. Renamed DispatchWorkflowsTests to ExecuteWorkflowsTests for clarity.
2024-11-21 16:44:25 +01:00
AlmightyLks 153397e63c
Fix all broken summary xml docs (#6123)
* Add proper summary tags back

* Apply suggestions from code review

Co-authored-by: Robin Sue <robinsue@live.de>

* 69 more files fixed

---------

Co-authored-by: Robin Sue <robinsue@live.de>
2024-11-18 13:41:27 +01:00
Sipke Schoorstra 9ecbb3f360 Comment out scope disposal and update server configurations
Commented out Scope disposal to prevent test runner hangs in component tests. Adjusted Cluster class to use property initializers instead of constructor. Updated workflow server connection strings and added detailed SQL Server configuration in Program.cs.
2024-11-01 18:02:47 +01:00
Sipke Schoorstra c49a92b5b7 Refactor signals and activity cancellation logic
Changed static signals to instance variables in test classes for better encapsulation. Increased default timeout in `SignalManager` to 60 seconds. Removed unused cancellation token logic in `ActivityExecutionContext`.
2024-10-30 19:04:48 +01:00
Sipke Schoorstra ca78f73e0a
Introduce Log Persistence Strategy (#6057)
* Implement log persistence strategy management

Added interfaces, services, and strategies for log persistence. Introduced new endpoint to list available log persistence strategies. Updated configurations and dependency injections accordingly.

* Refactor log record methods to asynchronous

Updated methods for extracting and persisting log records to be asynchronous, enhancing performance and scalability. This change includes modifying interfaces and implementations for better async support in workflow execution logging.

* Remove commented code

* Support nullable values in ActivityState dictionaries

Update ActivityState to support nullable values by changing type to 'IDictionary<string, object?>'. Enhanced DefaultActivityExecutionMapper to handle multiple persistence strategies for logging inputs and outputs.

* Rename ShouldPersistAsync to GetPersistenceModeAsync

Refactor method names for log persistence strategies to improve readability and consistency. Added summary comments for clarification and removed redundant configurations from appsettings.json. Added implicit uses and updated namespaces for better maintainability.

* Refactor activity payload and output retrieval logic

Extract payload and output retrieval into `GetPayload` and `GetOutputs` methods respectively. This modularizes the code for better readability and maintainability, and allows for potential reusability of these methods in other parts of the codebase.

* Add new project reference and update PostgreSQL provider usage

Added a project reference to Elsa.Agents.Persistence.EntityFrameworkCore.PostgreSql in the test project file. Also modified the WorkflowServer setup to specify the assembly in the PostgreSQL provider configuration.

* Add agent persistence to WorkflowServer

Integrated agent support and persistence using PostgreSQL in WorkflowServer. This includes adding necessary project references and configuring agents in the workflow server setup.
2024-10-25 19:41:10 +02:00
Sipke Schoorstra 32bd3ddfea Integrate improved workflow handling in bulk dispatch tests
Updated workflow creation and dispatching methods to streamline bulk dispatch tests. Modified signal triggering in `TriggerSignal` activity and upgraded `ElsaStudioVersion` for compatibility improvements. Removed unused imports for cleaner code structure.
2024-10-23 19:12:10 +02:00
Sipke Schoorstra 3208be7b89 Merge remote-tracking branch 'origin/blueberry' 2024-10-23 16:07:32 +02:00
Sipke Schoorstra 52516aa0ff
Fix Bulk Dispatch Missing Input (#6052)
* Remove outdated sample workflows and activities

Deleted CompositeExample, SampleWorkflow, and SlowActivity classes as they are no longer needed. These deletions help clean up the codebase and maintain relevance in the code structure.

* Handle null WorkflowBuilderType in ClrWorkflowMaterializer

Introduce a fallback NotFoundWorkflowbuilder class when WorkflowBuilderType is null. This ensures the MaterializeAsync method functions even if the provided context lacks a specific workflow builder type.

* Add merging of itemDictionary to input

This change ensures that the itemDictionary's contents are also merged into the input dictionary, preventing possible data loss. It builds on existing logic by adding an additional merge operation to incorporate all necessary data.

* Add new workflows to BulkDispatchWorkflows and modify tests

Introduce `FruitWorkflow` and `MixFruitsWorkflow` for bulk dispatch scenarios. Refactor `BulkDispatchWorkflowsTests` to include new tests and rename signals for clarity. Optimize `BulkDispatchWorkflows` activity by removing redundant dictionary merge.
2024-10-23 16:06:28 +02:00
Sipke Schoorstra 0c10113d7b Remove outdated database migration files
Deleted obsolete migration files from SQL Server, MySQL, and SQLite projects. This cleanup removes unused schema definitions and indexes, ensuring the repository remains up to date and free of unnecessary files.
2024-10-23 14:32:50 +02:00
Sipke Schoorstra 7e7a899bbf
Implement multitenant HTTP routing (#6031)
* Add tenant awareness to bookmark handling and route resolution

Added tenant ID support across various components, including bookmark updates, route resolution, and middleware processing. This ensures that bookmark and route operations can now appropriately handle tenant-specific data, improving the system's multitenancy capabilities.

* Add Multitenant HTTP Routing feature to Tenants module

Introduced a new MultitenantHttpRoutingFeature class to the Elsa.Tenants.AspNetCore module, enhancing the tenant resolution capabilities. Moved RoutePrefixTenantResolver from Elsa.Http to Elsa.Tenants.AspNetCore and updated relevant project references and namespaces accordingly. This refactor improves modularity and separation of concerns between HTTP and tenancy features.

* Refactor route handling and tenant configuration

Removed redundant `RouteTableExtensions` and replaced with new route providers and updaters, enhancing flexibility and modularity. Introduced tenant-specific HTTP endpoint configurations for better customization and configuration management.

* Rename HttpEndpointBookmarkStimulus to HttpEndpointBookmarkPayload

Refactor various classes and methods to reflect the renaming from `HttpEndpointBookmarkStimulus` to `HttpEndpointBookmarkPayload`. Add and configure new extension methods for tenant route handling, update the route provider to support multi-tenancy, and adjust the tenants provider to bind configuration properly.

* Add HeaderTenantResolver and refactor Http namespace.

Introduce HeaderTenantResolver to resolve tenants via HTTP headers. Refactor multiple classes and interfaces to move from the Elsa.Http.Models namespace directly into Elsa.Http for clarity and consistency.

* Add Host-based tenant resolution

Implemented a HostTenantResolver to resolve tenants based on the request's host and updated tenant configurations with host information. Modified the tenant resolver pipeline and added the new host resolver to the service registrations.

* Add tenant-aware caching and accessor support

Enhanced caching by incorporating tenant identifiers into cache keys for more granular cache management. Introduced ITenantAccessor dependencies in various services to retrieve the current tenant information. This ensures that cache entries are correctly isolated per tenant.

* Reorder tenant resolvers for pipeline setup.

Reordered the tenant resolvers in the pipeline to prioritize HostTenantResolver before RoutePrefixTenantResolver. This ensures that tenant resolution is correctly aligned with host-based resolving before checking the route prefix.

* Remove unused imports

This commit eliminates redundant `using` directives across multiple files to streamline the codebase. This cleanup helps improve code readability and maintainability by removing unnecessary dependencies.
2024-10-14 21:27:11 +02:00
Sipke Schoorstra a5cc3fc9e8
Refactor Tenant Resolution to Use Async Local Storage for Operation-wide Access (#6022)
* Remove obsolete tenant-related classes and add ASP.NET Core middleware

Refactored tenant resolution by removing obsolete interfaces and classes, such as `IAmbientTenantAccessor` and `ITenantResolutionStrategy`. Introduced new ASP.NET Core middleware for tenant resolution, encapsulated in the new `Elsa.Tenants.AspNetCore` project. Updated related usage in various parts of the application to align with these changes.

* Remove HttpContextTenantResolver.

Removed HttpContextTenantResolver from the multitenancy pipeline and related service registrations. This simplifies the tenant resolution by relying on remaining resolvers like ClaimsTenantResolver and RoutePrefixTenantResolver.

* Add Elsa solution definition file

This commit introduces the main solution file, Elsa.slnx, defining the folder structure, projects, and configuration for the Elsa repository. This includes folders for Docker, documentation, pipelines, samples, scripts, source code, and tests.

* Refactor DefaultAccessTokenIssuer for clarity and efficiency

Refactored the DefaultAccessTokenIssuer class by simplifying its constructor and utilizing scoped variables for token options. Improved token creation logic by adding a dedicated method to configure token options, enhancing code readability and maintainability.

* Remove Elsa.slnx solution file

No dotnet build support yet.

* Refactor tenant resolver service registrations

Updated the service registrations to use interfaces for DefaultTenantResolver and DefaultTenantResolverPipelineInvoker. This improves the code's flexibility, making it easier to replace or extend these implementations in the future.

* Add multitenancy support and tenant scope management

Introduced ITenantScopeFactory and related implementations for tenant scope management across the application. Enhanced the HTTP workflows middleware to handle tenants and updated relevant configurations and extension methods to support tenant resolution.

* Remove unnecessary folder inclusion

The <Folder> tag for "Modules\Modules\" was redundant and has been removed to clean up the project file. This change will not affect the existing functionality or project structure.

* Rename Create to CreateScope and improve authorization.

Updated the method name from Create to CreateScope for better clarity in the TenantScopeFactory. Fixed a logical error in the authorization process, ensuring proper status code setting for unauthorized requests, and refactored token expiration calculation for clarity.

* Add tenant agnostic filters and remove tenant setup

This commit introduces tenant agnostic filters in AutoUpdateTests to ensure workflows can trigger regardless of tenant. Additionally, it removes tenant configuration from WorkflowServer setup as it is no longer required for the current tests.
2024-10-12 12:08:09 +02:00
Sipke Schoorstra ff487fad80 Merge remote-tracking branch 'origin/blueberry' into enh/multi-tenancy-refactorings 2024-10-11 19:45:40 +02:00
Sipke Schoorstra dd812645c7 Refactor imports to reduce use of Elsa.Common.Contracts
Consolidate imports by replacing Elsa.Common.Contracts with Elsa.Common and Elsa.Common.Multitenancy. This update streamlines import statements across various modules, improving code readability and maintainability.
2024-10-05 18:30:45 +02:00
Sipke Schoorstra 41e1062148 Enhance JSON deserialization to include service provider
Updated ReadAsJsonAsync method to accept an IServiceProvider parameter for improved JSON deserialization. Modified corresponding tests and helper classes to utilize the service provider for more flexible JSON serialization options.
2024-10-03 19:12:00 +02:00
Sipke Schoorstra 631036f404
Proto.Actor implementation for ChangeTokenSignalPublisher (#5817)
* **Refactor ProtoActor modules and integrate new core module**

Removed obsolete proto actor-related files and introduced a new core module under `Elsa.ProtoActor.Core` to centralize ProtoActor functionalities. Updated services and extensions to align with the new core structure, focusing on efficient persistence and actor system configurations.

* Add Proto.Actor-based distributed caching module

Introduces a new module `Elsa.Caching.Distributed.ProtoActor` for Proto.Actor-based distributed caching, including configuration extensions, proto files, and required services. Refactors some existing Proto.Actor-related features and updates Dockerfile and example projects to use the new module.

* Refactor ProtoActor cache handling and virtual actor setup.

Reorganize the distributed caching by introducing LocalCacheVirtualActorProvider and StartLocalCacheActor. Update WorkflowInstanceVirtualActorProvider for better cluster kind handling. Adjust namespaces in Protobuf definitions for consistency.

* Refactor LocalCacheImpl to use IChangeTokenSignalInvoker

Replace IChangeTokenSignaler with IChangeTokenSignalInvoker to align with updated dependency contract. Adjust method call to use InvokeAsync for triggering token signals with cancellation support.

* Refactor ConfigureClusterConfig and config mutation

Change ConfigureClusterConfig from Action to Func for better flexibility. Update clusterConfig and remoteConfig to support reassignment from configuration methods.

* Add ProtoActor support for distributed caching

Introduced ProtoActor as a new distributed caching transport option. Updated the configuration and workflow runtime settings to utilize ProtoActor. Added necessary project reference for Elsa.Caching.Distributed.ProtoActor in the .csproj file.

* Add LocalNodeStrategy and integrate it in actor provider

Introduced `LocalNodeStrategy` to handle member placement on the current node. Integrated the new strategy in `LocalCacheVirtualActorProvider`, ensuring it uses `LocalNodeStrategy` for member management.

* Prevent duplicate member additions based on ID.

Updated the member checking logic to include member IDs. In addition, this change improves the robustness of the member management in `LocalNodeStrategy.cs`.

* Refactor virtual actor configuration into a separate method

Moved virtual actor setup logic from `ProtoActorFeature` to a new `AddVirtualActors` method to improve code readability and reusability. Updated related files to maintain consistency and enhance documentation clarity.

* Refactor LocalCache to use PubSub for change token signals

Replaced direct event stream usage with PubSub in LocalCache implementation. Updated service and hosted service to support PubSub subscription and publishing. Removed obsolete Start and Stop RPC methods from LocalCache service definition.

* Add UsedImplicitly attribute to notification handler

This change introduces the [UsedImplicitly] attribute to the DistributedWorkflowDefinitionNotificationsHandler class. The attribute is intended to prevent any accidental removal by static analysis tools, ensuring the class remains available for dynamic usage scenarios.

* Refactor caching and signal handling mechanisms

Replaced `TriggerChangeTokenSignalConsumer` with `ChangeTokenSignalInvoker` and added new decorators for change token handling. Renamed namespaces and file paths for better consistency and clarity. Updated test files to align with these changes.

* Remove unnecessary interface dependencies from services

Eliminated the ISignalManager and related interfaces to streamline dependency management. Updated services and test components to use concrete implementations directly, reducing complexity and improving maintainability.

* Remove Shared.proto and associated imports

Deleted the Shared.proto file and removed related import statements across multiple files. This cleanup also involved modifying the proto actor provider and project file to exclude references to Shared.proto.

* Simplify namespaces in component test helpers

Consolidated several namespaces into 'Elsa.Workflows.ComponentTests.Helpers' to reduce redundancy and improve maintainability. Removed unnecessary using directives in multiple test files for cleaner and more readable code.

* Refactor imports in component tests

Consolidated various helper imports in component tests by removing redundant specific references and utilizing general 'Elsa.Workflows.ComponentTests.Helpers'. This change simplifies the dependency management and ensures cleaner and more maintainable code.

* Remove redundant state persistence calls

Eliminated multiple calls to PersistStateAsync in WorkflowInstanceImpl.cs as they were unnecessary given that the WorkflowRunner already invokes the commit handler. This change simplifies the workflow execution and cancellation logic by avoiding redundant state persistence operations.

* Add workflowInstanceId to response mapping

Updated methods to include workflowInstanceId in response mapping functions for consistency and clarity. Additionally, fixed project reference paths and added error handling for missing workflow variables in tests.

* Remove unnecessary variable existence check

Removed a redundant check for the existence of the "Workflow1:variable-1" key in the variables dictionary. This streamlines the test and relies on the assumption that the key exists as expected without explicit validation.

* Add Kubernetes deployment and service configurations

Introduced a Deployment and Service configuration for the Kubernetes cluster. Updated Dockerfiles and build script to align with port 8080 configuration and renamed images for consistency. Updated solution file to include new deployment files.

* Add Kubernetes cluster integration

Introduced Kubernetes cluster provider for Proto.Actor and configured the application to use it if running in a Kubernetes environment. Added necessary RBAC roles, role bindings, and service accounts to support Kubernetes integration. Updated deployment configuration and package references to include Proto.Cluster.Kubernetes.

* Refactor deployment configurations and add service support.

Reorganized deployment YAML files into designated subdirectories for elsa-server, postgres, plant-uml, and trace-lens. Introduced new configuration maps, service accounts, roles, and service bindings. Updated .NET environment variables and solution structure to reflect these changes.

* Update service configurations and environment variables

Renamed and split services in trace-lens to isolate the OTEL collector. Updated environment variables in elsa-server to enhance instrumentation, connection strings, and profiling settings. Adjusted OTEL exporter endpoint to match the new service naming.

* Increase deployment replicas to 3

Updated the 'replicas' field in the deployment configuration to enhance the system's availability and load balancing. This change ensures that three instances of 'elsa-server' will be running simultaneously.

* Rename LocalCacheImpl to LocalCache and add logging

Renamed `LocalCacheImpl` class to `LocalCache` to better reflect its purpose. Added a logging statement in `OnReceive` method to log incoming `ProtoTriggerChangeTokenSignal` messages. These changes improve code readability and debugging.

* Disable OTEL console exporters and set session affinity

Disabled console exporters for logs, metrics, and traces in the OTEL configuration to reduce unnecessary console output. Additionally, set session affinity to 'None' in the elsa-server service configuration for load balancing.

* Rename WorkflowInstanceImpl to WorkflowInstance

Updated the class name from WorkflowInstanceImpl to WorkflowInstance for clarity and simplicity. Adjusted all relevant references and instances in the codebase to match the new class name.

* Remove ActivityIncidentStateMapper and Update ProtoBuf Mappings

Removed the unused ActivityIncidentStateMapper class to streamline the codebase. Updated all related ProtoBuf mappings and imports to ensure consistency and remove redundancy across the project.

* Remove duplicate actor spawn verification timeout setting

The code had a redundant setting for actor spawn verification timeout, which was specified twice. This commit removes the duplicate line to ensure cleaner and more maintainable configuration.

* Remove unused imports from Program.cs

Eliminated unnecessary imports for ActivityExecution, WorkflowExecution, and k8s libraries. This cleanup helps reduce the code footprint and may improve compile time.

* Remove debug logging from LocalCache actor

The `Console.WriteLine` statement was removed from the `OnReceive` method in `LocalCache.cs`. This change eliminates unnecessary console output during the token signal handling, improving performance and reducing log clutter.
2024-07-24 22:23:33 +02:00
Sipke Schoorstra 5586151bf1 Remove IWorkflowExecutionContextStore and related classes
Removed the IWorkflowExecutionContextStore interface and its implementations, along with the associated handlers and registrations. This change simplifies the codebase by eliminating unnecessary components and their dependencies.
2024-07-18 18:57:42 +02:00
Sipke Schoorstra 827680b3c4 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-18 18:25:49 +02:00
Sipke Schoorstra f448e9520a
Fix reloading logic for workflow definitions (#5781)
* Add reloading logic for workflow definitions

Introduced a new mechanism to handle reloaded workflow definitions. This includes creating a "ReloadedWorkflowDefinition" model, updating the caching logic, and modifying notification handlers to work with the enhanced workflow reloading logic. This ensures workflow definitions are updated and managed correctly when published, retracted, or deleted.

* Improve RefreshActivityRegistry documentation

Updated the XML documentation to clarify that `RefreshActivityRegistry` refreshes the `IActivityRegistry` for `WorkflowDefinitionActivityProvider` whenever workflow definitions are reloaded, instead of when they are published, retracted, or deleted.

* Add 'materializer' to user dictionary

The term 'materializer' has been added to the user dictionary to improve code spelling and naming consistency. This change ensures that 'materializer' is recognized as a correct term in the codebase.

* Organize test files by adding a 'Workflows' directory

Renamed 'http-workflow.json' to indicate it belongs under 'Workflows'. This improves file organization and clarity within the 'WorkflowDefinitionReload' scenario.

* Add TestWorkflowMaterializer and TestWorkflowProvider

Introduce `TestWorkflowMaterializer` for deserializing workflows from `TestWorkflowProvider`. Added integration of these new components in the `ReloadWorkflowTests` and `WorkflowServer`. Also renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests`.

* Rename and expand workflow reload tests

Renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests` to better reflect its purpose and expanded with additional test cases. Added tests to verify workflow and activity registry updates after source provider changes and workflow reloads.

* Update copy settings for workflow test scenarios

Reorganized and added 'CopyToOutputDirectory' settings for JSON files in workflow test scenarios. Ensured all necessary files are correctly included and copied during output directory builds to maintain test consistency.
2024-07-17 09:51:45 +02:00