Backport to release/3.8.0 so Elsa.Api.Client 3.8.0-rc2 exposes the
Resources/OutputConverters surface that Elsa Studio's release/3.8.0 branch
already consumes. Without it, Studio cannot build against a released client:
it was green against 3.8.0-preview.5397 (built from main) and broke when its
pin moved to 3.8.0-rc1 (built from this branch).
(cherry picked from commit d698e6b005)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Avoid null endpoint DTO metadata in tests
* Enforce console logs hub read permission
* Remove unused console logs hub import
* Support mapped endpoint metadata in auth tests
* Reduce console log capture throughput impact
* Address Copilot console logs review
* Refactor task scheduling to support tenant-level background work and enhance logging functionality.
* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.
* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.
* Add Ansi SGR parser for console logs and associated unit tests
* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.
* Address console logs code quality feedback
* Address PR review feedback
* Preserve console logs extension points
* Stabilize console logs host lifecycle
* Address final automated review comments
* Tighten console log capture shutdown
* Address console log review feedback
* Address follow-up review feedback
* Cover final review feedback
* Avoid recursive console provider initialization
* Guard console host lease shutdown
* Preserve console log scope and provider lifetime
* Correlate console log scope fallback
* Tighten console scope correlation
* Expose host services during provider construction
* Redact ANSI-normalized console lines
* Add OpenTelemetry diagnostics backend foundation
* Add OTLP HTTP ingestion parsing
* Document OpenTelemetry diagnostics setup
* Enforce OpenTelemetry hub permissions
* Remove `ConsoleCaptureTee` and related services and tests
* Add OpenTelemetry HTTP ingestion integration test
* Use pipeline contributors for console log context
* Update CShells package versions to 0.0.24-preview.132
* Add OpenTelemetry ingestion security tests
* Add OpenTelemetry API authorization tests
* Filter live console logs by workflow instance
* Add OpenTelemetry hub tests
* Add OpenTelemetry gRPC metadata hook
* Assert OpenTelemetry workflow tags survive ingestion
* Mark OpenTelemetry core build verified
* Enhance console logging with activity execution metadata and extend test coverage.
* Address console logs stream consumption comment
* Wire OpenTelemetry diagnostics into core sample
* Address Core diagnostics review feedback
* Address Core Copilot follow-up feedback
* Add OpenTelemetry metric instrument names
* Address Core Copilot provider feedback
* Address Core Copilot diagnostics follow-up
* Address Core Copilot live feed feedback
* Address Core Copilot store feedback
* Integrate OpenTelemetry for logging, tracing, and metrics in ModularServer and update launch settings and docker-compose configuration.
* Refactor to replace `ConsoleLogStream.Core` with `ConsoleLogStreaming.Core` across codebase and update `ConsoleStreamHook` installation.
* Add diagnostics OpenTelemetry backend
* Fix OpenTelemetry live hub subscription
* Fix modular OpenTelemetry exporter endpoints
* Add CShells logging configuration in appsettings.json
* Remove obsolete unit tests and helper classes
* Restore default activity exception handling
* Simplify type serialization and alias management
This commit refactors the internal type serialization and alias management system to reduce boilerplate, improve robustness, and simplify the developer experience:
- Removed numerous explicit `ExpressionOptions` type alias registrations across various modules.
- Updated `TypeJsonConverter` and polymorphic serialization to reliably handle types using assembly-qualified names when a short alias is not explicitly registered.
- Streamlined `ExcludeFromHashConverter` to strictly adhere to `ExcludeFromHashAttribute` for hash calculations, removing complex `JsonIgnoreCondition` logic.
- Eliminated several helper classes (`WorkflowJsonTypeResolver`, `WorkflowTypeValidator`, `IWorkflowTypeRegistry`, `WorkflowFactoryDictionary`, `JavaScriptExceptionTypeAliasRegistrar`, `WorkflowRuntimeTypeAliasRegistrar`) and their associated unit tests, simplifying the codebase.
Additionally, this commit introduces a comprehensive markdown document (`product-website-feature-source.md`) outlining Elsa's core features, Studio capabilities, extension ecosystem, and architectural selling points, intended as source material for the product website.
* Refine type serialization for improved robustness and alias handling
This commit further enhances the type serialization and deserialization mechanisms:
* Centralizes type resolution and alias management through `IWellKnownTypeRegistry` and `WorkflowJsonTypeResolver`.
* Prioritizes registered type aliases when serializing type metadata in `PolymorphicObjectConverter`, resulting in more concise JSON output.
* Enhances deserialization in `PolymorphicObjectConverter` and `VariableMapper` to gracefully handle unknown or non-instantiable types, providing fallbacks and logging warnings.
* Simplifies `TypeJsonConverter` by delegating complex type resolution logic to the `WorkflowJsonTypeResolver`.
* Adds `JsonArray` to the well-known type aliases for direct recognition.
* Fix console logs packaging and workflow type resolution
* Fix console log metadata and type resolution
* Address Copilot review feedback
* Enhance type resolution, improve console log handling, and update tests
- Streamlined `WorkflowDictionaryExtensions` for better workflow registration validation.
- Refined `ConsoleLogsAuthorizationTests` with the new `SetJsonRequest` helper to improve test requests handling.
- Updated `OrderDefinition` to ignore JSON serialization for `KeySelector`.
- Enhanced `WorkflowRuntimeFeature` for improved workflow registration and type alias configuration.
- Added tests to ensure `ConsoleLogProvider` metadata filtration in various scenarios.
- Improved type serialization logic in `WorkflowJsonTypeResolver`.
- Updated README to fix references related to diagnostics.
- Optimized `ExcludeFromHashConverter` for property serialization conditions.
- Modified `TriggerIndexer` for streamlined trigger management.
- Tested payload checks in `PublishEventTests`.
- Adjusted `Endpoint` in `ConsoleLogs` for automatic JSON request handling.
- Ensured registration of workflow type aliases in `WorkflowsFeature`.
* Restore CLR workflow registration compatibility
* Align JSON island serialization fixtures
* Add Console Logs Services and Enhance Endpoint Handling
- Introduced `ActivityExecutionsEndpointTests` to validate route exposure.
- Added `ConsoleLogCaptureHostedService` for console log streaming.
- Implemented `ConsoleStreamJsonConverter` for JSON conversion of console streams.
- Developed `ElsaConsoleLogRecentBuffer` to handle recent log buffering.
- Updated `ConsoleLogsAuthorizationTests` with new test cases for stream filter mapping.
- Consolidated console log provider dependencies and registration, including recent buffering.
- Enhanced `ElsaConsoleLogProvider` to use recent buffer for filtering.
- Adjusted `Program.cs` for streamlined logging service setup.
* Enhance type resolution and test coverage; streamline console log integration
- Added `ConsoleStreamHook` for streamlined log streaming.
- Updated `WorkflowJsonTypeResolverTests` to improve type resolution and test new scenarios.
- Simplified type resolution by removing trusted assembly checks.
* Fix CI smoke and package restore failures
* Fix Docker smoke image project paths
* Fix Docker Python runtime packages
* Fix Docker CA smoke teardown
* Refresh Elsa roadmap
* Implement background processors and mediation coordination
- Added `BackgroundCommandProcessor`, `BackgroundJobProcessor`, and `BackgroundNotificationProcessor` classes for handling commands, jobs, and notifications, respectively.
- Introduced `MediatorBackgroundProcessingCoordinator` to coordinate the execution of all background processors.
- Implemented `MediatorBackgroundTask` for wrapping `MediatorBackgroundProcessingCoordinator` in `BackgroundTask`.
- Added unit tests for `MediatorBackgroundTask` to ensure proper start and stop behavior.
- Refactored `BackgroundCommandSenderHostedService` to utilize `BackgroundCommandProcessor`.
- Introduced 'elsa-roadmap-refresh' skill configuration for roadmap updates.
* Address workflow type resolution review feedback
* Address follow-up review feedback
* Restore recent console logs execute path
* Address Copilot follow-up review
* Decouple workflow JSON aliases from expressions
* Fix workflow management unit test setup
* Fix console logs recent endpoint handler shape
* Respect workflow JSON strict type aliases
* Remove unused console log contracts reference
* Address Copilot review feedback
* Address Copilot follow-up comments
* Synchronize ring buffer dropped count
* Address background processor strategy replay
* Optimize workflow definition sync lookups
Replace repeated linear scans in workflow definition synchronization with a single hash set lookup. This keeps the existing replacement behavior while avoiding quadratic work as the number of workflow definitions grows.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Stabilize bulk dispatch component test
Wait for child workflow completion through the workflow instance store instead of relying on in-memory event delivery, and avoid using the scheduled Delay activity for the fire-and-forget child workflow timing check.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Stabilize dispatch component test
Use persisted workflow instance state when waiting for fire-and-forget child workflow completion, and avoid scheduled Delay for the timing check.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add missing using for merge conflict resolution
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Remove unused bulk dispatch using
Drop an unnecessary using from the component test file so the PR branch has a new maintainer-authored head commit for CI.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Stabilize bulk dispatch fire-and-forget test
* Improve bulk dispatch timeout diagnostics
* Stabilize bulk dispatch polling test
* Reduce bulk dispatch test polling pressure
* refactor(deps): use local CShells project refs
Replace CShells NuGet package references with direct project references to the local CShells source to enable developing and testing against local changes and simplify build integration across modules.
* Handle assembly load errors in feature discovery
Added error handling for assembly load failures in feature discovery to improve resilience. Also updated configuration for identity token options and removed unused service bus consumer dependencies. Simplified project structure by moving and cleaning up `Directory.Build.targets` files.
* Refactor configuration and service extension methods.
Moved `ShellSettingsExtensions` and `ShellConfiguration` to `CShells.Abstractions` for better modularity. Added new `ServiceCollectionFeatureExtensions` to improve options registration. Updated appsettings and references to support these changes.
* Introduce ManagementServiceCollectionExtensions to streamline activity and variable registration
Added `ManagementServiceCollectionExtensions` for registering Elsa activity types and variable descriptors, providing a modular and shell-feature-compatible approach to configuration. Updated relevant features to utilize these new extension methods, enhancing code modularity and reducing redundancy.
* Add resilience strategy registration to HTTP feature
Introduced `ResilienceServiceCollectionExtensions` to register resilience strategies within the `Elsa.Resilience.Core` module. Updated `HttpFeature` to incorporate resilience strategies, enhancing HTTP-related resilience configuration leveraging the new extension methods.
* Add new configuration options to JavaScriptFeature
Implemented multiple properties in `JavaScriptFeature` to enhance JavaScript execution: `AllowClrAccess`, `AllowConfigurationAccess`, `ScriptCacheTimeout`, `DisableWrappers`, and `DisableVariableCopying`. These additions enable more flexible and secure configuration of the Jint JavaScript engine.
* refactor(workflows): unify graph caching
Resolve workflow definitions first and store graphs under stable per-version-ID cache keys so different lookup paths share entries.
Centralize cache creation and change-token registration to remove duplicated caching logic.
Skip materializer-unavailable definitions to avoid caching null graphs and simplify flow.
* refactor(tests): centralize default IDs and materializer setup
Introduce constants for default definition and version IDs, and materializer name. Refactor tests to use these constants, streamline graph and definition resolution, and improve cache key creation by sharing logic across tests. Extend tests to check scenarios with unavailable materializers, ensuring caching only occurs for valid cases.
* extend(tests): enhance cache key verification in AutoUpdateTests
Added checks for both workflow definition and version cache keys in AutoUpdateTests to ensure comprehensive cache validation, improving test reliability and coverage.
* refactor(projects): update CShells project paths and solution configuration
Revised project reference paths in `Elsa.ModularServer.Web.csproj` for CShells projects and updated `Elsa.sln` to include new CShells projects, streamlining project organization and build configuration.
* Add `IWorkflowReferenceGraphBuilder` to `WorkflowManagementFeature`; rename `ResilienceShellFeature` to `ResilienceFeature`.
* Refactor `HttpFeature` to use `IMiddlewareShellFeature`, include `HttpWorkflowsMiddleware`, and update `HttpActivityOptions` defaults.
* Add `AddTypeAlias` and `AddVariableTypeAndAlias` extension methods to service collections
- Introduced `AddTypeAlias<T>` method in `ServiceCollectionExtensions.cs` for adding type aliases.
- Added `AddVariableTypeAndAlias<T>` method in `ManagementServiceCollectionExtensions.cs` to add variable types with aliases.
* Remove shell reload API endpoints, orchestrator, and associated tests from the codebase.
* Introduce `DefaultAdminUser` options and refactor `AdminUserInitializer` to use `IOptions`.
* Add user management endpoints: Delete, List, Update with enhanced user store functionality
* Implement `DefaultAdminUser` feature for initial admin bootstrap, decouple `SecurityRoot` from user management endpoints, update related documentation and permissions.
* Add role management endpoints: Delete, List, and Update, including role data models and handle obsolete SecurityRoot policy.
* Update CShells package references to version 0.0.12-preview.66 and refactor `TenantTaskManager` for improved task lifecycle management.
* Replace project references with package references in csproj files and remove unused folders.
* Integrate Nuplane features, add sample packages, and update dependency handling within ModularServer Web.
* Improve `CShells` startup endpoint registration and resolver handling
- Address duplicate endpoint registration by adding state-aware tracking and deduplication
- Resolve `WebRoutingShellResolver` constructor ambiguity by switching to factory-based registration
- Implement a startup-specific filter to prevent redundant endpoint remapping during `ShellsReloaded`
- Update project to use project references for `CShells` and `Nuplane` components in csproj files.
* Update logging configuration in appsettings for Development and Production
- Change default log level to 'Warning' in Development settings
- Adjust Microsoft.Hosting and Elsa.SamplePackage log levels to 'Information'
- Remove Microsoft.EntityFrameworkCore log level entry from Production settings
* Refactor assembly retrieval methods and update endpoint calls for consistency.
* Add `SampleEndpointFeature` and enhance logging and service integration
- Implement `SampleEndpointFeature` with a new endpoint for handling requests.
- Log endpoint access and integrate `ISampleService` with method `DoSomething`.
- Update logging configuration to include `CShells` and `Nuplane` log levels in Development settings.
- Update `Elsa.SamplePackage` to version 1.0.1 and manage dependencies with project and assembly references.
- Modify JSON configuration for `SampleEndpoint`.
* Update package versions for `CShells` to 0.0.13 and `Nuplane` to 0.0.1-preview.15 in props file.
* Refactor `DefaultAdminUserFeature` by renaming `ConfigureServices` to `Apply` and adjusting service registration method.
* Replace project references with package references across multiple projects and remove obsolete cshells-related solution entries.
* Remove `SampleCatalogEndpointExtensions.cs` and related endpoint mappings.
* Improve `TenantTaskManager` by using `TryRemove` for state clean-up and clarify `SemaphoreSlim` disposal behavior.
* Remove hardcoded default admin credentials and add warning for unconfigured AdminRoleName in admin user setup.
* Address unresolved review comments: fix doc comments, security defaults, compilation issue, and restore reload response contracts
Agent-Logs-Url: https://github.com/elsa-workflows/elsa-core/sessions/34eb1e13-833f-4b3c-9db6-2e9221d221b9
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Refine reload endpoints: use specific exceptions, add error messages, rename ReloadedAt to Timestamp, remove unused model
Agent-Logs-Url: https://github.com/elsa-workflows/elsa-core/sessions/34eb1e13-833f-4b3c-9db6-2e9221d221b9
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Potential fix for pull request finding 'Generic catch clause'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* Potential fix for pull request finding 'Generic catch clause'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* Potential fix for pull request finding 'Generic catch clause'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* Add `ExceptionExtensions` with `IsFatal` method and simplify exception handling in `TenantTaskManager`. Remove unused properties from `Directory.Build.props`.
* Add unit tests for `TenantTaskManager` and fix potential state orphaning issue.
* Potential fix for pull request finding 'Generic catch clause'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* Fix logger dependency in `SampleEndpointFeature` constructor to use correct type.
* Add unit tests for Elsa Shells API endpoints and update solution configuration.
* Refactor ShellReload models: remove ShellReloadItemResult, update ShellReloadResponse properties.
* Potential fix for pull request finding 'Generic catch clause'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* refactor(deps): use local CShells project refs
Replace CShells NuGet package references with direct project references to the local CShells source to enable developing and testing against local changes and simplify build integration across modules.
* Handle assembly load errors in feature discovery
Added error handling for assembly load failures in feature discovery to improve resilience. Also updated configuration for identity token options and removed unused service bus consumer dependencies. Simplified project structure by moving and cleaning up `Directory.Build.targets` files.
* Refactor configuration and service extension methods.
Moved `ShellSettingsExtensions` and `ShellConfiguration` to `CShells.Abstractions` for better modularity. Added new `ServiceCollectionFeatureExtensions` to improve options registration. Updated appsettings and references to support these changes.
* Introduce ManagementServiceCollectionExtensions to streamline activity and variable registration
Added `ManagementServiceCollectionExtensions` for registering Elsa activity types and variable descriptors, providing a modular and shell-feature-compatible approach to configuration. Updated relevant features to utilize these new extension methods, enhancing code modularity and reducing redundancy.
* Add resilience strategy registration to HTTP feature
Introduced `ResilienceServiceCollectionExtensions` to register resilience strategies within the `Elsa.Resilience.Core` module. Updated `HttpFeature` to incorporate resilience strategies, enhancing HTTP-related resilience configuration leveraging the new extension methods.
* Add new configuration options to JavaScriptFeature
Implemented multiple properties in `JavaScriptFeature` to enhance JavaScript execution: `AllowClrAccess`, `AllowConfigurationAccess`, `ScriptCacheTimeout`, `DisableWrappers`, and `DisableVariableCopying`. These additions enable more flexible and secure configuration of the Jint JavaScript engine.
* refactor(workflows): unify graph caching
Resolve workflow definitions first and store graphs under stable per-version-ID cache keys so different lookup paths share entries.
Centralize cache creation and change-token registration to remove duplicated caching logic.
Skip materializer-unavailable definitions to avoid caching null graphs and simplify flow.
* refactor(tests): centralize default IDs and materializer setup
Introduce constants for default definition and version IDs, and materializer name. Refactor tests to use these constants, streamline graph and definition resolution, and improve cache key creation by sharing logic across tests. Extend tests to check scenarios with unavailable materializers, ensuring caching only occurs for valid cases.
* extend(tests): enhance cache key verification in AutoUpdateTests
Added checks for both workflow definition and version cache keys in AutoUpdateTests to ensure comprehensive cache validation, improving test reliability and coverage.
* refactor(projects): update CShells project paths and solution configuration
Revised project reference paths in `Elsa.ModularServer.Web.csproj` for CShells projects and updated `Elsa.sln` to include new CShells projects, streamlining project organization and build configuration.
* Add `IWorkflowReferenceGraphBuilder` to `WorkflowManagementFeature`; rename `ResilienceShellFeature` to `ResilienceFeature`.
* Refactor `HttpFeature` to use `IMiddlewareShellFeature`, include `HttpWorkflowsMiddleware`, and update `HttpActivityOptions` defaults.
* Add `AddTypeAlias` and `AddVariableTypeAndAlias` extension methods to service collections
- Introduced `AddTypeAlias<T>` method in `ServiceCollectionExtensions.cs` for adding type aliases.
- Added `AddVariableTypeAndAlias<T>` method in `ManagementServiceCollectionExtensions.cs` to add variable types with aliases.
* Update CShells package versions to 0.0.11 and replace ProjectReferences with PackageReferences in project files
* Initial plan
* Add consumers API endpoint and enhance export with IncludeConsumingWorkflows option
- New endpoint: GET /workflow-definitions/{definitionId}/consumers
- Export request model: added IncludeConsumingWorkflows boolean property
- Export endpoint: recursive consumer discovery and inclusion in exports
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Address greptile-apps feedback: fix double-fetch, remove redundant HashSet, add version comment, remove unused params
- Extract WriteZipResponseAsync helper to avoid double-fetching definitions from DB
- Remove redundant existingDefinitionIds HashSet in IncludeConsumersAsync
- Add XML doc comment documenting that consumers are resolved at VersionOptions.Latest
- Remove unused constructor parameters (workflowDefinitionService, variableDefinitionMapper)
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Address greptile-apps round 2: 404 for unknown definitions, deterministic ZIP response, parallel BFS
- Consumers endpoint now returns 404 when the definition ID doesn't exist
- Single-workflow export always returns ZIP when includeConsumingWorkflows=true
- BFS traversal processes each frontier level concurrently via Task.WhenAll
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Implement workflow reference graph feature
- Added `IWorkflowReferenceGraphBuilder` interface for building complete graphs of workflow references.
- Created `WorkflowReferenceGraph` and `WorkflowReferenceEdge` models.
- Implemented `WorkflowReferenceGraphBuilder` service for recursive graph building.
- Replaced previous workflow reference query logic with the new graph-based approach in consumers.
* Refactor workflow consumers to utilize recursive graph-based approach for retrieving consumer definitions.
* Enhance workflow export by adding support for including consuming workflows and update nullable default values across endpoints and models.
* Add WorkflowReferenceGraphOptions for depth and definition limit configuration
- Introduced `WorkflowReferenceGraphOptions` to configure max depth and definition limits for reference graph building.
- Updated `WorkflowManagementFeature` to support configuring these options.
- Enhanced `WorkflowReferenceGraphBuilder` to respect configuration limits during graph construction process.
* Refactor `WorkflowReferenceGraphBuilder` to utilize target-typed new expressions for cleaner code.
* Add Workflow Reference Graph tests and scenarios
Introduced JSON files for parent-child-grandchild workflow hierarchy tests and implemented unit tests for `WorkflowReferenceGraphBuilder`. Also added component tests for workflow export and consumers endpoint validation. Updated project configuration for workflow JSON to always copy to output directory.
* Include DefinitionId in exported workflow definition filenames for uniqueness.
* Remove unused models and constants from `WorkflowReferenceGraphTests`.
* Refactor `WorkflowReferenceGraphBuilderTests`: streamline test setup with `SetupGraph`, replace repeated assertions with helper methods.
* Deterministic ZIP export and 404 test coverage for consumers endpoint (#7312)
* Initial plan
* Fix deterministic ZIP export and add 404 test for consumers endpoint
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Add ADR for Null Tenant ID, implement tenant-agnostic logic
Introduce ADR-0009 to document the use of `null` for tenant-agnostic entities, enhancing multitenancy handling. Update multitenancy features across the codebase, including EF Core query filters and ActivityRegistry, to handle null as a tenant ID, ensuring tenant-agnostic entities are accessible across all tenants.
* Add multitenancy support in `ActivityTestFixture` by registering `ITenantAccessor`.
* Update src/modules/Elsa.Common/Multitenancy/Implementations/DefaultTenantAccessor.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Remove unused `using Elsa.Common.Multitenancy;` from WorkflowDefinitionActivityDescriptorFactory (#7230)
* Initial plan
* Remove unused using Elsa.Common.Multitenancy statement
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Optimize ActivityRegistry.Find to prefer tenant-specific descriptors without performance regression (#7227)
* Initial plan
* Optimize Find(string type) to prefer tenant-specific descriptors with single-pass iteration
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Apply review feedback: combine if statements and add comprehensive unit tests
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Refactor tests for DRYness using theories and helper methods
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Clean up extra whitespace in test file
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Fix default tenant data visibility leak by removing NullIfEmpty conversion (#7229)
* Initial plan
* Remove NullIfEmpty conversion to align with ADR-0008 and ADR-0009
- Updated ElsaDbContextBase to use empty string for default tenant
- Updated ApplyTenantId to stop converting empty string to null
- Updated TenantAwareDbContextFactory to preserve empty string for default tenant
- Updated Store.cs to preserve empty string for default tenant
- This ensures: null = tenant-agnostic (visible to all), "" = default tenant
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Add database migration to convert null TenantId to empty string for SqlServer
- Added Management migration to convert null to "" for WorkflowDefinitions and WorkflowInstances
- Added Runtime migration to convert null to "" for all runtime entities
- This ensures existing default tenant data is properly migrated per ADR-0008
- Note: Similar migrations needed for PostgreSql, MySql, Sqlite, and Oracle providers
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Clarify tenant handling logic in `ElsaDbContextBase` with new ADR references.
* Add tenant ID convention analysis documents and quick reference guide
* Implement tenant-agnostic functionality across modules
- Introduce `AgnosticTenantId` constant to manage tenant-agnostic entities.
- Modify entity handling logic to respect tenant-agnostic designations.
- Adjust workflow processing to include tenant-agnostic workflows.
- Update caching and activity descriptor logic to accommodate the `AgnosticTenantId`.
* Refactor tenant management and registry logic in `ActivityRegistry` for improved clarity and separation of tenant-specific and tenant-agnostic activity descriptors. Remove `TestTenantResolver` and update workflow definition handling for tenant support.
* Refactor `ActivityRegistry`: prioritize tenant-specific descriptors over tenant-agnostic and simplify descriptor retrieval logic.
* Improve async handling in `CommandHandlerInvokerMiddleware` to await tasks without blocking
* Update ADR to use asterisk as sentinel value for tenant-agnostic entities
Replace the previous convention of using `null` for tenant-agnostic entities with an asterisk (`"*"`) for improved clarity and system architecture. Updated ADR documentation, TOC, and dependency graph accordingly.
* Remove migration `ConvertNullTenantIdToEmptyString` and its associated designer file to clean up the codebase.
* Refactor `ActivityRegistry`: streamline activity descriptor removal logic and simplify tenant ID checks.
* Update Elsa.sln
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Simplify `RefreshDescriptorsAsync` by removing unnecessary local variable `currentTenantId`.
* Remove unused `currentTenantId` variable from `ActivityRegistry`.
* Add detailed semantic flow and key points to ADR 0009
Document the tenant ID flow from entity creation to query, emphasizing normalization and tenant-agnostic workflows. Update semantic flow diagrams and provide testing considerations for preserving `"*"` values in multi-tenant scenarios.
* Remove outdated Tenant ID Analysis and associated documents
* Add security-by-default design for tenant-agnostic entities in ADR
Enhance Architecture Decision Record to detail explicit requirements for tenant-agnostic database entities, highlighting differences between in-memory activity descriptors and persistent entities. Emphasize importance of setting `TenantId = "*"` to prevent accidental data leakage.
* Normalize tenant ID grouping in `ActivityRegistry` to unify null and agnostic IDs, reducing redundant processing.
* Refactor `SignalManager`: improve timeout handling and streamline signal task cancellation.
* Update src/modules/Elsa.Workflows.Core/Models/TenantRegistryData.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Workflows.Core/Services/ActivityRegistry.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor tests to use `Tenant.AgnosticTenantId` instead of `null` for tenant-agnostic descriptors.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Enhance logging in recurring tasks: add error handling and logger support to prevent crashes in scheduled timers.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
- Set `useMultitenancy` to `false` in `Program.cs`.
- Remove tenant filtering from `WorkflowDefinitionActivityProvider`.
- Enhance `DeleteWorkflowTests` by ensuring registry refresh without delay.
- Adjust tenant configuration in `DeleteWorkflow` for test accuracy.
- Lower `Threshold` in test project and update GitHub Actions versions.
* Enable multitenancy support and normalize tenant ID handling.
- Activate multitenancy in `Program.cs`.
- Introduce `NormalizeTenantId` method for consistent tenant ID usage.
- Update tenant-related classes and features to support normalization logic.
* Add ADR for adopting empty string as the default tenant ID
- Standardized the tenant ID for the default tenant to use an empty string (`""`) instead of `null`.
- Documented the rationale and migration considerations in ADR 0007.
- Updated ADR table of contents and graph for new entry.
* Apply suggestion from @sfmskywalker
* Update doc/adr/graph.dot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Normalize spacing and improve readability in `Program.cs`. Fix multitenancy condition formatting.
* Fix ADR numbering and update TOC
* Add ADRs for flowchart execution model, tenant deletion event, merge modes, and default tenant ID
- Introduced ADR 0005: Token-centric flowchart execution model for improved loop and join handling.
- Added ADR 0006: Tenant Deleted event for distinct handling of tenant removal.
- Documented ADR 0007: Explicit merge modes for flowchart joins, improving reliability and configurability.
- Included ADR 0008: Standardization of empty string as the default tenant ID for consistency and clarity.
* Add unit tests for tenant ID normalization and multitenancy pipeline invoker
- Added comprehensive unit tests for tenant ID normalization to ensure consistent handling of null, empty, and valid IDs.
- Introduced tests for the multitenancy pipeline invoker covering various tenant resolution scenarios.
- Updated solution to include new unit testing projects for `Elsa.Tenants` and `Elsa.Common`.
* Update unit tests for `ActivityConstructionResult`
- Refactor test parameterization to verify `HasExceptions` property more explicitly.
- Simplify exception creation logic in helper methods.
- Improve test assertions by combining act and assert phases where applicable.
* Enable configuration-based multitenancy with tenant-specific settings
- Introduced a configuration-based tenant provider to streamline tenant initialization and customization.
- Added tenant ID handling filters to ensure tenant ID is applied and filtered automatically.
- Deprecated the `CommonPersistenceFeature` in favor of modular persistence feature extension.
* Update database indexes to include `TenantId` for multitenancy support
- Added `TenantId` to unique constraints on `Triggers` table across all EFCore providers.
- Adjusted index names to reflect the updated constraints.
- Updated trigger configuration to ensure uniqueness includes `TenantId`.
* Add tenant filtering to `DefaultWorkflowDefinitionStorePopulator`
- Introduced `ITenantAccessor` to support tenant-specific filtering of workflow definitions.
- Updated logic to skip workflows not matching the current tenant.
* Update doc/adr/toc.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Remove `CommonPersistenceFeature` as it has been deprecated
* Add tenant-specific filtering to workflow import logic in `DefaultWorkflowDefinitionStorePopulator`
* Replace hardcoded tenant ID with `Tenant.DefaultTenantId` in integration tests
* Update database indexes and migration logic to support `TenantId` for multitenancy
- Added `TenantId` to unique constraints on the `Triggers` table and updated index names.
- Included logic to drop outdated indexes without `TenantId` during migration.
- Adjusted tests to account for `TenantId` in workflow identity and indexing scenarios.
* Remove `TenantId` from workflow identity construction in concurrent trigger indexing tests
* Introduce `SelectiveMockLockProvider` for precise lock mocking in tests
- Added `SelectiveMockLockProvider` to allow targeted lock mocking without affecting unrelated background operations.
- Updated test services to use `SelectiveMockLockProvider` in place of `TestDistributedLockProvider`.
- Refactored `DistributedLockResilienceTests` to support selective mocking for deterministic and reliable assertions.
* Update Elsa.sln
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` for consistent filtering
* Refactor `TenantResolverResult` to support explicit resolved/unresolved state handling
- Updated `TenantResolverResult` to include an explicit `_isResolved` property.
- Adjusted `ResolveTenantId()` and `IsResolved` logic for improved clarity and robustness.
- Simplified tenant resolution invocation in `TenantResolverBase`.
- Removed redundant normalization in `DefaultTenantResolverPipelineInvoker`.
* Normalize tenant ID handling in `DefaultWorkflowDefinitionStorePopulator` and `ClrWorkflowsProvider`.
* Refactor `DefaultWorkflowDefinitionStorePopulatorTests`: streamline object initializations and add tenant-specific test coverage for `PopulateStoreAsync`.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add retry mechanism for distributed locks with transient error handling and logging
- Introduced Polly-based retry pipeline for distributed lock acquisition in `DistributedWorkflowClient` to handle transient errors such as network issues or database connection failures.
- Added detailed logging for retry attempts and lock release errors.
- Updated project dependencies to include Polly.
* Refactor transient exception handling to shared resilience module.
Migrated transient exception detection logic from scheduling module to a new shared resilience module. Updated services, jobs, and features to utilize the centralized `ITransientExceptionDetectionService`. This change improves maintainability and promotes reusability across modules.
* Add unit tests for transient exception detection and resilience strategy evaluation.
- Introduced comprehensive unit tests for `DefaultTransientExceptionDetector`, `ResilienceStrategyCatalog`, `ResilienceStrategyConfigEvaluator`, and `TransientExceptionDetectionService`.
- Added helper classes and test data factories to facilitate reusable test patterns for resilience modules.
- Updated solution to include `Elsa.Resilience.Core.UnitTests` project.
* Add component tests for distributed lock resilience
- Introduced new tests to verify retry behavior during transient lock acquisition and release failures.
- Added `TestDistributedLockProvider` and related mocks for simulating transient failures.
- Updated `WorkflowServer` test services to support the new distributed lock test scenarios.
* Refactor distributed lock resilience tests
- Consolidated test logic: streamlined test providers, injected services, and reusable test patterns.
- Simplified `TestDistributedLockProvider` implementation with enhanced initialization and failure simulation.
- Reorganized tests for transient acquisition/release failures to use parameterized `Theory` for improved maintainability.
* Refactor transient exception handling: rename interfaces and classes for consistency, update references across codebase, and improve code readability.
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Simplify WorkflowServer setup and DistributedLockResilienceTests by replacing IDistributedLockProvider with TestDistributedLockProvider.
* Remove unused `using` directives in unit tests to improve code cleanliness.
* Remove `TransientExceptionTypes` helper and inline its usage in tests for improved maintainability.
* Add descriptive `DisplayName` attributes to unit tests for improved test clarity.
* Fix redundant exception checking in TransientExceptionDetector (#7162)
* Initial plan
* Fix redundant exception checking in TransientExceptionDetector
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Extract MaxRetryAttempts constant in DistributedLockResilienceTests (#7164)
* Initial plan
* Extract MaxRetryAttempts constant to eliminate hardcoded magic number
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Make TestDistributedLockProvider thread-safe with Interlocked operations (#7163)
* Initial plan
* Make TestDistributedLockProvider thread-safe using Interlocked operations
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/DistributedLockResilience/DistributedLockResilienceTests.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Resilience.Core/Services/DefaultTransientExceptionStrategy.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Include `CancellationToken` in distributed lock handling methods for improved cancellation support.
* Initial plan (#7168)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* Cache detector list in TransientExceptionDetector to avoid repeated allocations (#7167)
* Initial plan
* Cache detector list in field to avoid repeated allocations
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Use IReadOnlyList instead of List for better intent expression
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Fix TestDistributedLockProvider registration to properly decorate IDistributedLockProvider (#7166)
* Initial plan
* Fix TestDistributedLockProvider registration to use Decorate pattern and fix variable reference bug
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Add runtime check for TestDistributedLockProvider registration
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Refactor `DistributedWorkflowClient` to simplify `Lazy<ResiliencePipeline>` initialization.
* Add integration tests for DistributedWorkflowClient lock resilience (#7165)
* Initial plan
* Fix compilation error: use correct parameter name transientExceptionDetector
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Add integration tests for DistributedWorkflowClient lock resilience
- Add SimpleWorkflow for testing distributed lock scenarios
- Add tests exercising RunInstanceAsync with transient lock failures
- Verify retry logic works correctly with actual workflow execution
- Test both acquisition and release failure scenarios
- Decorate IDistributedLockProvider to use TestDistributedLockProvider
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Address code review feedback
- Add explanatory comment for TestDistributedLockProvider cast
- Remove unnecessary blank line for consistent formatting
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* Simplify transient exception strategy by refactoring message pattern matching logic.
* Enhance distributed lock mock to support per-lock failure configuration and improve resilience tests.
* Refactor `TestDistributedLockProvider` to streamline failure handling logic and improve code clarity.
* Remove unused methods and redundant test case from `DistributedLockResilienceTests`.
* Refactor `DistributedLockResilienceTests` to simplify workflow client creation, consolidate assertion logic, and remove redundant test cases.
* Format `ResilienceStrategyCatalogTests` by removing redundant line breaks in test setup.
* Refactor `TransientExceptionDetectorTests` to simplify test setup, consolidate test cases, and remove redundant logic.
* Handle `InvalidOperationException` in `XunitLogger` to suppress logging errors during inactive tests.
* Update workflows to use .NET 10 and adjust resilience tests project configuration.
* Refactor distributed runtime feature and integrations to improve resilience handling, configure services fluently, and add cancellation safeguards in background services.
* Update `base_version` to `3.7.0` in GitHub workflow configuration.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
* Add support for activity host registration across workflows
Introduced new APIs and updates to enable registering custom activity hosts in the workflow management system. This includes modifications to attributes, service registrations, and extensions to streamline integration for advanced activity hosting scenarios.
* Remove unused `using` directives across Workflow Management module
* Add support for host method activity registration and description
Introduce new APIs to enable activity registration from public async methods (Task/Task<T>) on CLR types. Includes `HostMethodActivitiesOptions`, `HostMethodActivity`, `HostMethodActivityProvider`, and `HostMethodActivityDescriber` for dynamic activity generation and execution.
* Refactor host method activity execution and cleanup.
Reworked `HostMethodActivity` to support resumable workflows, improved parameter handling with pluggable value providers, and removed obsolete `AgentExecutionContext`. Enhanced method resolution, async handling, and input/output descriptor logic for better flexibility and maintainability.
* Refactor `Bookmark` model to use mutable properties and update XML documentation.
* Refactor `BookmarkExecutionContextExtensions` to improve structure, add `GenerateBookmarkTriggerToken` method, and enhance maintainability.
* Add extensibility for host method parameter binding with pluggable value providers
Introduced `IHostMethodParameterValueProvider` interface for custom parameter resolution, along with `DefaultHostMethodParameterValueProvider`, `DelegateHostMethodParameterValueProvider`, and `FromServicesAttribute` for flexible binding options. Enhances host method activity execution by supporting DI resolution and workflow input handling.
* Refactor nullable usage and improve bookmark management logic
Updated null assignment for consistency across files and refined logic for detecting and handling newly added bookmarks. Adjusted method signatures and parameters in the DecoratedStoryWriterAgent class for more explicit input handling. These changes enhance code readability, maintainability, and robustness.
* Update src/modules/Elsa.Workflows.Management/Activities/CodeFirst/HostMethodActivityProvider.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Ensure `CallbackMethodName` is set and skip bookmarks with empty values
* Update src/modules/Elsa.Workflows.Management/Features/WorkflowManagementFeature.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Workflows.Management/Contracts/IHostMethodActivityDescriber.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Workflows.Core/Attributes/InputAttribute.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor `CodeFirst` namespace to `HostMethod` for improved clarity and align with updated activity execution logic. Enhance DI-based parameter resolution and update XML documentation for `HostMethodActivitiesOptions`.
* Add `Penguin` activity host with sample activity methods and register in Elsa pipeline
* Add `TestHostMethod` activities and corresponding component tests. Register `TestHostMethod` as an activity host in the workflow server.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Add Docker logs capturing step in GitHub Actions workflow for better debugging.
- Update PostgreSQL test container to use `postgres:16-alpine` with enhanced configuration options.
- Set database connection `Max Pool Size` to 20 in component tests.
* Add component tests for concurrent trigger indexing to prevent duplicate trigger registration in multi-engine workflows (#7130)
* Pin `dotnet-ef` version in `.config/dotnet-tools.json` to address migration bug, update EF Core script modules, and enhance `DesignTimeDbContextFactoryBase` with improved option description.
* Add migrations for EF Core SQLite and MySQL to include unique indexing on triggers.
* Update ConcurrentTriggerIndexing test to verify unique constraint enforcement and throw `DbUpdateException` for duplicate triggers
* Update src/modules/Elsa.Persistence.EFCore/Modules/Runtime/TriggerStore.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/modules/Elsa.Persistence.EFCore/Modules/Runtime/TriggerStore.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update EF Core script to include `Management` module and restore `Sqlite` provider in migration process
* Rename `SimpleHttpApiWorkflow.cs` to `HttpWorkflow.cs` in ConcurrentTriggerIndexing test.
* Remove 'locks' folder from project
Removed the 'locks' folder from the project structure.
* Clean up migration script by removing redundant column type changes
* Update EF Core Oracle migrations to change NVARCHAR2 fields to NCLOB for larger data storage
* Update EF Core Oracle migrations to use NCLOB for larger data storage
* Remove retry logic and logger dependency from EFCoreTriggerStore in TriggerStore implementation
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
