Replaced direct implementation of `IPropertyUIHandler` with a new `PropertyUIHandlerBase` abstract class to simplify UI handler management and introduce a priority mechanism. Removed outdated `SampleWorkflow` and `SlowActivity` code to clean up the repository. Introduced a custom checklist options provider and registered it within the application services.
* Update default initializations and input parameters to `null`
Replaced `default!` with explicit `null` for input parameters and properties throughout various classes. Adjusted constructors' default values for consistency and readability. This change ensures better clarity and alignment with nullable reference types.
* Add Polly-based resilience integration for retry tracking
Introduce Polly diagnostics to log retry events in the execution context. Updated resilience strategy interfaces and implementations to support Polly's context and retry event tracking.
* Refactor resilience and retry handling, add flaky endpoint.
Removed custom Polly-based diagnostic listeners and observers in favor of a transient status code utility class. Introduced a mock "flaky" endpoint for testing failure scenarios and updated configuration for resilience strategies. Minor namespace fixes
* Add retry attempt recording to resilience feature
Introduce `IRetryAttemptRecorder` and its implementations to enable recording of retry attempts during activity execution. Updated `ResilientActivityInvoker` to persist retry attempts and modified `ResilienceFeature` to support configurable retry attempt recorders.
* Add retry attempt tracking and retrieval functionality
Introduced mechanisms to track and fetch retry attempts, including new interfaces, reader implementations, API endpoints, and related models. These enhancements improve resilience tracking and data access for activity execution across workflows.
* Add GetOutcome method to RetryAttempt model
Introduce a GetOutcome method to encapsulate logic for determining the retry attempt's outcome. It prioritizes the Result, falls back to the Exception message, or defaults to "Unknown" if neither is available. This improves clarity and reusability of the outcome evaluation.
* Add scoped registration for _retryAttemptReader
This change ensures that _retryAttemptReader is registered in the DI container as a scoped service.
* Refactor retry mechanism to support detailed retry metadata
Introduced a `CollectRetryDetails` method to `IResilientActivity` for enhanced retry data collection. Updated `RetryAttemptRecord` to include a `Details` dictionary for capturing metadata, replacing previous `Result` and `Exception` fields. These changes simplify the retry recording process and improve extensibility for tracking retry details across activities.
* Add support for capturing background activity properties
Introduced functionality to capture and persist background activity properties during workflow execution. This includes defining a key for properties, capturing them in middleware, and storing them in the workflow execution context. These changes ensure properties are handled consistently alongside other activity data.
* Add support for storing and propagating activity execution properties
Introduced a `Properties` dictionary to track additional metadata in activity execution records and stats, enabling richer diagnostics and tracing. Refactored resilience logic to improve retry handling and propagate retry-related flags in workflows. Enhanced database queries to map serialized properties for execution summaries.
* Add retry propagation for background activity execution
Introduced a mechanism to propagate the retry-attempted flag across activity execution contexts. Added a new notification `BackgroundActivityExecutionCompleted` and updated related middleware to send this notification. Enhanced resilience features to handle and propagate retry state effectively.
* Refactor default parameters and values to use 'null'.
Replaced 'default' with 'null' for optional parameters and values in `AddExecutionLogEntry`, improving clarity and ensuring semantic consistency with nullable types. No functional changes were introduced.
* Refactor flaky endpoint and enhance resilience support.
Replaced the "Flaky" endpoint with a more robust "SimulateResponseEndpoint" under a new module. Introduced a status code lookup utility and improved resilience strategies with configurable backoff types. Updated serialization to support enum conversions and enhanced caching behavior for response simulation.
* Update activity execution models with nullable properties
Replaced `default!` initializations with `null!` to ensure correct handling of nullable string properties in `ActivityExecutionRecord`. Added a new `Properties` dictionary to `ActivityExecutionRecordSummary` to store additional activity execution data. This enhances model flexibility and data extensibility.
* Add support for recording resilience strategy in context
Introduced a new method to store resilience strategy details in the activity execution context for enhanced diagnostics. Updated `ResilientActivityInvoker` to serialize and set the resilience strategy using this method, leveraging `JsonSerializer`.
* Remove redundant PropertyNamingPolicy assignment
The PropertyNamingPolicy was set to the default value (CamelCase), making the assignment unnecessary. This change simplifies the code while maintaining existing functionality.
* Set JSON property naming policy to camelCase
Updated JSON serialization settings to use camelCase naming for property names. This improves consistency with standard JSON naming conventions and ensures compatibility with camelCase-based APIs.
* Remove unused Endpoints folder reference from project file
The Endpoints folder reference in the project file was unnecessary and has been removed. This cleanup helps maintain a tidy and accurate project structure.
* Remove unused RetryAttemptFilter and add Polly packages
Removed the obsolete RetryAttemptFilter class as it was no longer in use. Added Polly and Polly.Extensions packages to the project to support resilience and fault-handling strategies. This update aligns with keeping dependencies relevant and reducing unused artifacts.
* Add resilience integration test for FlowSendHttpRequest (#6692)
* Refactor and fix resilience test cases for clarity and accuracy
Simplified imports, adjusted code structure, and corrected attempt indexing logic in resilience tests. These changes improve readability, maintainability, and ensure accurate validation of retry attempts in test scenarios.
Updated default property values across activities to use `null!` instead of `default!`, ensuring clearer nullability intent. Also adjusted constructor parameter defaults to `null` for consistency.
* Await HTTP resume callbacks
* Update HttpEndpoint.cs
* Update HttpEndpointActivityExecutionContextExtensions.cs
* Refactor to use `WaitForHttpRequestAsync` consistently.
Replaced `WaitForHttpRequest` with `WaitForHttpRequestAsync` across the codebase to align with asynchronous naming conventions. This change improves clarity and maintains consistency in method naming.
* Add 'codex/*' branch to workflow path filters
This update ensures that actions in the workflow are triggered for changes in branches following the 'codex/*' naming convention. It aligns the path filters with the branching strategy and improves CI/CD coverage.
* Add resilience module with core interfaces and services
Introduced a new `Elsa.Resilience` module and its core components to support resilient services and activities. This includes resilience strategies, providers, and attributes, along with integration into the existing HTTP module for enhanced fault tolerance. Added solution and project references for proper dependency management.
* Add resilience strategy framework with HTTP strategy support
Introduced a resilience strategy architecture, including a configurable `HttpResilienceStrategy` with retry capabilities, strategy serialization, and integration with existing modules. Enhanced ResilienceFeature to support registration of strategy types and updated application configuration to enable resilience strategies. This change ensures more robust and fault-tolerant HTTP request handling.
* Add JSON serialization support for resilience configuration
Introduced `ConfigurationExtensions` to enable JSON serialization of configuration sections. Updated resilience strategies to utilize the new extension methods and adjusted JSON serialization logic to support polymorphism with `$type` discriminator. Minor modifications were made to support deserialization and property mutability.
* Add resilience strategy support to workflows and API clients
Introduced resilience strategy configuration, serialization, and execution support across workflows and API clients. Added new APIs, models, and services to enhance fault tolerance capabilities for activities and HTTP interactions.
* Refactor resilience services for improved modularity.
Replaced `IResilienceService` with new modular interfaces (`IResilienceStrategyCatalog`, `IResilienceStrategyConfigEvaluator`, `IResilientActivityInvoker`) and corresponding implementations. Enhanced maintainability by simplifying components and responsibilities, ensuring better separation of concerns.
* Rename methods in ResilienceStrategyCatalog for clarity
Updated method names in `ResilienceStrategyCatalog` and its interfaces for better readability and alignment with naming conventions. Replaced `GetAllStrategiesAsync` with `ListAsync` and `GetStrategyAsync` with `GetAsync` across the codebase.
* Refactor resilience handling in HTTP activities.
Replaced `ResilienceCategory` property with `ResilienceCategoryAttribute` for a cleaner implementation. Updated `IResilientActivity` to simplify its interface and adjusted related modifications accordingly. Introduced `IResilientActivityInvoker` to enhance resilience strategy execution.
* Add support for additional resilience and scripting features
Extended resilience strategy handling with serialization support, added `HttpResilienceStrategy` type in JavaScript handler, and refined object conversion logic for interfaces. Minor adjustments to `Expression` class properties for consistency.
* Remove `UseResilience` call from Program.cs
This call was redundant and no longer necessary for the application. Its removal simplifies the code and ensures only required middleware is used.
* Remove commented-out JSON converter code in serializer setup
Cleaned up unused and commented-out converter initialization code in `ResilienceStrategySerializer`. This improves readability and removes unnecessary clutter from the file.
* Fix typo in method names from 'Resiliency' to 'Resilience'
Renamed methods to maintain consistency in naming conventions across the codebase. This change ensures clarity and alignment with established terminology.
* Refactor namespace for ConfigurationResilienceStrategySource
Updated the namespace of ConfigurationResilienceStrategySource to "StrategySources" for better alignment with naming conventions and structure. Removed an unused namespace reference in ResilienceFeature for cleanup.
* Mark EnableResiliency as obsolete in SendHttpRequestBase.
The EnableResiliency property is now marked with the [Obsolete] attribute. Developers are encouraged to use the common Resilience Strategy setting instead for managing HTTP request resiliency. This change ensures better consistency and alignment with the broader resilience strategy.
* Restrict ResilienceCategoryAttribute to class targets only
Removed support for using ResilienceCategoryAttribute on properties. This change enforces a stricter and more focused usage of the attribute, ensuring it applies only to class-level declarations.
* Add documentation for IResilientActivityInvoker interface
Include summaries and parameter descriptions for the `InvokeAsync` method. This improves code clarity and helps developers understand the functionality and usage of the resilient activity invocation process.
* Fix logical operator precedence in type comparison check
Parentheses were added to ensure correct evaluation of conditions when checking type compatibility. This prevents potential logical errors when determining the target type in object conversions.
* Add support for resilience source identification
Introduce the `ResilienceSourceNameAttribute` to allow naming of resilience sources. Updated `ResilienceStrategyCatalog` to utilize the attribute for prefixing strategy IDs, improving source identification and traceability. Applied the attribute to `ConfigurationResilienceStrategySource` as an example.
* Revert "Add support for resilience source identification"
This reverts commit 19b4e7121d6330b5de4f692b78da4c1e4a2d1f67.
* Reapply "Add support for resilience source identification"
This reverts commit 8bcba9d040c4eb247077aec6d90dc02817adcbd5.
* Revert "Reapply "Add support for resilience source identification""
This reverts commit ee04d35e7930956c752dda3ed150ca34a535e66c.
Updated the class properties' default initializers from `default!` to `null!` for better clarity and null handling. This ensures consistency in initialization and aligns with modern C# practices.
* Refactor bookmark naming to use "Name" instead of "ActivityTypeName".
Replaces usages of "ActivityTypeName" with "Name" across relevant classes, filters, and database mappings for clarity and consistency. Maintains backward compatibility where necessary and updates corresponding indices, filters, and methods for proper functionality.
* Add migration for V3_5 with schema updates for EF Core
This migration modifies the `Triggers` and `Bookmarks` tables by adding the `Name` column, updating its nullable state, and creating corresponding indexes. Changes apply to both SQLite and MySQL contexts, ensuring compatibility across databases.
* Refactor bookmark filtering and tenant events handling.
Unified the bookmark filtering logic with overloads that accept multiple names, improving flexibility and reusability. Simplified object initializations in tenant events to enhance code readability and reduce verbosity. Added task continuation logic to background task execution for better task sequencing and error handling.
* Refactor bookmark creation to use target-typed `new` expressions.
Replaces explicit `CreateBookmarkArgs` instantiations with concise target-typed `new` expressions for improved readability and reduced redundancy. This does not alter functionality but simplifies the code structure.
* Make callback parameters optional and adjust OTEL settings
Updated methods to allow optional callbacks for improved flexibility. Refactored Delay activity to reuse helper methods. Adjusted OTEL instrumentation settings to enable console exporter and disable manual instrumentation.
* Refactor stimulus handling and streamline event workflows.
Introduces `WaitForEvent` and `GetEventInput` extensions to improve ActivityExecutionContext usability. Replaces generic filter methods with targeted single-name filtering, simplifying scheduling logic. Updates stimulus names for consistency and deprecates obsolete methods to enhance clarity and maintainability.
* Refactor `Event` activity handling and event stimulus logic.
Replaces inline event stimulus creation with a new `GetEventStimulus` helper method for cleaner code. Simplifies event execution handling by removing redundant logic in `ExecuteAsync`. Extends `WaitForEvent` to handle workflow triggers more efficiently.
* Refactor and enhance Timer and Delay execution logic
Introduced `TimerBase` for shared timer functionality and refactored `Timer` to extend it. Improved method names for clarity, replacing `ResumeIn`/`ResumeAt` with `DelayFor`/`DelayUntil`. Enhanced flexibility in bookmark handling and activity execution context extensions.
* Add custom activities and refactor HTTP stimulus handling
Introduce new custom activities (CustomDelay, CustomEvent, CustomHttpEndpoint, CustomTimer) to enhance workflow functionality. Refactor HTTP stimulus handling by replacing activity type names with a new centralized HttpStimulusNames constant, improving consistency and maintainability. Additionally, streamline HTTP endpoint logic with new helper extensions and simplify related services to reduce redundancy.
* Add HttpEndpointBase abstraction to simplify HTTP endpoints
Introduce a new `HttpEndpointBase` class to centralize common logic for HTTP endpoint activities. Refactored `CustomHttpEndpoint` to inherit from this new base class, reducing redundancy and improving maintainability.
* Refactor events framework with base class for event activities
Introduce `EventBase` to streamline implementations of event-driven activities. Updated `CustomEvent` to inherit from `EventBase`, reducing duplicate logic and improving maintainability. Removed unnecessary dependencies in `CustomTimer`.
* Refactor bookmark creation to use target-typed `new` expressions.
Replaces explicit `CreateBookmarkArgs` instantiations with concise target-typed `new` expressions for improved readability and reduced redundancy. This does not alter functionality but simplifies the code structure.
Make callback parameters optional and adjust OTEL settings
Updated methods to allow optional callbacks for improved flexibility. Refactored Delay activity to reuse helper methods. Adjusted OTEL instrumentation settings to enable console exporter and disable manual instrumentation.
Refactor stimulus handling and streamline event workflows.
Introduces `WaitForEvent` and `GetEventInput` extensions to improve ActivityExecutionContext usability. Replaces generic filter methods with targeted single-name filtering, simplifying scheduling logic. Updates stimulus names for consistency and deprecates obsolete methods to enhance clarity and maintainability.
Refactor `Event` activity handling and event stimulus logic.
Replaces inline event stimulus creation with a new `GetEventStimulus` helper method for cleaner code. Simplifies event execution handling by removing redundant logic in `ExecuteAsync`. Extends `WaitForEvent` to handle workflow triggers more efficiently.
Refactor and enhance Timer and Delay execution logic
Introduced `TimerBase` for shared timer functionality and refactored `Timer` to extend it. Improved method names for clarity, replacing `ResumeIn`/`ResumeAt` with `DelayFor`/`DelayUntil`. Enhanced flexibility in bookmark handling and activity execution context extensions.
Add custom activities and refactor HTTP stimulus handling
Introduce new custom activities (CustomDelay, CustomEvent, CustomHttpEndpoint, CustomTimer) to enhance workflow functionality. Refactor HTTP stimulus handling by replacing activity type names with a new centralized HttpStimulusNames constant, improving consistency and maintainability. Additionally, streamline HTTP endpoint logic with new helper extensions and simplify related services to reduce redundancy.
Add HttpEndpointBase abstraction to simplify HTTP endpoints
Introduce a new `HttpEndpointBase` class to centralize common logic for HTTP endpoint activities. Refactored `CustomHttpEndpoint` to inherit from this new base class, reducing redundancy and improving maintainability.
Refactor events framework with base class for event activities
Introduce `EventBase` to streamline implementations of event-driven activities. Updated `CustomEvent` to inherit from `EventBase`, reducing duplicate logic and improving maintainability. Removed unnecessary dependencies in `CustomTimer`.
* Move HttpEndpointOptions model to its own file
The HttpEndpointOptions class was moved from an extension file to its own dedicated file for better organization and modularity. This model defines HTTP endpoint properties such as path, methods, authorization, policies, request timeout, and size limit. The change improves code clarity and structure.
* Fix unnecessary whitespace in Timer.cs
Removed an extra whitespace line in the Timer.cs file to maintain code formatting consistency. No functional changes were made to the code.
* Remove extraneous whitespace in IStimulusSender.cs file
Eliminate unnecessary blank line in the IStimulusSender interface for improved code cleanliness. This change enhances readability and aligns with coding standards.
Refactored `TriggerIndexingContext` to enhance immutability and streamline trigger handling logic. Added support for generating trigger names dynamically and improved bookmark creation in HTTP workflows. Removed unused Docker Compose file and updated the base version in the build workflow to `3.5.0`.
Fixes#6512
Refactor the bulk delete endpoint to include a new primary route while deprecating the old one. Expand the request model to support workflow definition filters and adjust response instantiation for clarity. Update obsolete API route prefix guidance for consistency across modules.
Removed outer timeout since it was not working in the current set-up
Reduced maximum retry attempts to reduce the maximum amount of time spent on the request
Replaced specific transient status code check for HttpRequestException with a more generalized handling approach. This ensures all HTTP exceptions are retried, improving robustness and simplifying the logic.
Updated comments to correct total delay and grace period calculations for clarity and accuracy. This ensures consistency in expected retry behavior and improves maintainability.
Added a link to the Polly retry strategy documentation for clarity and future reference within the resiliency pipeline builder code. This improves code maintainability and helps developers quickly access relevant information.
Lowered MaxRetryAttempts from 6 to 4 and reduced the outer timeout to 60 seconds to align with the updated exponential backoff total of 32 seconds. These changes aim to improve efficiency and reduce unnecessary waiting time during transient failures.
Updated the retry logic by increasing the max retry attempts from 4 to 6 and reducing the delay per attempt to 1 second. Adjusted the outer timeout to accommodate the new backoff configuration, allowing for a longer retry grace period.
Increased max retry attempts to 4, removed jitter, and adjusted delay and backoff settings for clearer and more predictable behavior. Extended outer timeout to align with retry duration and added 409 Conflict to transient errors. Simplified logic for identifying transient network failures.
Cleaned up extra blank lines in the code to improve readability and maintain consistent formatting. These changes do not impact functionality or behavior of the code.
Introduce configurable resiliency mechanisms for HTTP requests, including retries, circuit breakers, and timeouts, leveraging Microsoft.Extensions.Resilience and Polly. Refactor `SendHttpRequestBase` to include an `EnableResiliency` input and encapsulate resiliency logic in a dedicated pipeline. Update project references to include necessary dependencies.
Replaced mediator requests and commands with direct store interactions in the workflow management module. Simplified service dependencies, eliminating unnecessary handlers and requests, and streamlined workflow-related operations for better maintainability and performance.
Reorganized workflow handling to enhance clarity and maintainability, including introducing dedicated request handlers, refactoring middleware, and improving definition publishing. Deprecated `ValidateWorkflowResponse` and updated relevant namespaces for notifications handlers.
Simplified route normalization logic by removing unnecessary lowercase conversion and adjusted usages accordingly. These changes improve code readability, maintain consistency and fixes that route values are stored lower-cased instead of original-cased.
* Work in progress: Add DefaultTenantService for tenant management
Introduce `DefaultTenantService` and its corresponding interface `ITenantService` to manage tenant operations such as finding, getting, and listing tenants. Update `MultitenantBackgroundService` to utilize `DefaultTenantService` for handling tenant lifecycle events. This enhancement standardizes tenant operations and improves the maintainability of the multitenancy feature.
* WIP
* Add multitenancy event handlers and task interfaces
Implemented new interfaces IBackgroundTaskStarter and ITaskExecutor to manage task lifecycle events efficiently. Introduced new classes such as RunBackgroundTasks, RunStartupTasks, and StartRecurringTasks for handling tenant activation and deactivation events. Modified TaskExecutor to implement these interfaces and adjusted tenant registration logic to invoke these new handlers.
* Refactor multitenancy and task management services.
Remove background and recurring task runners, and integrate tenant activation and deactivation into the multitenancy feature. Enable multitenancy in the server application and create a new service for tenant activation and deactivation. This refactor simplifies the management of tenant-specific tasks and enhances the modularity of the platform.
* Refactor background service to use startup tasks
Replaced hosted service implementation with startup tasks for executing multi-tenant tasks and EF Core migrations. Introduced `PriorityAttribute` to manage task execution order, ensuring migrations run before other services that require database access. This simplifies tenant activation with an ordered task execution and removes redundant classes.
* Refactor MultitenancyFeature service registrations
Reorganized service registrations for better clarity and maintainability. Changed the registration of some services to use factory delegates for retrieving existing services to ensure correct dependencies. This refactor improves the flexibility of the tenant lifecycle event handling.
* Update V3_3 migration files
* Add tenant management endpoints and enhance tenant handling
Implemented tenant management endpoints including Add, Get, List, and Update. Enhanced tenant handling by introducing configuration and store-based providers, and improved error logging for tenant updates. Adjusted various internal functionalities to better support multitenancy features through different persistence providers.
* Implement tenant deletion endpoint and refactor migration setup.
Introduce a new API endpoint to handle tenant deletions while providing appropriate responses based on successful or unsuccessful attempts. Refactor migration handling by replacing startup tasks with hosted services across various modules to streamline the migration execution process.
* Add and integrate ConfigurationJsonConverter
Introduce a `ConfigurationJsonConverter` to handle JSON serialization and deserialization of `IConfiguration` objects. This change centralizes configuration serialization logic, leading to cleaner and more maintainable code. Updated various parts of the codebase to use the new serialization utility, ensuring a consistent approach throughout the application.
* Refactor JSON conversion and update tenant endpoint.
Removed unused workflow references and streamlined JSON handling in `ConfigurationJsonConverter`. Simplified tenant ID handling by removing `IIdentityGenerator` and setting a default value for `UpdatedTenant.Id`.
* Add logging for cancelled recurring tasks
Integrated ILogger to log information when a recurring task is canceled due to an OperationCanceledException. This change enhances troubleshooting by providing clearer insights into task cancellations and their underlying reasons, improving maintainability and observability of the task execution process.
* Disable multitenancy support and adjust default Tenant ID.
Multitenancy is now disabled by setting 'useMultitenancy' to false in the configuration. Additionally, the default Tenant's ID has been changed from null to an empty string to prevent potential null reference issues.
* Remove MultitenantHostedService abstraction file
The MultitenantHostedService.cs file was removed as it is no longer necessary. Its responsibilities have likely been refactored or integrated into another service, indicating a simplification or restructuring of the multitenancy handling in the codebase.
* Rename PriorityAttribute to OrderAttribute for clarity.
This change improves the clarity of the code by renaming PriorityAttribute to OrderAttribute, reflecting its actual purpose. All occurrences of the attribute in the codebase have been updated accordingly to maintain consistency. This makes the intent of the code more understandable for future maintenance and development.
* Fix message key retrieval in ProduceMessage activity
Update the ProduceMessage activity to use GetOrDefault for retrieving the message key. This change ensures that a null key is used if no explicit key is provided or if the key is empty or whitespace, preventing potential errors during message production.
* Refactor multitenancy and scheduling services.
Removed DefaultTenantContextInitializer interface and class, refactored tenant activation/deactivation to use try-catch logging, and updated tenant context handling to use IDisposable for context push. New activities and workflows added in Elsa.Server.Web, and scheduling services enhanced to schedule jobs with explicit job keys and groups. Also, adjusted configurations to enable multitenancy, providing improved maintainability and flexibility.
* Remove Example1 activities and disable multitenancy
Deleted Example1Activity, Example1Workflow, and FirstActivity classes to clean up unused code and simplify the codebase. Disabled multitenancy by setting useMultitenancy to false, likely to streamline configuration and resource utilization.
* Fix and normalize URL path concatenation.
Ensure that the base URLs in both base path providers consistently end with a forward slash. This normalization prevents potential issues with endpoint routing and path concatenation, improving overall URL construction robustness.
Updated the URL trimming logic to ensure consistent handling of base URLs and paths, removing trailing slashes where necessary. This change improves the consistency and reliability of URL generation.
* Add Kafka module with integration and example setup
Introduced the Kafka module providing consumer integration and activities into the project. This includes new classes for consumer handling, configuration, and activities. An example setup using Docker Compose is also added to facilitate development and testing.
* Refactor KafkaTransportMessage to inline Timestamp namespace
Simplify the namespace usage for the Timestamp type within the KafkaTransportMessage record. This change eliminates the need for a separate using directive for Timestamp, enhancing code readability and maintainability.
* Add support for handling Kafka transport messages
This commit introduces the capability to handle and trigger workflows based on Kafka transport messages. It adds a new handler, notifications, and updates the message stimulus to include correlating fields. Additionally, the Kafka consumers are now managed more modularly with updated startup tasks and mediator integration.
* Enable Kafka integration and fix Kafka options naming
Added support for Kafka integration in Elsa.Server.Web by setting up Kafka configurations in appsettings.json and updating Program.cs. Also, renamed `ConsumerConfigs` to `ConsumerDefinitions` in Kafka options for clarity.
* Add consumer definition enumeration and dropdown support
Introduced `IConsumerDefinitionEnumerator` for managing consumer definitions across providers and implemented in `ConsumerDefinitionEnumerator` class. Enhanced `KafkaFeature` to register these services and updated the `MessageReceived` activity to use a dropdown UI hint for consuming definitions. Improved `StartConsumersTask` by refactoring consumer definition retrieval logic.
* Add SendMessage activity and refine Kafka messaging
Introduce a new SendMessage activity for Kafka, enabling message publishing to specific topics. Refine KafkaTransportMessage model by removing headers and timestamp fields. Adjust the StimulusSender logic to streamline the bookmark queuing process and fix key-value pairing in dropdown options. Update appsettings for corrected Kafka bootstrap server and topic configurations.
* Add producer and topic management support
Introduced interfaces and implementations for managing producer and topic definitions along with their respective enumerators and list providers. Updated `SendMessage` activity to include producer selection and refactored consumer definition providers for better consistency.
* Refactor Kafka configuration property names
Renamed Kafka configuration properties for better consistency and readability across the codebase. Updated property names from `ProducerDefinitions` to `Producers`, `ConsumerDefinitions` to `Consumers`, and `TopicDefinitions` to `Topics`. Added missing input attribute in `SendMessage.cs` and registered additional handlers in `KafkaFeature.cs`.
* Add custom serializers for Kafka message handling
Introduced `DefaultSerializers` class for custom serialization and deserialization of Kafka messages. Updated `MessageReceived` and `SendMessage` activities to use these custom serializers, and modified `KafkaOptions` to include them.
* Fix ExpandoObject serialization method parameter
Changed the serialization type from `ExpandoObject` to the actual type of the object to ensure proper serialization. This ensures that derived types are correctly handled during the serialization process.
* Add Producer and Consumer workflows for Kafka
Introduced two new workflows: `ProducerWorkflow` and `ConsumerWorkflow` for handling Kafka messages. Updated `DefaultSerializers` to use camelCase property naming and modified `appsettings.json` to include `topic-2` and format entries.
* Add JSON serialization to log output in ConsumerWorkflow
This change enhances the log output by serializing messages to JSON format before writing them. The addition of System.Text.Json ensures that the message content is presented in a structured and standardized format in logs.
* Add correlation strategies and update Kafka features
Implemented HeaderCorrelationStrategy and NullCorrelationStrategy, and updated KafkaFeature to support customizable correlation strategies. Added correlation ID handling to Kafka transport messages and updated config and handlers accordingly.
* Add tenant accessor to ConsumerDefinitionWorkflowContextProvider
Integrated ITenantAccessor to the provider to support tenant-specific context loading. Updated the constructor and LoadAsync method to retrieve the tenant information and use it for context-specific operations.
* Switch to MySQL and disable Kafka
This commit changes the SQL database provider from SQLite to MySQL and disables Kafka use. It also includes necessary adjustments such as adding MySQL handling in configuration and connection setups, updating `docker-compose` to include MySQL services, and referencing MySQL projects in the `.csproj` file.
* Add UI property handlers to multiple features
This commit introduces various UI property handlers across several features such as Python, JavaScript, CSharp, and Workflow features to enhance user interface property handling. It also updates the property UI handler resolution logic to better manage cases where providers are not available. Furthermore, adjustments were made in the server configuration to switch database providers and enable Kafka.
* Refactor property UI handler retrieval logic
Modified the logic to fetch property UI handlers by preloading them into a list and then filtering. This change improves readability and potentially performance by reducing repetitive service provider calls.
* Incremental work on Kafka workers and predicate evaluation
* Merge BookmarkInvoker with BookmarkResumer
* Register IWorkerManager
* Change lifetime scope of WorkerManager to Singleton
* **Introduce topic subscription handling for Kafka workers**
Added `IWorkerTopicSubscriber` interface and its implementation for managing topic subscriptions. Enhanced workers to bind triggers and bookmarks dynamically based on existing data. Updated several classes and methods to support topic-based subscriptions and headers.
* Refactor trigger matching logic.
Extract trigger matching conditions into `IsMatchAsync` method for reuse. This enhances code maintainability and readability by reducing redundancy. The new `GetTopic` helper method isolates the topic retrieval logic.
* Add Name property to MassTransitActivityTypeProvider
This commit inserts the Name property in the returned object within the MassTransitActivityTypeProvider class. It ensures that the typeName is included, providing a clearer definition of the activity type.
* Add handling for deleted bookmarks and refactor bookmark removal
Added a new event handler for `BookmarksDeleted` to ensure removed bookmarks are processed correctly. Refactored the bookmark removal logic into a helper method to reduce code duplication and streamline the workflow.
* Switch to asynchronous bookmark queue processing
Refactored the `TriggerWorkflows` handler to use `IBookmarkQueue` instead of directly invoking the `IBookmarkResumer`. This change aims to improve scalability by queueing bookmark resumption requests, enabling better load distribution and async processing. Added necessary helpers and configuration options to support this functionality.
* Remove unused IBookmarkResumer dependency
Simplify the constructor by removing the unused IBookmarkResumer dependency. This cleanup reduces potential confusion and improves code maintainability without impacting functionality.
* Add support for local message processing
Introduced an `IsLocal` property to `MessageReceivedStimulus` for determining if the message event is local to a specific workflow instance. Updated `BookmarkBinding` and related handler methods to utilize `CorrelationId` for local event matching. Removed unused `CorrelatingFields` from `MessageReceived` activity.
* Add nullability checks to IWorker retrieval methods
Updated `GetWorker` methods to return nullable `IWorker` to handle cases where a worker might not exist. Modified code to include null checks and conditional operations to prevent potential null reference exceptions when accessing worker methods.
* Add filtering based on activity type name for triggers and bookmarks
This commit introduces filtering for triggers and bookmarks based on the `MessageReceived` activity type name. It also adds an option to mark messages as local in the `SendMessage` activity, where local messages are delivered to the current workflow instance only. These changes help enhance the management and targeted delivery of messages within the workflow framework.
* Add new Kafka topics and clean up producers config
New topics "topic-3" and "topic-4" were added to the Kafka settings. Unused topic references were removed from the producers configuration to simplify and improve clarity.
* Add predicate to KafkaConsumerActivity in ConsumerWorkflow
Introduced a predicate to the KafkaConsumerActivity using JavaScript expressions to filter messages based on OrderId. This ensures only relevant messages are processed in the workflow.
* Add proper summary tags back
* Apply suggestions from code review
Co-authored-by: Robin Sue <robinsue@live.de>
* 69 more files fixed
---------
Co-authored-by: Robin Sue <robinsue@live.de>
Assign order properties to input attributes in `SendHttpRequestBase` and `FlowSendHttpRequest` classes. This change ensures a consistent input display order in the generated UI.
Updated the GetContentAsDictionary method to handle different types of content more gracefully. Changed the type from Dictionary to IDictionary to increase compatibility. Adjusted content serialization to accommodate varying input formats reliably.
* Remove initial migrations
Deleted obsolete initial migration files from multiple databases: MySQL, SQL Server, SQLite, and PostgreSQL. This cleanup helps maintain a streamlined and updated migration history.
* Add Document base class and create tenant-specific indices
Introduced a new abstract `Document` base class to unify common properties. Implemented tenant-specific unique indices across multiple collections by including `TenantId` alongside `Id` to ensure uniqueness within tenant scopes.
* Remove outdated migration files
Deleted various migration files under MySql, PostgreSql, Sqlite, and SqlServer directories. This cleanup removes unnecessary schema definitions and helps to streamline the codebase.
* Refactor workflow identity assignment logic
Streamline workflow identity handling to ensure consistent assignment of Id, DefinitionId, and TenantId values. This change integrates tenant prefix and version suffix cleanly, enhancing clarity and maintainability.
* Enable multitenancy support
Added configuration for a new tenant (tenant-1) in appsettings.json and enabled multitenancy feature in Program.cs. This change allows the application to support multiple tenants, with specific configurations for each.
* Refactor route table update to run as startup task
Replaced `UpdateRouteTableHostedService` with `UpdateRouteTableStartupTask` to ensure route table updates are executed during application startup instead of as a hosted service. Updated configuration in `HttpFeature` and adjusted trigger validation logic in `ValidateWorkflowRequestHandler`.
* Add recurring task scheduling and single-node task support.
Introduce `IntervalExpressionType`, recurring task scheduling classes, and `SingleNodeTaskAttribute`. Update `RecurringTasksRunner` to handle schedules and add single-node task logic to `StartupTasksRunner`. Ensure proper namespace changes and configure sample recurring tasks.
* Refactor recurring tasks scheduling system
Replaced existing scheduling classes with a more modular and granular approach. Introduced new classes and interfaces like `ISchedule`, `CronSchedule`, `IntervalSchedule`, and `RecurringTaskScheduleManager`. Updated related methods and code to comply with the new design.
* Refactor background task management
Removed `ExpiredSecretsHostedService` and refactored it into a recurring task. Introduced `TaskExecutor` for shared task execution logic. Updated and renamed feature classes to better represent their purpose, improving task scheduling and execution management.
* Add BackgroundTask abstract class to Elsa.Common module
This new abstract class implements the IBackgroundTask interface with default methods for executing, starting, and stopping tasks asynchronously. It provides a basic framework for background task management in the Elsa.Common module.
* Switch to CreateAsyncScope in DefaultTenantScopeFactory
Updated the CreateScope method to use CreateAsyncScope instead of CreateScope. This change improves asynchronous handling of service scopes within the DefaultTenantScopeFactory class.
* Add tenant handling and move StartWorkers background task
Introduce ITenantAccessor in Worker class for multitenancy support. Rename and relocate StartWorkers service to BackgroundTask, ensuring smoother workflow initialization. Also, update the configuration to support Azure Service Bus connection string.
* Add tenant support and refactor ProtoActor client
Integrated ITenantAccessor in ProtoActorWorkflowClient class to handle multi-tenancy. Refactored methods in the client to support custom headers and added async disposable pattern in various services for proper resource management. Additionally, enabled Azure Service Bus and updated related documentation.
* Add support for custom headers in ProtoActor grain methods
Introduced a T4 template to generate grain methods with custom headers, enabling the use of tenant ID in requests. Updated `ProtoActorWorkflowClient` to employ these methods, removing redundant code and directly utilizing the client for various workflow operations.
* Add tenant middleware to MassTransit configurations
Introduced multitenancy middleware for MassTransit message handling. Added new message type `OrderReceived` and updated RabbitMQ setup in Elsa Server. Applied middleware to configure tenant data on send, publish, and consume operations.
* Add new product workflow and streamline ID handling
Introduced a new `RequestResponseWorkflow` for handling product requests. Simplified ID handling in `WorkflowBuilder` and `ClrWorkflowsProvider` by defaulting to empty strings and adding a version prefix. Enhanced `HttpWorkflowsMiddleware` to correctly parse full request paths.
* Remove redundant files and update configuration
Deleted unused files `Product.cs` and `RequestResponseWorkflow.cs` to clean up the codebase. Updated `Program.cs` configuration: switched MassTransitBroker to Memory and disabled multitenancy.
* Remove MultitenantRecurringTaskService and update AzureServiceBus
Removed `MultitenantRecurringTaskService` and adjusted related code for Azure Service Bus to work without it. This includes removal of tenant accessor dependency from `Worker` and cleanup of service configuration flags in `Program.cs`.
* Increase signal wait timeout to 10000 milliseconds.
Extended the default timeout for signal awaiting methods from 8000 to 10000 milliseconds. This change ensures more flexible and resilient waiting periods, reducing timeout occurrences in scenarios with longer processing times.
* Refactor scheduling service to be a background task
Renamed `CreateSchedulesHostedService` to `CreateSchedulesBackgroundTask` and refactored it to inherit from `BackgroundTask` instead of `BackgroundService`. Simplified the constructor by injecting the required dependencies directly, eliminating the need for a scoped factory.
* Refactor workflow version suffix formatting
Changed the version suffix format from `:v{version}` to `v{version}` and adjusted the ID concatenation accordingly. This improves consistency and readability of workflow IDs.
* Enable multitenancy support in Quartz scheduler
Added `TenantJobListener` to inject tenant context into jobs. Modified `QuartzWorkflowScheduler` to incorporate tenant IDs into job data maps and adjusted the configuration to acknowledge multitenancy settings.
* Remove ConfigureSchedulerHostedService and TenantJobListener
Consolidated tenant resolution logic into JobExecutionExtensions class. Updated ResumeWorkflowJob and RunWorkflowJob to use the new extension method for tenant retrieval. This simplifies the QuartzSchedulerFeature setup by removing the hosted service configuration.
* Refactor HTTP feature and update route table task
Move 'UpdateRouteTableStartupTask' from 'HostedServices' to 'Tasks' and update dependency injection configurations accordingly. Simplify 'DefaultRouteTableUpdater' by removing unnecessary options and tenant-agnostic settings from filters.
* Disable multitenancy in Program.cs
The useMultitenancy flag has been changed from true to false. This update affects the Elsa.Server.Web application configuration.
* Simplify variable usage in HttpWorkflowsMiddleware
Replaced 'fullPath' variable with 'path' to streamline code. This change enhances readability by reducing redundancy and ensures consistency in variable naming throughout the method.
* Enable multitenancy and refactor tenant handling logic
Enable multitenancy in the application and refactor tenant handling logic to use ITenantFinder and ITenantContextInitializer interfaces. Added header constants, updated middleware to use these interfaces, and moved extension methods to the appropriate namespace.
* Add input validation to user registration form
Implemented checks to ensure all required fields are filled and that input data adheres to format requirements. This change reduces errors and enhances form reliability.
* Remove unused import from TenantPrefixHttpEndpointRoutesProvider
This change cleans up the code by removing an unnecessary import statement. It improves code readability and reduces clutter, making future maintenance easier. The functionality remains unchanged.
* Rename filter scope to "tenantPublish" in Probe method
Updated the Probe method in TenantPublishMiddleware.cs to use "tenantPublish" instead of "tenantSend" for better clarity. Ensures consistency with the method's context and aligns with naming conventions.
* Refactor: Remove extraneous whitespace
Eliminate unnecessary whitespace in ProtoActorWorkflowClient.cs for cleaner code. This change helps maintain consistent formatting and improves readability.
* Refactor DefaultRegistriesPopulator for cleaner initialization
Converted constructor to use read-only fields directly, removing unnecessary instance variables. This change simplifies the code by reducing redundancy and making the constructor cleaner.
Introduce DefaultHttpEndpointBasePathProvider and TenantPrefixHttpEndpointBasePathProvider to manage HTTP endpoint base paths. Update ITenantAccessor property from CurrentTenant to Tenant for consistency across the codebase.
* Refactor: Update namespaces and add TenantExtensions
Updated namespaces throughout the project to improve clarity and consistency by moving from 'Common' to appropriate modules. Added TenantExtensions class to simplify fetching connection strings for tenants.
* Implement multitenant DB connection strings
Redesign tenant-specific classes to support multitenancy more effectively. Introduce `MultitenantBackgroundService` and `MultitenantHostedService` for handling tasks per tenant.
* Refactor constructors and remove redundant code
Simplified the constructor parameters for `MultitenantBackgroundService` and `List` class. Removed the unused parameter in `MultitenantBackgroundService` and redundant folder inclusion in the project file. Updated the method calls to use direct parameters in `List` class.
* Remove unused import in ActivityDescriptors Endpoint
The Elsa.Common.Multitenancy import was removed as it is unused in the List/Endpoint.cs file. Removing unused imports helps to improve code readability and maintainability. This change does not affect functionality.
* Add tenant awareness to bookmark handling and route resolution
Added tenant ID support across various components, including bookmark updates, route resolution, and middleware processing. This ensures that bookmark and route operations can now appropriately handle tenant-specific data, improving the system's multitenancy capabilities.
* Add Multitenant HTTP Routing feature to Tenants module
Introduced a new MultitenantHttpRoutingFeature class to the Elsa.Tenants.AspNetCore module, enhancing the tenant resolution capabilities. Moved RoutePrefixTenantResolver from Elsa.Http to Elsa.Tenants.AspNetCore and updated relevant project references and namespaces accordingly. This refactor improves modularity and separation of concerns between HTTP and tenancy features.
* Refactor route handling and tenant configuration
Removed redundant `RouteTableExtensions` and replaced with new route providers and updaters, enhancing flexibility and modularity. Introduced tenant-specific HTTP endpoint configurations for better customization and configuration management.
* Rename HttpEndpointBookmarkStimulus to HttpEndpointBookmarkPayload
Refactor various classes and methods to reflect the renaming from `HttpEndpointBookmarkStimulus` to `HttpEndpointBookmarkPayload`. Add and configure new extension methods for tenant route handling, update the route provider to support multi-tenancy, and adjust the tenants provider to bind configuration properly.
* Add HeaderTenantResolver and refactor Http namespace.
Introduce HeaderTenantResolver to resolve tenants via HTTP headers. Refactor multiple classes and interfaces to move from the Elsa.Http.Models namespace directly into Elsa.Http for clarity and consistency.
* Add Host-based tenant resolution
Implemented a HostTenantResolver to resolve tenants based on the request's host and updated tenant configurations with host information. Modified the tenant resolver pipeline and added the new host resolver to the service registrations.
* Add tenant-aware caching and accessor support
Enhanced caching by incorporating tenant identifiers into cache keys for more granular cache management. Introduced ITenantAccessor dependencies in various services to retrieve the current tenant information. This ensures that cache entries are correctly isolated per tenant.
* Reorder tenant resolvers for pipeline setup.
Reordered the tenant resolvers in the pipeline to prioritize HostTenantResolver before RoutePrefixTenantResolver. This ensures that tenant resolution is correctly aligned with host-based resolving before checking the route prefix.
* Remove unused imports
This commit eliminates redundant `using` directives across multiple files to streamline the codebase. This cleanup helps improve code readability and maintainability by removing unnecessary dependencies.
* Remove obsolete tenant-related classes and add ASP.NET Core middleware
Refactored tenant resolution by removing obsolete interfaces and classes, such as `IAmbientTenantAccessor` and `ITenantResolutionStrategy`. Introduced new ASP.NET Core middleware for tenant resolution, encapsulated in the new `Elsa.Tenants.AspNetCore` project. Updated related usage in various parts of the application to align with these changes.
* Remove HttpContextTenantResolver.
Removed HttpContextTenantResolver from the multitenancy pipeline and related service registrations. This simplifies the tenant resolution by relying on remaining resolvers like ClaimsTenantResolver and RoutePrefixTenantResolver.
* Add Elsa solution definition file
This commit introduces the main solution file, Elsa.slnx, defining the folder structure, projects, and configuration for the Elsa repository. This includes folders for Docker, documentation, pipelines, samples, scripts, source code, and tests.
* Refactor DefaultAccessTokenIssuer for clarity and efficiency
Refactored the DefaultAccessTokenIssuer class by simplifying its constructor and utilizing scoped variables for token options. Improved token creation logic by adding a dedicated method to configure token options, enhancing code readability and maintainability.
* Remove Elsa.slnx solution file
No dotnet build support yet.
* Refactor tenant resolver service registrations
Updated the service registrations to use interfaces for DefaultTenantResolver and DefaultTenantResolverPipelineInvoker. This improves the code's flexibility, making it easier to replace or extend these implementations in the future.
* Add multitenancy support and tenant scope management
Introduced ITenantScopeFactory and related implementations for tenant scope management across the application. Enhanced the HTTP workflows middleware to handle tenants and updated relevant configurations and extension methods to support tenant resolution.
* Remove unnecessary folder inclusion
The <Folder> tag for "Modules\Modules\" was redundant and has been removed to clean up the project file. This change will not affect the existing functionality or project structure.
* Rename Create to CreateScope and improve authorization.
Updated the method name from Create to CreateScope for better clarity in the TenantScopeFactory. Fixed a logical error in the authorization process, ensuring proper status code setting for unauthorized requests, and refactored token expiration calculation for clarity.
* Add tenant agnostic filters and remove tenant setup
This commit introduces tenant agnostic filters in AutoUpdateTests to ensure workflows can trigger regardless of tenant. Additionally, it removes tenant configuration from WorkflowServer setup as it is no longer required for the current tests.
Replaced hardcoded System.Text.Json version 8.0.4 with a variable $(SystemTextJsonVersion) across multiple project files. This improves maintainability and consistency when updating the package version.
Consolidate imports by replacing Elsa.Common.Contracts with Elsa.Common and Elsa.Common.Multitenancy. This update streamlines import statements across various modules, improving code readability and maintainability.
* Add secret scripting integration for JavaScript
Introduced a new `Elsa.Secrets.Scripting` module that provides secret management capabilities within JavaScript workflows. This includes configuring the Jint engine to use workflow variables, adding new type and variable definition providers, and integrating with existing secret management features.
* Refactor secret name extraction to a separate method
Moved the logic for extracting secret names from the main method to a dedicated private method `GetSecretNamesFromExpression`. This improves code readability and maintains the single responsibility principle by delegating secret name extraction to its own method.
* Add input evaluation, sensitive input handling, and middleware refactor
Introduced methods for evaluating activity input properties and handling inputs marked as sensitive. Refactored `ExecutionLogMiddleware` constructor for consistency. Enhanced `SendHttpRequestBase` to mark authorization inputs as potentially containing secrets. Removed obsolete entries and adjusted persistence logic for clarity.
* Refactor IActivityStateProtector interface
Remove unused using directives and unnecessary comments. Simplify the definition of the `ProtectedActivityStateContext` record.
* Add activity state filtering mechanism
Introduce an abstract filter base class, context, and result models to enable filtering of activity state. Implement a default filter manager to run these filters and apply a specific filter for obfuscating HTTP request headers. Update necessary dependencies and extension methods to integrate the new filtering functionality.
* Add expired secrets management
Implemented services to manage expired secrets by periodically checking and updating their status. Introduced a new hosted service to perform the sweep and configurable options for the sweep interval. Updated related classes and configurations accordingly.
* Update SweepInterval in appsettings.json
Changed the Secrets Management SweepInterval from 30 seconds to 4 hours. This adjustment aims to reduce the frequency of sweep operations and improve overall system performance.
* Update comment to reflect configuring engine with secrets
The comment was changed to better describe the handler's function, specifying that it configures the Jint engine with secrets instead of workflow variables. This clarifies the purpose and usage of the handler in the context of the code.
* Remove unused inputDescriptors variable
This commit removes the inputDescriptors variable, which was declared but never used in DefaultActivityExecutionMapper.cs. This helps in cleaning up the code and potentially reducing memory usage. Ensuring that all declared variables are utilized can improve code readability and maintainability.
Reorder parameters in StartWorkflowAsync and ResumeWorkflowAsync for consistency. Update FindBookmarksAsync and ExecuteWorkflowAsync to include workflowInstanceId and correlationId, improving workflow lookup and execution logic.
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
