Fixes that composite activities' Root property (by means of the `JsonIgnoreCompositeRootConverterFactory`) does not get serialized, just like the `WorkflowDefinitionActivity`.
Composite activities added to a workflow should not include their children when being serialized.
Introduce the ability to pass variables into workflow invocations. Updates include adding variables to workflow execution options, requests, and scheduling logic. This enhancement allows dynamic variable management during workflow execution and instance creation.
Introduced the ability to set and persist custom names for workflow instances. Updated relevant classes, services, and tests to ensure proper handling of the `Name` property.
This commit introduces migration files for multiple database providers (MySQL, SQL Server, PostgreSQL, SQLite, and Oracle) to support schema changes for version 3.4. The migrations primarily involve mapping entity schemas, indexes, and table structures for consistency across all supported database systems.
Introduce `WorkflowHeartbeatMiddleware` to monitor workflow liveness and update timestamps during execution. Extend WorkflowInstanceStore implementations to support updating the `UpdatedAt` timestamp across various data stores. Enhance Dapper query builder and workflow execution pipeline with related functionality.
Introduced a mechanism to identify and restart interrupted workflows. This includes a new `IWorkflowRestarter` contract, its default implementation, and a recurring task for handling restarts. Additionally, updated configurations and added extensions to improve workflow instance filtering and liveness tracking.
Replaced mediator requests and commands with direct store interactions in the workflow management module. Simplified service dependencies, eliminating unnecessary handlers and requests, and streamlined workflow-related operations for better maintainability and performance.
Introduced `ElsaWorkflowInstanceStore` for managing workflow instances, including querying, saving, updating, and deleting. Added associated index provider and schema changes for indexing workflow instances, enabling efficient data retrieval. Updated startup and manifest files to register these new functionalities.
Refactored multiple classes to align with modern C# coding practices such as object initializers and nullable type handling. Added a new base class for checklist dropdown providers to enhance UI configuration extensibility. Simplified constructors for feature classes by adopting record-like syntax, improving readability and maintainability.
Implemented `NewAsync` method in `IWorkflowDefinitionPublisher` for asynchronous workflow creation. Added content item-based workflow definition management, including publishing, retracting, and draft management. Updated workflows API and integrated serialization improvements for better content handling.
Introduced a `ContentItemWorkflowDefinitionStore` and related mappers, extensions, and index updates to manage workflow definitions as content items. Refactored the system to replace existing handlers with a store-based approach and optimized querying and mapping workflows.
Replaced individual parameters with WorkflowDefinitionHandle in workflow-related methods and requests to centralize and simplify logic. Updated related services, models, and handlers to align with this new structure, improving code maintainability and consistency.
Replaced direct store interactions in workflow management with mediator-based requests using `IMediator`, improving consistency and modularity. Added new handlers and requests for workflow definition operations, and updated existing services to align with mediator patterns.
Introduced new filtering capabilities such as `Names`, `Statuses`, `SubStatuses`, and `SearchTerm` for workflow instance filters. Simplified object initialization syntax and removed unnecessary attributes, improving code readability. Added missing dependencies and API service registrations to ensure completeness.
Updated the object initialization syntax for cleaner code and removed an unused import in WorkflowValidator.cs to improve code readability and maintainability.
Introduces a `ValidateWorkflow` notification handler to enforce validation of unique input and output names in workflows. Adds validation errors when duplicate names are detected, improving the integrity of workflow definitions. Integrates the handler into the workflow management feature.
Improved the log message to include specific details about the failed conversion, such as the default value, variable name, and type. This enhances debugging by providing more context about the error.
Refactored the `VariableDefinitionMapper` to utilize dependency injection for `IWellKnownTypeRegistry` and added `ILogger` for improved error handling. Updated value conversion logic to log warnings when conversion fails, ensuring better traceability. Simplified return statements and replaced redundant collections with modern syntax.
Reorganized workflow handling to enhance clarity and maintainability, including introducing dedicated request handlers, refactoring middleware, and improving definition publishing. Deprecated `ValidateWorkflowResponse` and updated relevant namespaces for notifications handlers.
Updated the logic to use `MakeArrayType` for array handling instead of generic collection types, ensuring consistency and better alignment with expected type structures. Adjustments were made in both the `VariableDefinitionMapper` and the `TypeJsonConverter`.
Replaced absolute expiration with sliding expiration for cache entries to ensure the cache is kept active during frequent access. This change optimizes cache behavior and improves resource utilization.
Replaced asynchronous instance creation methods with streamlined synchronous alternatives where applicable. Introduced `CreateAndCommitWorkflowInstanceAsync` for combined instantiation and persistence, along with a separate `CreateWorkflowInstance` method for non-committal instantiation. Refactored related code to improve readability, maintainability, and runtime performance.
Add the name property to the workflow instance filter to make use of the name property that can be used in the Request class for the list endpoint. This allows us to filter purely on workflow instance name without using the SearchTerm
Updated the target framework from .NET 8 to .NET 9 for future compatibility and language improvements. Adjusted null handling in `WorkflowDefinitionImporter` for better readability and correctness. Added a cancellation token to `ReadToEndAsync` for improved async operation control.
Updated the `StringData` property to allow null values, improving flexibility and aligning with usage scenarios where the property may not always contain a value. This change ensures better handling of optional data.
Introduce a new integration test to verify workflow serialization functionality, ensuring that newly created workflow definitions can be serialized and deserialized correctly. Adjust default nullability for certain fields in `WorkflowDefinition` to improve consistency and prevent null reference issues. Minor argument update in `New` method of `WorkflowDefinitionPublisher`.
* Work in progress: Add DefaultTenantService for tenant management
Introduce `DefaultTenantService` and its corresponding interface `ITenantService` to manage tenant operations such as finding, getting, and listing tenants. Update `MultitenantBackgroundService` to utilize `DefaultTenantService` for handling tenant lifecycle events. This enhancement standardizes tenant operations and improves the maintainability of the multitenancy feature.
* WIP
* Add multitenancy event handlers and task interfaces
Implemented new interfaces IBackgroundTaskStarter and ITaskExecutor to manage task lifecycle events efficiently. Introduced new classes such as RunBackgroundTasks, RunStartupTasks, and StartRecurringTasks for handling tenant activation and deactivation events. Modified TaskExecutor to implement these interfaces and adjusted tenant registration logic to invoke these new handlers.
* Refactor multitenancy and task management services.
Remove background and recurring task runners, and integrate tenant activation and deactivation into the multitenancy feature. Enable multitenancy in the server application and create a new service for tenant activation and deactivation. This refactor simplifies the management of tenant-specific tasks and enhances the modularity of the platform.
* Refactor background service to use startup tasks
Replaced hosted service implementation with startup tasks for executing multi-tenant tasks and EF Core migrations. Introduced `PriorityAttribute` to manage task execution order, ensuring migrations run before other services that require database access. This simplifies tenant activation with an ordered task execution and removes redundant classes.
* Refactor MultitenancyFeature service registrations
Reorganized service registrations for better clarity and maintainability. Changed the registration of some services to use factory delegates for retrieving existing services to ensure correct dependencies. This refactor improves the flexibility of the tenant lifecycle event handling.
* Update V3_3 migration files
* Add tenant management endpoints and enhance tenant handling
Implemented tenant management endpoints including Add, Get, List, and Update. Enhanced tenant handling by introducing configuration and store-based providers, and improved error logging for tenant updates. Adjusted various internal functionalities to better support multitenancy features through different persistence providers.
* Implement tenant deletion endpoint and refactor migration setup.
Introduce a new API endpoint to handle tenant deletions while providing appropriate responses based on successful or unsuccessful attempts. Refactor migration handling by replacing startup tasks with hosted services across various modules to streamline the migration execution process.
* Add and integrate ConfigurationJsonConverter
Introduce a `ConfigurationJsonConverter` to handle JSON serialization and deserialization of `IConfiguration` objects. This change centralizes configuration serialization logic, leading to cleaner and more maintainable code. Updated various parts of the codebase to use the new serialization utility, ensuring a consistent approach throughout the application.
* Refactor JSON conversion and update tenant endpoint.
Removed unused workflow references and streamlined JSON handling in `ConfigurationJsonConverter`. Simplified tenant ID handling by removing `IIdentityGenerator` and setting a default value for `UpdatedTenant.Id`.
* Add logging for cancelled recurring tasks
Integrated ILogger to log information when a recurring task is canceled due to an OperationCanceledException. This change enhances troubleshooting by providing clearer insights into task cancellations and their underlying reasons, improving maintainability and observability of the task execution process.
* Disable multitenancy support and adjust default Tenant ID.
Multitenancy is now disabled by setting 'useMultitenancy' to false in the configuration. Additionally, the default Tenant's ID has been changed from null to an empty string to prevent potential null reference issues.
* Remove MultitenantHostedService abstraction file
The MultitenantHostedService.cs file was removed as it is no longer necessary. Its responsibilities have likely been refactored or integrated into another service, indicating a simplification or restructuring of the multitenancy handling in the codebase.
* Rename PriorityAttribute to OrderAttribute for clarity.
This change improves the clarity of the code by renaming PriorityAttribute to OrderAttribute, reflecting its actual purpose. All occurrences of the attribute in the codebase have been updated accordingly to maintain consistency. This makes the intent of the code more understandable for future maintenance and development.
* Fix message key retrieval in ProduceMessage activity
Update the ProduceMessage activity to use GetOrDefault for retrieving the message key. This change ensures that a null key is used if no explicit key is provided or if the key is empty or whitespace, preventing potential errors during message production.
* Refactor multitenancy and scheduling services.
Removed DefaultTenantContextInitializer interface and class, refactored tenant activation/deactivation to use try-catch logging, and updated tenant context handling to use IDisposable for context push. New activities and workflows added in Elsa.Server.Web, and scheduling services enhanced to schedule jobs with explicit job keys and groups. Also, adjusted configurations to enable multitenancy, providing improved maintainability and flexibility.
* Remove Example1 activities and disable multitenancy
Deleted Example1Activity, Example1Workflow, and FirstActivity classes to clean up unused code and simplify the codebase. Disabled multitenancy by setting useMultitenancy to false, likely to streamline configuration and resource utilization.
* Fix and normalize URL path concatenation.
Ensure that the base URLs in both base path providers consistently end with a forward slash. This normalization prevents potential issues with endpoint routing and path concatenation, improving overall URL construction robustness.
* Add log persistence configuration and strategy evaluation
Introduced `LogPersistenceConfiguration` to support dynamic strategy and expression-based log persistence evaluations. Updated `Default default values.
* Add LogPersistenceConfiguration and enum support
Introduced LogPersistenceConfiguration class and LogPersistenceEvaluationMode enum in both Elsa.Api.Client and Elsa.Workflows.Runtime. Updated JavaScript services to handle enums correctly and register LogPersistenceMode.
* Rename log persistence key for consistency
Updated the log persistence key from `LogPersistenceStrategyKey` to `LogPersistenceConfigKey` to maintain consistency across the codebase. This change ensures that property access aligns with the updated naming conventions used in the application's configuration.
* Reuse JSON serializer options
* Update log persistence config structure in comments
This commit revises the JSON example in code comments to reflect the updated structure of the log persistence configuration. The changes include updated evaluation modes and strategy types for default, inputs, and outputs sections. These modifications aim to enhance clarity and provide accurate documentation of the expected configuration format.
Ensure that only the latest version of each workflow definition is processed by grouping and selecting the highest version. This change helps prevent redundant updates and ensures data consistency.
* Add Kafka module with integration and example setup
Introduced the Kafka module providing consumer integration and activities into the project. This includes new classes for consumer handling, configuration, and activities. An example setup using Docker Compose is also added to facilitate development and testing.
* Refactor KafkaTransportMessage to inline Timestamp namespace
Simplify the namespace usage for the Timestamp type within the KafkaTransportMessage record. This change eliminates the need for a separate using directive for Timestamp, enhancing code readability and maintainability.
* Add support for handling Kafka transport messages
This commit introduces the capability to handle and trigger workflows based on Kafka transport messages. It adds a new handler, notifications, and updates the message stimulus to include correlating fields. Additionally, the Kafka consumers are now managed more modularly with updated startup tasks and mediator integration.
* Enable Kafka integration and fix Kafka options naming
Added support for Kafka integration in Elsa.Server.Web by setting up Kafka configurations in appsettings.json and updating Program.cs. Also, renamed `ConsumerConfigs` to `ConsumerDefinitions` in Kafka options for clarity.
* Add consumer definition enumeration and dropdown support
Introduced `IConsumerDefinitionEnumerator` for managing consumer definitions across providers and implemented in `ConsumerDefinitionEnumerator` class. Enhanced `KafkaFeature` to register these services and updated the `MessageReceived` activity to use a dropdown UI hint for consuming definitions. Improved `StartConsumersTask` by refactoring consumer definition retrieval logic.
* Add SendMessage activity and refine Kafka messaging
Introduce a new SendMessage activity for Kafka, enabling message publishing to specific topics. Refine KafkaTransportMessage model by removing headers and timestamp fields. Adjust the StimulusSender logic to streamline the bookmark queuing process and fix key-value pairing in dropdown options. Update appsettings for corrected Kafka bootstrap server and topic configurations.
* Add producer and topic management support
Introduced interfaces and implementations for managing producer and topic definitions along with their respective enumerators and list providers. Updated `SendMessage` activity to include producer selection and refactored consumer definition providers for better consistency.
* Refactor Kafka configuration property names
Renamed Kafka configuration properties for better consistency and readability across the codebase. Updated property names from `ProducerDefinitions` to `Producers`, `ConsumerDefinitions` to `Consumers`, and `TopicDefinitions` to `Topics`. Added missing input attribute in `SendMessage.cs` and registered additional handlers in `KafkaFeature.cs`.
* Add custom serializers for Kafka message handling
Introduced `DefaultSerializers` class for custom serialization and deserialization of Kafka messages. Updated `MessageReceived` and `SendMessage` activities to use these custom serializers, and modified `KafkaOptions` to include them.
* Fix ExpandoObject serialization method parameter
Changed the serialization type from `ExpandoObject` to the actual type of the object to ensure proper serialization. This ensures that derived types are correctly handled during the serialization process.
* Add Producer and Consumer workflows for Kafka
Introduced two new workflows: `ProducerWorkflow` and `ConsumerWorkflow` for handling Kafka messages. Updated `DefaultSerializers` to use camelCase property naming and modified `appsettings.json` to include `topic-2` and format entries.
* Add JSON serialization to log output in ConsumerWorkflow
This change enhances the log output by serializing messages to JSON format before writing them. The addition of System.Text.Json ensures that the message content is presented in a structured and standardized format in logs.
* Add correlation strategies and update Kafka features
Implemented HeaderCorrelationStrategy and NullCorrelationStrategy, and updated KafkaFeature to support customizable correlation strategies. Added correlation ID handling to Kafka transport messages and updated config and handlers accordingly.
* Add tenant accessor to ConsumerDefinitionWorkflowContextProvider
Integrated ITenantAccessor to the provider to support tenant-specific context loading. Updated the constructor and LoadAsync method to retrieve the tenant information and use it for context-specific operations.
* Switch to MySQL and disable Kafka
This commit changes the SQL database provider from SQLite to MySQL and disables Kafka use. It also includes necessary adjustments such as adding MySQL handling in configuration and connection setups, updating `docker-compose` to include MySQL services, and referencing MySQL projects in the `.csproj` file.
* Add UI property handlers to multiple features
This commit introduces various UI property handlers across several features such as Python, JavaScript, CSharp, and Workflow features to enhance user interface property handling. It also updates the property UI handler resolution logic to better manage cases where providers are not available. Furthermore, adjustments were made in the server configuration to switch database providers and enable Kafka.
* Refactor property UI handler retrieval logic
Modified the logic to fetch property UI handlers by preloading them into a list and then filtering. This change improves readability and potentially performance by reducing repetitive service provider calls.
* Incremental work on Kafka workers and predicate evaluation
* Merge BookmarkInvoker with BookmarkResumer
* Register IWorkerManager
* Change lifetime scope of WorkerManager to Singleton
* **Introduce topic subscription handling for Kafka workers**
Added `IWorkerTopicSubscriber` interface and its implementation for managing topic subscriptions. Enhanced workers to bind triggers and bookmarks dynamically based on existing data. Updated several classes and methods to support topic-based subscriptions and headers.
* Refactor trigger matching logic.
Extract trigger matching conditions into `IsMatchAsync` method for reuse. This enhances code maintainability and readability by reducing redundancy. The new `GetTopic` helper method isolates the topic retrieval logic.
* Add Name property to MassTransitActivityTypeProvider
This commit inserts the Name property in the returned object within the MassTransitActivityTypeProvider class. It ensures that the typeName is included, providing a clearer definition of the activity type.
* Add handling for deleted bookmarks and refactor bookmark removal
Added a new event handler for `BookmarksDeleted` to ensure removed bookmarks are processed correctly. Refactored the bookmark removal logic into a helper method to reduce code duplication and streamline the workflow.
* Switch to asynchronous bookmark queue processing
Refactored the `TriggerWorkflows` handler to use `IBookmarkQueue` instead of directly invoking the `IBookmarkResumer`. This change aims to improve scalability by queueing bookmark resumption requests, enabling better load distribution and async processing. Added necessary helpers and configuration options to support this functionality.
* Remove unused IBookmarkResumer dependency
Simplify the constructor by removing the unused IBookmarkResumer dependency. This cleanup reduces potential confusion and improves code maintainability without impacting functionality.
* Add support for local message processing
Introduced an `IsLocal` property to `MessageReceivedStimulus` for determining if the message event is local to a specific workflow instance. Updated `BookmarkBinding` and related handler methods to utilize `CorrelationId` for local event matching. Removed unused `CorrelatingFields` from `MessageReceived` activity.
* Add nullability checks to IWorker retrieval methods
Updated `GetWorker` methods to return nullable `IWorker` to handle cases where a worker might not exist. Modified code to include null checks and conditional operations to prevent potential null reference exceptions when accessing worker methods.
* Add filtering based on activity type name for triggers and bookmarks
This commit introduces filtering for triggers and bookmarks based on the `MessageReceived` activity type name. It also adds an option to mark messages as local in the `SendMessage` activity, where local messages are delivered to the current workflow instance only. These changes help enhance the management and targeted delivery of messages within the workflow framework.
* Add new Kafka topics and clean up producers config
New topics "topic-3" and "topic-4" were added to the Kafka settings. Unused topic references were removed from the producers configuration to simplify and improve clarity.
* Add predicate to KafkaConsumerActivity in ConsumerWorkflow
Introduced a predicate to the KafkaConsumerActivity using JavaScript expressions to filter messages based on OrderId. This ensures only relevant messages are processed in the workflow.
* Add proper summary tags back
* Apply suggestions from code review
Co-authored-by: Robin Sue <robinsue@live.de>
* 69 more files fixed
---------
Co-authored-by: Robin Sue <robinsue@live.de>
Updated method names in `IWorkflowInstanceVariableManager` and `WorkflowInstanceVariableManager` to use the `Async` suffix for consistency. This change improves code readability and aligns with async programming conventions.
* Regenerate 3.3 EF Core migrations
* Refactor variable handling in API responses.
Updated `ApiResponseExtensions` to handle potential null issue flagged by ReSharper. Refactored `Endpoint.cs` in WorkflowInstances to use `ResolvedVariableModel` for better type safety and clarity.
* Add API endpoint to retrieve workflow variables
Introduce a new endpoint to get workflow instance variables in `IWorkflowInstancesApi`. Added `ResolvedVariable` model and adjusted `WorkflowInstances` endpoint to return a list of these variables.
* Add endpoint for updating workflow instance variables
Introduce a new API endpoint to update workflow instance variables using the DefaultWorkflowInstanceVariableWriter. Also creating the necessary interfaces and classes, and renaming existing ones for consistency.
* Add IWorkflowInstanceVariableManager interface and implementation
Introduce the IWorkflowInstanceVariableManager interface and its implementation, enabling variable management for workflow instances. Refactor existing endpoints to utilize this new manager, streamlining the variable retrieval and update processes.
