44 commits

Author	SHA1	Message	Date
Sipke Schoorstra	74fc0c93c2	Update JavaScript evaluator and enable caching ... Removed unused imports in JavaScript evaluator and instantiated ObjectWrapper using static Create method. This simplifies the object instantiation process. Additionally, updated the mediator reference and activated the useCaching feature on the Elsa server to optimize performance.	2024-06-15 18:29:23 +02:00
Sipke Schoorstra	2aa914044d	Multitenancy (#5159) ... * Feature/multitenancy (#4739) * feat(multi-tenancy): add tenantId to entities and additionnal configuration to dbContext * feat(multi-tenancy): split code in a Elsa.Tenants project, add some configuration to the DbContext for strategies * feat(multi-tenancy): fix queryfilter to split data between tenants * feat(multi-tenancy): add tenantId to entity, generate migrations and create a sample * feat(multi-tenancy): add strategies to always have tenantId when saving * feat(multi-tenancy): add external user provider support for tenant * feat(multi-tenancy): fix dbcontext filter on tenantid * feat(multi-tenancy): manage background execution of workflows * feat(multi-tenancy): change tenant accessor and middlewares * feat(multi-tenancy): fix tenantId missing in some cases * feat(multitenancy): fix efcore store for multitenant * feat(multi-tenancy): fix some comment of the pullrequest (naming, use Sqlite for example, etc.) and split efcore from tenant project * Move Elsa.Samples.AspNet.Tenants and Elsa.Samples.AspNet.Tenants.External projects * Refactor tenant middleware and enhance code documentation Multiple files including middleware and entity classes related to tenants have been refactored for more straightforward implementation. Unnecessary code has been removed and constructor parameters have been directly utilized, increasing the code's readability and efficiency. Moreover, documentation for classes and interfaces has been enhanced providing better understanding of their role and function. * Refactor tenant-related classes and move to Elsa.Tenants module Moved ITenantAccessor and related classes from Elsa.Common to Elsa.Tenants module. Adjusted necessary references in files where these classes were used. Made corresponding changes to the various services where these classes were instantiated or used. Updated project references accordingly. * Refactor DbContext strategies and streamline code The DbContextStrategy interfaces were moved from Abstractions to Contracts for better semantics. Async methods in ElsaDbContextBase and IBeforeSavingDbContextStrategy were made synchronous to simplify usage. This included the alteration of methods in related classes to reflect the changes. Unnecessarily complex lines of code were also rewritten for better readability and standardization. * Refactor multi-tenancy feature and rename workflow provider interfaces The update refers to multi-tenancy and workflow providers. For the multi-tenancy feature, the 'ConfigurationTenantProvider' was refactored to 'ConfigurationTenantsProvider' and used in 'TenantsFeature'. It was consolidated into one function 'UseConfigurationBasedTenantsProvider'. Middleware to handle this scenario was also adjusted. Regarding workflow providers, the 'IWorkflowProvider' has been renamed to 'IWorkflowsProvider' and corresponding changes made in classes consuming this interface. Name adjustments also affected functions and configuration options. Various non-related code cleanups and removals have also been made. * Replace initial migration files for various databases The replacement ensures that TenantId columns will be added when users migrate to 3.1. * Remove old migration files This commit removes multiple old database migration files from the source. These files are related to the Elsa.EntityFrameworkCore module and they cover database alterations and management. They are no longer needed due to changes in the application's data structure and updates in the database schema. * Update database model and migration snapshot This commit updates the database model and migration snapshot in Elsa.EntityFrameworkCore.MySql. The "ProductVersion" annotation has been updated, and the "Status" property's type has been changed from int to string. Furthermore, a "TenantId" property has been added which is indexed for faster lookups. * Add TenantId to app settings This update modifies the appsettings.json file to include a "TenantId" for Administrators, Users, and Applications. Moreover, the key names are updated to follow the PascalCase convention consistently. * Refactored workflow command handler and added indexes to entities The DispatchWorkflowRequestHandler has been replaced with DispatchWorkflowCommandHandler in the WorkflowRuntimeFeature. Also removed the DispatchTenantWorkflowRequestHandler from the TenantsFeature. In the entity framework, added indexes to SerializedKeyValuePair's Key and TenantId in the Configurations. The SerializedKeyValuePair class now inherits from Entity. * Reset migrations to Initial * "Added TenantId column and altered status column type in AlterationPlans and AlterationJobs tables" This commit introduces new TenantId columns to the AlterationPlans and AlterationJobs tables across different DB contexts (MySql, SqlServer, Sqlite, PostgreSql). The type of the Status column has also been changed from int to string in both tables. Indexes for the new TenantId column are also created as part of this change. * Update authentication settings and improve code structure Several modifications have been made to improve our code and update our authentication setup. Made changes to disable Dapper and utilize JWT Bearer for authentication in the Elsa.Server.Web program settings. Additionally, we've updated a method on IdentityTokenOptions to be internal and adjusted token option assignments in ModuleExtensions. Syntax improvements were also made in ApplicationProviderExtensions for better consistency with best practices. * Remove V3_1 migration from different databases The V3_1 migration from MySql, SqlServer, Sqlite, and PostgreSQL databases has been removed. This includes alterations such as the addition of the "TenantId" column and changes in the "Status" column type within Elsa's schema. * Remove old and add new tenant handling components Removed files related to the old way of handling tenants in the application and added new ones for improved management. This change introduces a tenant resolution strategy pipeline, allowing different strategies to be used in sequence to resolve the current tenant. Various schemes have been implemented, including resolving tenants based on route prefixes, the currently authenticated user, and the user's claims. The update also enhances the way entities are handled when working with SQLite and Oracle databases. * Added SQLite setup to PersistenceFeatureBase The code now adds a line to include SetupForSqlite in PersistenceFeatureBase's services list. The SetupForOracle class was removed from the SetupForSqlite file - it is possibly now dedicated to SQLite setup only. * Add CommonPersistenceFeature, refactor related classes Implemented a new class, CommonPersistenceFeature, which handles aspects previously managed in PersistenceFeatureBase. ServiceScope registrations were removed from PersistenceFeatureBase and added to the new CommonPersistenceFeature. In addition, the tenant filter in SetTenantIdFilter has been simplified for more direct accessibility. Changes were also made in WorkflowManagementPersistenceFeature to accommodate these implementations. * Refactor multi-tenancy configuration and resolution Updated the code to use "MultiTenancy" section instead of "Tenants" section in all configuration files. Also, refactored the Store class to use IServiceProvider instead of IDbContextFactory and added tenant resolution logic. Renamed classes and variables accordingly to better represent multi-tenancy. * Add Tenant Dymanic Filter options for Workflow Definition Tenant Dymanic Filter provides the ability to include or exclude tenant matching in the filter. The corresponding queries and method signatures were modified to account for this new feature. Added `HttpContextTenantExtensions` and `HttpContextTenantResolver` classes to enable tenant-based filtering of HTTP context. --------- Co-authored-by: Jean-Baptiste Dalle <jean-baptiste.dalle@stereograph.fr> Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com> * Remove authenticating services and activities The AuthenticatingMediator, GetPlacedCallHubSpotId activity, and IAuthenticatingMediator interface have been removed. Middleware for authorization context workflow execution and tenant providers have been added to enhance multi-tenancy support. * Add tenant resolution strategies and update namespaces Introduce tenant resolution strategies in the Identity module and refactor namespaces across modules to align with the new multi-tenancy structure. This includes moving various tenant-related classes from the Tenants module to Common and Identity, adjusting references, and consolidating tenant constants and options. * Remove IdentityOptions and refactor token claims Deleted `IdentityOptions` and moved `TenantIdClaimsType` to `IdentityTokenOptions`. Renamed `ClaimConstants` to `CustomClaimTypes` and updated references across multiple services and samples accordingly. Adjusted `ClaimsTenantResolver` to use `IdentityTokenOptions` and removed deprecated method `Deconstruct` from `IdentityTokenOptions`. * Update SystemClock registration to singleton Changed the dependency injection scope for ISystemClock from scoped to singleton to ensure a single instance is used application-wide. This ensures consistent time references across different components and enhances performance by reducing the number of instances created. * Change service registrations to singleton The `IQueueProvider`, `ITopicProvider`, and `ISubscriptionProvider` are now registered as singletons to ensure single instances throughout the application. Additionally, dependency resolution for `ITriggerStore` and `IBookmarkStore` is moved to a scoped lifetime within `StartWorkers` to maintain correct scope per operation. * Refactor tenant ID claim retrieval Simplify the extraction of tenant ID from claims by removing the fallback to a default type, given that it is non-nullable. * Remove unused Constants import in ClaimsTenantResolver The import of Elsa.Identity.Constants was removed as it was not used in the ClaimsTenantResolver class. This cleanup helps maintain code clarity and reduces unnecessary dependencies. * Remove outdated workflow activities Removed several deprecated workflow activities pertaining to lead and deal updates in preparation for new workflow structure implementation. * Reset EF Core migrations to 3.0 * Add missing default value in constructor A default value was added to the WorkflowDefinitionModel constructor to align with the expected parameter count. Additionally, removed unused tenant-related code from ElsaDbContextBase and redundant entity configurations from DbContext specific to Oracle, simplifying the model setup. * Fix interface list type in store populator Adjusted the type passed to the DefaultWorkflowDefinitionStorePopulator constructor to use the correct IWorkflowsProvider interface. This change aligns the type expectation with the implemented providers list, ensuring proper dependency resolution. * Update model snapshots to v7.0.14 and add new properties The model snapshots for various database providers have been updated to reflect the new product version 7.0.14. Additionally, new properties and indices for `IsSystem`, `TenantId`, and compression information have been introduced across entities such as `WorkflowDefinition`, `WorkflowInstance`, `Label`, and others to support multi-tenancy and system-level workflows. * Add tenant ID and alter status column Migration V3_1 adds a tenant ID column and indexes, and changes the status column's type to a string across all database providers. It also renames the SerializedWorkflowInstanceIds to SerializedWorkflowInstanceFilter. * Refactor Dapper integration for multi-tenancy support Updated migration versions and internalized records to extend from a new base Record class with tenant ID property for multi-tenancy. Added TenantId across stores and migrations to support tenant-specific data isolation within the Dapper module. * Add multi-tenancy support to Dapper provider Introduced TenantId to various tables and records to enable multi-tenancy. Migration scripts have been added across modules for managing TenantId columns effectively. * Introduce multi-tenancy support for MongoDB provider Optimized TenantResolver for caching current tenant, refactored MongoDB stores to include tenant resolution, and updated index creation to add tenant-specific indices. Enforced multi-tenancy across various Elsa modules ensuring tenant isolation and efficient data retrieval. * Refactor bookmark ID references to use 'Id' property Changed usage of BookmarkId to Id across several files, ensuring consistent referencing of the bookmark identifier for clarity and future deprecation of obsolete properties. * Refactor MongoDB store to support tenant-agnostic queries Provide an option for tenant-agnostic operations across all MongoDB store methods, including Find, Count, Any, and Delete operations. Default behavior respects tenant boundaries, but adding a flag allows operations to ignore tenant constraints. Removed unnecessary using directive. * Add Dapper persistence stores and refactorings Added Dapper persistence stores for various entities and refactored existing stores for better maintainability. * Add tenant-agnostic query support to Elsa Dapper Introduced an option to perform tenant-agnostic queries in Elsa Dapper, allowing retrieval of records without tenant filtering. Refactored query methods to include a tenantAgnostic parameter and applied tenant filters conditionally based on this flag. This supports scenarios where tenant-specific data isolation is not required. * Refactor tenant resolution in Store.cs Changed the code to asynchronously resolve the tenant and handle possible null values before assigning the tenant ID. This improvement ensures tenant resolution is done properly and enhances code reliability by checking for nulls. * Switch Mongo collections to IMongoCollection Refactored Mongo collection references to use IMongoCollection interface across various modules. This change allows for more consistent and flexible database operations. Additionally, updated program configuration to enable MongoDB and disable Dapper. * Remove ac-call-ring-group workflow The workflow file ac-call-ring-group.json has been deleted, as it is no longer needed in our project. This may be due to changes in requirements or workflow optimizations. * Reset EF Core migrations to 3.1 * Update migration scripts and clean up Elsa.Tenants dependencies Removed outdated migration scripts (efcore-3.0.sh, efcore-3.1.sh, efcore-3.1-sql.sh) and added new versions (generate.sh, generate-sql.sh). Also optimized the dependency list in Elsa.Tenants.csproj - switched individual package references into a single reference to Microsoft.AspNetCore.App. This streamlined the project's dependencies and reduced redundancy. * Add TenantId to multiple tables and update primary keys The commit introduces a 'TenantId' column to several tables in the database schema including 'WorkflowInboxMessages', 'WorkflowExecutionLogRecords', 'Triggers', 'KeyValuePairs', 'Bookmarks', and 'ActivityExecutionRecords'. Additionally, the primary key for the 'KeyValuePairs' table was updated and the 'BookmarkId' column in the 'Bookmarks' table was renamed to 'Id'. These changes were necessary to support new functionality and requirements in the application. * Update "MultiTenancy" to "Multitenancy" This change reflects the renaming of "MultiTenancy" to "Multitenancy" in multiple instances throughout the codebase. This update is made in the 'appsettings.json' configuration file and also in 'Program.cs' where the renamed section is being accessed. * Remove generate-migrations-initial copy.sh script This commit deletes the generate-migrations-initial copy.sh script from the project. The script was previously used for managing database migrations across different providers but it's no longer needed. This change simplifies the codebase and reduces maintenance work. * Fix typo in configuration key of Multitenancy The typo in the configuration key "MultiTenancy" has been corrected to "Multitenancy" in both Program.cs and appsettings.json files. This ensures that the program correctly reads the multitenancy configuration from the appsettings.json file. * Remove unused imports in ElsaStudioWebAssembly The unnecessary namespaces Elsa.Studio.Workflows.Extensions and System.Text.Json have been removed from the ElsaStudioWebAssembly Program.cs file. This change helps to keep the code concise and increase readability. * Simplify comment * Update TenantResolutionStrategyBase class description The class summary for TenantResolutionStrategyBase has been updated to more accurately reflect its purpose. It is now described as a base class intended for implementing a tenant resolution strategy, providing a more clear understanding of its functionality. * Remove GetTenantId method from WorkflowInstanceStore This update removes the unused GetTenantId method from the WorkflowInstanceStore.cs file. Additionally, minor formatting changes have been made to clean up extra spaces and align the syntax for better readability. * Update ApplyTenantId class description The class description in the ApplyTenantId file has been updated for clarity. This handler is designed to apply the tenant ID to an entity before changes are saved. * Refactor TenantId filter application in Elsa module The SetTenantIdFilter class in the Elsa.EntityFrameworkCore.Common module has been refactored to improve readability and performance. Flattening of control blocks has been done for better comprehensibility. Additionally, the logic has been revised to apply a TenantId filter only if an entity is assignable to the shared Entity base class. * Remove MustHaveTenantException The file MustHaveTenantException has been deleted. This exception is no longer needed in the project, due to changes in tenant handling. * Remove unused Migrations\Alterations folder reference The reference to the Migrations\Alterations folder in the Elsa.EntityFrameworkCore.MySql project file has been removed. This folder is not being used anywhere in the project hence the reference was unnecessary. This change simplifies the project structure. * Refactor code and annotate methods in WorkflowInstanceStore The update includes adding 'System.Diagnostics.CodeAnalysis' namespace and a number of attributes 'RequiresUnreferencedCode' to various methods because they call 'Elsa.Workflows.Contracts.IWorkflowStateSerializer.SerializeAsync(WorkflowState, CancellationToken)'. Additionally, removed the 'GetTenantId' method as it is no longer used. * Refactor MongoWorkflowInstanceStore for cleaner code The constructor of MongoWorkflowInstanceStore has been simplified and the use of a private MongoDbStore<WorkflowInstance> has been removed. This results in the use of mongoDbStore directly in calls instead of referencing a private instance, leading to a leaner and more readable implementation. The method GetTenantId has also been removed as well. * Rename 'strategies' to 'resolvers' in TenantResolution code The term 'strategies' has been replaced with 'resolvers' throughout the TenantResolution code, to better reflect its purpose. All references, variable names and comments have been changed accordingly. The change also includes error messages and function inputs, making the codebase more consistent and understandable. * Rename MultiTenancyOptions to MultitenancyOptions All references to MultiTenancyOptions have been changed to MultitenancyOptions as part of a codebase-wide renaming effort. This is done to ensure consistent naming conventions across the project. * Update attribute message in ConfigurationTenantsProvider The attribute message associated with the FindAsync method in the ConfigurationTenantsProvider class has been changed. It now indicates that the caller of the method may require dynamic access to the tenant properties. * Remove tenant ID method from IWorkflowDefinitionStore This commit removes the GetTenantId method from the IWorkflowDefinitionStore interface. The method, which used to find the tenant ID of a workflow definition for a specified ID, is no longer necessary in the updated design. * Remove tenantId retrieval methods The methods for retrieving tenantId have been removed from several modules across the application. This decision was based on a shift in architecture design, thus limiting the need for these specific methods. * Refactor EFCoreWorkflowDefinitionStore constructor parameters The EFCoreWorkflowDefinitionStore constructor parameters are refactored to no longer be private field properties. This change streamlines the code by eliminating redundancy. Also, now all method calls within the class use these directly passed parameters. * Refactor MemoryBookmarkStore for code simplification Revised MemoryBookmarkStore to remove the private field and directly use the constructor parameter instead, achieving a more straightforward structure. This modification simplifies the code, making it easier to understand and maintain. * Refactor ServiceBus integration tests for readability This commit simplifies and improves the readability of the code in ServiceBus integration tests. The changes primarily involve reformatting method call arguments for better visibility and removing superfluous white space. Additionally, a null-check operator has been added for safety. * Remove redundant release configurations The commit removes duplicate lines for the release configuration of the project with ID {99F2B1DA-2F69-4D70-A2A3-AC985AD91EC4}. The project solution (.sln) file is now cleaner and easier to read, reducing potential confusion when reviewing or modifying configurations. * Remove Debug build configuration for project This commit removes the debug build configuration setting from the Elsa.sln. The unnecessary setting for Debug Any CPU build for the specific project has been deleted, aiming to streamline and maintain only needed configurations. * Update NuGet.Packaging and NuGet.Protocol versions The versions of NuGet.Packaging and NuGet.Protocol packages were updated from 6.8.0 to 6.9.1. This update is aimed at keeping compatibility in our project with the latest updates and features available in these packages. * Add caching to workflow runtime and workflow management stores The update introduces caching to workflow runtime and workflow management stores to enhance performance. This is achieved by adding decorators for several stores, which cache records to reduce database fetches. Additionally, a signaler for change tokens allows for cache invalidation when changes occur. The MemoryCache feature has also been updated to include Scrutor for decoration and the caching duration can be configured through the new CachingOptions class. * Refactor WorkflowsMiddleware for HTTP Endpoint bookmarks and triggers The WorkflowsMiddleware has been extensively refactored to handle HTTP Endpoint bookmarks and triggers. This involves breaking down the InvokeAsync method by extracting parts of its functionality into separate helper methods such as FindTriggersAsync and FindBookmarksAsync. Moreover, Assist with authorization checks, workflow execution within request timeout, and handling of workflow faults has been improved to be more efficient and clearly segmented. * Update HTTP endpoint authorization to use Workflow context The authorization process in the AuthenticationBasedHttpEndpointAuthorizationHandler class has been updated to use the Workflow context instead of the WorkflowInstanceId string. The AuthorizeHttpEndpointContext model has been correspondingly changed to include a Workflow property, thereby strengthening the link between authorization and specific workflows. * Add FindAsync methods to trigger and bookmark stores The code adjustments add new FindAsync methods to the trigger and bookmark store contracts as well as all their concrete implementations (MongoDb, Memory and EFCore). These methods support fetching the first record matching a given filter. The adjustments also include minor syntax improvements and the addition of [UsedImplicitly] attributes where needed. * Refactor WorkflowsMiddleware for improved workflow handling The code was refactored to simplify the flow of handling workflows in the WorkflowsMiddleware class. Specifically, the methods to start and resume a workflow have been extracted to improve code readability. Further, handleErrorMiddlewares was also updated to better manage instances where no valid workflows or base paths are found. * Add caching functionality to WorkflowsMiddleware Added IMemoryCache usage in the WorkflowsMiddleware to cache lookup results for workflows and their associated triggers. This will reduce the number of database operations required when searching for workflows, thus improving performance. The cache is maintained for one minute before it is refreshed. * Implement dynamic cache duration for workflows The code has been updated to have a dynamic cache duration for the workflows instead of a hardcoded one minute. By using the CachingOptions service, the cache duration can now be set in the configuration making it more flexible and adaptable to different performance needs. * Add HttpWorkflowsCacheManager for caching HTTP workflows This commit includes the implementation of IHttpWorkflowsCacheManager for caching of HTTP workflows. New handlers have been added to invalidate cache on workflow updates. Additionally, WorkflowsMiddleware has been renamed to HttpWorkflowsMiddleware. * Refactor workflow trigger handling and caching The refactoring includes deletion of `IndexWorkflowTriggersHandler.cs` and creation of `IndexTriggers.cs` thus revising the workflow trigger indexing approach. Also, revamped the `ITriggerIndexer` interface which now handles deletion of triggers with specific workflow and filter. Furthermore, the caching mechanism in `HttpWorkflowsCacheManager.cs` is modified to handle eviction of workflow definitions and triggers separately boosting its efficiency. * Add summary to IndexedWorkflowTriggers A summary has been added to the 'IndexedWorkflowTriggers' class, providing a brief description. This description outlines that it represents a collection of indexed workflow triggers, promoting clearer understanding for future reference. * Refactor memory caching feature into separate module This commit separates the memory caching feature from the Elsa.Common module into a distinct Elsa.Caching module. This includes moving and renaming related files, such as the MemoryCacheFeature class and associated dependencies. The references in other modules and in the main solution file have been updated accordingly to include the new Elsa.Caching module. * Add distributed caching and update async methods Introduced a distributed caching feature with extensible change token signal publishing. Updated various cache-related methods to be asynchronous for improved performance and responsiveness. Also updated some workflow identity references for clarity. * Add distributed caching with MassTransit support This addition includes the implementation of a distributed caching system with MassTransit transport. The changes introduce necessary interfaces and services, new distributed caching feature along with the support for MassTransit as a transport option. Moreover, the instance management feature has been renamed to clustering feature for better clarity. * Refactor queue naming and scope of MassTransitChangeTokenSignalPublisher Queue name construction is adjusted in the RabbitMqServiceBusFeature and AzureServiceBusFeature modules for better organization and readability. MassTransitChangeTokenSignalPublisher is now a singleton service, ensuring the signal publisher can be shared across the application, increasing efficiency and performance. Also, introduced use of DistributedCacheFeature in MassTransitDistributedCacheFeature module for better modularization. * Add caching capabilities to workflow definition service This commit introduces caching to the workflow definition service, improving the performance for retrieving workflow definitions. 4 new classes have been created (`CachingWorkflowDefinitionService`, `EvictWorkflowDefinitionServiceCache`, `WorkflowDefinitionCacheManager`, and `IWorkflowDefinitionCacheManager`), and several existing classes have been updated to support caching. The caching also includes invalidation mechanisms, ensuring data consistency. * Refactor caching mechanism in workflow definition In the workflow definition module, the explicit caching functionality related to workflow definition versioning has been removed in favor of a more streamlined approach. Additionally, the manner in which services are registered has been altered. As a result, the caching now directly involves the overall workflow definition rather than individual versions, simplifying the caching logic and potentially improving the performance. * Update MassTransitBroker and enable RealTimeWorkflows and SignalRHubs The MassTransitBroker has been updated to Memory from RabbitMq. In addition, the RealTimeWorkflows and UseWorkflowsSignalRHubs features are now enabled in the code. This change will impact how the service communicates and processes real-time requests for workflow operations. * Reformat variable types in HttpFeature The reformatting involves a list of variable types in the HttpFeature module. Each type now appears on a new line for improved readability, making the code easier to maintain and review. * Update HTTP workflows cache invalidation handler XML comment * Remove unnecessary using directives Unnecessary using directives were deleted across several files in the Elsa.Http module. This simplifies the code and will possibly improve execution speed. Specific deletions include those for Encoding, Unicode, Extensions, Collections.Generic, Linq, Text, and Tasks namespaces. * Refactor HttpWorkflowsMiddleware constructor This commit simplifies the HttpWorkflowsMiddleware class constructor. It removes the intermediary variables `_next` and `_options` and directly uses the passed arguments in the constructor. Now, the `next` and `options` parameters are used directly throughout the middleware. * Simplify workflow retrieval in HttpWorkflowsMiddleware This refactoring replaces the use of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync with a single method, FindWorkflowAsync. This simplifies the middleware code and likely improves performance by reducing the number of database queries or service calls required to retrieve a workflow. * Refactor workflow retrieval in HttpBookmarkProcessor Simplified the workflow retrieval process in HttpBookmarkProcessor.cs. Replaced FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods with a single FindWorkflowAsync call. This reduces the complexity and improves efficiency in retrieving workflow. * Optimize FindWorkflowAsync method in HttpWorkflowsCacheManager Removed redundant lines of code to simplify workflow search functionality. This change simplified the FindWorkflowAsync method by directly calling the FindWorkflowAsync function in the workflowDefinitionService, thus increasing code readability and efficiency. * Refactor Endpoint.cs for workflow retrieval The method for obtaining a workflow in the Endpoint.cs script has been refactored and streamlined. The 'GetWorkflowDefinition' method is replaced by the 'GetWorkflowAsync' method which directly retrieves the workflow, without the intermediate step of materializing the workflow definition. This shortens the code and simplifies the process. * Refactor InputFunctionsDefinitionProvider constructor The constructor for InputFunctionsDefinitionProvider has been simplified by removing unnecessary private fields. Services are now directly used in the method instead of being stored in fields. This improves readability and reduces complexity in the class structure. * Refactor WorkflowInstance with improved state handling Simplified the methods for handling workflow and workflow state in the WorkflowInstance class. The refactoring also included some code clean-ups and variable renaming. The new implementation provides better readability and maintainability of the code by reducing unnecessary lines and improving structuring of objects and responses. * Remove unused IBookmarkManager and update workflow functions IBookmarkManager from ProtoActorWorkflowRuntime.cs file is removed due to its redundant status. Additionally, the "FindAsync" method has been updated to use a cancellation token. Also, annotations were added to the "ExportWorkflowStateAsync" and "ImportWorkflowStateAsync" methods to flag calls to functions that require unreferenced code. * Remove unused ReSharper directive Unused ReSharper directive in the file IndexTriggers.cs was identified and therefore removed. This change makes the code cleaner and easier to read. * Update activity invocation in workflow runtime Updated the DefaultBackgroundActivityInvoker service in the Elsa.Workflows.Runtime module to annotate the ExecuteAsync method with "RequiresUnreferencedCode" attribute. This change is made considering the potential code trimming issue. Additionally, simplified the process of fetching workflow by directly using FindWorkflowAsync method instead of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods. * Refactor code to simplify workflow definition loading The code for finding and materializing workflow definitions has been simplified. Instead of loading the definition and materializing it into a workflow in separate steps, a new method called FindWorkflowAsync has been introduced to perform both actions at once. This reduces redundancy and makes the code more readable. * Refactor WorkflowHostFactory to streamline workflow creation This commit simplifies the workflow creation process in WorkflowHostFactory. It removes redundant code and extraneous methods, specifically the overloaded CreateAsync method which used WorkflowDefinition. Now, it directly finds and uses the Workflow instance, thereby simplifying the code base and improving maintainability. * Refactor workflow retrieval in WorkflowInstance.cs Changed the way workflow instances are retrieved from the WorkflowDefinitionService. Instead of obtaining the workflow definition and then materializing the workflow from it, the workflow is directly retrieved using the FindWorkflowAsync function. This simplifies the code and avoids unnecessary null-checks. * Remove unnecessary whitespace in WorkflowInstance.cs An extraneous whitespace character was identified and removed in the WorkflowInstance.cs file. This change contributes towards maintaining clean and readable code in the Elsa.ProtoActor module. * Remove unnecessary comment in ProtoActorWorkflowRuntime The unnecessary comment ("Load the workflow definition.") in the method TryStartWorkflowAsync of the ProtoActorWorkflowRuntime.cs file was removed. This is part of an ongoing effort to keep the codebase clean and readable. * Update workflow management features and handlers Added explicit notification handlers for DeleteWorkflowInstances and RefreshActivityRegistry in WorkflowManagementFeature.cs. Also, renamed RefreshActivityRegistryHandler.cs to RefreshActivityRegistry.cs for better clarity. * Add multiple log record support to workflow execution log stores The major change of this commit is the addition of methods to add multiple log records in the WorkflowExecutionLogStore, across different storage modules such as EntityFramework, MongoDB, Dapper, Elasticsearch and Memory. This ensures consistency and uniform behavior across different storage types. Furthermore, some reformatting and tidying up of the code were undertaken to maintain readability and clarity. * Remove redundant workflow definition check The workflow definition existence check and related service retrieval were removed from HttpWorkflowsMiddleware.cs. It was determined that this check was unnecessary as the workflow definition's existence is guaranteed at this point in the process, reducing redundancy in the code. * Improve cancellation token usage in workflow execution This commit refines the usage of cancellation tokens during the execution of workflows in Elsa.Server and HttpWorkflowsMiddleware. Previously, a cancellation token pair was created before ExecuteWithinTimeoutAsync was called, which limited duration control solely to that method. Now, cancellation tokens are included within ExecuteWithinTimeoutAsync method. This allows the method to observe any cancellation initiated by outer scopes, enhancing control over the timeout of operations. * Remove unused EntityFrameworkCore import The Microsoft.EntityFrameworkCore namespace was included but not used in ElsaDbContextOptions.cs. This commit removes that unnecessary import to simplify the code and improve clarity. * Add PersistStateAsync method to WorkflowHost A new PersistStateAsync method has been added to the WorkflowHost, which enables the host to directly persist its own state. The method has been integrated into the DefaultWorkflowRuntime and HttpWorkflowsMiddleware. This update eliminates the need to continuously get instances of IWorkflowInstanceManager to save state, which improves efficiency and code readability. * Refactor DefaultAlterationRunner service This update simplifies the DefaultAlterationRunner service, reducing the number of code lines and removing unnecessary references. The workflow materialization step has been merged with the find workflow step, and unused namespaces have been dropped. * Refine wording in IWorkflowHost interface documentation The documentation for the 'CanStartWorkflowAsync' method in the IWorkflowHost interface has been cleaned up. The superfluous "or not" verbiage has been removed, making it easier to understand the method's function. * Remove CancellationTokens struct and simplify cancellation handling Removed the CancellationTokens struct and all its references across the code base. Simplified cancellation handling by using standard CancellationToken only. Also cleaned up redundant usages and unnecessary namespaces across various modules. This simplification better aligns with standard .NET conventions and reduces the code complexity. * Implement ActivityHandle for better activity identification The ActivityHandle class has been introduced to consolidate various activity identification parameters such as ActivityId, ActivityNodeId, ActivityInstanceId, and ActivityHash. This makes it easier to track and manage activities by reducing the number of parameters needed for identification. Adoption of ActivityHandle has been implemented across the codebase. * Remove AzureContainerApps related code This commit includes the removal of all the AzureContainerApps associated code from the ProtoActor Cluster. The deletion includes multiple files which had housed the AzureContainerApps cluster provider and all its supporting services, util functions, options, contracts, and models. This move might be towards refactoring the structure or removal of unwanted dependencies. * Add distributed execution runtime and client Removed some unused classes and created a distributed execution runtime using distributed locking and persistence. Introduced an interface for workflow clients and implemented two versions, a local one and one using `Proto.Actor`. Made some changes in other modules to add the necessary interfaces and methods. Also, added `DistributedLockingRuntime` module to implement distributed execution. * Remove WorkflowClient.cs from Elsa.Workflows.Runtime The WorkflowClient.cs file was removed from the Elsa.Workflows.Runtime project. It held placeholder methods that had not yet been implemented, making it unnecessary in the current codebase. * Add ProtoActor implementation for workflow execution Added ProtoActor implementation for executing workflow instances. This includes ProtoBuf message definitions, grain interfaces and implementations, and various ProtoActor services. Extended WorkflowHost and related components to provide more execution details in order to better support distributed processing via ProtoActor. * Refactor workflow parameters to workflow requests The refactor includes renaming old 'WorkflowParams' classes to 'WorkflowRequest' to better represent their usage. The changes have been made across all the involved modules ensuring that the project maintains consistency. New properties, methods, and class names were updated accordingly. * Add ProtoActor implementation for data mappers This update adds the ProtoActor implementation for data mappers and updates the DI configuration accordingly. It also updates the client interfaces for a more streamlined usage, and refactors the WorkflowHost to adapt to these changes. All changes were implemented in accordance with the new bookmark information model. * Add functionality to create a new workflow instance This commit introduces the capability to create a new workflow instance in the Elsa Workflow Runtime by enhancing existing workflow classes and creating the WorkflowClientFactoryExtensions class, in addition to adding the "IsNewInstance" parameter to multiple workflows and requests. It provides more flexibility in managing workflow instances and aids in building workflow processes that require instantiation of a new workflow. * Remove Elsa.Runtimes.DistributedLockingRuntime module This commit deletes the Elsa.Runtimes.DistributedLockingRuntime module from the project. Changes include removed code files related to features, services and commands under this module. The Elsa.sln file has been updated to reflect these changes, excluding the removed module from the solution. * Add dynamic client type to WorkflowClientFactory The WorkflowClientFactory now accepts a client Type parameter for dynamic client creation. This enables the factory to create different subclasses of IWorkflowClient based on the provided Type. The CreateClient function is updated in the WorkflowClientFactoryExtensions and IWorkflowClientFactory interface, and applied in HttpWorkflowsMiddleware. * Implement Proto.Actor support in Elsa A comprehensive change that introduces support for Proto.Actor clustering to Elsa. It includes a redesign of the workflow client invocation model, transitioning the methods from synchronous to asynchronous. Also, modification to the Proto.Actor-based 'WorkflowGrain' was made to handle state recovery and execution processing. The structural enhancements improve scalability and performance capabilities. * Refactor null-checks in SaveSnapshotAsync method Simplified the null-checking in the method SaveSnapshotAsync within WorkflowGrain.cs. The check on _workflowHost and workflowState has been shortened using conditional access operator, improving the code readability. * Refactor code to enhance database configuration logic Removed the 'IModelCreatingHandler' file as it's no longer needed and instead introduced enums for 'SqlDatabaseProvider' and 'PersistenceProvider'. The enums replace previous hard-coded boolean flags for database options, making the code cleaner and more scalable. Simplified and improved the efficiency of Database Connection Handling by optimizing the timing of connection creation. * Improve tenant ID retrieval in HttpContext The method GetTenantId has been updated in HttpContextTenantExtensions. Instead of accessing the "TenantId" key directly in HttpContext.Items, it now uses TryGetValue. This change improves error handling for cases when the "TenantId" key is not found in the collection. * Remove trailing comma in IdentityTokenOptions The trailing comma at the end of the NameClaimType assignment in the IdentityTokenOptions.cs file has been removed. This change ensures proper usage of the syntax and maintains the cleanliness of the codebase. * Remove unused imports Several unused imports were detected and removed from the codebase. These imports were scattered across multiple files and did not contribute to the functionality of the system. By removing them, we decrease clutter and make the codebase easier to navigate and maintain. * Refactor code and update packages The codebase has undergone a major refactoring to improve code readability and consistency, with non-essential methods being removed and variable names being optimized for clarity. Package references for Proto.Actor and related packages have been also updated to the latest versions. * Add Proto.Cluster.AzureContainerApps package to Directory.Packages.props The Proto.Cluster.AzureContainerApps package with version 1.6.0 has been included in the Directory.Packages.props file. This addition extends the range of Proto.Cluster packages used in the project. * Remove redundant Proto.Actor implementation files This commit removes several redundant files related to the Proto.Actor implementation. These files were unnecessary and were cluttering up the codebase. With this removal, the project has become leaner and easier to maintain. It also eliminates potential confusion for future developers working on this project. * Add WorkflowMatcher and replace BookmarkHasher with StimulusHasher This commit introduces a new class, WorkflowMatcher, which represents a contract for finding triggers and bookmarks associated with workflow activities. It also renames and replaces all instances of BookmarkHasher with StimulusHasher to reflect a more accurate function relating to stimulus rather than bookmarks. Other necessary changes were made to ensure consistency with these updates across different classes. It's important to note that WorkflowInbox interface is now marked as obsolete. * Incremental work on stimuli refactoring * Refactor codebase to support new IWorkflowInvoker and invoke workflow logic Multiple files updated to introduce a `IWorkflowInvoker` interface, implementation and supporting classes. This simplified how workflows are instantiated and invoked. Consequently, necessary adjustments made across multiple entities and endpoints to account for the updated invocation procedure. Old logic related to counting running workflows removed. Introduced stimulus concept while deprecating non-compliant classes and methods. * bumped versions to fix dependency vulnerabilities (#5256) * Update patch version in GitHub workflows The version number used in the branch checking step of the GitHub workflows has been updated. Instead of scanning for the branch containing the patch version 3.1.2, it now scans for the branch that contains version 3.1.3. This change is aligned with the updated product version. * Update git branch grep pattern in workflow file The git grep pattern has been corrected to properly identify tagged versions in the GitHub Actions workflow. The correction ensures that the workflow script fetches the right branches as per the release tag instead of patch. * Update grep command in packages workflow The grep command used in the 'packages.yml' GitHub workflow was previously looking for the exact 'refs/tags/3.1.3' string. This commit simplifies the command by making it only look for '3.1.3'. This adjustment will streamline the process and potentially prevent issues with branch recognition. * Update package versions and refactor code for Elasticsearch and JavaScript modules Updated versions of numerous packages in the Directory.Packages.props file to their latest stable releases. This includes updates to Elasticsearch, JavaScript, and MongoDB packages among others. Additionally, refactored parts of the code in the WorkflowInstanceConfiguration and JintJavaScriptEvaluator within the Elasticsearch and JavaScript modules, respectively, to improve index management and script preparation. The WorkflowInstanceStore also saw a minor adjustment. * Refactor workflow management with workflow definition handles The existing workflow management has been significantly refactored, introducing the concept of "Workflow Definition Handles". These handles allow for consistent management whether we're dealing with a specific workflow definition, a version of a definition, or more flexible version constraints. This refactor also adjusts how workflow instances are created, now using a more intuitive and detailed "WorkflowInstanceOptions" approach. * Add ResumeBookmarkResult and update related methods Implemented a ResumeBookmarkResult class to handle bookmark resumption results. Methods related to resuming bookmarks have been refactored to return this new class, providing more information such as bookmarks' matched state. Also, some methods were optimized to break the loop early if no bookmarks were found, improving code efficiency. * Update workflow definition, execution and correlation This commit focuses on enhancements and adjustments to workflow definition and execution. Notable changes include the addition of ExecuteResponse model and new test workflow scenarios. Refactoring has been performed to improve readability and efficiency in various components. It also includes an important fix for the correct application of WorkflowDefinition filters, primarily using both DefinitionId and DefinitionVersionId for more accurate results. * Refactor runtime codebase for better structure and workflow control This commit involves changes to functionally reorganize the runtime codebase for improved structure. It also allows for better handling of workflows, particularly through the addition of original bookmarks in the workflow execution context. Removed unnecessary dependencies and ensured more efficient management of bookmarks in the workflow running process. Several method and class names were also updated to better reflect their purpose. * Refactor WorkflowInvoker and remove 'OriginalBookmarks' Optimized the constructor of 'WorkflowInvoker' by using 'IServiceScopeFactory' to get instances of required services. Removed the 'OriginalBookmarks' property from 'WorkflowStateExtractor' and 'WorkflowState', a subsequent change included in 'WorkflowGraphBuilder' as well. Reorganized the namespaces in 'Elsa.Workflows.ComponentTests' project. Added 'UseCache' in WorkflowServer configurations. * Removed RunWorkflowParams class This update deletes the RunWorkflowParams class within the Runtime Requests of the Elsa.Workflows module. The class was no longer needed, hence the elimination and cleanup in the codebase. * Update RunWorkflowParamsMapper to handle null or empty fields This commit modifies the RunWorkflowParamsMapper in the Elsa.ProtoActor module. It primarily treats the BookmarkId and TriggerActivityId fields to return null if they are empty, boosting the application's robustness against potential null or empty field issues. * Refactor workflow handling and improve null checks In this update, the handling of workflows was refactored to improve efficiency. The 'RunAsync' method now correctly uses 'WorkflowGraph' as a parameter, instead of 'Workflow'. Additionally, null checks for 'bookmarkId' and 'activityHandle.ActivityInstanceId' have been improved to avoid null and empty strings. Finally, when '_workflowInstanceId' is null in the 'WorkflowGrain' class, it is now properly initialized by parsing the cluster identity. * Enable ProtoActor in Elsa.Server.Web This commit turns on the use of ProtoActor within the Elsa.Server.Web bundle. This switch may affect the system's behavior and performance. * Added new workflow scheduling and management features Implemented new features for creating, running, and scheduling workflow instances. The implementation added new files for handling workflow runtime and scheduling features, including creating and running a workflow instance request, a mapper for the request, and handler services. Modified files include updating method calls according to the new requests, updating workflow definitions, and adjustment to method arguments in Hangfire job class. * Handle null or empty workflow instance IDs and correlation IDs This commit introduces null checks for workflow instance IDs and correlation IDs in the workflow infrastructure. Previously, the code assumed that the IDs were provided. It now gracefully handles cases where they might be null or empty, preventing possible null reference exceptions and ensuring more robust workflow execution. * Update mapping details in ResumeWorkflowJob Removed unnecessary using statement for Elsa.Workflows.Runtime.Requests in ResumeWorkflowJob.cs. Updated references from DispatchWorkflowInstanceRequest to ScheduleExistingWorkflowInstanceRequest for retrieving ActivityHandle and WorkflowInstanceId. * Refactor AzureServiceBus module and integrate into web project In this commit, changes were made to the AzureServiceBus module to use Topic definitions for subscriptions. The Subscriptions property in the AzureServiceBusOptions and the Topic property in the SubscriptionDefinition class are marked as obsolete and suggestions to use TopicDefinition.Subscriptions instead have been added. The AzureServiceBus module was also integrated into the web project and configured to use options from the appsettings.json. * Refactor code to use async scopes and improve service dependencies Refactored code to use async scopes for improved task management. Also organized service dependencies better by moving service fetching inside methods where they are needed and propagating necessary dependencies through method parameters for cleaner code. * Add Azure Service Bus workflow component tests This commit introduces a set of workflow component tests for Azure Service Bus integration. These tests encompass scenarios like message receiving, sending messages with correlation IDS, and sending single messages. In addition, 'SignalResetEvent' test helper was removed, a new test helper 'TriggerSignal' was added, and a Mock ServiceBusClient and ServiceBusAdministrationClient were added to the WorkflowServer fixture. Lastly, the NSubstitute package was added to the test project's dependencies. * Add support for deferred tasks in workflow execution context Added support for deferred tasks in the workflow execution context. This implementation allows tasks to be deferred and executed right after persistence of bookmarks in the workflow. A new middleware, ExecuteDeferredActivityTasks, is introduced to handle the execution of these deferred tasks. * Refactor TriggerSignal and SendMessage activity execution Refactored the execution of TriggerSignal and SendMessage activities by using context.DeferTask to ensure the activities run asynchronously. Also, made code format modifications and simplifications such as handling ApplicationProperties better in the SendMessage activity, and other minor changes. * Update workflow ID generation method The workflow ID generation method has been updated in the WorkflowBuilder. Previously, it was always generated by _identityGenerator. Now, it only gets generated if the definitionId or Id is null or empty, otherwise it uses the existing value. This change brings the ID generation practice in line with how we handle definitionId. * Add support for service bus testing in workflow tests Two new helper classes, DictionaryExtensions and MockServiceBusProcessor, have been added to support mocking Azure service bus in workflow component tests. Also, the tests have been updated to use SignalManager to ensure proper order of execution. The WorkflowServer test fixture has been extensively refactored to create mock instances of service bus senders and processors. Furthermore, some configurations have been commented out and new configurations related to MemoryTriggerStore and MemoryBookmarkStore have been added. * Enhanced workflow correlation and caching in Elsa Workflows This update improves Elsa Workflows by enabling caching and enhancing correlation in Correlate.cs. It expands functionality by adding new methods that accept various input types and provide more options for activities correlation. Correlation improvements also extend to AzureServiceBusTests, which were updated to test workflow instances by correlation ID. Furthermore, the test workflow received a name change and new correlation mechanics based on Azure messaging. * Renamed method argument from 'payload' to 'stimulus' The method argument 'payload' in various functions across 'MessageReceived.cs' and 'ActivityExecutionContext.cs' files has been renamed to 'stimulus'. This change was made to improve code readability and understanding by using a more context-specific term. * Refactor Workflow APIs and enhance logging Refactored Workflow APIs by removing the TriggerActivityId from StimulusMetadata and introducing Direct Triggers. Refactored Reenter method in WorkflowGrain to be more concise and straightforward. Enhanced logging by adding additional logging in ActivityExecutionPipeline and LogLevel in appsettings. Also, updated ProtoActorFeature for better log level management. * Refactor AzureServiceBusTests and add workflow completion signal The AzureServiceBusTests class is refactored to separate and encapsulate concerns. SignalManager and WorkflowEvents are now class-level variables. Also, a signal to indicate the completion of a workflow is added. This allows the test to wait for workflow completion in the sequence of its operations. Additionally, the workflow definition ID in MessageReceivedTriggerWorkflow is now static and fixed, rather than dynamically derived from the class name. * Refactor methods to streamline workflow creation and execution The changes primarily consolidate the process of creating and running a workflow instance into a single operation. Specifically, the 'CreateAndRunWorkflowInstanceRequest' class is utilized in multiple modules to simplify and streamline the workflow creation process. Also, several redundant and inefficient methods were removed in the 'WorkflowGrain' module, and the remaining methods were updated to return the required response directly, resulting in cleaner, more efficient code. * Refactor asynchronous serialization to synchronous Simplified serialization by converting all asynchronous tasks in the workflow state serializer to synchronous ones. This change affects Elsa's Core, Dapper, EntityFrameworkCore, Management, and ProtoActor modules, as well as the Workflow State Serializer - switching all async workflow state serialization methods to their synchronous equivalents. * Removed Elsa.ServiceBus.IntegrationTests project The Elsa.ServiceBus.IntegrationTests project and all related files and references were removed from the solution. This update affects the main application and several workflow files, scenarios, and helper methods. * Refactor WorkflowGrain and update ProtoActor timeouts Renamed the Method OnStopped to async and replaced Context.Stop with Context.Poison in the Method Stop within the WorkflowGrain.cs. Also, adjusted the ActorRequestTimeout to a shorter duration and commented out the LegacyRequestTimeoutBehavior() in ProtoActorFeature.cs. These changes aim to enhance the efficiency and performance of the system. * Refactor Workflow execution and ProtoActor interaction This commit refactors the execution of workflows to manage re-entrancy and improve sequential calls. It also modifies the interaction model between ProtoActor grains and clients by following the ask-pattern. The ProtoWorkflowSubStatus enumeration has been extended to include a "Pending" state. The ActorRequestTimeout has been increased for better debugging. * Remove unused queue and receive timeout in WorkflowGrain The _executionQueue was initialized but never used in the WorkflowGrain class. This removal leads to cleaner and less confusing code. Additionally, the Context.SetReceiveTimeout method call has been removed from OnStarted method as it's no longer needed. * Add ProtoActor to WorkflowServer runtime In the WorkflowServer of Elsa Workflows Component Tests, the ProtoActor has been added to the runtime settings. This enhances the overall functionality and efficiency of the server. * Add new component tests for Elsa.AzureServiceBus and remove old unit tests In this commit, a new set of component tests for Elsa.AzureServiceBus have been added, providing more detailed and reliable testing. Simultaneously, several old unit test files and projects have been removed as they're no longer relevant or useful. These include tests in the 'Elsa.Workflows.Runtime.UnitTests1', 'Elsa.Activities.UnitTests' and 'Elsa.JavaScript.UnitTests' namespaces among others. The decision to remove these tests is motivated by the desire to streamline the testing process and focus on the most meaningful and reliable tests. * Rename GlobalUsings.cs to Usings.cs in integration tests Renamed the GlobalUsings.cs file to Usings.cs in the Elsa.Alterations.IntegrationTests project to better reflect its purpose. This change is intended to improve clarity within the codebase. * Refactor Azure service bus testing setup to separate extension This commit abstracts the setup for Azure service bus testing into a separate extension named AzureServiceBusServiceCollectionExtensions. The code has been removed from the WorkflowServer class, contributing to a cleaner and more modular codebase. This enhancement will facilitate better unit testing and reduce redundancy in test setup. * Uncomment 'Description' and remove 'OptionsProvider' and 'OptionsMethod' In the 'InputAttribute.cs' file, the 'Description' property has been uncommented to allow for a brief description of properties during workflow tooling. Furthermore, the 'OptionsProvider' and 'OptionsMethod' properties have been removed, simplifying the attribute options handling. * Enable Azure Service Bus module The change updates the 'useAzureServiceBusModule' constant from false to true in the Elsa.Server.Web Program.cs file. This adjustment allows the application to employ the Azure Service Bus module. * Refactor ProtoActor module for workflow instance focus The main changes in this commit revolve around renaming and refactoring to orient the ProtoActor module toward handling workflow instances. In the process, unnecessary imports have been removed, classes have been renamed to reflect their new focus on workflow instances, and various related elements such as protobuf files and services have also been renamed and refactored to align with these changes. * Change AnalysisModeDocumentation to 'AllDisabledByDefault' The AnalysisModeDocumentation setting in Directory.Build.props has been updated to 'AllDisabledByDefault' from 'Default'. This change disables all analysis by default in the documentation generation process. * Update default value for Content and modify build properties Modified the default value for the property 'Content' in the 'Message' class. Also updated the build properties by changing the 'AnalysisModeDocumentation' to "Default", disabling 'EnableTrimAnalyzer', and adding exception warning codes 'CS0162' and 'CS1591' to 'NoWarn'. * Disable Azure Service Bus and initialize Customer fields With this commit, the Azure Service Bus module usage has been turned off. In addition, the initial fields of the Customer entity have been set to their default values for safer initialization and to avoid potential null reference exceptions. * Add distributed workflow services and configurations Implemented basic functionalities in DistributedWorkflowClient and DistributedWorkflowRuntime. Updated Web Application 'Program.cs' to support different WorkflowRuntimes as per the configuration. Renamed WorkflowRuntimeFeature.cs to DistributedRuntimeFeature.cs and refactored the code accordingly. Removed unnecessary method 'UseDefaultWorkflowRuntime' from ModuleExtensions.cs. * Refactor ReceivedServiceBusMessageModel from record to class Converted ReceivedServiceBusMessageModel from a record to a class and updated relevant initialization code. Change was implemented because of the polymorphic serialization incabability of dealing with $type properties with records. * Refactor WorkflowInstanceImpl for improved workflow management This change refactors the WorkflowInstanceImpl class to improve workflow management. It implements queuing of RunWorkflowOptions while a workflow is running and ensures state before any workflow operations. Furthermore, workflow host creation is replaced with directly creating and managing workflow instances. * Add DefaultFormattersFeature and update dependencies A new feature, DefaultFormattersFeature, has been added to the Elsa.Common module. The WorkflowsFeature in the Elsa.Workflows.Core module has been updated to depend on this new feature. This will ensure that default JSON formatters are available across the application. * Refactor JsonFormatter with JsonSerializerOptions property Improved the JsonFormatter class by introducing a private JsonSerializerOptions property. The new implementation uses this property in the FromStringAsync method instead of initializing a new JsonSerializerOptions each time, improving efficiency. The JsonStringEnumConverter has also been moved to the class constructor. * Refactor WorkflowInstanceImpl for improved code clarity The refactor includes replacing direct field access with properties in multiple places for WorkflowGraph and WorkflowState. It also makes _queuedRunWorkflowOptions field readonly and removes unnecessary newline characters. These changes aim to improve readability, maintainability and encapsulation. * Add and update methods to workflow instances This commit introduces add and update functionalities to the workflow instances across the application. Additionally, it includes enhancements to the workflow execution, such as distributed locking and expanded logging capabilities for better debugging and tracking. Lastly, it introduces new configurations in application settings to fine-tune the workflow runtime environment. * Refactor worker management in AzureServiceBus module The codebase has been simplified by removing the unnecessary management of worker ref counts in the AzureServiceBus module. Previously, to manage worker instances, we kept a count of references to each worker and removed it when the count dropped to zero. This complexity has been entirely removed to make the module simpler and easier to maintain. Now we just create a worker when necessary, without tracking its usage across the codebase. * Refactor workflow runtime with distributed locking and state checking The workflow runtime has been refactored for better concurrency control and state management. The changes primarily include the introduction of distributed locking in action methods of the DistributedWorkflowClient and the handling of workflow instance states in the LocalWorkflowClient. This entails changes to the logic in the RunInstanceAsync, CreateAndRunInstanceAsync, CancelAsync, ExportStateAsync and ImportStateAsync methods. * Refactor DispatchWorkflowInstance in MassTransitWorkflowDispatcher The DispatchWorkflowInstance model in the MassTransitWorkflowDispatcher has been refactored. Previous detailed activity parameters (ActivityId, ActivityNodeId, ActivityInstanceId, ActivityHash) were replaced with a single ActivityHandle property. An additional Properties field was added. * Update test in BulkDispatchWorkflowsTests The DispatchAndWaitWorkflow test case in the BulkDispatchWorkflowsTests has been updated to use the new workflow client creation and instance running methods. Necessary namespace imports have also been added in the process to support the changes. * Remove snapshot and persistence functionality from WorkflowInstanceImpl This commit eliminates snapshot creation and persistence from the WorkflowInstanceImpl class. Methods related to these processes have been removed, including SaveSnapshotAsync(), ApplySnapshot(), and GetState(). This has also impacted the constructor and OnStarted() method where certain calls were made related to persistence. The need to manage and recover state asynchronously was eliminated, simplifying the implementation. * Refactor workflow client implementations and update WorkflowStateMapper Workflow clients in the modules Elsa.Workflows.Runtime and Elsa.ProtoActor have been refactored for better maintainability and readability. The WorkflowStateMapper in the Elsa.Workflows.Management is updated to include an "Apply" method, splitting the mapping functionality into smaller, more manageable methods. Also, usage of the Azure Service Bus Module in Elsa.Server.Web has been turned off. * Update BulkDispatchWorkflowsTests specification This commit updates the test specification in BulkDispatchWorkflowsTests by removing an unnecessary import and updating the WorkflowDefinitionHandle in the DispatchAndWaitWorkflow test method. Also, superfluous comments have been removed for readability. * Update workflow definition in BulkDispatchWorkflowsTests Changed the referenced workflow definition in the test scenario from EmployeeGreetingWorkflow to GreetEmployeesWorkflow. This update reflects more precise naming in our component tests for the BulkDispatchWorkflows scenario. * Refactor BulkDispatchWorkflows and simplify error handling The BulkDispatchWorkflows class in Elsa Workflows Runtime module has been refactored to simplify it. The detailed error handling with a list of errors has been removed and it is now directly dispatching child workflows without checking for a successful dispatch. The ProcessItem method has been removed which handled the errors previously, and the DispatchChildWorkflowAsync method no longer returns a dispatch response. This substantially simplifies the code and reduces its complexity. * Update workflow runtime and distributed locking configuration settings In the web server settings, the workflow runtime was changed from 'Distributed' to 'ProtoActor'. Also, the distributed lock provider setting has been moved into a new 'DistributedLocking' section in the configuration settings, allowing more detailed lock options to be set, including a lock acquisition timeout. * Replace ProtoActor with DistributedRuntime in WorkflowServer The current commit modifies our WorkflowServer configuration under our component tests. The "UseProtoActor" method which was previously used for runtime configuration has been commented out and replaced with "UseDistributedRuntime". This change suggests a shift towards a distributed runtime environment. * Add new services and classes for workflow messaging This commit adds new classes and services to facilitate workflow messaging. These include 'BroadcastWorkflowInboxMessageOptions', 'IWorkflowInbox', 'NewWorkflowInboxMessage', 'SubmitWorkflowInboxMessageResult', 'WorkflowInboxMessage', 'WorkflowInboxMessageDeliveryParams', and 'WorkflowInboxMessageFilter'. Additionally, changes were made in 'HttpWorkflowsMiddleware.cs' and 'WorkflowRuntimeFeature.cs' to use 'IWorkflowRunner' for executing workflows, and deprecated 'IWorkflowHostFactory' and 'WorkflowInvoker'. The messaging-related classes provide methods and properties to manage and manipulate workflow messages, delivery options and results. 'IWorkflowInbox' provides an interface for delivering messages to workflow instances while 'IWorkflowRunner' fulfills running workflows. * Refactor WorkflowCancellationService for cleaner syntax Aesthetic adjustments were made to the WorkflowCancellationService code to improve readability and maintainability. This includes tidying up line breaks, reformatting lists, correcting spelling in comments, and consolidating parameters in a function call. * Remove unused import in MassTransitWorkflowCancellationDispatcher The 'Elsa.Workflows.Runtime.Contracts' namespace was removed from the MassTransitWorkflowCancellationDispatcher file because it was not being utilized. This ensures a clean and efficient codebase by removing unnecessary imports. * Remove Class1 from Elsa.Testing.Shared.Component This commit deletes Class1.cs as it was no longer serving any purpose in the Elsa.Testing.Shared.Component. This removal helps keep the codebase clean and maintainable. * Reduce default timeout in ISignalManager interface The default millisecond timeout for the WaitAsync functions in the ISignalManager interface has been reduced from 5000ms to 1000ms. This change will speed up signal wait times in our testing framework. * Refactor syntax representation in SendMessage activity Simplified the syntax representation in the SendMesssage activity in the Elsa Azure Service Bus module. The SupportedSyntaxes property now utilizes a more concise array initialization. * Removed obsolete 'Stimulus' property and adjusted consumers The 'Stimulus' property in DispatchResumeWorkflows and DispatchTriggerWorkflowsRequest classes has been removed as it was marked obsolete. The DispatchWorkflowRequestConsumer has been adjusted to only use the 'BookmarkPayload' property. This ensures a cleaner code base and removes potential confusion between the properties. * Rename GetNamedWorkflowGrain to GetNamedWorkflowInstanceClient The method name GetNamedWorkflowGrain in both ProtoActorWorkflowClient and ClusterExtensions has been renamed to GetNamedWorkflowInstanceClient to improve code clarity. This change provides better semantics of what the function is purposed for. * Remove unused snapshot classes The commit removes the WorkflowGrainSnapshot and WorkflowInstanceGrainSnapshot classes from the Elsa.ProtoActor module. These classes were part of an older architecture and are no longer required. * Remove unused field from Azure ServiceBus Worker The _refCount field in the Worker class from the Azure ServiceBus module was unused. To maintain a clean codebase and improve readability, this field has been removed. * Remove WorkflowInboxMessageRecord from Elsa.Dapper module The commit includes the deletion of the entire WorkflowInboxMessageRecord class from Elsa.Dapper module. This file was managing various activities related to the workflow inbox messages, such as delivering messages to a workflow instance. * Add new V3_2 migrations for all database providers This commit introduces new V3_2 migrations for MySQL, SQLite, PostgreSQL, and SQLServer database providers. Included are `Up` and `Down` migrations, the corresponding Designer files, and specific Alterations and Runtime changes. SqlDbType specifications were also added for each. * Refactor code and update namespaces in multiple modules The changes update the namespaces of various files and refactor code for efficiency and readability. In addition, some unnecessary comments and redundant namespaces were removed, and import statements were corrected. Functions were also simplified by removing unneeded parameters and streamlining logic. * Refactor namespace from Elsa.Tenants.Contracts to Elsa.Tenants The refactor operation includes the removal of 'Contracts' from the namespace under the Elsa.Tenants module. This modification affects several files across different modules where the namespace is being referenced. In addition, a new folder for Multitenancy scenarios was created in the Elsa.Workflows.ComponentTests project. * Remove V3_2 database migration files The commit includes removal of EntityFrameworkCore database migrations files across various databases such as MySql, PostgreSql, Sqlite, SqlServer etc. The V3_2 migration files were presumably outdated, unnecessary or causing issues in the project. * Add TenantId field and index to various tables This update adds a new 'TenantId' column to several tables in the database including 'WorkflowExecutionLogRecords', 'Triggers', 'KeyValuePairs', 'Bookmarks', and 'ActivityExecutionRecords'. It also creates indices for this new column in each corresponding table. This enhancement is added across various database types including MySQL, SQL Server, SQLite and PostgreSQL. * Added TenantId column to multiple tables In the database migration script, the "TenantId" column has been added to the Triggers, Bookmarks, WorkflowExecutionLogRecords, ActivityExecutionRecords, and KeyValuePairs tables. Also, the process of removal of the same column from these tables has been scripted for rollback scenarios. * Update migration version number in V3_2.cs The commit contains an update in the version number of the migration Elsa:Runtime:V3.2 in V3_2.cs file. The previous version was 20002; it has now been updated to 20003. * Add tenantID support to workflow builder The update introduces tenantID support to the workflow builder. This allows workflows to be associated with different tenants, improving the multi-tenant support. The changes include adding a `TenantId` property and a `WithTenantId` method, and adjusting the workflow identity creation to include the tenant ID. * Add tenant functionality to WorkflowServer The WorkflowServer now supports multi-tenancy. Added `UseTenants` extension method where new TestTenantsProvider is configured which will provide multiple tenants. A TenantResolutionStrategy is appended to tenant options to decide the tenant context in runtime. Also, `ITenantResolutionStrategy` is registered in the dependency injection container. * Add multitenancy tests and utility classes Added test scenarios for multitenancy in the Elsa workflow component. Utility classes for tenant resolution strategy and tenants provider are also added to help perform the tests. Different workflows for different tenants are set up for testing. * Handle OperationCanceledException in BackgroundEventPublisherHostedService An additional catch block has been added to handle OperationCanceledException within the BackgroundEventPublisherHostedService. This provides a more specific message when an operation is cancelled during the queue processing. * Refactor WorkflowDefinitionFilter and update related modules Empty lines were removed between property declarations to improve readability in WorkflowDefinitionFilter. An error where 'filter.IsReadonly' was used instead of 'IsReadonly' was also corrected to ensure proper functionality. Additionally, WorkflowDefinitionFilter was imported in the Delete and Revert endpoint modules of the Elsa.Workflow.Api class to maintain consistency across the codebase. * Change MongoUserStore to non-abstract class The MongoUserStore class was previously tagged as an abstract class in the Identity module of Elsa.MongoDb. This update changes the MongoUserStore class from an abstract to a non-abstract (concrete) class to enable direct instantiation. * Add ForwardedType attribute and update bookmarks This commit introduces a new `ForwardedType` attribute to aid in forwarding types to new types when deserializing JSON. Additionally, the existing bookmarks related to workflow runtime activities have been updated. These changes also require modifying the `PolymorphicSerializer` in the Elsa.MongoDb project to handle types using the new `TypeHelper.GetLatestType` method. * Correct typos in code comments Fixed typographical errors in comments of the "JobId" property in both BackgroundActivityStimulus and BackgroundActivityBookmark classes. This minor change ensures accuracy and clarity in the code documentation. * Refactor Workflow API by altering import statements This commit modifies the import statements in various files under the Workflow API endpoints. Most changes involve replacing Elsa.Workflows.Management.Contracts import with Elsa.Workflows.Management. This refactoring ensures the correct classes and interfaces are used from the updated module path. Few methods have additional parameters added as well, enhancing their functionality. * Implement DefaultTenantResolver and update tenant resolution logic This commit introduces a new DefaultTenantResolver class which always returns the default tenant. The previous TenantResolver is renamed to PipelinedTenantResolver. The ITenantResolver interface has also been moved to a more general namespace. All services using ITenantResolver are updated accordingly. Furthermore, TenantResolverFeature is now a dependency of WorkflowsFeature. * Fix logger reference in exception handling The logger reference used in the catch block of the DefaultTriggerScheduler.cs file was incorrect. This commit corrects it by using the appropriate logger variable for logging any potential cron expression format errors. * Refactor ElsaDbContextBase constructor and remove unused usings Removed unnecessary using directives and modified the `ElsaDbContextBase` constructor. Introduced `IServiceProvider` dependency to the constructor and removed `SaveChangesAsync` method. Also, a set of `ModifiedEntityStates` was added. * Add async SaveChangesAsync method in ElsaDbContextBase An override for the SaveChangesAsync method has been added to the ElsaDbContextBase class. This method ensures OnBeforeSavingAsync is called prior to the base.SaveChangesAsync method, allowing for pre-save operations to be queued and executed asynchronously. * Add Identity module to Workflow Server configurations The Identity module from Elsa.EntityFrameworkCore is now added to the Workflow Server configurations in both Elsa.Workflows.ComponentTests and Elsa.AzureServiceBus.ComponentTests projects. This module is configured to use Entity Framework Core with PostgreSQL as a data provider. * Replace key with ID in key-value pair handling The key identifier from various key-value pair references has been replaced with ID in several modules, as reflected in the recent code changes. The 'Key' field was removed in the SerializedKeyValuePair class, related entity configurations, and multiple store methods. It's expected to improve the consistency of data identification across different components in the application. * Remove unnecessary code and refactor namespace usage Deleted the obsolete DictionaryExtensions.cs file, and did extensive refactoring of namespace usage. Some classes have been moved to new namespaces while others have been removed. Also removed methods that were not used or duplicated elsewhere in the codebase. * Update import in AzureServiceBusServiceCollectionExtensions The import in AzureServiceBusServiceCollectionExtensions test file was changed. Elsa.Extensions was used instead of Elsa.Workflows.ComponentTests, adjusting the dependencies for the test configuration. * Remove outdated migrations for Elsa.EntityFrameworkCore This commit removes old migration files pertaining to different databases in the Elsa.EntityFrameworkCore module. These outdated migrations versioned as 'V3_2' are no longer necessary in the current context of the application. The removal helps to keep the codebase cleaner and easier to manage. * Update migration scripts and include in Elsa solution Deleted previous versions of migration scripts and created new versions for EFCore 3.0, 3.1, and 3.3. These scripts have been included in the Elsa.sln, to facilitate regular and consistent updates of database schemas across various provider types (MySQL, SqlServer, Sqlite, PostgreSQL). * Add TenantId to multiple tables and remove WorkflowInboxMessages table The commit introduces a new TenantId column to several tables including WorkflowExecutionLogRecords, Triggers, KeyValuePairs, Bookmarks, and ActivityExecutionRecords in different database providers (MySql, PostgreSql, SqlServer, Sqlite). The changes will allow isolating data based on tenant ids. This commit also removes the WorkflowInboxMessages table which is not needed anymore. * Add PostgreSQL support to workflow servers The code includes additional extensions to Elsa workflow server configurations. The changes involve the alteration modules, enabling them to use EntityFrameworkCore with PostgreSQL. This was applied in two workflow servers for component testing, i.e., Elsa.Workflows.ComponentTests and Elsa.AzureServiceBus.ComponentTests. --------- Co-authored-by: jeanbaptistedalle <jean-baptiste.dalle@laposte.net> Co-authored-by: Jean-Baptiste Dalle <jean-baptiste.dalle@stereograph.fr> Co-authored-by: Steve Taylor <stevetayloruk@users.noreply.github.com>	2024-06-10 21:30:53 +02:00
Sipke Schoorstra	77a71afc7a	Refactor Workflow Runtimes (#5444) ... * Add caching to workflow runtime and workflow management stores The update introduces caching to workflow runtime and workflow management stores to enhance performance. This is achieved by adding decorators for several stores, which cache records to reduce database fetches. Additionally, a signaler for change tokens allows for cache invalidation when changes occur. The MemoryCache feature has also been updated to include Scrutor for decoration and the caching duration can be configured through the new CachingOptions class. * Refactor WorkflowsMiddleware for HTTP Endpoint bookmarks and triggers The WorkflowsMiddleware has been extensively refactored to handle HTTP Endpoint bookmarks and triggers. This involves breaking down the InvokeAsync method by extracting parts of its functionality into separate helper methods such as FindTriggersAsync and FindBookmarksAsync. Moreover, Assist with authorization checks, workflow execution within request timeout, and handling of workflow faults has been improved to be more efficient and clearly segmented. * Update HTTP endpoint authorization to use Workflow context The authorization process in the AuthenticationBasedHttpEndpointAuthorizationHandler class has been updated to use the Workflow context instead of the WorkflowInstanceId string. The AuthorizeHttpEndpointContext model has been correspondingly changed to include a Workflow property, thereby strengthening the link between authorization and specific workflows. * Add FindAsync methods to trigger and bookmark stores The code adjustments add new FindAsync methods to the trigger and bookmark store contracts as well as all their concrete implementations (MongoDb, Memory and EFCore). These methods support fetching the first record matching a given filter. The adjustments also include minor syntax improvements and the addition of [UsedImplicitly] attributes where needed. * Refactor WorkflowsMiddleware for improved workflow handling The code was refactored to simplify the flow of handling workflows in the WorkflowsMiddleware class. Specifically, the methods to start and resume a workflow have been extracted to improve code readability. Further, handleErrorMiddlewares was also updated to better manage instances where no valid workflows or base paths are found. * Add caching functionality to WorkflowsMiddleware Added IMemoryCache usage in the WorkflowsMiddleware to cache lookup results for workflows and their associated triggers. This will reduce the number of database operations required when searching for workflows, thus improving performance. The cache is maintained for one minute before it is refreshed. * Implement dynamic cache duration for workflows The code has been updated to have a dynamic cache duration for the workflows instead of a hardcoded one minute. By using the CachingOptions service, the cache duration can now be set in the configuration making it more flexible and adaptable to different performance needs. * Add HttpWorkflowsCacheManager for caching HTTP workflows This commit includes the implementation of IHttpWorkflowsCacheManager for caching of HTTP workflows. New handlers have been added to invalidate cache on workflow updates. Additionally, WorkflowsMiddleware has been renamed to HttpWorkflowsMiddleware. * Refactor workflow trigger handling and caching The refactoring includes deletion of `IndexWorkflowTriggersHandler.cs` and creation of `IndexTriggers.cs` thus revising the workflow trigger indexing approach. Also, revamped the `ITriggerIndexer` interface which now handles deletion of triggers with specific workflow and filter. Furthermore, the caching mechanism in `HttpWorkflowsCacheManager.cs` is modified to handle eviction of workflow definitions and triggers separately boosting its efficiency. * Add summary to IndexedWorkflowTriggers A summary has been added to the 'IndexedWorkflowTriggers' class, providing a brief description. This description outlines that it represents a collection of indexed workflow triggers, promoting clearer understanding for future reference. * Refactor memory caching feature into separate module This commit separates the memory caching feature from the Elsa.Common module into a distinct Elsa.Caching module. This includes moving and renaming related files, such as the MemoryCacheFeature class and associated dependencies. The references in other modules and in the main solution file have been updated accordingly to include the new Elsa.Caching module. * Add distributed caching and update async methods Introduced a distributed caching feature with extensible change token signal publishing. Updated various cache-related methods to be asynchronous for improved performance and responsiveness. Also updated some workflow identity references for clarity. * Add distributed caching with MassTransit support This addition includes the implementation of a distributed caching system with MassTransit transport. The changes introduce necessary interfaces and services, new distributed caching feature along with the support for MassTransit as a transport option. Moreover, the instance management feature has been renamed to clustering feature for better clarity. * Refactor queue naming and scope of MassTransitChangeTokenSignalPublisher Queue name construction is adjusted in the RabbitMqServiceBusFeature and AzureServiceBusFeature modules for better organization and readability. MassTransitChangeTokenSignalPublisher is now a singleton service, ensuring the signal publisher can be shared across the application, increasing efficiency and performance. Also, introduced use of DistributedCacheFeature in MassTransitDistributedCacheFeature module for better modularization. * Add caching capabilities to workflow definition service This commit introduces caching to the workflow definition service, improving the performance for retrieving workflow definitions. 4 new classes have been created (`CachingWorkflowDefinitionService`, `EvictWorkflowDefinitionServiceCache`, `WorkflowDefinitionCacheManager`, and `IWorkflowDefinitionCacheManager`), and several existing classes have been updated to support caching. The caching also includes invalidation mechanisms, ensuring data consistency. * Refactor caching mechanism in workflow definition In the workflow definition module, the explicit caching functionality related to workflow definition versioning has been removed in favor of a more streamlined approach. Additionally, the manner in which services are registered has been altered. As a result, the caching now directly involves the overall workflow definition rather than individual versions, simplifying the caching logic and potentially improving the performance. * Update MassTransitBroker and enable RealTimeWorkflows and SignalRHubs The MassTransitBroker has been updated to Memory from RabbitMq. In addition, the RealTimeWorkflows and UseWorkflowsSignalRHubs features are now enabled in the code. This change will impact how the service communicates and processes real-time requests for workflow operations. * Reformat variable types in HttpFeature The reformatting involves a list of variable types in the HttpFeature module. Each type now appears on a new line for improved readability, making the code easier to maintain and review. * Update HTTP workflows cache invalidation handler XML comment * Remove unnecessary using directives Unnecessary using directives were deleted across several files in the Elsa.Http module. This simplifies the code and will possibly improve execution speed. Specific deletions include those for Encoding, Unicode, Extensions, Collections.Generic, Linq, Text, and Tasks namespaces. * Refactor HttpWorkflowsMiddleware constructor This commit simplifies the HttpWorkflowsMiddleware class constructor. It removes the intermediary variables `_next` and `_options` and directly uses the passed arguments in the constructor. Now, the `next` and `options` parameters are used directly throughout the middleware. * Simplify workflow retrieval in HttpWorkflowsMiddleware This refactoring replaces the use of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync with a single method, FindWorkflowAsync. This simplifies the middleware code and likely improves performance by reducing the number of database queries or service calls required to retrieve a workflow. * Refactor workflow retrieval in HttpBookmarkProcessor Simplified the workflow retrieval process in HttpBookmarkProcessor.cs. Replaced FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods with a single FindWorkflowAsync call. This reduces the complexity and improves efficiency in retrieving workflow. * Optimize FindWorkflowAsync method in HttpWorkflowsCacheManager Removed redundant lines of code to simplify workflow search functionality. This change simplified the FindWorkflowAsync method by directly calling the FindWorkflowAsync function in the workflowDefinitionService, thus increasing code readability and efficiency. * Refactor Endpoint.cs for workflow retrieval The method for obtaining a workflow in the Endpoint.cs script has been refactored and streamlined. The 'GetWorkflowDefinition' method is replaced by the 'GetWorkflowAsync' method which directly retrieves the workflow, without the intermediate step of materializing the workflow definition. This shortens the code and simplifies the process. * Refactor InputFunctionsDefinitionProvider constructor The constructor for InputFunctionsDefinitionProvider has been simplified by removing unnecessary private fields. Services are now directly used in the method instead of being stored in fields. This improves readability and reduces complexity in the class structure. * Refactor WorkflowInstance with improved state handling Simplified the methods for handling workflow and workflow state in the WorkflowInstance class. The refactoring also included some code clean-ups and variable renaming. The new implementation provides better readability and maintainability of the code by reducing unnecessary lines and improving structuring of objects and responses. * Remove unused IBookmarkManager and update workflow functions IBookmarkManager from ProtoActorWorkflowRuntime.cs file is removed due to its redundant status. Additionally, the "FindAsync" method has been updated to use a cancellation token. Also, annotations were added to the "ExportWorkflowStateAsync" and "ImportWorkflowStateAsync" methods to flag calls to functions that require unreferenced code. * Remove unused ReSharper directive Unused ReSharper directive in the file IndexTriggers.cs was identified and therefore removed. This change makes the code cleaner and easier to read. * Update activity invocation in workflow runtime Updated the DefaultBackgroundActivityInvoker service in the Elsa.Workflows.Runtime module to annotate the ExecuteAsync method with "RequiresUnreferencedCode" attribute. This change is made considering the potential code trimming issue. Additionally, simplified the process of fetching workflow by directly using FindWorkflowAsync method instead of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods. * Refactor code to simplify workflow definition loading The code for finding and materializing workflow definitions has been simplified. Instead of loading the definition and materializing it into a workflow in separate steps, a new method called FindWorkflowAsync has been introduced to perform both actions at once. This reduces redundancy and makes the code more readable. * Refactor WorkflowHostFactory to streamline workflow creation This commit simplifies the workflow creation process in WorkflowHostFactory. It removes redundant code and extraneous methods, specifically the overloaded CreateAsync method which used WorkflowDefinition. Now, it directly finds and uses the Workflow instance, thereby simplifying the code base and improving maintainability. * Refactor workflow retrieval in WorkflowInstance.cs Changed the way workflow instances are retrieved from the WorkflowDefinitionService. Instead of obtaining the workflow definition and then materializing the workflow from it, the workflow is directly retrieved using the FindWorkflowAsync function. This simplifies the code and avoids unnecessary null-checks. * Remove unnecessary whitespace in WorkflowInstance.cs An extraneous whitespace character was identified and removed in the WorkflowInstance.cs file. This change contributes towards maintaining clean and readable code in the Elsa.ProtoActor module. * Remove unnecessary comment in ProtoActorWorkflowRuntime The unnecessary comment ("Load the workflow definition.") in the method TryStartWorkflowAsync of the ProtoActorWorkflowRuntime.cs file was removed. This is part of an ongoing effort to keep the codebase clean and readable. * Update workflow management features and handlers Added explicit notification handlers for DeleteWorkflowInstances and RefreshActivityRegistry in WorkflowManagementFeature.cs. Also, renamed RefreshActivityRegistryHandler.cs to RefreshActivityRegistry.cs for better clarity. * Add multiple log record support to workflow execution log stores The major change of this commit is the addition of methods to add multiple log records in the WorkflowExecutionLogStore, across different storage modules such as EntityFramework, MongoDB, Dapper, Elasticsearch and Memory. This ensures consistency and uniform behavior across different storage types. Furthermore, some reformatting and tidying up of the code were undertaken to maintain readability and clarity. * Remove redundant workflow definition check The workflow definition existence check and related service retrieval were removed from HttpWorkflowsMiddleware.cs. It was determined that this check was unnecessary as the workflow definition's existence is guaranteed at this point in the process, reducing redundancy in the code. * Improve cancellation token usage in workflow execution This commit refines the usage of cancellation tokens during the execution of workflows in Elsa.Server and HttpWorkflowsMiddleware. Previously, a cancellation token pair was created before ExecuteWithinTimeoutAsync was called, which limited duration control solely to that method. Now, cancellation tokens are included within ExecuteWithinTimeoutAsync method. This allows the method to observe any cancellation initiated by outer scopes, enhancing control over the timeout of operations. * Add PersistStateAsync method to WorkflowHost A new PersistStateAsync method has been added to the WorkflowHost, which enables the host to directly persist its own state. The method has been integrated into the DefaultWorkflowRuntime and HttpWorkflowsMiddleware. This update eliminates the need to continuously get instances of IWorkflowInstanceManager to save state, which improves efficiency and code readability. * Refactor DefaultAlterationRunner service This update simplifies the DefaultAlterationRunner service, reducing the number of code lines and removing unnecessary references. The workflow materialization step has been merged with the find workflow step, and unused namespaces have been dropped. * Refine wording in IWorkflowHost interface documentation The documentation for the 'CanStartWorkflowAsync' method in the IWorkflowHost interface has been cleaned up. The superfluous "or not" verbiage has been removed, making it easier to understand the method's function. * Remove CancellationTokens struct and simplify cancellation handling Removed the CancellationTokens struct and all its references across the code base. Simplified cancellation handling by using standard CancellationToken only. Also cleaned up redundant usages and unnecessary namespaces across various modules. This simplification better aligns with standard .NET conventions and reduces the code complexity. * Implement ActivityHandle for better activity identification The ActivityHandle class has been introduced to consolidate various activity identification parameters such as ActivityId, ActivityNodeId, ActivityInstanceId, and ActivityHash. This makes it easier to track and manage activities by reducing the number of parameters needed for identification. Adoption of ActivityHandle has been implemented across the codebase. * Remove AzureContainerApps related code This commit includes the removal of all the AzureContainerApps associated code from the ProtoActor Cluster. The deletion includes multiple files which had housed the AzureContainerApps cluster provider and all its supporting services, util functions, options, contracts, and models. This move might be towards refactoring the structure or removal of unwanted dependencies. * Add distributed execution runtime and client Removed some unused classes and created a distributed execution runtime using distributed locking and persistence. Introduced an interface for workflow clients and implemented two versions, a local one and one using `Proto.Actor`. Made some changes in other modules to add the necessary interfaces and methods. Also, added `DistributedLockingRuntime` module to implement distributed execution. * Remove WorkflowClient.cs from Elsa.Workflows.Runtime The WorkflowClient.cs file was removed from the Elsa.Workflows.Runtime project. It held placeholder methods that had not yet been implemented, making it unnecessary in the current codebase. * Add ProtoActor implementation for workflow execution Added ProtoActor implementation for executing workflow instances. This includes ProtoBuf message definitions, grain interfaces and implementations, and various ProtoActor services. Extended WorkflowHost and related components to provide more execution details in order to better support distributed processing via ProtoActor. * Refactor workflow parameters to workflow requests The refactor includes renaming old 'WorkflowParams' classes to 'WorkflowRequest' to better represent their usage. The changes have been made across all the involved modules ensuring that the project maintains consistency. New properties, methods, and class names were updated accordingly. * Add ProtoActor implementation for data mappers This update adds the ProtoActor implementation for data mappers and updates the DI configuration accordingly. It also updates the client interfaces for a more streamlined usage, and refactors the WorkflowHost to adapt to these changes. All changes were implemented in accordance with the new bookmark information model. * Add functionality to create a new workflow instance This commit introduces the capability to create a new workflow instance in the Elsa Workflow Runtime by enhancing existing workflow classes and creating the WorkflowClientFactoryExtensions class, in addition to adding the "IsNewInstance" parameter to multiple workflows and requests. It provides more flexibility in managing workflow instances and aids in building workflow processes that require instantiation of a new workflow. * Remove Elsa.Runtimes.DistributedLockingRuntime module This commit deletes the Elsa.Runtimes.DistributedLockingRuntime module from the project. Changes include removed code files related to features, services and commands under this module. The Elsa.sln file has been updated to reflect these changes, excluding the removed module from the solution. * Add dynamic client type to WorkflowClientFactory The WorkflowClientFactory now accepts a client Type parameter for dynamic client creation. This enables the factory to create different subclasses of IWorkflowClient based on the provided Type. The CreateClient function is updated in the WorkflowClientFactoryExtensions and IWorkflowClientFactory interface, and applied in HttpWorkflowsMiddleware. * Implement Proto.Actor support in Elsa A comprehensive change that introduces support for Proto.Actor clustering to Elsa. It includes a redesign of the workflow client invocation model, transitioning the methods from synchronous to asynchronous. Also, modification to the Proto.Actor-based 'WorkflowGrain' was made to handle state recovery and execution processing. The structural enhancements improve scalability and performance capabilities. * Refactor null-checks in SaveSnapshotAsync method Simplified the null-checking in the method SaveSnapshotAsync within WorkflowGrain.cs. The check on _workflowHost and workflowState has been shortened using conditional access operator, improving the code readability. * Refactor code and update packages The codebase has undergone a major refactoring to improve code readability and consistency, with non-essential methods being removed and variable names being optimized for clarity. Package references for Proto.Actor and related packages have been also updated to the latest versions. * Add Proto.Cluster.AzureContainerApps package to Directory.Packages.props The Proto.Cluster.AzureContainerApps package with version 1.6.0 has been included in the Directory.Packages.props file. This addition extends the range of Proto.Cluster packages used in the project. * Remove redundant Proto.Actor implementation files This commit removes several redundant files related to the Proto.Actor implementation. These files were unnecessary and were cluttering up the codebase. With this removal, the project has become leaner and easier to maintain. It also eliminates potential confusion for future developers working on this project. * Add WorkflowMatcher and replace BookmarkHasher with StimulusHasher This commit introduces a new class, WorkflowMatcher, which represents a contract for finding triggers and bookmarks associated with workflow activities. It also renames and replaces all instances of BookmarkHasher with StimulusHasher to reflect a more accurate function relating to stimulus rather than bookmarks. Other necessary changes were made to ensure consistency with these updates across different classes. It's important to note that WorkflowInbox interface is now marked as obsolete. * Incremental work on stimuli refactoring * Refactor codebase to support new IWorkflowInvoker and invoke workflow logic Multiple files updated to introduce a `IWorkflowInvoker` interface, implementation and supporting classes. This simplified how workflows are instantiated and invoked. Consequently, necessary adjustments made across multiple entities and endpoints to account for the updated invocation procedure. Old logic related to counting running workflows removed. Introduced stimulus concept while deprecating non-compliant classes and methods. * bumped versions to fix dependency vulnerabilities (#5256) * Update patch version in GitHub workflows The version number used in the branch checking step of the GitHub workflows has been updated. Instead of scanning for the branch containing the patch version 3.1.2, it now scans for the branch that contains version 3.1.3. This change is aligned with the updated product version. * Update git branch grep pattern in workflow file The git grep pattern has been corrected to properly identify tagged versions in the GitHub Actions workflow. The correction ensures that the workflow script fetches the right branches as per the release tag instead of patch. * Update grep command in packages workflow The grep command used in the 'packages.yml' GitHub workflow was previously looking for the exact 'refs/tags/3.1.3' string. This commit simplifies the command by making it only look for '3.1.3'. This adjustment will streamline the process and potentially prevent issues with branch recognition. * Update package versions and refactor code for Elasticsearch and JavaScript modules Updated versions of numerous packages in the Directory.Packages.props file to their latest stable releases. This includes updates to Elasticsearch, JavaScript, and MongoDB packages among others. Additionally, refactored parts of the code in the WorkflowInstanceConfiguration and JintJavaScriptEvaluator within the Elasticsearch and JavaScript modules, respectively, to improve index management and script preparation. The WorkflowInstanceStore also saw a minor adjustment. * Refactor workflow management with workflow definition handles The existing workflow management has been significantly refactored, introducing the concept of "Workflow Definition Handles". These handles allow for consistent management whether we're dealing with a specific workflow definition, a version of a definition, or more flexible version constraints. This refactor also adjusts how workflow instances are created, now using a more intuitive and detailed "WorkflowInstanceOptions" approach. * Add ResumeBookmarkResult and update related methods Implemented a ResumeBookmarkResult class to handle bookmark resumption results. Methods related to resuming bookmarks have been refactored to return this new class, providing more information such as bookmarks' matched state. Also, some methods were optimized to break the loop early if no bookmarks were found, improving code efficiency. * Update workflow definition, execution and correlation This commit focuses on enhancements and adjustments to workflow definition and execution. Notable changes include the addition of ExecuteResponse model and new test workflow scenarios. Refactoring has been performed to improve readability and efficiency in various components. It also includes an important fix for the correct application of WorkflowDefinition filters, primarily using both DefinitionId and DefinitionVersionId for more accurate results. * Refactor runtime codebase for better structure and workflow control This commit involves changes to functionally reorganize the runtime codebase for improved structure. It also allows for better handling of workflows, particularly through the addition of original bookmarks in the workflow execution context. Removed unnecessary dependencies and ensured more efficient management of bookmarks in the workflow running process. Several method and class names were also updated to better reflect their purpose. * Refactor WorkflowInvoker and remove 'OriginalBookmarks' Optimized the constructor of 'WorkflowInvoker' by using 'IServiceScopeFactory' to get instances of required services. Removed the 'OriginalBookmarks' property from 'WorkflowStateExtractor' and 'WorkflowState', a subsequent change included in 'WorkflowGraphBuilder' as well. Reorganized the namespaces in 'Elsa.Workflows.ComponentTests' project. Added 'UseCache' in WorkflowServer configurations. * Removed RunWorkflowParams class This update deletes the RunWorkflowParams class within the Runtime Requests of the Elsa.Workflows module. The class was no longer needed, hence the elimination and cleanup in the codebase. * Update RunWorkflowParamsMapper to handle null or empty fields This commit modifies the RunWorkflowParamsMapper in the Elsa.ProtoActor module. It primarily treats the BookmarkId and TriggerActivityId fields to return null if they are empty, boosting the application's robustness against potential null or empty field issues. * Refactor workflow handling and improve null checks In this update, the handling of workflows was refactored to improve efficiency. The 'RunAsync' method now correctly uses 'WorkflowGraph' as a parameter, instead of 'Workflow'. Additionally, null checks for 'bookmarkId' and 'activityHandle.ActivityInstanceId' have been improved to avoid null and empty strings. Finally, when '_workflowInstanceId' is null in the 'WorkflowGrain' class, it is now properly initialized by parsing the cluster identity. * Enable ProtoActor in Elsa.Server.Web This commit turns on the use of ProtoActor within the Elsa.Server.Web bundle. This switch may affect the system's behavior and performance. * Added new workflow scheduling and management features Implemented new features for creating, running, and scheduling workflow instances. The implementation added new files for handling workflow runtime and scheduling features, including creating and running a workflow instance request, a mapper for the request, and handler services. Modified files include updating method calls according to the new requests, updating workflow definitions, and adjustment to method arguments in Hangfire job class. * Handle null or empty workflow instance IDs and correlation IDs This commit introduces null checks for workflow instance IDs and correlation IDs in the workflow infrastructure. Previously, the code assumed that the IDs were provided. It now gracefully handles cases where they might be null or empty, preventing possible null reference exceptions and ensuring more robust workflow execution. * Update mapping details in ResumeWorkflowJob Removed unnecessary using statement for Elsa.Workflows.Runtime.Requests in ResumeWorkflowJob.cs. Updated references from DispatchWorkflowInstanceRequest to ScheduleExistingWorkflowInstanceRequest for retrieving ActivityHandle and WorkflowInstanceId. * Refactor AzureServiceBus module and integrate into web project In this commit, changes were made to the AzureServiceBus module to use Topic definitions for subscriptions. The Subscriptions property in the AzureServiceBusOptions and the Topic property in the SubscriptionDefinition class are marked as obsolete and suggestions to use TopicDefinition.Subscriptions instead have been added. The AzureServiceBus module was also integrated into the web project and configured to use options from the appsettings.json. * Refactor code to use async scopes and improve service dependencies Refactored code to use async scopes for improved task management. Also organized service dependencies better by moving service fetching inside methods where they are needed and propagating necessary dependencies through method parameters for cleaner code. * Add Azure Service Bus workflow component tests This commit introduces a set of workflow component tests for Azure Service Bus integration. These tests encompass scenarios like message receiving, sending messages with correlation IDS, and sending single messages. In addition, 'SignalResetEvent' test helper was removed, a new test helper 'TriggerSignal' was added, and a Mock ServiceBusClient and ServiceBusAdministrationClient were added to the WorkflowServer fixture. Lastly, the NSubstitute package was added to the test project's dependencies. * Add support for deferred tasks in workflow execution context Added support for deferred tasks in the workflow execution context. This implementation allows tasks to be deferred and executed right after persistence of bookmarks in the workflow. A new middleware, ExecuteDeferredActivityTasks, is introduced to handle the execution of these deferred tasks. * Refactor TriggerSignal and SendMessage activity execution Refactored the execution of TriggerSignal and SendMessage activities by using context.DeferTask to ensure the activities run asynchronously. Also, made code format modifications and simplifications such as handling ApplicationProperties better in the SendMessage activity, and other minor changes. * Update workflow ID generation method The workflow ID generation method has been updated in the WorkflowBuilder. Previously, it was always generated by _identityGenerator. Now, it only gets generated if the definitionId or Id is null or empty, otherwise it uses the existing value. This change brings the ID generation practice in line with how we handle definitionId. * Add support for service bus testing in workflow tests Two new helper classes, DictionaryExtensions and MockServiceBusProcessor, have been added to support mocking Azure service bus in workflow component tests. Also, the tests have been updated to use SignalManager to ensure proper order of execution. The WorkflowServer test fixture has been extensively refactored to create mock instances of service bus senders and processors. Furthermore, some configurations have been commented out and new configurations related to MemoryTriggerStore and MemoryBookmarkStore have been added. * Enhanced workflow correlation and caching in Elsa Workflows This update improves Elsa Workflows by enabling caching and enhancing correlation in Correlate.cs. It expands functionality by adding new methods that accept various input types and provide more options for activities correlation. Correlation improvements also extend to AzureServiceBusTests, which were updated to test workflow instances by correlation ID. Furthermore, the test workflow received a name change and new correlation mechanics based on Azure messaging. * Renamed method argument from 'payload' to 'stimulus' The method argument 'payload' in various functions across 'MessageReceived.cs' and 'ActivityExecutionContext.cs' files has been renamed to 'stimulus'. This change was made to improve code readability and understanding by using a more context-specific term. * Refactor Workflow APIs and enhance logging Refactored Workflow APIs by removing the TriggerActivityId from StimulusMetadata and introducing Direct Triggers. Refactored Reenter method in WorkflowGrain to be more concise and straightforward. Enhanced logging by adding additional logging in ActivityExecutionPipeline and LogLevel in appsettings. Also, updated ProtoActorFeature for better log level management. * Refactor AzureServiceBusTests and add workflow completion signal The AzureServiceBusTests class is refactored to separate and encapsulate concerns. SignalManager and WorkflowEvents are now class-level variables. Also, a signal to indicate the completion of a workflow is added. This allows the test to wait for workflow completion in the sequence of its operations. Additionally, the workflow definition ID in MessageReceivedTriggerWorkflow is now static and fixed, rather than dynamically derived from the class name. * Refactor methods to streamline workflow creation and execution The changes primarily consolidate the process of creating and running a workflow instance into a single operation. Specifically, the 'CreateAndRunWorkflowInstanceRequest' class is utilized in multiple modules to simplify and streamline the workflow creation process. Also, several redundant and inefficient methods were removed in the 'WorkflowGrain' module, and the remaining methods were updated to return the required response directly, resulting in cleaner, more efficient code. * Refactor asynchronous serialization to synchronous Simplified serialization by converting all asynchronous tasks in the workflow state serializer to synchronous ones. This change affects Elsa's Core, Dapper, EntityFrameworkCore, Management, and ProtoActor modules, as well as the Workflow State Serializer - switching all async workflow state serialization methods to their synchronous equivalents. * Removed Elsa.ServiceBus.IntegrationTests project The Elsa.ServiceBus.IntegrationTests project and all related files and references were removed from the solution. This update affects the main application and several workflow files, scenarios, and helper methods. * Refactor WorkflowGrain and update ProtoActor timeouts Renamed the Method OnStopped to async and replaced Context.Stop with Context.Poison in the Method Stop within the WorkflowGrain.cs. Also, adjusted the ActorRequestTimeout to a shorter duration and commented out the LegacyRequestTimeoutBehavior() in ProtoActorFeature.cs. These changes aim to enhance the efficiency and performance of the system. * Refactor Workflow execution and ProtoActor interaction This commit refactors the execution of workflows to manage re-entrancy and improve sequential calls. It also modifies the interaction model between ProtoActor grains and clients by following the ask-pattern. The ProtoWorkflowSubStatus enumeration has been extended to include a "Pending" state. The ActorRequestTimeout has been increased for better debugging. * Remove unused queue and receive timeout in WorkflowGrain The _executionQueue was initialized but never used in the WorkflowGrain class. This removal leads to cleaner and less confusing code. Additionally, the Context.SetReceiveTimeout method call has been removed from OnStarted method as it's no longer needed. * Add ProtoActor to WorkflowServer runtime In the WorkflowServer of Elsa Workflows Component Tests, the ProtoActor has been added to the runtime settings. This enhances the overall functionality and efficiency of the server. * Add new component tests for Elsa.AzureServiceBus and remove old unit tests In this commit, a new set of component tests for Elsa.AzureServiceBus have been added, providing more detailed and reliable testing. Simultaneously, several old unit test files and projects have been removed as they're no longer relevant or useful. These include tests in the 'Elsa.Workflows.Runtime.UnitTests1', 'Elsa.Activities.UnitTests' and 'Elsa.JavaScript.UnitTests' namespaces among others. The decision to remove these tests is motivated by the desire to streamline the testing process and focus on the most meaningful and reliable tests. * Rename GlobalUsings.cs to Usings.cs in integration tests Renamed the GlobalUsings.cs file to Usings.cs in the Elsa.Alterations.IntegrationTests project to better reflect its purpose. This change is intended to improve clarity within the codebase. * Refactor Azure service bus testing setup to separate extension This commit abstracts the setup for Azure service bus testing into a separate extension named AzureServiceBusServiceCollectionExtensions. The code has been removed from the WorkflowServer class, contributing to a cleaner and more modular codebase. This enhancement will facilitate better unit testing and reduce redundancy in test setup. * Uncomment 'Description' and remove 'OptionsProvider' and 'OptionsMethod' In the 'InputAttribute.cs' file, the 'Description' property has been uncommented to allow for a brief description of properties during workflow tooling. Furthermore, the 'OptionsProvider' and 'OptionsMethod' properties have been removed, simplifying the attribute options handling. * Enable Azure Service Bus module The change updates the 'useAzureServiceBusModule' constant from false to true in the Elsa.Server.Web Program.cs file. This adjustment allows the application to employ the Azure Service Bus module. * Refactor ProtoActor module for workflow instance focus The main changes in this commit revolve around renaming and refactoring to orient the ProtoActor module toward handling workflow instances. In the process, unnecessary imports have been removed, classes have been renamed to reflect their new focus on workflow instances, and various related elements such as protobuf files and services have also been renamed and refactored to align with these changes. * Change AnalysisModeDocumentation to 'AllDisabledByDefault' The AnalysisModeDocumentation setting in Directory.Build.props has been updated to 'AllDisabledByDefault' from 'Default'. This change disables all analysis by default in the documentation generation process. * Update default value for Content and modify build properties Modified the default value for the property 'Content' in the 'Message' class. Also updated the build properties by changing the 'AnalysisModeDocumentation' to "Default", disabling 'EnableTrimAnalyzer', and adding exception warning codes 'CS0162' and 'CS1591' to 'NoWarn'. * Disable Azure Service Bus and initialize Customer fields With this commit, the Azure Service Bus module usage has been turned off. In addition, the initial fields of the Customer entity have been set to their default values for safer initialization and to avoid potential null reference exceptions. * Add distributed workflow services and configurations Implemented basic functionalities in DistributedWorkflowClient and DistributedWorkflowRuntime. Updated Web Application 'Program.cs' to support different WorkflowRuntimes as per the configuration. Renamed WorkflowRuntimeFeature.cs to DistributedRuntimeFeature.cs and refactored the code accordingly. Removed unnecessary method 'UseDefaultWorkflowRuntime' from ModuleExtensions.cs. * Refactor ReceivedServiceBusMessageModel from record to class Converted ReceivedServiceBusMessageModel from a record to a class and updated relevant initialization code. Change was implemented because of the polymorphic serialization incabability of dealing with $type properties with records. * Refactor WorkflowInstanceImpl for improved workflow management This change refactors the WorkflowInstanceImpl class to improve workflow management. It implements queuing of RunWorkflowOptions while a workflow is running and ensures state before any workflow operations. Furthermore, workflow host creation is replaced with directly creating and managing workflow instances. * Add DefaultFormattersFeature and update dependencies A new feature, DefaultFormattersFeature, has been added to the Elsa.Common module. The WorkflowsFeature in the Elsa.Workflows.Core module has been updated to depend on this new feature. This will ensure that default JSON formatters are available across the application. * Refactor JsonFormatter with JsonSerializerOptions property Improved the JsonFormatter class by introducing a private JsonSerializerOptions property. The new implementation uses this property in the FromStringAsync method instead of initializing a new JsonSerializerOptions each time, improving efficiency. The JsonStringEnumConverter has also been moved to the class constructor. * Refactor WorkflowInstanceImpl for improved code clarity The refactor includes replacing direct field access with properties in multiple places for WorkflowGraph and WorkflowState. It also makes _queuedRunWorkflowOptions field readonly and removes unnecessary newline characters. These changes aim to improve readability, maintainability and encapsulation. * Add and update methods to workflow instances This commit introduces add and update functionalities to the workflow instances across the application. Additionally, it includes enhancements to the workflow execution, such as distributed locking and expanded logging capabilities for better debugging and tracking. Lastly, it introduces new configurations in application settings to fine-tune the workflow runtime environment. * Refactor worker management in AzureServiceBus module The codebase has been simplified by removing the unnecessary management of worker ref counts in the AzureServiceBus module. Previously, to manage worker instances, we kept a count of references to each worker and removed it when the count dropped to zero. This complexity has been entirely removed to make the module simpler and easier to maintain. Now we just create a worker when necessary, without tracking its usage across the codebase. * Refactor workflow runtime with distributed locking and state checking The workflow runtime has been refactored for better concurrency control and state management. The changes primarily include the introduction of distributed locking in action methods of the DistributedWorkflowClient and the handling of workflow instance states in the LocalWorkflowClient. This entails changes to the logic in the RunInstanceAsync, CreateAndRunInstanceAsync, CancelAsync, ExportStateAsync and ImportStateAsync methods. * Refactor DispatchWorkflowInstance in MassTransitWorkflowDispatcher The DispatchWorkflowInstance model in the MassTransitWorkflowDispatcher has been refactored. Previous detailed activity parameters (ActivityId, ActivityNodeId, ActivityInstanceId, ActivityHash) were replaced with a single ActivityHandle property. An additional Properties field was added. * Update test in BulkDispatchWorkflowsTests The DispatchAndWaitWorkflow test case in the BulkDispatchWorkflowsTests has been updated to use the new workflow client creation and instance running methods. Necessary namespace imports have also been added in the process to support the changes. * Remove snapshot and persistence functionality from WorkflowInstanceImpl This commit eliminates snapshot creation and persistence from the WorkflowInstanceImpl class. Methods related to these processes have been removed, including SaveSnapshotAsync(), ApplySnapshot(), and GetState(). This has also impacted the constructor and OnStarted() method where certain calls were made related to persistence. The need to manage and recover state asynchronously was eliminated, simplifying the implementation. * Refactor workflow client implementations and update WorkflowStateMapper Workflow clients in the modules Elsa.Workflows.Runtime and Elsa.ProtoActor have been refactored for better maintainability and readability. The WorkflowStateMapper in the Elsa.Workflows.Management is updated to include an "Apply" method, splitting the mapping functionality into smaller, more manageable methods. Also, usage of the Azure Service Bus Module in Elsa.Server.Web has been turned off. * Update BulkDispatchWorkflowsTests specification This commit updates the test specification in BulkDispatchWorkflowsTests by removing an unnecessary import and updating the WorkflowDefinitionHandle in the DispatchAndWaitWorkflow test method. Also, superfluous comments have been removed for readability. * Update workflow definition in BulkDispatchWorkflowsTests Changed the referenced workflow definition in the test scenario from EmployeeGreetingWorkflow to GreetEmployeesWorkflow. This update reflects more precise naming in our component tests for the BulkDispatchWorkflows scenario. * Refactor BulkDispatchWorkflows and simplify error handling The BulkDispatchWorkflows class in Elsa Workflows Runtime module has been refactored to simplify it. The detailed error handling with a list of errors has been removed and it is now directly dispatching child workflows without checking for a successful dispatch. The ProcessItem method has been removed which handled the errors previously, and the DispatchChildWorkflowAsync method no longer returns a dispatch response. This substantially simplifies the code and reduces its complexity. * Update workflow runtime and distributed locking configuration settings In the web server settings, the workflow runtime was changed from 'Distributed' to 'ProtoActor'. Also, the distributed lock provider setting has been moved into a new 'DistributedLocking' section in the configuration settings, allowing more detailed lock options to be set, including a lock acquisition timeout. * Replace ProtoActor with DistributedRuntime in WorkflowServer The current commit modifies our WorkflowServer configuration under our component tests. The "UseProtoActor" method which was previously used for runtime configuration has been commented out and replaced with "UseDistributedRuntime". This change suggests a shift towards a distributed runtime environment. * Add new services and classes for workflow messaging This commit adds new classes and services to facilitate workflow messaging. These include 'BroadcastWorkflowInboxMessageOptions', 'IWorkflowInbox', 'NewWorkflowInboxMessage', 'SubmitWorkflowInboxMessageResult', 'WorkflowInboxMessage', 'WorkflowInboxMessageDeliveryParams', and 'WorkflowInboxMessageFilter'. Additionally, changes were made in 'HttpWorkflowsMiddleware.cs' and 'WorkflowRuntimeFeature.cs' to use 'IWorkflowRunner' for executing workflows, and deprecated 'IWorkflowHostFactory' and 'WorkflowInvoker'. The messaging-related classes provide methods and properties to manage and manipulate workflow messages, delivery options and results. 'IWorkflowInbox' provides an interface for delivering messages to workflow instances while 'IWorkflowRunner' fulfills running workflows. * Refactor WorkflowCancellationService for cleaner syntax Aesthetic adjustments were made to the WorkflowCancellationService code to improve readability and maintainability. This includes tidying up line breaks, reformatting lists, correcting spelling in comments, and consolidating parameters in a function call. * Remove unused import in MassTransitWorkflowCancellationDispatcher The 'Elsa.Workflows.Runtime.Contracts' namespace was removed from the MassTransitWorkflowCancellationDispatcher file because it was not being utilized. This ensures a clean and efficient codebase by removing unnecessary imports. * Remove Class1 from Elsa.Testing.Shared.Component This commit deletes Class1.cs as it was no longer serving any purpose in the Elsa.Testing.Shared.Component. This removal helps keep the codebase clean and maintainable. * Reduce default timeout in ISignalManager interface The default millisecond timeout for the WaitAsync functions in the ISignalManager interface has been reduced from 5000ms to 1000ms. This change will speed up signal wait times in our testing framework. * Refactor syntax representation in SendMessage activity Simplified the syntax representation in the SendMesssage activity in the Elsa Azure Service Bus module. The SupportedSyntaxes property now utilizes a more concise array initialization. * Removed obsolete 'Stimulus' property and adjusted consumers The 'Stimulus' property in DispatchResumeWorkflows and DispatchTriggerWorkflowsRequest classes has been removed as it was marked obsolete. The DispatchWorkflowRequestConsumer has been adjusted to only use the 'BookmarkPayload' property. This ensures a cleaner code base and removes potential confusion between the properties. * Rename GetNamedWorkflowGrain to GetNamedWorkflowInstanceClient The method name GetNamedWorkflowGrain in both ProtoActorWorkflowClient and ClusterExtensions has been renamed to GetNamedWorkflowInstanceClient to improve code clarity. This change provides better semantics of what the function is purposed for. * Remove unused snapshot classes The commit removes the WorkflowGrainSnapshot and WorkflowInstanceGrainSnapshot classes from the Elsa.ProtoActor module. These classes were part of an older architecture and are no longer required. * Remove unused field from Azure ServiceBus Worker The _refCount field in the Worker class from the Azure ServiceBus module was unused. To maintain a clean codebase and improve readability, this field has been removed. * Remove WorkflowInboxMessageRecord from Elsa.Dapper module The commit includes the deletion of the entire WorkflowInboxMessageRecord class from Elsa.Dapper module. This file was managing various activities related to the workflow inbox messages, such as delivering messages to a workflow instance. * Add new V3_2 migrations for all database providers This commit introduces new V3_2 migrations for MySQL, SQLite, PostgreSQL, and SQLServer database providers. Included are `Up` and `Down` migrations, the corresponding Designer files, and specific Alterations and Runtime changes. SqlDbType specifications were also added for each. * Refactor WorkflowDefinitionFilter and update related modules Empty lines were removed between property declarations to improve readability in WorkflowDefinitionFilter. An error where 'filter.IsReadonly' was used instead of 'IsReadonly' was also corrected to ensure proper functionality. Additionally, WorkflowDefinitionFilter was imported in the Delete and Revert endpoint modules of the Elsa.Workflow.Api class to maintain consistency across the codebase. * Change MongoUserStore to non-abstract class The MongoUserStore class was previously tagged as an abstract class in the Identity module of Elsa.MongoDb. This update changes the MongoUserStore class from an abstract to a non-abstract (concrete) class to enable direct instantiation. * Add ForwardedType attribute and update bookmarks This commit introduces a new `ForwardedType` attribute to aid in forwarding types to new types when deserializing JSON. Additionally, the existing bookmarks related to workflow runtime activities have been updated. These changes also require modifying the `PolymorphicSerializer` in the Elsa.MongoDb project to handle types using the new `TypeHelper.GetLatestType` method. * Fix logger reference in exception handling The logger reference used in the catch block of the DefaultTriggerScheduler.cs file was incorrect. This commit corrects it by using the appropriate logger variable for logging any potential cron expression format errors. * Refactor component tests and improve code cleanliness This commit refactor the component tests, introduced new interfaces and events related to workflow definition and trigger change token signals. Removed an unused file, DictionaryExtensions.cs and refactored the GetOrAdd method to better handle null values. Some minor changes and improvements were also made in existing files to enhance overall code cleanliness. --------- Co-authored-by: Steve Taylor <stevetayloruk@users.noreply.github.com>	2024-06-10 19:36:51 +02:00
Sipke Schoorstra	0422435be9	Fix workflow variable scope inconsistency (#5558) ... * Refactor variable handling in ExpressionExecutionContextExtensions The core change in this commit is the refactoring of the handling of variables within the ExpressionExecutionContextExtensions. The methods GetVariable, CreateVariable, and GetVariableBlock have been altered for clarity and simplified reducing redundant code. Unnecessary parameters and returns in method documentation have been removed, and overall code formatting has been improved to enhance readability. * Simplify MemoryRegister creation in Workflow.cs The creation of the MemoryRegister object in the file Workflow.cs was simplified to one line. The previous method, which declared a new object then called the Declare method before returning, was removed. * Map controller routes in server web program Added a line of code in the Elsa.Server.Web program.cs file to map controller routes. This change ensures that HTTP requests are correctly directed to their corresponding controller actions. * Update workflow state extraction logic The logic in the WorkflowStateExtractor has been updated to retain the root Workflow activity context even if it's completed. This change is necessary to keep workflow-level variables accessible. * Remove RequiresUnreferencedCode attribute from ConvertTo method The RequiresUnreferencedCode attribute was removed from the ConvertTo method in the ObjectConverter class. * Remove unused services and rename test file Unused services in the AutoUpdateTests.cs class were removed, reducing clutter and improving code readability. Additionally, the DeleteWorkflow_Clustered.cs test file has been renamed to DeleteWorkflowClustered.cs for better naming consistency. * Add CountdownStep activity and CountdownWorkflow for testing This commit introduces new component tests for simulations involving counters. It includes a new CountdownStep activity that decrements a counter variable, as well as a CountdownWorkflow which consists of a loop based on the aforementioned activity. It also involves a CountdownWorkflowTests class for testing counter persistence across workflow runs. * Remove unnecessary whitespace in CountdownWorkflowTests This commit eliminates the superfluous whitespace in the CountdownWorkflowTests.cs file. It maintains the proper formatting and ensures code consistency across the test component. * Refactor CountdownWorkflowTests constructor Simplified the constructor of the CountdownWorkflowTests class. The changes remove the unnecessary constructor body and pass the 'app' object directly to the base AppComponentTest class, enhancing the code's readability and maintainability. * Add application roles and configure them in MassTransit A new enum ApplicationRole has been added for distinguishing among different roles (Hybrid, Api, Worker) an application can take. In the configuration of MassTransit, it is now possible to disable the consumers based on application role, which can help optimize the usage of resources and increase application efficiency. * Update Program.cs Switch to Memory broker * Update Program.cs Simplify DisableConsumers assignment. * Update ApplicationRole.cs Rename Hybrid to Default. * Update appsettings.json	2024-06-10 10:51:23 +02:00
Sipke Schoorstra	29742db513	Add application roles and configure them in MassTransit (#5561) ... A new enum ApplicationRole has been added for distinguishing among different roles (Hybrid, Api, Worker) an application can take. In the configuration of MassTransit, it is now possible to disable the consumers based on application role, which can help optimize the usage of resources and increase application efficiency.	2024-06-10 10:45:12 +02:00
rosca-sabina	da7bff83b7	Added MongoDB implementation for alteration storage (#5555)	2024-06-09 15:18:30 +02:00
Sipke Schoorstra	35143d99ec	Disable Quartz, ZipCompression and switch MassTransitBroker to Memory ... Quartz and ZipCompression have been turned off by changing their use constants to false. Simultaneously, the MassTransitBroker has been switched from RabbitMq to Memory for optimization.	2024-06-07 18:24:09 +02:00
Raymond den Haan	48523c4ecd	Add HTTP cache invalidation on auto-update of consuming workflows	2024-06-07 14:01:14 +02:00
MariusVuscanNx	d817d7ccaf	Global read-only and read-only workflows (#5406) ... * Implemented global readonly and workflow readonly * Fixed bug of unfilled versions * Fixed failing test * Updated pipeline * Revert * Pushed fixes based on the PR comments and fixed history bug * Implemented changes based on PR comments * Fixed issue in caused by AuthorizationHandler not being able to receive null resources * Remove unused Google.Protobuf dependency in Program.cs An unused Protobuf library import was found in Program.cs file. In order to clean up the code and improve readability, this import statement has been removed. No other changes have been made. * Refactor code to reduce redundancy and improve readability Optimize code by removing unnecessary constructor assignments and simplify function calls. The simplification is mainly achieved by directly passing the required services as constructor parameters instead of assigning them to private fields first. This results in a more readable and less redundant code. --------- Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>	2024-05-27 10:06:39 +02:00
Sipke Schoorstra	5241bd790d	Update configuration for DistributedLockProvider ... The configuration for DistributedLockProvider has been changed from the root level to under the Runtime section in appsettings.json. The corresponding call to get this value in code was also updated to reflect this change.	2024-05-21 21:33:14 +02:00
Sipke Schoorstra	6a0f74e06c	Fix FlowJoin Activity NRE Bug (#5349) ... * Update property retrieval in FlowJoin activity This commit modifies the way flowScope is retrieved within the FlowJoin activity in Elsa.Workflows.Core. Instead of directly calling GetProperty, a fallback value is now being provided in case the desired property is not found. This reduces the risk of null reference exceptions. * Add FlowJoins component tests Two new files have been created to facilitate component testing for the FlowJoins scenarios in the Elsa Workflows. The `Tests.cs` file includes a Fact to validate the successful execution of a Flowchart with a single FlowJoin. The `Workflows.cs` file defines a single join workflow for these tests. * Add RabbitMq support to ComponentTests Added RabbitMq to Infrastructure.cs for component testing, allowing both RabbitMqContainer and DbContainer to start and stop asynchronously. Also, adjusted WorkflowServer.cs to configure RabbitMq mass transit alongside existing PostgreSql support, aiming to improve testing robustness and coverage. * Refactor placement of RemoveOrphanedSubscriptions service Move the implementation of AddNotificationHandler<RemoveOrphanedSubscriptions>() from the notifier block to the singleton section in AzureServiceBusFeature.cs. This will help keep all service registration related to notifications in one place. * Increase prefetch count	2024-05-08 09:29:15 +02:00
Sipke Schoorstra	b6689c7e2b	Fix Race Condition in WorkflowDefinitionActivity (#5315) ... * Refactor hash method in Hasher.cs The Hash method in the Hasher.cs file has been refactored to use more efficient code. Instead of creating a SHA256 instance, it directly hashes the data using SHA256.HashData. As a result, the private Hash method and its usage, which only has relevance to the previous approach, have been removed. * Replace GetHashCode with SHA256 for script caching Two unused namespaces, Esprima and Esprima.Ast, have been removed, and two new ones, System.Security.Cryptography and System.Text, have been added. These changes in Elsa.JavaScript's JintJavaScriptEvaluator file were made to replace the basic GetHashCode method, which was used earlier to generate a cache key for JavaScript scripts, with SHA256 hash function to ensure uniqueness and avoid possible hash collisions. This greatly increases the reliability of the caching mechanism. * Set HTTP client timeout in WorkflowServer The client timeout in WorkflowServer has been set to 1 minute for `ResrService` and `client` methods. This was done to manage long running requests and prevent timeouts. * Refactor code and add validation in ScheduleActivityAsync methods An unnecessary line of code in the ScheduleActivityAsync method under the ActivityExecutionContext file has been removed to simplify the function. Meanwhile, validation has been added to ensure that specified activities are part of the workflow. This improves code clarity and prevents potential errors caused by incorrect activity scheduling. * Refactor ActivityVisitor to use ActivityVisitorContext The ActivityVisitor class in Elsa.Workflows.Core was refactored to use an ActivityVisitorContext class. The introduction of this context class replaced the multiple hashsets that were used within function signatures, consolidating them into a single object and reducing function complexity. This change improves readability and code organization. * Refactor WorkflowDefinitionActivity to use WorkflowGraph Adjusted the WorkflowDefinitionActivity class in the Elsa.Workflows.Management module. The changes include migration from using Workflow to WorkflowGraph objects and condensing code blocks for clearer readability. Additionally, a new 'IsInitialized' field is added to eliminate potential race conditions during the graph construction process. * Add activity validation to WorkflowExecutionContext A check has been implemented in the WorkflowExecutionContextExtensions to validate that a specified activity is part of the workflow. This prevents incorrect activity references when scheduling tasks in the workflow. * Refactor to use WorkflowGraph instead of Workflow The codebase has been refactored to utilize the WorkflowGraph instead of the Workflow while running and manipulating workflows. Additional changes include restructuring WorkflowRunner and WorkflowDefinitionService classes, introducing WorkflowGraphBuilder usage, and mapping updates in WorkflowDefinitionMapper. The WorkflowHost, WorkflowDispatcher, and WorkflowExecutionContext have also been updated accordingly. * Add WorkflowGraph and related services This commit introduces the IWorkflowGraphBuilder interface, the WorkflowGraph model, and an implementation of the interface in the WorkflowGraphBuilder class. The purpose of these additions is to establish the building and structure of a workflow graph. The WorkflowGraph model also includes activity node handling and hashing capabilities. * Add tests to ensure exception when scheduling an activity not part of workflow Several unit tests were written to ensure that the right behavior is exhibited when scheduling an activity that is not part of the workflow. An exception is expected to be thrown in this case. Additionally, new service definitions, workflow definitions and workflow queries were added for more comprehensive testing. * Update WorkflowGraph class and add comment descriptions This commit updates the WorkflowGraph class by extending its descriptions and implementing an explicit mention to the Workflow reference. Additionally, more attribute descriptions have been added to increase code readability and comprehension. * Removed unnecessary brackets * Remove unused services from workflow management This commit deletes "ExpressionDescriptorRegistryPopulator.cs" and "ScopedWorkflowDefinitionLookup.cs" files from Elsa.Workflows.Management.Services. These files, containing obsolete services, are no longer used in the workflow management process. The services' registration has been removed as well from "WorkflowManagementFeature.cs". * Remove TestWorkflowDefinitionService.cs from component tests A file, TestWorkflowDefinitionService.cs, was removed under Elsa.Workflows.ComponentTests. This is part of the improvement process where inefficient or unnecessary test files are cleaned up. * Add comments to WorkflowDefinitionActivityTests This commit adds explanatory comments to the WorkflowDefinitionActivityTests file. The comments provide information about the purpose of these tests and a reference to a related issue in the project's Github repository. * Updated duplicate and missing package references --------- Co-authored-by: Raymond den Haan <raymond.den.haan@nexxbiz.io>	2024-05-01 14:53:37 +02:00
Sipke Schoorstra	406469825c	Add Component Testing Framework (#5261) ... * Add application component tests Multiple new test files were added to deliver application component tests. This move improves testing by adding integration tests that cover overall system behavior and checking end-to-end actions. Ensuring the system functions correctly as a whole. In the process, updating some package versions to maintain compatibility. * Add RefitSettings helper and revise API client service configuration The commit introduces a 'RefitSettingsHelper' for Elsa API client and revises the way the API client services are configured. It also makes improvements to the WorkflowServerTestWebAppFactory for component testing. Some endpoint contracts related to workflow execution are also updated to have optional parameters. * Remove old tests and add new workflow tests This commit removes old, unnecessary tests and incorporates new workflow tests. It also improves the Elsa API client JSON serializer and adds a helper for HttpResponseMessage. Lastly, the commit introduces changes to properly configure the test logging and to manage application settings. * Add HttpHelloWorld workflow tests A new component test scenario, HttpHelloWorldTests, has been created for testing an HttpHelloWorld workflow. This involves asserting if a workflow responds correctly with "Hello World". Furthermore, an HTTP workflow client has been introduced in the WorkflowServerTestWebAppFactory class to provide a base address for workflow API calls. * Add new test file and update workflow execution tests This change adds a new test file "fork-1.json" to the Elsa.Workflows.Api.ComponentTests project. Also, updates were made throughout the tests to replace the WorkflowServerTestWebAppFactory with a fixture, allowing the tests to run in parallel. Lastly, unnecessary warning suppression was removed from the Elsa.Workflows.Core extension method. * Add filter for .json and .elsa files in BlobStorageWorkflowProvider This change adds a BrowseFilter in the BlobStorageWorkflowProvider options. This filter checks for files that end with .json or .elsa and includes only these files when browsing through the blob storage. This filter helps prioritize specific workflow file types. * Rename WorkflowServerTestWebAppFactoryFixture and update usage The old class name "WorkflowServerTestWebAppFactoryFixture" has been replaced with the more accurate "WorkflowServerWebAppFactoryFixture". All references to the previous name in other classes were also updated accordingly. In addition, the directory key in the method "CreateConvoyOptionsBuilder" has been updated from "Workflows" to "Scenarios". * Update test fixture in workflow tests The commit updates the test fixture in two test classes: HttpHelloWorldTests and HelloWorldTests. The former test fixture, WorkflowServerTestWebAppFactoryFixture, was replaced by WorkflowServerWebAppFactoryFixture to accurately match the testing needs. * Update .csproj file paths and reorganize tests The commit modifies the file paths for several test scenario files in the Elsa.Workflows.Api.ComponentTests.csproj, reflecting a reorganization of the tests. Previously static paths have been updated to new paths under 'Scenarios'. Additionally, two new test files related to 'LogPersistenceModes' have been included in the project. * Add tests for log persistence modes This commit introduces two new test scenarios for logging persistence modes and includes a related test called 'HelloWorldWorkflow'. These tests cover scenarios where certain workflow inputs should be stored and others shouldn't, thereby testing the log persistence feature. This ensures that the logging behavior respects the specified persistence mode. * Add log persistence tests and update LogPersistenceMode enum The commit contains the addition of new log persistence tests for verifying correctness of log persistence behavior. Furthermore, the LogPersistenceMode enum has been updated, replacing 'Default' with 'Inherit'. This change makes the mode's purpose clearer. Lastly, new test scenarios and test data files were added for more comprehensive testing. * Remove obsolete component tests and support files The files removed are no longer necessary for the current state of the application. They include various component tests and their related support files within the Elsa.Workflows.Api.ComponentTests project. By removing these, the project structure is cleaner and only contains relevant tests. * Add dispatch workflow scenario tests and necessary helper classes This commit includes two new tests for dispatching workflows, along with the creation of new 'ChildWorkflow' and 'DispatchAndWaitWorkflow' classes. Auxiliary helpers and services have been added to aid in managing workflow events and signals for these tests. The 'ComponentTest' has also been upgraded to support disposal handling. * Remove ITestOutputHelper dependency from test classes Removed the dependency on ITestOutputHelper in multiple test classes across various workflow scenarios. This change simplifies the test class constructors by reducing the number of required dependencies, contributing to cleaner and leaner code. * Add 'Hello World' scenario to WorkflowCompletion tests The 'Hello World' scenario was moved into WorkflowCompletion tests, along with changes in workflow definition identifiers. As part of these changes, the 'hello-world.json' file was updated; a new file under the same name was created in the WorkflowCompletion area and the workflow identifiers in basic and workflow completion tests were updated accordingly. Additionally, 'fork-1.json' has been renamed to 'fork.json'. * Add support for cluster hosting tests This commit introduces a suite of integration tests designed to validate the behaviour of hosting multiple instances of Elsa in a clustered environment. These tests simulate a typical clustered hosting scenario by using 'App', 'Cluster', and 'Infrastructure' objects to emulate different instances of the Elsa workflow engine running on separate servers. Name changes were made to certain classes and methods to reflect their new scopes and roles within the testing environment. * Add performance tests and improve component tests Added a new performance tests project scaffold, complete with its own project file, build properties file, and a dummy test. Updated component tests to improve multi-pod testing, primarily through the addition of additional service scopes and asserting activity registry synchronization. These changes also required updates to existing project and props files as well as the solution file. * Update ActivityRegistrySyncTests and Infrastructure Added a reference to Services in ActivityRegistrySyncTests and removed unnecessary whitespace in both files. The test component Elsa.Workflows has been modified to import newly added services, ensuring all tests are running with the expected resources and services. * Fix comment * Add NOOP implementations for stores * Update PostgreSQL image and adjust test timings The PostgreSQL image used for testing has been updated to the latest version from 13.3-alpine. Timeouts in ISignalManager and DispatchWorkflowsTests have been reduced for efficiency. A delay in the ChildWorkflow has also been decreased. Additionally, an 'ImportWorkflowActivity' test in ActivityRegistrySyncTests has been marked as not yet implemented.	2024-04-26 15:49:14 +02:00
Sipke Schoorstra	21c54f583e	Fix WorkflowActivity to Use Cached Workflow Definitions for Consistent Behavior (#5223) ... * Replace IServiceScopeFactory with IServiceProvider in WorkflowRunner Unused dependencies were removed from the workflow runner service. The IServiceScopeFactory was replaced with IServiceProvider to better handle the creation and deletion of service scopes, resulting in cleaner code with less manual scope management. Microsoft.Extensions.DependencyInjection and System.Diagnostics.CodeAnalysis were removed as they were no longer necessary. * Refactor WorkflowDefinitionActivity to use WorkflowDefinitionService The WorkflowDefinitionActivity class has been refactored to make use of the WorkflowDefinitionService instead of the WorkflowDefinitionStore. This fixes #5222 by ensuring the same activity instances are used in the graph model of the workflow execution context. * Add workflow filtering and caching functionality Added methods to `WorkflowDefinitionService` to find workflow definitions and workflows using filter criteria. A key generation method for caching filtered workflows was also added to `WorkflowDefinitionCacheManager`. The implementation includes generating a hash of the filter parameters and using this hash as a cache key, providing efficient caching functionality for filtered searches. * Refactor TriggerIndexer to handle only ITrigger activities The code in TriggerIndexer has been refactored to deal specifically with ITrigger activities, streamlining its behavior. Removed code related to handling non-ITrigger activities and simplified the workflow creation process. The extraction of "startable" nodes now directly filters and casts to ITrigger, reducing complexity and increasing readability. * Update caching service to support filter-based search The CachingWorkflowDefinitionService has been updated to support workflow definition and workflow search based on filter criteria. The update also includes change of class scope from public to internal. Further, it resolves the missing reference by switching from Elsa.Caching.Contracts to Elsa.Caching. * Optimize Elsa project imports and use explicit cache variable names This commit removes superfluous import references, relocates the 'IChangeTokenSignaler' contract into the 'Elsa.Caching' namespace, and replaces ambiguous 'cache' variable names with more explicit 'memoryCache' across several files. Additionally, new package references have been added and access modifiers have been changed to improve encapsulation. Cleanup enhances readability and maintainability of the codebase. * Add .DotSettings file to Elsa.Caching module A new .DotSettings file has been added to the Elsa.Caching module. This file is used for namespace configuration, specifically to skip the "contracts" folder in code inspections. * Update workflow interfaces to support filter queries The update extends `IWorkflowDefinitionCacheManager` and `IWorkflowDefinitionService` interfaces. Functions are added to allow creating filter cache keys and finding workflow definitions and workflows using a new `WorkflowDefinitionFilter`. This enhances querying flexibility by enabling filtered searches. * Add WorkflowDefinitionVersionId to WorkflowTriggerEqualityComparer A new property, WorkflowDefinitionVersionId, has been added to the object being serialized in WorkflowTriggerEqualityComparer. This change allows for a more accurate comparison between workflow triggers, considering not just the workflow definition ID but also its version. * Update service registration types in WorkflowsFeature Changed the registration type for both IHasher and IBookmarkHasher services from Scoped to Singleton in the workflows feature configuration. This alteration aims to improve application performance and manage service lifetimes more efficiently. * Remove Open.Linq.AsyncExtensions dependency The Open.Linq.AsyncExtensions package reference was removed across the project. The usage within the CachingWorkflowDefinitionStore was updated accordingly to maintain functionality. * Move System.Linq.Dynamic.Core package reference The System.Linq.Dynamic.Core package reference was moved from the Directory.Build.props file to the Elsa.Workflows.Management.csproj file. This change reflects the specific dependency of the Elsa.Workflows.Management module on System.Linq.Dynamic.Core, without impacting other modules. * Implement caching for HTTP workflows This update introduces caching mechanisms for HTTP workflows, which significantly improves their performance. The changes involve creating a `CacheManager` and `CachingHttpWorkflowLookupService`, and modifying some existing components to use the new caching mechanism. Additionally, the `HttpWorkflowsCacheManager` was renamed to `HttpWorkflowsCacheInvalidationManager` to better reflect its role. * Refactor cache management across modules This commit refactor the cache management across various modules. The 'ICacheManager' interface now includes methods for triggering and getting change tokens, and the 'HttpWorkflowsCacheInvalidationManager' has been renamed to 'HttpWorkflowsCacheManager'. The caching functionality in 'WorkflowDefinitionService' and other similar services have been updated to use these new methods, improving consistency and maintainability. * Enable caching in Elsa.Server.Web The "useCaching" variable has been set to true to enable caching. Simultaneously, the method name "UseCachingStores" has been refactored to "UseCache". Conditional statements have been added to check the "useCaching" variable before invoking caching. * Rename method UseCaching to UseCache In the Elsa.Server.Web and Elsa.Http project files, the method UseCaching has been renamed to UseCache. This modification is aimed at bridging naming inconsistencies and maintaining naming standards across the application. * Update HttpCacheFeature class description The class summary for HttpCacheFeature has been revised. Originally, it stated that the class was used for installing services related to HTTP services and activities, but it actually focuses more on HTTP caching. * Remove unused Configure method from HttpCacheFeature The Configure method in HttpCacheFeature was found to be redundant as it wasn't doing any significant work or contributing to any functionality. It has therefore been removed to clean up the code and avoid confusion. * Add 'bug/*' to workflow triggers This commit includes 'bug/*' to the list of triggers in our GitHub Actions workflow. Now, any push or pull request under a 'bug/*' branch will trigger the workflow.	2024-04-15 10:06:27 +02:00
Sipke Schoorstra	8bcab1b832	Fix LogPersistenceMode default mapping logic (#5218) ... * Fix LogPersistenceMode default mapping logic Fixes #5217 * Format adjustment in DefaultActivityExecutionMapper.cs Applied minor code reformatting in the DefaultActivityExecutionMapper.cs file to maintain consistency and improve readability. This was mostly centered around adding spaces for proper code alignment and indentation to match the overall code style in the project. No functional changes were made.	2024-04-12 15:00:00 +02:00
Sipke Schoorstra	9f55e831e6	Refactor workflow dispatch and instance creation process (#5213) ... * Refactor workflow dispatch and instance creation process This update splits the process of dispatching a workflow into two steps: Initialization and Execution. Now, first, a new workflow instance is created and saved with the input parameters. Second, the new workflow instance is dispatched for execution. This process ensures that the size of the message dispatched does not exceed acceptable limits and enhances workflow dispatch efficiency. It also helps avoid data loss in case of premature process termination or failure in the initial stages of execution. * Change default workflow substatus to 'Pending' The code changes involve modifying the default WorkflowSubStatus from 'Executing' to 'Pending'. This minor adjustment is implemented to represent a more accurate initial state of a new workflow instance in the WorkflowManagement module in Elsa. * Add support for existing workflow instances in WorkflowInstance grain The change extends the WorkflowInstance grain to include an IWorkflowInstanceStore to support workflows that already exist. Additionally, the CreateWorkflowHostAsync method is enhanced to rebuild a workflow host if the instance already exists. * Simplify XML comments format * Refactor comments in `StartWorkflowHostParams` class Removed unnecessary comment tags in the `StartWorkflowHostParams` class for better readability and simplicity. Also, unused lines of code from the `IWorkflowRuntime` interface have been commented out to enhance clarity and cleanliness in the codebase. * Remove unnecessary comment in MassTransitWorkflowDispatcher The obsolete comment about attaching a version header to the message was removed from 'MassTransitWorkflowDispatcher.cs'. This comment was no longer relevant as the dispatcher no longer performs the action described.	2024-04-12 10:46:40 +02:00
Sipke Schoorstra	e603e5480b	Add caching to workflow runtime and workflow management stores (#5174) ... * Add caching to workflow runtime and workflow management stores The update introduces caching to workflow runtime and workflow management stores to enhance performance. This is achieved by adding decorators for several stores, which cache records to reduce database fetches. Additionally, a signaler for change tokens allows for cache invalidation when changes occur. The MemoryCache feature has also been updated to include Scrutor for decoration and the caching duration can be configured through the new CachingOptions class. * Refactor WorkflowsMiddleware for HTTP Endpoint bookmarks and triggers The WorkflowsMiddleware has been extensively refactored to handle HTTP Endpoint bookmarks and triggers. This involves breaking down the InvokeAsync method by extracting parts of its functionality into separate helper methods such as FindTriggersAsync and FindBookmarksAsync. Moreover, Assist with authorization checks, workflow execution within request timeout, and handling of workflow faults has been improved to be more efficient and clearly segmented. * Update HTTP endpoint authorization to use Workflow context The authorization process in the AuthenticationBasedHttpEndpointAuthorizationHandler class has been updated to use the Workflow context instead of the WorkflowInstanceId string. The AuthorizeHttpEndpointContext model has been correspondingly changed to include a Workflow property, thereby strengthening the link between authorization and specific workflows. * Add FindAsync methods to trigger and bookmark stores The code adjustments add new FindAsync methods to the trigger and bookmark store contracts as well as all their concrete implementations (MongoDb, Memory and EFCore). These methods support fetching the first record matching a given filter. The adjustments also include minor syntax improvements and the addition of [UsedImplicitly] attributes where needed. * Refactor WorkflowsMiddleware for improved workflow handling The code was refactored to simplify the flow of handling workflows in the WorkflowsMiddleware class. Specifically, the methods to start and resume a workflow have been extracted to improve code readability. Further, handleErrorMiddlewares was also updated to better manage instances where no valid workflows or base paths are found. * Add caching functionality to WorkflowsMiddleware Added IMemoryCache usage in the WorkflowsMiddleware to cache lookup results for workflows and their associated triggers. This will reduce the number of database operations required when searching for workflows, thus improving performance. The cache is maintained for one minute before it is refreshed. * Implement dynamic cache duration for workflows The code has been updated to have a dynamic cache duration for the workflows instead of a hardcoded one minute. By using the CachingOptions service, the cache duration can now be set in the configuration making it more flexible and adaptable to different performance needs. * Add HttpWorkflowsCacheManager for caching HTTP workflows This commit includes the implementation of IHttpWorkflowsCacheManager for caching of HTTP workflows. New handlers have been added to invalidate cache on workflow updates. Additionally, WorkflowsMiddleware has been renamed to HttpWorkflowsMiddleware. * Refactor workflow trigger handling and caching The refactoring includes deletion of `IndexWorkflowTriggersHandler.cs` and creation of `IndexTriggers.cs` thus revising the workflow trigger indexing approach. Also, revamped the `ITriggerIndexer` interface which now handles deletion of triggers with specific workflow and filter. Furthermore, the caching mechanism in `HttpWorkflowsCacheManager.cs` is modified to handle eviction of workflow definitions and triggers separately boosting its efficiency. * Add summary to IndexedWorkflowTriggers A summary has been added to the 'IndexedWorkflowTriggers' class, providing a brief description. This description outlines that it represents a collection of indexed workflow triggers, promoting clearer understanding for future reference. * Refactor memory caching feature into separate module This commit separates the memory caching feature from the Elsa.Common module into a distinct Elsa.Caching module. This includes moving and renaming related files, such as the MemoryCacheFeature class and associated dependencies. The references in other modules and in the main solution file have been updated accordingly to include the new Elsa.Caching module. * Add distributed caching and update async methods Introduced a distributed caching feature with extensible change token signal publishing. Updated various cache-related methods to be asynchronous for improved performance and responsiveness. Also updated some workflow identity references for clarity. * Add distributed caching with MassTransit support This addition includes the implementation of a distributed caching system with MassTransit transport. The changes introduce necessary interfaces and services, new distributed caching feature along with the support for MassTransit as a transport option. Moreover, the instance management feature has been renamed to clustering feature for better clarity. * Refactor queue naming and scope of MassTransitChangeTokenSignalPublisher Queue name construction is adjusted in the RabbitMqServiceBusFeature and AzureServiceBusFeature modules for better organization and readability. MassTransitChangeTokenSignalPublisher is now a singleton service, ensuring the signal publisher can be shared across the application, increasing efficiency and performance. Also, introduced use of DistributedCacheFeature in MassTransitDistributedCacheFeature module for better modularization. * Add caching capabilities to workflow definition service This commit introduces caching to the workflow definition service, improving the performance for retrieving workflow definitions. 4 new classes have been created (`CachingWorkflowDefinitionService`, `EvictWorkflowDefinitionServiceCache`, `WorkflowDefinitionCacheManager`, and `IWorkflowDefinitionCacheManager`), and several existing classes have been updated to support caching. The caching also includes invalidation mechanisms, ensuring data consistency. * Refactor caching mechanism in workflow definition In the workflow definition module, the explicit caching functionality related to workflow definition versioning has been removed in favor of a more streamlined approach. Additionally, the manner in which services are registered has been altered. As a result, the caching now directly involves the overall workflow definition rather than individual versions, simplifying the caching logic and potentially improving the performance. * Update MassTransitBroker and enable RealTimeWorkflows and SignalRHubs The MassTransitBroker has been updated to Memory from RabbitMq. In addition, the RealTimeWorkflows and UseWorkflowsSignalRHubs features are now enabled in the code. This change will impact how the service communicates and processes real-time requests for workflow operations. * Reformat variable types in HttpFeature The reformatting involves a list of variable types in the HttpFeature module. Each type now appears on a new line for improved readability, making the code easier to maintain and review. * Update HTTP workflows cache invalidation handler XML comment * Remove unnecessary using directives Unnecessary using directives were deleted across several files in the Elsa.Http module. This simplifies the code and will possibly improve execution speed. Specific deletions include those for Encoding, Unicode, Extensions, Collections.Generic, Linq, Text, and Tasks namespaces. * Refactor HttpWorkflowsMiddleware constructor This commit simplifies the HttpWorkflowsMiddleware class constructor. It removes the intermediary variables `_next` and `_options` and directly uses the passed arguments in the constructor. Now, the `next` and `options` parameters are used directly throughout the middleware. * Simplify workflow retrieval in HttpWorkflowsMiddleware This refactoring replaces the use of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync with a single method, FindWorkflowAsync. This simplifies the middleware code and likely improves performance by reducing the number of database queries or service calls required to retrieve a workflow. * Refactor workflow retrieval in HttpBookmarkProcessor Simplified the workflow retrieval process in HttpBookmarkProcessor.cs. Replaced FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods with a single FindWorkflowAsync call. This reduces the complexity and improves efficiency in retrieving workflow. * Optimize FindWorkflowAsync method in HttpWorkflowsCacheManager Removed redundant lines of code to simplify workflow search functionality. This change simplified the FindWorkflowAsync method by directly calling the FindWorkflowAsync function in the workflowDefinitionService, thus increasing code readability and efficiency. * Refactor Endpoint.cs for workflow retrieval The method for obtaining a workflow in the Endpoint.cs script has been refactored and streamlined. The 'GetWorkflowDefinition' method is replaced by the 'GetWorkflowAsync' method which directly retrieves the workflow, without the intermediate step of materializing the workflow definition. This shortens the code and simplifies the process. * Refactor InputFunctionsDefinitionProvider constructor The constructor for InputFunctionsDefinitionProvider has been simplified by removing unnecessary private fields. Services are now directly used in the method instead of being stored in fields. This improves readability and reduces complexity in the class structure. * Refactor WorkflowInstance with improved state handling Simplified the methods for handling workflow and workflow state in the WorkflowInstance class. The refactoring also included some code clean-ups and variable renaming. The new implementation provides better readability and maintainability of the code by reducing unnecessary lines and improving structuring of objects and responses. * Remove unused IBookmarkManager and update workflow functions IBookmarkManager from ProtoActorWorkflowRuntime.cs file is removed due to its redundant status. Additionally, the "FindAsync" method has been updated to use a cancellation token. Also, annotations were added to the "ExportWorkflowStateAsync" and "ImportWorkflowStateAsync" methods to flag calls to functions that require unreferenced code. * Remove unused ReSharper directive Unused ReSharper directive in the file IndexTriggers.cs was identified and therefore removed. This change makes the code cleaner and easier to read. * Update activity invocation in workflow runtime Updated the DefaultBackgroundActivityInvoker service in the Elsa.Workflows.Runtime module to annotate the ExecuteAsync method with "RequiresUnreferencedCode" attribute. This change is made considering the potential code trimming issue. Additionally, simplified the process of fetching workflow by directly using FindWorkflowAsync method instead of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods. * Refactor code to simplify workflow definition loading The code for finding and materializing workflow definitions has been simplified. Instead of loading the definition and materializing it into a workflow in separate steps, a new method called FindWorkflowAsync has been introduced to perform both actions at once. This reduces redundancy and makes the code more readable. * Refactor WorkflowHostFactory to streamline workflow creation This commit simplifies the workflow creation process in WorkflowHostFactory. It removes redundant code and extraneous methods, specifically the overloaded CreateAsync method which used WorkflowDefinition. Now, it directly finds and uses the Workflow instance, thereby simplifying the code base and improving maintainability. * Refactor workflow retrieval in WorkflowInstance.cs Changed the way workflow instances are retrieved from the WorkflowDefinitionService. Instead of obtaining the workflow definition and then materializing the workflow from it, the workflow is directly retrieved using the FindWorkflowAsync function. This simplifies the code and avoids unnecessary null-checks. * Remove unnecessary whitespace in WorkflowInstance.cs An extraneous whitespace character was identified and removed in the WorkflowInstance.cs file. This change contributes towards maintaining clean and readable code in the Elsa.ProtoActor module. * Remove unnecessary comment in ProtoActorWorkflowRuntime The unnecessary comment ("Load the workflow definition.") in the method TryStartWorkflowAsync of the ProtoActorWorkflowRuntime.cs file was removed. This is part of an ongoing effort to keep the codebase clean and readable. * Update workflow management features and handlers Added explicit notification handlers for DeleteWorkflowInstances and RefreshActivityRegistry in WorkflowManagementFeature.cs. Also, renamed RefreshActivityRegistryHandler.cs to RefreshActivityRegistry.cs for better clarity. * Add multiple log record support to workflow execution log stores The major change of this commit is the addition of methods to add multiple log records in the WorkflowExecutionLogStore, across different storage modules such as EntityFramework, MongoDB, Dapper, Elasticsearch and Memory. This ensures consistency and uniform behavior across different storage types. Furthermore, some reformatting and tidying up of the code were undertaken to maintain readability and clarity. * Remove redundant workflow definition check The workflow definition existence check and related service retrieval were removed from HttpWorkflowsMiddleware.cs. It was determined that this check was unnecessary as the workflow definition's existence is guaranteed at this point in the process, reducing redundancy in the code. * Improve cancellation token usage in workflow execution This commit refines the usage of cancellation tokens during the execution of workflows in Elsa.Server and HttpWorkflowsMiddleware. Previously, a cancellation token pair was created before ExecuteWithinTimeoutAsync was called, which limited duration control solely to that method. Now, cancellation tokens are included within ExecuteWithinTimeoutAsync method. This allows the method to observe any cancellation initiated by outer scopes, enhancing control over the timeout of operations. * Add PersistStateAsync method to WorkflowHost A new PersistStateAsync method has been added to the WorkflowHost, which enables the host to directly persist its own state. The method has been integrated into the DefaultWorkflowRuntime and HttpWorkflowsMiddleware. This update eliminates the need to continuously get instances of IWorkflowInstanceManager to save state, which improves efficiency and code readability. * Refactor DefaultAlterationRunner service This update simplifies the DefaultAlterationRunner service, reducing the number of code lines and removing unnecessary references. The workflow materialization step has been merged with the find workflow step, and unused namespaces have been dropped. * Refine wording in IWorkflowHost interface documentation The documentation for the 'CanStartWorkflowAsync' method in the IWorkflowHost interface has been cleaned up. The superfluous "or not" verbiage has been removed, making it easier to understand the method's function. * Remove Redis from DistributedCachingTransport The Redis option was removed from the DistributedCachingTransport enumeration. This transport isn't currently implemented. * Remove 'useDistributedCaching' constant The 'useDistributedCaching' constant was removed from `Program.cs`, and conditional logic was updated to use `distributedCachingTransport != DistributedCachingTransport.None`. A new option 'None' was added to the `DistributedCachingTransport` enum to facilitate this change. * Update package tags in MassTransit project file The package tags in the Elsa.Caching.Distributed.MassTransit project file was updated to consolidate the tags, changing 'mass-transit' to 'masstransit'. This change better aligns with standard naming conventions and improves searchability. * Refactor distributed caching implementation This commit involves an extensive refactor of the distributed caching implementation. Distributed caching related code and resources were moved into an independent 'Elsa.Caching.Distributed' module. The interface 'IDistributedChangeTokenSignaler' was deleted and its functionality was replaced by 'IChangeTokenSignalInvoker'. * Refactor order of parameters in GetOrCreateAsync method The order of parameters in the GetOrCreateAsync method within the CachingWorkflowDefinitionStore class has been changed. This change ensures that the `key` parameter is now first, followed by the `factory` parameter. This improves code readability and aligns with standard coding practices. * Refactor cache retrieval in Workflow service Refactoring was done to streamline the way objects are retrieved from cache in the Workflow service. Duplicated code was condensed into a new `GetFromCacheAsync` method, which is now called in the existing methods, thus increasing maintainability and reducing the possibility of errors. * Update method descriptions and fix comments formatting Method descriptions in various contracts have been updated to more accurately reflect their function regarding record addition and updating in the persistence store. All double comment markers (/// ///) have also been corrected to the standard (///) across multiple classes. * Remove unused caching methods in ModuleExtensions The commit removes the unused methods, `UseMemoryCache` and `UseDistributedCache` from the `ModuleExtensions.cs` file. The removal is part of a wider cleanup and refactoring effort to streamline the codebase and improve legibility. * Remove redundant PrimaryKeyName in DapperWorkflowExecutionLogStore The "PrimaryKeyName" constant was removed in DapperWorkflowExecutionLogStore. This change simplifies the initialization of the '_store' property, reducing unnecessary redundancy and complexity. The refactored code maintains the same functionality but improves readability and maintainability. * Refactor SaveAsync methods in Elsa.Dapper Store The SaveAsync functions have been updated in the Store.cs file inside the Elsa.Dapper module. They now include cancellation token parameters and specify that they add or update records, providing clearer distinction and flexibility. * Refactor store initialization in Elsa.Dapper modules Removed the redundant usage of primary keys during the store initialization across Elsa.Dapper module. Simplified the SaveAsync methods by removing the parameter for primary key, making the code cleaner and more maintainable. This refactoring does not affect the module's functionality. * Refactor UserStore in Elsa.Dapper module The code was adjusted to improve readability within the Elsa.Dapper module's UserStore. Two lines that were previously combined have now been separated into distinct lines, making the code structure more clear. * Refactor constructor arguments in MongoDb module Simplified several classes in the MongoDb module by injecting dependencies directly through the constructor instead of assigning them to private readonly fields. This improves readability and removes unnecessary code lines. Also added JetBrains.Annotations where applicable. * Fix comment syntax in IWorkflowInstanceStore A syntax error in the comments for the method SaveManyAsync (in IWorkflowInstanceStore interface) has been corrected. This change ensures that the remarks section of the method is properly formatted and correctly displayed in documentation. * Remove ComputeBookmarkHash from IHttpWorkflowsCacheManager The ComputeBookmarkHash method was removed from IHttpWorkflowsCacheManager to declutter the interface. The functionality was moved and adapted in the HttpWorkflowsMiddleware class to maintain the original functionality. * Add logging to HttpWorkflowsMiddleware In this update, the HttpWorkflowsMiddleware class has been modified to include logging. Specifically, warning logs have been added to track workflow-related processes and to notify if mentioned bookmarks or workflow instances are not found. * Update consumer configuration in MassTransitFeature This commit modifies the consumer configuration in the MassTransitFeature. Instead of hardcoding the consumer type to DispatchCancelWorkflowsRequestConsumer, it now uses the dynamic consumer type retrieved from the context, making the feature more adaptable for different scenarios. * Change default MassTransitBroker to Memory The default value for the variable useMassTransitBroker in Elsa.Server.Web's Program.cs file has been modified. It has been changed from RabbitMq to Memory to change the message broker used by MassTransit in the application. * Remove Datadog.Trace package from Directory.Packages.props The Datadog.Trace package with version 2.49.0 has been removed from the Directory.Packages.props file. This change reflects the fact that this package is no longer required in our project.	2024-04-10 11:51:40 +02:00
Sipke Schoorstra	c7481b2e64	Refactor synchronization provider selection in Program.cs ... Rearranged the order in which synchronization providers are processed and removed some unnecessary empty lines. Specifically, moved the "File" case above the "Noop" case to optimize the synchronization provider selection process, as "File" is expected to be a more common use case.	2024-03-30 21:59:23 +01:00
raymonddenhaan	9d3f685bb6	Implemented distributed cache refresh for workflow definitions (#5095) ... * Moved handler to correct namespace * Added version deletion messages to activity registry handler * Add MassTransit support for workflow management * Moved logic from Workflow.Management.MassTransit project into Elsa.MassTransit project * Added notifier to MT project for creating distributed messages * Rename and update workflow consumer class Renamed `WorkflowDefinitionConsumer` to `WorkflowDefinitionEventsConsumer` to better reflect its purpose. Updated references accordingly in the `MassTransitWorkflowManagementFeature` class to align with the new name. * Shortened instance (queue) names * Remove unnecessary comment in handler definition An unnecessary comment mark ("/") was removed in the definition of the DistributedWorkflowDefinitionNotificationsHandler class. This change contributes to code cleanup and increases code readability. * Add additional destinations in LoadBalancer settings Three new destinations have been added to the LoadBalancer settings in Elsa Server. These are primarily intended to provide broader network coverage and improve overall server performance. * Add MassTransit support in Elsa.Server A new using directive for Elsa.MassTransit.Extensions has been included at the start of the script. In addition, under certain conditions, the system now uses the MassTransit Dispatcher in the workflow management system. These changes provide support for using MassTransit in the Elsa.Server component. * Add MassTransitBroker enum for MassTransit setup A new enum called MassTransitBroker has been added to represent different types of messaging brokers used in MassTransit. This has also been integrated into the Web project's Program.cs file to allow conditional usage of brokers in the MassTransit setup, enabling more flexible and dynamic configuration. * Refactor WorkflowManagementFeature class The WorkflowManagementFeature class has been cleaned up and its formatting has been corrected. A blank line was removed, a line break was added for readability in the AddVariableType method, and several stray spaces were also removed. * Refactor RefreshActivityRegistryHandler.cs This commit removes unnecessary whitespace and improves the code readability in RefreshActivityRegistryHandler.cs. The changes include deletion of extra lines and adjustment of indentation. --------- Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>	2024-03-24 21:09:51 +01:00
Sipke Schoorstra	83dd776233	Add new docker-compose files, new activities and services ... Create new docker-compose files for non-data dog and data dog services. New services include Postgres, cockroachdb, rabbitmq, redis, and elsa-server. Along with the services, a new bash script for migration and various new activities and endpoints are also added for the Elsa server. This set of changes broadens the operational environment of Elsa.	2024-03-24 20:28:19 +01:00
Sipke Schoorstra	3140b72e15	Add Postgres support to Elsa.Server.Web ... Introduced support for PostgreSQL in the Elsa Server configuration and updated the docker-compose to reflect the new database option. Adjusted the dependencies to include the EntityFrameworkCore PostgreSQL project reference.	2024-03-19 11:38:51 +01:00
Sipke Schoorstra	20d936d3c2	Alterations with Queries (#5032) ... * Remove Elsa.Alteration Handlers and implement workflow alteration activities The Elsa.Alterations.Handlers, Services, and Endpoints have been removed and replaced with Elsa.Alterations.Activities for various steps in executing an alteration plan. These activities include submitting an alteration plan, generating alteration jobs, and dispatching alteration jobs. New models for ActivityFilter, AlterationWorkflowInstanceFilter, and alteration activity summaries are also introduced for better management of alteration plans. * Add 'PrintMessage' custom activity and update naming rules A new 'PrintMessage' activity has been added to the custom activities of the ConsoleApp, which simply writes a 'Hello world!' message. On top of this, the naming rules in the code styling settings have been expanded, introducing separate rules for instance and static private fields. * Remove V3_1 database migration files The V3_1 database migration files were deleted across all database providers, including MySql, SQL Server, SQLite and PostgreSQL. This is indicative of a rollback or significant revision to the data migration strategy. * Update product version and modify tables in Elsa DB context Updated "Elsa" DbSet product version from "7.0.11" to "7.0.14". Additionally, added new properties as well as modified existing ones in tables like "Elsa.KeyValues.Entities.SerializedKeyValuePair" to improve data structure. Also properly ignored certain entities in the Alterations DbContext. * Add V3_1 migration for Elsa This commit introduces the V3_1 migration for the Elsa module. Changes involve the introduction of new tables and alteration of existing tables. Specifically, a new 'KeyValuePairs' table was added and several columns on various tables were renamed or altered. * Remove multiple Elsa.EfCore V3_1 migration files The commit removes several migration files related to the V3_1 migration in multiple Elsa.EntityFrameworkCore modules including SqlServer, PostgreSQL, and SQLite. These files encompass changes in table structures and alterations. * Update default value for ArgumentDefinition Type The default value for the `Type` property in the `ArgumentDefinition` class was updated to `typeof(object)`. The `Type` property of `inputDefinition` in the `WorkflowBuilder` class was also modified to be set as `typeof(T)`. These modifications enable more flexible type handling. * Refactor SafeSerializer and serialization services The SafeSerializer service and related serialization services have been comprehensively refactored. This includes the removal of internal notifications, the incorporation of [RequiresUnreferencedCode] attributes, and changes in method signatures. Methods have been simplified to return ValueTasks directly from the JsonSerializer, and the notification sender dependency was removed. * Add JSON serialization for alterations Implemented a JSON serializer and a custom converter for the IAlteration interface in the Elsa.Alterations.Core.Serialization namespace. Also, introduced a new IJsonSerializer interface and an implementation for standard JSON serialization in Elsa.Common. These serialization changes enhance modularity and extensibility of JSON operations, and improve handling of IAlteration objects. * Add "IsSystem" field to Workflow Definitions and Instances This update introduces a new boolean field "IsSystem" to the Workflow Definition and Workflow Instance models. This field is designed to identify system workflows which aren't meant to be modified by users. Management and Runtime module code, API endpoints, database migrations, and related classes have been updated to include this new field. The MongoDB indices and Dapper migrations have been updated as well, and serialization/deserialization processes for these types have been modified to handle this new attribute. * Remove outdated database migrations Deleted database migration files from the MySql, SqlServer, and PostgreSql projects. * Add V3_1 migration for Elsa.EntityFrameworkCore.Sqlite This commit introduces the V3_1 migration for the Elsa.EntityFrameworkCore.Sqlite module. It includes changes in Alterations, Runtime, Management, and Identity contexts. These changes involve table alterations, addition of new columns, and creation of new tables. Each context alteration encapsulates relevant changes for specific functionalities of the module. * Add `IsSystem` property to workflow classes The `IsSystem` property has been added to several sections of the codebase related to workflows, such as `WorkflowStateMapper.cs`, `ListWorkflowDefinitionsRequest.cs`, `WorkflowDefinitionStore.cs`, and `ListWorkflowInstancesRequest.cs`. This property flag allows us to include or filter system workflows in our queries and results. * Add 'UsedImplicitly' attribute to Alteration Handlers and Types The 'UsedImplicitly' attribute from JetBrains.Annotations has been added to all Alteration Handlers and Alteration Types in the Elsa.Alterations module. This change specifies that these classes are being used implicitly and helps tools like ReSharper prevent false unused code detection. * Update migration script for efcore-3.1 The migration script efcore-3.1.sh has been updated. Module "Alterations" and provider "Sqlite" are added back into the lists while unused elements are removed for cleanup. The script now covers all necessary modules and providers, improving its overall effectiveness. * Remove PrintMessage class from Elsa.Server.Web bundle This commit deletes the PrintMessage.cs file that was previously part of the Elsa.Server.Web bundle. * Refactor MassTransit service configuration This commit reduces line code length by merging splitted long lines into one, improves maintainability of code. It also updates the Service configuration and removes unused namespaces across files, which help to simplify the codebase. Changes have also been made to switch the MassTransit broker from RabbitMq to memory and adjust the endpoint name formatter configuration. * Remove extra whitespace in Program.cs Removed a trailing whitespace on the line declaring the useSqlServer constant in the Program.cs file. No functionality changes were made, simply a minor formatting correction to maintain clean and consistent code. * Remove AlterationJsonConverter and update other files The AlterationJsonConverter class has been removed, simplifying the serialization process. The AlterationSerializer.cs file was updated with additional attribute tags, giving more information about serialization expectations. Lastly, the AlterationSerializationOptionConfigurator.cs was simplified by removing unnecessary overrides and the Elsa.Alterations.Core.csproj file was tidied up by removing unused services folder reference. * Refactor code for improved readability and maintainability The diff shows changes across multiple files in multiple modules. Primarily, these changes revolve around formatting edits such as removing unnecessary spaces or line breaks and adjusting alignments. A significant change was also made to the MySQL server version in the `DbContextOptionsBuilder.cs` file. These adjustments help improve the readability and maintainability of the code, making it easier to understand and modify in future iterations. * Remove unnecessary notification files and refactor code Several files pertaining to workflow serialization notifications were deemed unnecessary and removed. The Endpoint class in the `Elsa.Workflows.Api` module has been refactored for efficient usage of the `IWorkflowDefinitionStore` store, and similarly the `HttpFeature.cs` and `DesignTimeDbContextFactoryBase.cs` files have been slightly refactored for better readability and optimization. * Refactor HttpFeature class on variable types order The variable types order within the HttpFeature class in the Elsa Http module has been reorganized for cleaner code. No functional changes have been made - only the order of the types has been rearranged. This refactoring contributes to better readability and code maintenance. * Add 'issue/*' branch to GitHub workflow triggers This update includes 'issue/*' branches in the trigger configuration of GitHub workflows. Now, these branches will trigger specified GitHub actions, facilitating more effective issue tracking and resolution. * Change version option in WorkflowDefinitionManager The version option in the WorkflowDefinitionManager has been updated. Previously it was set to "LatestAndPublished", but this has been changed to "LatestOrPublished", adjusting the way the manager handles workflow versions. * Update GitHub Actions to remove duplicate branch pattern Removed the duplicated 'issue/*' entry from the branch trigger list in packages.yml to prevent redundancy in GitHub Actions workflow triggers. This ensures workflows are triggered correctly and efficiently for branch patterns intended for CI processes. * Refactor job generation in GenerateAlterationJobs Refactored the GenerateAlterationJobs activity to enhance code readability and maintainability. Broke down the ExecuteAsync method into smaller, purpose-specific private methods and added the missing `Elsa.Alterations.Core.Models` namespace. * Refactor consumer concatenation logic in MassTransitFeature Removed the commented-out code that filtered temporary queues, simplifying the concatenation of consumer types with workflow message consumers. * Refactor whitespace and improve readability Removed extraneous whitespaces and reformatted the `IsEmpty` property for better code readability. This enhances the code structure without affecting functionality. * Refactor MassTransitFeature configuration Moved transport configuration logic into the dedicated `ConfigureInMemoryTransport` method and removed unused usings to simplify the `MassTransitFeature` class setup. This streamlines setting up MassTransit in-memory transport and improves code maintainability. --------- Co-authored-by: Raymond den Haan <raymond.den.haan@nexxbiz.io> Co-authored-by: Marius Vasile Vușcan <marius.vuscan@nexxbiz.io>	2024-03-13 20:34:46 +01:00
Sipke Schoorstra	13304cdb64	Add Properties field to ActivityExecutionRecord and Regenerate V3_1 migrations (#5039)	2024-03-08 08:54:08 +01:00
Sipke Schoorstra	66a620b7d9	Disable Dapper and integrate EFCore.BulkExtensions.MIT package (#5005) ... The changes involve turning off the use of Dapper in Elsa.Server.Web project for database operations. In addition, the `EFCore.BulkExtensions.MIT` has been added in the Elsa.EntityFrameworkCore.Common project as a package reference. The BulkUpsertAsync function has been refactored to utilize the BulkInsertOrUpdateAsync function provided by the BulkExtensions package, streamlining the operation. The related package versions have also been introduced in the Directory.Packages.props file.	2024-02-28 09:12:00 +01:00
Sipke Schoorstra	330dc1a0d7	Fix Dapper Provider (#4998) ... * Update Dapper for better support and pagination The update includes a significant enhancement to the Dapper module. Improvements were made to support SQL Server and enhanced SQL query pagination. Additionally, table column size was increased for various fields to handle more extensive data. Minor refactoring was also done to improve code readability. * Toggle SQL Server usage in Elsa Server The previous setting had SQL Server usage in Elsa server enabled by default. This commit changes the useSqlServer constant in Elsa.Server.Web to false, effectively switching SQL Server off unless manually activated.	2024-02-25 20:36:31 +01:00
Sipke Schoorstra	e48c5e2075	Enable Dapper and update its service registration ... The Dapper usage flag has been set to true in the Elsa.Server.Web program. Additionally, in the Dapper feature, the DbConnectionProvider is now registered as a singleton instead of scoped, reflecting changes in the database connection strategy.	2024-02-25 14:23:46 +01:00
Sipke Schoorstra	662a076726	Add simulated latency middleware ... A new middleware, SimulatedLatencyMiddleware, has been added to simulate latency in a designated range. This can be used for testing purposes by delaying a request for a random amount of time. Additionally, an extension method for IApplicationBuilder has been implemented to conveniently add this functionality to the application's pipeline.	2024-02-21 09:25:59 +01:00
Sipke Schoorstra	99d7d0009b	Update MassTransitBroker to use Memory ... The code previously had the MassTransitBroker set to AzureServiceBus. This commit changes it to use in-memory MassTransitBroker. An "Memory" enumeration value is also added in the MassTransitBroker enumeration.	2024-02-16 22:46:29 +01:00
Sipke Schoorstra	ef3431c57a	Implement dispatch channels (#4949) ... * Add DispatchWorkflowOptions and update MassTransit configuration Introduced a new class `DispatchWorkflowOptions` to provide workflow dispatch options. Updated MassTransit configuration to include NET6.0 and NET7.0 support, and ensure correct MassTransit version usage per target framework version. * Add support for configurable MassTransit message dispatching Implemented a feature which allows for configurable MassTransit message dispatching. Added support for specifying channels and message brokers. Message dispatching code was massively refactored and relevant endpoints and response models were updated to support new message dispatching features. * Add IEndpointChannelFormatter interface and implementation An interface for formatting channel queue names, 'IEndpointChannelFormatter', has been added, along with its default implementation 'DefaultEndpointChannelFormatter'. The code that uses hardcoded queue name formatting has been modified to use the new formatter instead, making it more configurable and reusable. The implementation of the formatter uses the 'Humanizer' library to kebab-case the channel names. * Add channel dispatch option to workflow activities Introduced `WorkflowDispatcherChannelOptionsProvider` to provide dropdown channel options for workflow dispatch-related activities. Updated `DispatchWorkflow` and `BulkDispatchWorkflows` activities to include a new dropdown input for specifying a dispatch channel. Also, included the selected channel name in the `DispatchWorkflowOptions` during workflow dispatching process. * Update MassTransit configurations and remove unused code The MassTransit setup in Elsa.MassTransit module has been simplified by removing conditional code for different .NET versions. Additionally, unused parameter '__X_Channel' in 'DispatchWorkflowDefinition' was removed. Lastly, the MassTransit broker in Elsa.Server.Web was switched from RabbitMq to AzureServiceBus. * Refactor dispatch workflow classes and methods Simplified the class, method and variable names related to workflow dispatching in the Elsa.Workflows.Runtime module. For example, the 'WorkflowDispatcherChannelDescriptor' class was renamed to 'DispatcherChannel'. This refactoring was performed to make code more readable and maintainable by removing redundant wording in the naming convention. * Update GitHub Workflow to support feature and issue branches The workflow changes add support for feature and issue branches. Now, it extracts the branch name and verifies the commit exists in the given branch rather than just 'main'. The versioning scheme is also modified to include the branch name and not just the run number. * Add 'bug/*' to triggering branches in packages workflow The 'bug/*' pattern was missing from the triggers that initiate the GitHub actions within our packages workflow. This update includes any branch with a 'bug/' prefix to the list, allowing bug-related branches to start jobs in our CI/CD pipeline. * Remove 'issue/*' and 'bug/*' branches from packages workflow The 'issue/*' and 'bug/*' branches have been removed from the GitHub action workflow for packages. This change was made to simplify the workflow and optimize the triggering of package building. * Update GitHub workflow to handle main branch versioning This commit modifies the GitHub workflow script to accommodate changes when the branch name is "main." If the branch name is "main", a preview version is used. It also updates script execution to print the branch name for easier debugging and verifies commit existence on the correct branch instead of dispatch channels. * Enclose branch names in quotes in packages.yml The update modifies the branch names in the packages.yml GitHub Actions workflow file. The change consists of enclosing the branch names 'main' and 'feature/*' in single quotes, ensuring compatibility and preventing potential string interpretation issues. * Update GitHub workflows package configuration The workflows package configuration has been updated to specifically watch for changes on 'feature/dispatch-channels' rather than on all feature branches. This change will prevent unnecessary builds on less relevant feature branches. * Update trigger branches in packages workflow The triggering branches in the packages workflow have been updated. Previously, only changes in the 'main' and 'feature/dispatch-channels' would trigger the workflow, now any 'feature/*' branch will. This will cause more frequent and comprehensive testing. * Add 'patch/*' to workflow trigger branches This update adds 'patch/*' to the list of branches in .github/workflows/packages.yml that can trigger the workflow. It will allow the workflow to be initiated not just for main and feature branches, but also for patches. * Add 'preview/*' to workflow triggers This commit adds a new trigger for the GitHub Actions workflow. It now also responds to push events on 'preview/*' branches, allowing for automated testing and building of these preview branches. * Update branch name extraction in GitHub Actions The extraction of the branch name has been slightly modified in the packages.yml GitHub workflow file. This alteration ensures the correct branch name is obtained for further processing within the workflow without any discrepancy. * Update branch name extraction in packages.yml Corrected the syntax for extracting the branch name within the packages.yml github workflow file. Added an extra line to print out the ref which might be useful for debugging. * Update branch name extraction in GitHub workflow The commit simplifies the way the branch name is being extracted from the GitHub ref in the packages.yml workflow file. The new method employs straightforward string manipulation, making it easier to understand and debug in case of potential issues. * Add extraction of branch name in workflow Added a new line in the GitHub workflow file (.github/workflows/packages.yml) to extract the last part after the final slash from the branch name. This enhancement allows cleaner naming conventions, especially in cases where branches are named feature/issue-123, as it will only retain 'issue-123'. * Update package naming in Github workflow The Github workflow has been updated to handle package naming more effectively. Previously, the branch name was used directly for package versioning. Now, the last part of the branch name is extracted and used as the package prefix. If the branch name is "main", the package prefix is set to "preview". * Move and add environment variable assignments The placement of the assignment for BRANCH_NAME environment variable was moved for better readability. Additionally, the PACKAGE_PREFIX environment variable was also added. These environment variables are crucial for subsequent steps in the GitHub workflow. * Add workflow dispatch validation and response handling Removed several specific dispatch response classes and consolidated all types of dispatch responses into a single DispatchWorkflowResponse class. Added a new ValidatingWorkflowDispatcher service to validate dispatch requests before they're sent. Updated several classes to work with these changes, including the BackgroundWorkflowDispatcher, MassTransitWorkflowDispatcher, and the API endpoint class. * Handle dispatch workflow failures with exceptions The DispatchWorkflow and BulkDispatchWorkflows activities now throw a FaultException when the dispatch operations fail. Previously, these operations were not checking for success and could fail silently. Now, an unsuccessful dispatch response results in a FaultException with an error message from the dispatch response.	2024-02-16 10:56:30 +01:00
Sipke Schoorstra	d4e1cf0985	Fix consumption of event payload (#4909) ... The 'input' parameter in 'PublishAsync', 'DispatchAsync' and related methods across multiple classes has been replaced with 'payload'. The renaming of 'input' to 'payload' provides a more intuitive understanding of the parameter as the actual content or data of the event being published or dispatched. It also includes changes in 'PublishInternalAsync' where the workflow input dictionary is now being populated with the payload.	2024-02-09 08:20:54 +01:00
Sipke Schoorstra	611c72fa26	Implement Zstd compression and update related components ... Removed ICompressionStrategyResolver interface and file, and added ICompressionCodec and ICompressionCodecResolver interface. Updated relevant classes for the new interface. Specifically, added Zstd class under Compression as a new compression method. Also modified WorkflowInstanceStore.cs, None.cs, EFCoreWorkflowInstanceStore.cs, GZip.cs, and ActivityExecutionLogStore.cs for uniform compression terminology.	2024-02-04 12:18:55 +01:00
Sipke Schoorstra	0bc46d3f74	Remove migration files for Management module ... Deleted all migration files related to the Management module across MySql, SqlServer, Sqlite and PostgreSql data providers. These files included data compression algorithm and data format column additions in the WorkflowInstances table.	2024-02-04 01:24:03 +01:00
Sipke Schoorstra	383eeab751	Add data compression and format fields to WorkflowInstances ... This commit introduces two new fields to the WorkflowInstances table: "DataCompressionAlgorithm" and "DataFormat". These changes allow storing additional contextual information about the payloads of workflow instances and enhance future handling and processing of this data.	2024-02-04 00:06:41 +01:00
Sipke Schoorstra	5436d954d0	Merge branch 'v3.0.5'	2024-02-01 20:11:01 +01:00
Sipke Schoorstra	1e1abfa100	Add support for list/array conversion in JavaScript evaluator ... A StringObjectDictionaryConverter has been added, which converts all IList fields of an object to array fields. This enhances the interaction of JS expressions with list types, allowing common Array methods to be applied directly. Furthermore, defaults have been added to the ExpressionEvaluatorOptions parameters to improve usability. Lastly, adjustments were made to ensure variables are correctly set while evaluating expressions.	2024-01-30 21:22:32 +01:00
Sipke Schoorstra	d0a72deadf	Merge branch 'v3.0.4'	2024-01-24 11:07:56 +01:00
Sipke Schoorstra	ac9428a6e0	Allow creation of variables dynamically via setVariable (#4800) ... * Disable MassTransitRabbitMq in Elsa.Server.Web. * Create variable if not exists in Workflow.Core The update introduced checks if a variable exists before attempting to set its value in the ExpressionExecutionContextExtensions. If the variable doesn't exist, it now creates one automatically. Fixes #4789 * Remove unnecessary comment The comment indicating to create a variable if it doesn't exist was deemed superfluous and removed from the ExpressionExecutionContextExtensions.cs file. This improves code cleanliness as the functionality is self-explanatory through the existing code structure.	2024-01-15 21:15:05 +01:00
Sipke Schoorstra	66d8e4b20f	Updated MassTransit sample project ... This commit updates the Elsa.Samples.AspNet.MassTransitActivities.csproj	2024-01-12 08:47:41 +01:00
Sipke Schoorstra	71690a2903	Merge 3.0.1 (#4759) ... * Update packages.yml for version 3.0.1 The packages.yml workflow file has been updated to target the v3.0.1 branch instead of main. The version also has been updated from 3.0.0 to 3.0.1 in the version prediction logic. Hotfix tags usage has been removed. * Update git branch for commit verification in workflow The Github workflow's step for verifying commits' existence has been updated. Instead of searching in the 'origin/main' branch, the workflow now checks in the 'origin/v3.0.1' branch. This modification ensures compatibility and consistency with the version being used. * Update Elsa.Studio package versions The Elsa.Studio and Elsa.Studio.Login.BlazorWasm packages in the Elsa.ServerAndStudio.Web and Elsa.Studio.Web projects have been updated from version 3.0.0-preview.177 to the stable version 3.0.0. This is to ensure we're using the stable and reliable versions of these packages in our projects. * Add background execution to activities and update HTTP requests Significantly enhanced the capabilities of background execution of activities. Included a change in activity type of "SendHttpRequest" from 'Task' to 'Action'. Introduced new classes for handling outcomes of context in background execution. Made some necessary adjustments to HTTP Request Task to handle sending HTTP requests from a background task. Updated several middleware classes to align with these modifications. * Add background execution handling to activity context This commit adds the ability to manage the background execution state directly within the activity execution context. This includes adding methods to set and verify the background execution state, and modifying the existing code to use these new methods. A method for handling activity scheduling during background execution has also been started, but its implementation is not finished yet. The HTTP Request activities were updated accordingly to reflect these changes. * Add scheduling function for background activities This commit achieves two main goals. Firstly, it introduces two new classes called ScheduledActivity and ScheduledActivityOptions to store scheduled activities' information. Secondly, it modifies how activities are executed in the background by capturing the scheduling information as a serializable format and storing it in the workflow execution context properties dictionary. This change allows the workflow execution context to resume the activity execution context. * Refactor HTTP request handling by removing SendHttpRequestTask SendHttpRequestTask was deleted and its functionality was merged into SendHttpRequestBase. This consolidation led to the addition of StatusCode and ResponseHeaders output fields in SendHttpRequestBase. Another change includes the update in FlowSendHttpRequest to indicate that it's no longer deprecated. Also, HttpHeaders class was extended to accommodate HttpResponseHeaders objects. The consolidation was done to streamline the HTTP request handling process. * Update GitHub Actions workflow for new release The GitHub Actions workflow configuration has been updated to target the '3.0.1' branch instead of 'main'. Furthermore, the preview version set in the workflow has been updated to '3.0.1-preview', changing from the previous '3.0.0-preview'. * Update branch verification in GitHub workflow The GitHub workflow configuration has been updated to verify that the commit exists in the branch 'origin/3.0.1' instead of 'origin/main'. This is done during the automated package generation process. * New options to control retry logic for transient failures (#4750) * Add an option to control the number of automatic retries for transient failures. * Add SleepDurationProvider option for ElsaClientBuilderOptions --------- Co-authored-by: admin <admin@admin.com> * Add IExecuteWorkflowApi interface and refine retry policy configuration A new interface, IExecuteWorkflowApi, was created to handle execution and dispatch of workflow definitions. This breaks down functionalities previously present in IWorkflowDefinitionsApi. Also, the retry policy configuration for HTTP requests has been refactored. Instead of hardcoding retry settings, now a delegate method can be optionally passed to customize the behavior. This makes it more flexible and shifts the responsibility of configuring retry policies to the client. * Add option for synchronous broadcast in WorkflowInbox This update introduces a new option to control synchronicity when broadcasting messages in the WorkflowInbox. The 'BroadcastWorkflowInboxMessageOptions' class allows the developer to specify whether the broadcasting will occur synchronously or asynchronously. The update also includes a new Endpoint and Workflow for demonstration and testing of this functionality. * Update FastEndpoints packages to version 5.21.2 The current commit updates the version of all FastEndpoints packages from 5.20.1.7-beta to 5.21.2 in the 'Elsa.Api.Common' project. This ensures we are using the most recent stable release of these packages. #4747 * Add decimal check in PolymorphicObjectConverter In the PolymorphicObjectConverter class, the check for primitive types and specific object types was updated to include decimal. Fixes #4714 * Refactor Dapper workflow and update migrations Modified the store service to optimize the SaveManyAsync method by converting input to list only once. Also, enhanced deletion query in the store service to enable usage of different primary keys. Made changes in the Dapper migrations, replacing "NodeId" with "ActivityNodeId". * Update workflow to use main branch and version 3.1.0 The workflow has been updated to work with updates on the 'main' branch rather than the 'v3.0.1' branch. Also, the version number for the 'VERSION' variable in preview mode has been updated to 3.1.0 from 3.0.1. * Remove unnecessary Elsa Server activities Deleted sample files: DataSourceActivity, MyEndpoint, and MyEventWorkflow from the Elsa Server Web bundle as part of our ongoing codebase optimization strategy. These files were no longer required and their removal simplifies our code structure. * Update package workflow to reference v3.0.1 The GitHub actions workflow has been updated to pull from branch v3.0.1 instead of main. This change affects the commit verification and version setting steps, now using version 3.0.1-preview in the workflow process. * Update Elsa.Studio packages to version 3.0.1-preview.196 This commit involves updating the versions of `Elsa.Studio`, `Elsa.Studio.Core.BlazorWasm`, and `Elsa.Studio.Login.BlazorWasm` packages in `Elsa.ServerAndStudio.Web.csproj` and `ElsaStudioWebAssembly.csproj` files to 3.0.1-preview.196. This update will incorporate the new changes and improvements included in this newer version. * Update Elsa.Studio package versions The Elsa.Studio and Elsa.Studio.Login.BlazorWasm package versions have been updated in the Elsa.Studio.Web project. Both package versions have been upgraded from 3.0.0 to 3.0.1-preview.196. * Remove unused Workflow models and simplify AddStorageDriver method The commit removes BackgroundExecutionOutcome and BackgroundExecutionResult models from Elsa.Workflows.Core, as they are no longer in use. Additionally, it simplifies the AddStorageDriver extension method in ModuleExtensions.cs, now it directly adds the service as an implementation of the IStorageDriver interface. * Remove unnecessary dependencies in DefaultBackgroundActivityInvoker Dependencies on IBookmarksPersister and IWorkflowStateExtractor have been removed in the DefaultBackgroundActivityInvoker.cs file. Additionally, an unused `using` statement for Elsa.Workflows.Helpers has been eliminated. This commit aims to declutter the code and increase its maintainability by eliminating unnecessary dependencies. --------- Co-authored-by: Night Wu <lofrank@outlook.com> Co-authored-by: admin <admin@admin.com>	2024-01-06 17:26:25 +01:00
Sipke Schoorstra	9e9b122c6e	Merge branch 'v3.0.1' into v3.1.0	2024-01-06 17:13:58 +01:00
Sipke Schoorstra	fa21cba51f	Add workflow registry ... This commit introduces a workflow registry for dynamically creating and registering workflows. It also includes a typed workflow materializer and extends existing services to support the new functionality. Changes are made across multiple files to integrate these new components and enhance the overall workflow functionalities.	2024-01-04 22:07:55 +01:00
Sipke Schoorstra	8002f20e50	Stash	2024-01-03 09:30:24 +01:00
Sipke Schoorstra	2a5cf04207	Update HTTP endpoint authorization handler ... The default authorization handler for HTTP endpoints is now the AuthenticationBasedHttpEndpointAuthorizationHandler instead of the AllowAnonymousHttpEndpointAuthorizationHandler. The configuration was updated in HttpFeature.cs and the override in Program.cs was removed. This provides more security to HTTP endpoints by requiring authentication.	2023-12-28 19:46:38 +01:00
Sipke Schoorstra	f17966c592	Refactor project structure and Docker configurations ... The changes included in this commit are made to refactor the project structure and Docker configurations. Previously, the application was structured around the AllInOneWeb solution, but it has been split into Server.Web and ServerAndStudio.Web solutions for better separation of concerns. Additionally, the Docker configurations have been updated to reflect the new structure, which should streamline the build and deployment process.	2023-12-27 20:30:33 +01:00

Renamed from src/bundles/Elsa.WorkflowServer.Web/Program.cs (Browse further)