SemianiakaVY/elsa-core - Forgejo: Beyond coding. We forge.

SemianiakaVY/elsa-core

11 lines

200 B

C#

Raw Normal View History

Add caching to workflow runtime and workflow management stores (#5174) * Add caching to workflow runtime and workflow management stores The update introduces caching to workflow runtime and workflow management stores to enhance performance. This is achieved by adding decorators for several stores, which cache records to reduce database fetches. Additionally, a signaler for change tokens allows for cache invalidation when changes occur. The MemoryCache feature has also been updated to include Scrutor for decoration and the caching duration can be configured through the new CachingOptions class. * Refactor WorkflowsMiddleware for HTTP Endpoint bookmarks and triggers The WorkflowsMiddleware has been extensively refactored to handle HTTP Endpoint bookmarks and triggers. This involves breaking down the InvokeAsync method by extracting parts of its functionality into separate helper methods such as FindTriggersAsync and FindBookmarksAsync. Moreover, Assist with authorization checks, workflow execution within request timeout, and handling of workflow faults has been improved to be more efficient and clearly segmented. * Update HTTP endpoint authorization to use Workflow context The authorization process in the AuthenticationBasedHttpEndpointAuthorizationHandler class has been updated to use the Workflow context instead of the WorkflowInstanceId string. The AuthorizeHttpEndpointContext model has been correspondingly changed to include a Workflow property, thereby strengthening the link between authorization and specific workflows. * Add FindAsync methods to trigger and bookmark stores The code adjustments add new FindAsync methods to the trigger and bookmark store contracts as well as all their concrete implementations (MongoDb, Memory and EFCore). These methods support fetching the first record matching a given filter. The adjustments also include minor syntax improvements and the addition of [UsedImplicitly] attributes where needed. * Refactor WorkflowsMiddleware for improved workflow handling The code was refactored to simplify the flow of handling workflows in the WorkflowsMiddleware class. Specifically, the methods to start and resume a workflow have been extracted to improve code readability. Further, handleErrorMiddlewares was also updated to better manage instances where no valid workflows or base paths are found. * Add caching functionality to WorkflowsMiddleware Added IMemoryCache usage in the WorkflowsMiddleware to cache lookup results for workflows and their associated triggers. This will reduce the number of database operations required when searching for workflows, thus improving performance. The cache is maintained for one minute before it is refreshed. * Implement dynamic cache duration for workflows The code has been updated to have a dynamic cache duration for the workflows instead of a hardcoded one minute. By using the CachingOptions service, the cache duration can now be set in the configuration making it more flexible and adaptable to different performance needs. * Add HttpWorkflowsCacheManager for caching HTTP workflows This commit includes the implementation of IHttpWorkflowsCacheManager for caching of HTTP workflows. New handlers have been added to invalidate cache on workflow updates. Additionally, WorkflowsMiddleware has been renamed to HttpWorkflowsMiddleware. * Refactor workflow trigger handling and caching The refactoring includes deletion of `IndexWorkflowTriggersHandler.cs` and creation of `IndexTriggers.cs` thus revising the workflow trigger indexing approach. Also, revamped the `ITriggerIndexer` interface which now handles deletion of triggers with specific workflow and filter. Furthermore, the caching mechanism in `HttpWorkflowsCacheManager.cs` is modified to handle eviction of workflow definitions and triggers separately boosting its efficiency. * Add summary to IndexedWorkflowTriggers A summary has been added to the 'IndexedWorkflowTriggers' class, providing a brief description. This description outlines that it represents a collection of indexed workflow triggers, promoting clearer understanding for future reference. * Refactor memory caching feature into separate module This commit separates the memory caching feature from the Elsa.Common module into a distinct Elsa.Caching module. This includes moving and renaming related files, such as the MemoryCacheFeature class and associated dependencies. The references in other modules and in the main solution file have been updated accordingly to include the new Elsa.Caching module. * Add distributed caching and update async methods Introduced a distributed caching feature with extensible change token signal publishing. Updated various cache-related methods to be asynchronous for improved performance and responsiveness. Also updated some workflow identity references for clarity. * Add distributed caching with MassTransit support This addition includes the implementation of a distributed caching system with MassTransit transport. The changes introduce necessary interfaces and services, new distributed caching feature along with the support for MassTransit as a transport option. Moreover, the instance management feature has been renamed to clustering feature for better clarity. * Refactor queue naming and scope of MassTransitChangeTokenSignalPublisher Queue name construction is adjusted in the RabbitMqServiceBusFeature and AzureServiceBusFeature modules for better organization and readability. MassTransitChangeTokenSignalPublisher is now a singleton service, ensuring the signal publisher can be shared across the application, increasing efficiency and performance. Also, introduced use of DistributedCacheFeature in MassTransitDistributedCacheFeature module for better modularization. * Add caching capabilities to workflow definition service This commit introduces caching to the workflow definition service, improving the performance for retrieving workflow definitions. 4 new classes have been created (`CachingWorkflowDefinitionService`, `EvictWorkflowDefinitionServiceCache`, `WorkflowDefinitionCacheManager`, and `IWorkflowDefinitionCacheManager`), and several existing classes have been updated to support caching. The caching also includes invalidation mechanisms, ensuring data consistency. * Refactor caching mechanism in workflow definition In the workflow definition module, the explicit caching functionality related to workflow definition versioning has been removed in favor of a more streamlined approach. Additionally, the manner in which services are registered has been altered. As a result, the caching now directly involves the overall workflow definition rather than individual versions, simplifying the caching logic and potentially improving the performance. * Update MassTransitBroker and enable RealTimeWorkflows and SignalRHubs The MassTransitBroker has been updated to Memory from RabbitMq. In addition, the RealTimeWorkflows and UseWorkflowsSignalRHubs features are now enabled in the code. This change will impact how the service communicates and processes real-time requests for workflow operations. * Reformat variable types in HttpFeature The reformatting involves a list of variable types in the HttpFeature module. Each type now appears on a new line for improved readability, making the code easier to maintain and review. * Update HTTP workflows cache invalidation handler XML comment * Remove unnecessary using directives Unnecessary using directives were deleted across several files in the Elsa.Http module. This simplifies the code and will possibly improve execution speed. Specific deletions include those for Encoding, Unicode, Extensions, Collections.Generic, Linq, Text, and Tasks namespaces. * Refactor HttpWorkflowsMiddleware constructor This commit simplifies the HttpWorkflowsMiddleware class constructor. It removes the intermediary variables `_next` and `_options` and directly uses the passed arguments in the constructor. Now, the `next` and `options` parameters are used directly throughout the middleware. * Simplify workflow retrieval in HttpWorkflowsMiddleware This refactoring replaces the use of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync with a single method, FindWorkflowAsync. This simplifies the middleware code and likely improves performance by reducing the number of database queries or service calls required to retrieve a workflow. * Refactor workflow retrieval in HttpBookmarkProcessor Simplified the workflow retrieval process in HttpBookmarkProcessor.cs. Replaced FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods with a single FindWorkflowAsync call. This reduces the complexity and improves efficiency in retrieving workflow. * Optimize FindWorkflowAsync method in HttpWorkflowsCacheManager Removed redundant lines of code to simplify workflow search functionality. This change simplified the FindWorkflowAsync method by directly calling the FindWorkflowAsync function in the workflowDefinitionService, thus increasing code readability and efficiency. * Refactor Endpoint.cs for workflow retrieval The method for obtaining a workflow in the Endpoint.cs script has been refactored and streamlined. The 'GetWorkflowDefinition' method is replaced by the 'GetWorkflowAsync' method which directly retrieves the workflow, without the intermediate step of materializing the workflow definition. This shortens the code and simplifies the process. * Refactor InputFunctionsDefinitionProvider constructor The constructor for InputFunctionsDefinitionProvider has been simplified by removing unnecessary private fields. Services are now directly used in the method instead of being stored in fields. This improves readability and reduces complexity in the class structure. * Refactor WorkflowInstance with improved state handling Simplified the methods for handling workflow and workflow state in the WorkflowInstance class. The refactoring also included some code clean-ups and variable renaming. The new implementation provides better readability and maintainability of the code by reducing unnecessary lines and improving structuring of objects and responses. * Remove unused IBookmarkManager and update workflow functions IBookmarkManager from ProtoActorWorkflowRuntime.cs file is removed due to its redundant status. Additionally, the "FindAsync" method has been updated to use a cancellation token. Also, annotations were added to the "ExportWorkflowStateAsync" and "ImportWorkflowStateAsync" methods to flag calls to functions that require unreferenced code. * Remove unused ReSharper directive Unused ReSharper directive in the file IndexTriggers.cs was identified and therefore removed. This change makes the code cleaner and easier to read. * Update activity invocation in workflow runtime Updated the DefaultBackgroundActivityInvoker service in the Elsa.Workflows.Runtime module to annotate the ExecuteAsync method with "RequiresUnreferencedCode" attribute. This change is made considering the potential code trimming issue. Additionally, simplified the process of fetching workflow by directly using FindWorkflowAsync method instead of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods. * Refactor code to simplify workflow definition loading The code for finding and materializing workflow definitions has been simplified. Instead of loading the definition and materializing it into a workflow in separate steps, a new method called FindWorkflowAsync has been introduced to perform both actions at once. This reduces redundancy and makes the code more readable. * Refactor WorkflowHostFactory to streamline workflow creation This commit simplifies the workflow creation process in WorkflowHostFactory. It removes redundant code and extraneous methods, specifically the overloaded CreateAsync method which used WorkflowDefinition. Now, it directly finds and uses the Workflow instance, thereby simplifying the code base and improving maintainability. * Refactor workflow retrieval in WorkflowInstance.cs Changed the way workflow instances are retrieved from the WorkflowDefinitionService. Instead of obtaining the workflow definition and then materializing the workflow from it, the workflow is directly retrieved using the FindWorkflowAsync function. This simplifies the code and avoids unnecessary null-checks. * Remove unnecessary whitespace in WorkflowInstance.cs An extraneous whitespace character was identified and removed in the WorkflowInstance.cs file. This change contributes towards maintaining clean and readable code in the Elsa.ProtoActor module. * Remove unnecessary comment in ProtoActorWorkflowRuntime The unnecessary comment ("Load the workflow definition.") in the method TryStartWorkflowAsync of the ProtoActorWorkflowRuntime.cs file was removed. This is part of an ongoing effort to keep the codebase clean and readable. * Update workflow management features and handlers Added explicit notification handlers for DeleteWorkflowInstances and RefreshActivityRegistry in WorkflowManagementFeature.cs. Also, renamed RefreshActivityRegistryHandler.cs to RefreshActivityRegistry.cs for better clarity. * Add multiple log record support to workflow execution log stores The major change of this commit is the addition of methods to add multiple log records in the WorkflowExecutionLogStore, across different storage modules such as EntityFramework, MongoDB, Dapper, Elasticsearch and Memory. This ensures consistency and uniform behavior across different storage types. Furthermore, some reformatting and tidying up of the code were undertaken to maintain readability and clarity. * Remove redundant workflow definition check The workflow definition existence check and related service retrieval were removed from HttpWorkflowsMiddleware.cs. It was determined that this check was unnecessary as the workflow definition's existence is guaranteed at this point in the process, reducing redundancy in the code. * Improve cancellation token usage in workflow execution This commit refines the usage of cancellation tokens during the execution of workflows in Elsa.Server and HttpWorkflowsMiddleware. Previously, a cancellation token pair was created before ExecuteWithinTimeoutAsync was called, which limited duration control solely to that method. Now, cancellation tokens are included within ExecuteWithinTimeoutAsync method. This allows the method to observe any cancellation initiated by outer scopes, enhancing control over the timeout of operations. * Add PersistStateAsync method to WorkflowHost A new PersistStateAsync method has been added to the WorkflowHost, which enables the host to directly persist its own state. The method has been integrated into the DefaultWorkflowRuntime and HttpWorkflowsMiddleware. This update eliminates the need to continuously get instances of IWorkflowInstanceManager to save state, which improves efficiency and code readability. * Refactor DefaultAlterationRunner service This update simplifies the DefaultAlterationRunner service, reducing the number of code lines and removing unnecessary references. The workflow materialization step has been merged with the find workflow step, and unused namespaces have been dropped. * Refine wording in IWorkflowHost interface documentation The documentation for the 'CanStartWorkflowAsync' method in the IWorkflowHost interface has been cleaned up. The superfluous "or not" verbiage has been removed, making it easier to understand the method's function. * Remove Redis from DistributedCachingTransport The Redis option was removed from the DistributedCachingTransport enumeration. This transport isn't currently implemented. * Remove 'useDistributedCaching' constant The 'useDistributedCaching' constant was removed from `Program.cs`, and conditional logic was updated to use `distributedCachingTransport != DistributedCachingTransport.None`. A new option 'None' was added to the `DistributedCachingTransport` enum to facilitate this change. * Update package tags in MassTransit project file The package tags in the Elsa.Caching.Distributed.MassTransit project file was updated to consolidate the tags, changing 'mass-transit' to 'masstransit'. This change better aligns with standard naming conventions and improves searchability. * Refactor distributed caching implementation This commit involves an extensive refactor of the distributed caching implementation. Distributed caching related code and resources were moved into an independent 'Elsa.Caching.Distributed' module. The interface 'IDistributedChangeTokenSignaler' was deleted and its functionality was replaced by 'IChangeTokenSignalInvoker'. * Refactor order of parameters in GetOrCreateAsync method The order of parameters in the GetOrCreateAsync method within the CachingWorkflowDefinitionStore class has been changed. This change ensures that the `key` parameter is now first, followed by the `factory` parameter. This improves code readability and aligns with standard coding practices. * Refactor cache retrieval in Workflow service Refactoring was done to streamline the way objects are retrieved from cache in the Workflow service. Duplicated code was condensed into a new `GetFromCacheAsync` method, which is now called in the existing methods, thus increasing maintainability and reducing the possibility of errors. * Update method descriptions and fix comments formatting Method descriptions in various contracts have been updated to more accurately reflect their function regarding record addition and updating in the persistence store. All double comment markers (/// ///) have also been corrected to the standard (///) across multiple classes. * Remove unused caching methods in ModuleExtensions The commit removes the unused methods, `UseMemoryCache` and `UseDistributedCache` from the `ModuleExtensions.cs` file. The removal is part of a wider cleanup and refactoring effort to streamline the codebase and improve legibility. * Remove redundant PrimaryKeyName in DapperWorkflowExecutionLogStore The "PrimaryKeyName" constant was removed in DapperWorkflowExecutionLogStore. This change simplifies the initialization of the '_store' property, reducing unnecessary redundancy and complexity. The refactored code maintains the same functionality but improves readability and maintainability. * Refactor SaveAsync methods in Elsa.Dapper Store The SaveAsync functions have been updated in the Store.cs file inside the Elsa.Dapper module. They now include cancellation token parameters and specify that they add or update records, providing clearer distinction and flexibility. * Refactor store initialization in Elsa.Dapper modules Removed the redundant usage of primary keys during the store initialization across Elsa.Dapper module. Simplified the SaveAsync methods by removing the parameter for primary key, making the code cleaner and more maintainable. This refactoring does not affect the module's functionality. * Refactor UserStore in Elsa.Dapper module The code was adjusted to improve readability within the Elsa.Dapper module's UserStore. Two lines that were previously combined have now been separated into distinct lines, making the code structure more clear. * Refactor constructor arguments in MongoDb module Simplified several classes in the MongoDb module by injecting dependencies directly through the constructor instead of assigning them to private readonly fields. This improves readability and removes unnecessary code lines. Also added JetBrains.Annotations where applicable. * Fix comment syntax in IWorkflowInstanceStore A syntax error in the comments for the method SaveManyAsync (in IWorkflowInstanceStore interface) has been corrected. This change ensures that the remarks section of the method is properly formatted and correctly displayed in documentation. * Remove ComputeBookmarkHash from IHttpWorkflowsCacheManager The ComputeBookmarkHash method was removed from IHttpWorkflowsCacheManager to declutter the interface. The functionality was moved and adapted in the HttpWorkflowsMiddleware class to maintain the original functionality. * Add logging to HttpWorkflowsMiddleware In this update, the HttpWorkflowsMiddleware class has been modified to include logging. Specifically, warning logs have been added to track workflow-related processes and to notify if mentioned bookmarks or workflow instances are not found. * Update consumer configuration in MassTransitFeature This commit modifies the consumer configuration in the MassTransitFeature. Instead of hardcoding the consumer type to DispatchCancelWorkflowsRequestConsumer, it now uses the dynamic consumer type retrieved from the context, making the feature more adaptable for different scenarios. * Change default MassTransitBroker to Memory The default value for the variable useMassTransitBroker in Elsa.Server.Web's Program.cs file has been modified. It has been changed from RabbitMq to Memory to change the message broker used by MassTransit in the application. * Remove Datadog.Trace package from Directory.Packages.props The Datadog.Trace package with version 2.49.0 has been removed from the Directory.Packages.props file. This change reflects the fact that this package is no longer required in our project. 2024-04-10 09:51:40 +00:00			`namespace Elsa.Server.Web;`

			`/// <summary>`
			`/// Represents the transport options for distributed caching.`
			`/// </summary>`
			`public enum DistributedCachingTransport`
			`{`
			`None,`
			`Memory,`
			`MassTransit`
			`}`