Commit graph

30 commits

Author SHA1 Message Date
Sipke Schoorstra e603e5480b
Add caching to workflow runtime and workflow management stores (#5174)
* Add caching to workflow runtime and workflow management stores

The update introduces caching to workflow runtime and workflow management stores to enhance performance. This is achieved by adding decorators for several stores, which cache records to reduce database fetches. Additionally, a signaler for change tokens allows for cache invalidation when changes occur. The MemoryCache feature has also been updated to include Scrutor for decoration and the caching duration can be configured through the new CachingOptions class.

* Refactor WorkflowsMiddleware for HTTP Endpoint bookmarks and triggers

The WorkflowsMiddleware has been extensively refactored to handle HTTP Endpoint bookmarks and triggers. This involves breaking down the InvokeAsync method by extracting parts of its functionality into separate helper methods such as FindTriggersAsync and FindBookmarksAsync. Moreover, Assist with authorization checks, workflow execution within request timeout, and handling of workflow faults has been improved to be more efficient and clearly segmented.

* Update HTTP endpoint authorization to use Workflow context

The authorization process in the AuthenticationBasedHttpEndpointAuthorizationHandler class has been updated to use the Workflow context instead of the WorkflowInstanceId string. The AuthorizeHttpEndpointContext model has been correspondingly changed to include a Workflow property, thereby strengthening the link between authorization and specific workflows.

* Add FindAsync methods to trigger and bookmark stores

The code adjustments add new FindAsync methods to the trigger and bookmark store contracts as well as all their concrete implementations (MongoDb, Memory and EFCore). These methods support fetching the first record matching a given filter. The adjustments also include minor syntax improvements and the addition of [UsedImplicitly] attributes where needed.

* Refactor WorkflowsMiddleware for improved workflow handling

The code was refactored to simplify the flow of handling workflows in the WorkflowsMiddleware class. Specifically, the methods to start and resume a workflow have been extracted to improve code readability. Further, handleErrorMiddlewares was also updated to better manage instances where no valid workflows or base paths are found.

* Add caching functionality to WorkflowsMiddleware

Added IMemoryCache usage in the WorkflowsMiddleware to cache lookup results for workflows and their associated triggers. This will reduce the number of database operations required when searching for workflows, thus improving performance. The cache is maintained for one minute before it is refreshed.

* Implement dynamic cache duration for workflows

The code has been updated to have a dynamic cache duration for the workflows instead of a hardcoded one minute. By using the CachingOptions service, the cache duration can now be set in the configuration making it more flexible and adaptable to different performance needs.

* Add HttpWorkflowsCacheManager for caching HTTP workflows

This commit includes the implementation of IHttpWorkflowsCacheManager for caching of HTTP workflows. New handlers have been added to invalidate cache on workflow updates. Additionally, WorkflowsMiddleware has been renamed to HttpWorkflowsMiddleware.

* Refactor workflow trigger handling and caching

The refactoring includes deletion of `IndexWorkflowTriggersHandler.cs` and creation of `IndexTriggers.cs` thus revising the workflow trigger indexing approach. Also, revamped the `ITriggerIndexer` interface which now handles deletion of triggers with specific workflow and filter. Furthermore, the caching mechanism in `HttpWorkflowsCacheManager.cs` is modified to handle eviction of workflow definitions and triggers separately boosting its efficiency.

* Add summary to IndexedWorkflowTriggers

A summary has been added to the 'IndexedWorkflowTriggers' class, providing a brief description. This description outlines that it represents a collection of indexed workflow triggers, promoting clearer understanding for future reference.

* Refactor memory caching feature into separate module

This commit separates the memory caching feature from the Elsa.Common module into a distinct Elsa.Caching module. This includes moving and renaming related files, such as the MemoryCacheFeature class and associated dependencies. The references in other modules and in the main solution file have been updated accordingly to include the new Elsa.Caching module.

* Add distributed caching and update async methods

Introduced a distributed caching feature with extensible change token signal publishing. Updated various cache-related methods to be asynchronous for improved performance and responsiveness. Also updated some workflow identity references for clarity.

* Add distributed caching with MassTransit support

This addition includes the implementation of a distributed caching system with MassTransit transport. The changes introduce necessary interfaces and services, new distributed caching feature along with the support for MassTransit as a transport option. Moreover, the instance management feature has been renamed to clustering feature for better clarity.

* Refactor queue naming and scope of MassTransitChangeTokenSignalPublisher

Queue name construction is adjusted in the RabbitMqServiceBusFeature and AzureServiceBusFeature modules for better organization and readability. MassTransitChangeTokenSignalPublisher is now a singleton service, ensuring the signal publisher can be shared across the application, increasing efficiency and performance. Also, introduced use of DistributedCacheFeature in MassTransitDistributedCacheFeature module for better modularization.

* Add caching capabilities to workflow definition service

This commit introduces caching to the workflow definition service, improving the performance for retrieving workflow definitions. 4 new classes have been created (`CachingWorkflowDefinitionService`, `EvictWorkflowDefinitionServiceCache`, `WorkflowDefinitionCacheManager`, and `IWorkflowDefinitionCacheManager`), and several existing classes have been updated to support caching. The caching also includes invalidation mechanisms, ensuring data consistency.

* Refactor caching mechanism in workflow definition

In the workflow definition module, the explicit caching functionality related to workflow definition versioning has been removed in favor of a more streamlined approach. Additionally, the manner in which services are registered has been altered. As a result, the caching now directly involves the overall workflow definition rather than individual versions, simplifying the caching logic and potentially improving the performance.

* Update MassTransitBroker and enable RealTimeWorkflows and SignalRHubs

The MassTransitBroker has been updated to Memory from RabbitMq. In addition, the RealTimeWorkflows and UseWorkflowsSignalRHubs features are now enabled in the code. This change will impact how the service communicates and processes real-time requests for workflow operations.

* Reformat variable types in HttpFeature

The reformatting involves a list of variable types in the HttpFeature module. Each type now appears on a new line for improved readability, making the code easier to maintain and review.

* Update HTTP workflows cache invalidation handler XML comment

* Remove unnecessary using directives

Unnecessary using directives were deleted across several files in the Elsa.Http module. This simplifies the code and will possibly improve execution speed. Specific deletions include those for Encoding, Unicode, Extensions, Collections.Generic, Linq, Text, and Tasks namespaces.

* Refactor HttpWorkflowsMiddleware constructor

This commit simplifies the HttpWorkflowsMiddleware class constructor. It removes the intermediary variables `_next` and `_options` and directly uses the passed arguments in the constructor. Now, the `next` and `options` parameters are used directly throughout the middleware.

* Simplify workflow retrieval in HttpWorkflowsMiddleware

This refactoring replaces the use of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync with a single method, FindWorkflowAsync. This simplifies the middleware code and likely improves performance by reducing the number of database queries or service calls required to retrieve a workflow.

* Refactor workflow retrieval in HttpBookmarkProcessor

Simplified the workflow retrieval process in HttpBookmarkProcessor.cs. Replaced FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods with a single FindWorkflowAsync call. This reduces the complexity and improves efficiency in retrieving workflow.

* Optimize FindWorkflowAsync method in HttpWorkflowsCacheManager

Removed redundant lines of code to simplify workflow search functionality. This change simplified the FindWorkflowAsync method by directly calling the FindWorkflowAsync function in the workflowDefinitionService, thus increasing code readability and efficiency.

* Refactor Endpoint.cs for workflow retrieval

The method for obtaining a workflow in the Endpoint.cs script has been refactored and streamlined. The 'GetWorkflowDefinition' method is replaced by the 'GetWorkflowAsync' method which directly retrieves the workflow, without the intermediate step of materializing the workflow definition. This shortens the code and simplifies the process.

* Refactor InputFunctionsDefinitionProvider constructor

The constructor for InputFunctionsDefinitionProvider has been simplified by removing unnecessary private fields. Services are now directly used in the method instead of being stored in fields. This improves readability and reduces complexity in the class structure.

* Refactor WorkflowInstance with improved state handling

Simplified the methods for handling workflow and workflow state in the WorkflowInstance class. The refactoring also included some code clean-ups and variable renaming. The new implementation provides better readability and maintainability of the code by reducing unnecessary lines and improving structuring of objects and responses.

* Remove unused IBookmarkManager and update workflow functions

IBookmarkManager from ProtoActorWorkflowRuntime.cs file is removed due to its redundant status. Additionally, the "FindAsync" method has been updated to use a cancellation token. Also, annotations were added to the "ExportWorkflowStateAsync" and "ImportWorkflowStateAsync" methods to flag calls to functions that require unreferenced code.

* Remove unused ReSharper directive

Unused ReSharper directive in the file IndexTriggers.cs was identified and therefore removed. This change makes the code cleaner and easier to read.

* Update activity invocation in workflow runtime

Updated the DefaultBackgroundActivityInvoker service in the Elsa.Workflows.Runtime module to annotate the ExecuteAsync method with "RequiresUnreferencedCode" attribute. This change is made considering the potential code trimming issue. Additionally, simplified the process of fetching workflow by directly using FindWorkflowAsync method instead of FindWorkflowDefinitionAsync and MaterializeWorkflowAsync methods.

* Refactor code to simplify workflow definition loading

The code for finding and materializing workflow definitions has been simplified. Instead of loading the definition and materializing it into a workflow in separate steps, a new method called FindWorkflowAsync has been introduced to perform both actions at once. This reduces redundancy and makes the code more readable.

* Refactor WorkflowHostFactory to streamline workflow creation

This commit simplifies the workflow creation process in WorkflowHostFactory. It removes redundant code and extraneous methods, specifically the overloaded CreateAsync method which used WorkflowDefinition. Now, it directly finds and uses the Workflow instance, thereby simplifying the code base and improving maintainability.

* Refactor workflow retrieval in WorkflowInstance.cs

Changed the way workflow instances are retrieved from the WorkflowDefinitionService. Instead of obtaining the workflow definition and then materializing the workflow from it, the workflow is directly retrieved using the FindWorkflowAsync function. This simplifies the code and avoids unnecessary null-checks.

* Remove unnecessary whitespace in WorkflowInstance.cs

An extraneous whitespace character was identified and removed in the WorkflowInstance.cs file. This change contributes towards maintaining clean and readable code in the Elsa.ProtoActor module.

* Remove unnecessary comment in ProtoActorWorkflowRuntime

The unnecessary comment ("Load the workflow definition.") in the method TryStartWorkflowAsync of the ProtoActorWorkflowRuntime.cs file was removed. This is part of an ongoing effort to keep the codebase clean and readable.

* Update workflow management features and handlers

Added explicit notification handlers for DeleteWorkflowInstances and RefreshActivityRegistry in WorkflowManagementFeature.cs. Also, renamed RefreshActivityRegistryHandler.cs to RefreshActivityRegistry.cs for better clarity.

* Add multiple log record support to workflow execution log stores

The major change of this commit is the addition of methods to add multiple log records in the WorkflowExecutionLogStore, across different storage modules such as EntityFramework, MongoDB, Dapper, Elasticsearch and Memory. This ensures consistency and uniform behavior across different storage types. Furthermore, some reformatting and tidying up of the code were undertaken to maintain readability and clarity.

* Remove redundant workflow definition check

The workflow definition existence check and related service retrieval were removed from HttpWorkflowsMiddleware.cs. It was determined that this check was unnecessary as the workflow definition's existence is guaranteed at this point in the process, reducing redundancy in the code.

* Improve cancellation token usage in workflow execution

This commit refines the usage of cancellation tokens during the execution of workflows in Elsa.Server and HttpWorkflowsMiddleware. Previously, a cancellation token pair was created before ExecuteWithinTimeoutAsync was called, which limited duration control solely to that method. Now, cancellation tokens are included within ExecuteWithinTimeoutAsync method. This allows the method to observe any cancellation initiated by outer scopes, enhancing control over the timeout of operations.

* Add PersistStateAsync method to WorkflowHost

A new PersistStateAsync method has been added to the WorkflowHost, which enables the host to directly persist its own state. The method has been integrated into the DefaultWorkflowRuntime and HttpWorkflowsMiddleware. This update eliminates the need to continuously get instances of IWorkflowInstanceManager to save state, which improves efficiency and code readability.

* Refactor DefaultAlterationRunner service

This update simplifies the DefaultAlterationRunner service, reducing the number of code lines and removing unnecessary references. The workflow materialization step has been merged with the find workflow step, and unused namespaces have been dropped.

* Refine wording in IWorkflowHost interface documentation

The documentation for the 'CanStartWorkflowAsync' method in the IWorkflowHost interface has been cleaned up. The superfluous "or not" verbiage has been removed, making it easier to understand the method's function.

* Remove Redis from DistributedCachingTransport

The Redis option was removed from the DistributedCachingTransport enumeration. This transport isn't currently implemented.

* Remove 'useDistributedCaching' constant

The 'useDistributedCaching' constant was removed from `Program.cs`, and conditional logic was updated to use `distributedCachingTransport != DistributedCachingTransport.None`. A new option 'None' was added to the `DistributedCachingTransport` enum to facilitate this change.

* Update package tags in MassTransit project file

The package tags in the Elsa.Caching.Distributed.MassTransit project file was updated to consolidate the tags, changing 'mass-transit' to 'masstransit'. This change better aligns with standard naming conventions and improves searchability.

* Refactor distributed caching implementation

This commit involves an extensive refactor of the distributed caching implementation. Distributed caching related code and resources were moved into an independent 'Elsa.Caching.Distributed' module. The interface 'IDistributedChangeTokenSignaler' was deleted and its functionality was replaced by 'IChangeTokenSignalInvoker'.

* Refactor order of parameters in GetOrCreateAsync method

The order of parameters in the GetOrCreateAsync method within the CachingWorkflowDefinitionStore class has been changed. This change ensures that the `key` parameter is now first, followed by the `factory` parameter. This improves code readability and aligns with standard coding practices.

* Refactor cache retrieval in Workflow service

Refactoring was done to streamline the way objects are retrieved from cache in the Workflow service. Duplicated code was condensed into a new `GetFromCacheAsync` method, which is now called in the existing methods, thus increasing maintainability and reducing the possibility of errors.

* Update method descriptions and fix comments formatting

Method descriptions in various contracts have been updated to more accurately reflect their function regarding record addition and updating in the persistence store. All double comment markers (/// ///) have also been corrected to the standard (///) across multiple classes.

* Remove unused caching methods in ModuleExtensions

The commit removes the unused methods, `UseMemoryCache` and `UseDistributedCache` from the `ModuleExtensions.cs` file. The removal is part of a wider cleanup and refactoring effort to streamline the codebase and improve legibility.

* Remove redundant PrimaryKeyName in DapperWorkflowExecutionLogStore

The "PrimaryKeyName" constant was removed in DapperWorkflowExecutionLogStore. This change simplifies the initialization of the '_store' property, reducing unnecessary redundancy and complexity. The refactored code maintains the same functionality but improves readability and maintainability.

* Refactor SaveAsync methods in Elsa.Dapper Store

The SaveAsync functions have been updated in the Store.cs file inside the Elsa.Dapper module. They now include cancellation token parameters and specify that they add or update records, providing clearer distinction and flexibility.

* Refactor store initialization in Elsa.Dapper modules

Removed the redundant usage of primary keys during the store initialization across Elsa.Dapper module. Simplified the SaveAsync methods by removing the parameter for primary key, making the code cleaner and more maintainable. This refactoring does not affect the module's functionality.

* Refactor UserStore in Elsa.Dapper module

The code was adjusted to improve readability within the Elsa.Dapper module's UserStore. Two lines that were previously combined have now been separated into distinct lines, making the code structure more clear.

* Refactor constructor arguments in MongoDb module

Simplified several classes in the MongoDb module by injecting dependencies directly through the constructor instead of assigning them to private readonly fields. This improves readability and removes unnecessary code lines. Also added JetBrains.Annotations where applicable.

* Fix comment syntax in IWorkflowInstanceStore

A syntax error in the comments for the method SaveManyAsync (in IWorkflowInstanceStore interface) has been corrected. This change ensures that the remarks section of the method is properly formatted and correctly displayed in documentation.

* Remove ComputeBookmarkHash from IHttpWorkflowsCacheManager

The ComputeBookmarkHash method was removed from IHttpWorkflowsCacheManager to declutter the interface. The functionality was moved and adapted in the HttpWorkflowsMiddleware class to maintain the original functionality.

* Add logging to HttpWorkflowsMiddleware

In this update, the HttpWorkflowsMiddleware class has been modified to include logging. Specifically, warning logs have been added to track workflow-related processes and to notify if mentioned bookmarks or workflow instances are not found.

* Update consumer configuration in MassTransitFeature

This commit modifies the consumer configuration in the MassTransitFeature. Instead of hardcoding the consumer type to DispatchCancelWorkflowsRequestConsumer, it now uses the dynamic consumer type retrieved from the context, making the feature more adaptable for different scenarios.

* Change default MassTransitBroker to Memory

The default value for the variable useMassTransitBroker in Elsa.Server.Web's Program.cs file has been modified. It has been changed from RabbitMq to Memory to change the message broker used by MassTransit in the application.

* Remove Datadog.Trace package from Directory.Packages.props

The Datadog.Trace package with version 2.49.0 has been removed from the Directory.Packages.props file. This change reflects the fact that this package is no longer required in our project.
2024-04-10 11:51:40 +02:00
Sipke Schoorstra 5f47cd0aab
Implement distributed lock for registry population (#4983)
* Implement distributed lock for registry population

Added distributed lock in 'PopulateRegistriesHostedService' to prevent concurrent registry updates. Also implemented a semaphore in 'DefaultWorkflowDefinitionStorePopulator' to control access to shared resources during add or update operations. This change helps to ensure the integrity and consistency of the workflow registries.

* Refactor dependency injection for IDistributedLockProvider

* Refactor option classes to parameter classes in workflow runtime

This refactoring enhances the clarity of the Elsa Workflow runtime by renaming "options" classes to "parameters" classes. The name "options" misrepresented the classes' role and created confusion, as they are used to parameterize method calls rather than to configure services. The change applies to various workflow methods and tests across the project.
2024-02-22 19:37:03 +01:00
raymonddenhaan c409414152
Workflow cancellation (#4813)
* Removed duplicate entries

* Prevented workflows and activities from starting when the parent workflow is being cancelled

* Added cancellation to execution contexts

* Added store for workflow execution contexts

* Added cancellation to workflowRuntime

* Removed calling BookmarkPersistedHandler when persisting bookmarks.

* Added endpoint for bulk cancelling tasks

* Added tests for cancelling workflows

* Prevented cancelling the cancellation process since it could have unwanted effects

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-01-23 12:05:03 +01:00
Sipke Schoorstra 71690a2903
Merge 3.0.1 (#4759)
* Update packages.yml for version 3.0.1

The packages.yml workflow file has been updated to target the v3.0.1 branch instead of main. The version also has been updated from 3.0.0 to 3.0.1 in the version prediction logic. Hotfix tags usage has been removed.

* Update git branch for commit verification in workflow

The Github workflow's step for verifying commits' existence has been updated. Instead of searching in the 'origin/main' branch, the workflow now checks in the 'origin/v3.0.1' branch. This modification ensures compatibility and consistency with the version being used.

* Update Elsa.Studio package versions

The Elsa.Studio and Elsa.Studio.Login.BlazorWasm packages in the Elsa.ServerAndStudio.Web and Elsa.Studio.Web projects have been updated from version 3.0.0-preview.177 to the stable version 3.0.0. This is to ensure we're using the stable and reliable versions of these packages in our projects.

* Add background execution to activities and update HTTP requests

Significantly enhanced the capabilities of background execution of activities. Included a change in activity type of "SendHttpRequest" from 'Task' to 'Action'. Introduced new classes for handling outcomes of context in background execution. Made some necessary adjustments to HTTP Request Task to handle sending HTTP requests from a background task. Updated several middleware classes to align with these modifications.

* Add background execution handling to activity context

This commit adds the ability to manage the background execution state directly within the activity execution context. This includes adding methods to set and verify the background execution state, and modifying the existing code to use these new methods. A method for handling activity scheduling during background execution has also been started, but its implementation is not finished yet. The HTTP Request activities were updated accordingly to reflect these changes.

* Add scheduling function for background activities

This commit achieves two main goals. Firstly, it introduces two new classes called ScheduledActivity and ScheduledActivityOptions to store scheduled activities' information. Secondly, it modifies how activities are executed in the background by capturing the scheduling information as a serializable format and storing it in the workflow execution context properties dictionary. This change allows the workflow execution context to resume the activity execution context.

* Refactor HTTP request handling by removing SendHttpRequestTask

SendHttpRequestTask was deleted and its functionality was merged into SendHttpRequestBase. This consolidation led to the addition of StatusCode and ResponseHeaders output fields in SendHttpRequestBase. Another change includes the update in FlowSendHttpRequest to indicate that it's no longer deprecated. Also, HttpHeaders class was extended to accommodate HttpResponseHeaders objects. The consolidation was done to streamline the HTTP request handling process.

* Update GitHub Actions workflow for new release

The GitHub Actions workflow configuration has been updated to target the '3.0.1' branch instead of 'main'. Furthermore, the preview version set in the workflow has been updated to '3.0.1-preview', changing from the previous '3.0.0-preview'.

* Update branch verification in GitHub workflow

The GitHub workflow configuration has been updated to verify that the commit exists in the branch 'origin/3.0.1' instead of 'origin/main'. This is done during the automated package generation process.

* New options to control retry logic for transient failures (#4750)

* Add an option to control the number of automatic retries for transient failures.

* Add SleepDurationProvider option for ElsaClientBuilderOptions

---------

Co-authored-by: admin <admin@admin.com>

* Add IExecuteWorkflowApi interface and refine retry policy configuration

A new interface, IExecuteWorkflowApi, was created to handle execution and dispatch of workflow definitions. This breaks down functionalities previously present in IWorkflowDefinitionsApi. Also, the retry policy configuration for HTTP requests has been refactored. Instead of hardcoding retry settings, now a delegate method can be optionally passed to customize the behavior. This makes it more flexible and shifts the responsibility of configuring retry policies to the client.

* Add option for synchronous broadcast in WorkflowInbox

This update introduces a new option to control synchronicity when broadcasting messages in the WorkflowInbox. The 'BroadcastWorkflowInboxMessageOptions' class allows the developer to specify whether the broadcasting will occur synchronously or asynchronously. The update also includes a new Endpoint and Workflow for demonstration and testing of this functionality.

* Update FastEndpoints packages to version 5.21.2

The current commit updates the version of all FastEndpoints packages from 5.20.1.7-beta to 5.21.2 in the 'Elsa.Api.Common' project. This ensures we are using the most recent stable release of these packages.

#4747

* Add decimal check in PolymorphicObjectConverter

In the PolymorphicObjectConverter class, the check for primitive types and specific object types was updated to include decimal.

Fixes #4714

* Refactor Dapper workflow and update migrations

Modified the store service to optimize the SaveManyAsync method by converting input to list only once. Also, enhanced deletion query in the store service to enable usage of different primary keys. Made changes in the Dapper migrations, replacing "NodeId" with "ActivityNodeId".

* Update workflow to use main branch and version 3.1.0

The workflow has been updated to work with updates on the 'main' branch rather than the 'v3.0.1' branch. Also, the version number for the 'VERSION' variable in preview mode has been updated to 3.1.0 from 3.0.1.

* Remove unnecessary Elsa Server activities

Deleted sample files: DataSourceActivity, MyEndpoint, and MyEventWorkflow from the Elsa Server Web bundle as part of our ongoing codebase optimization strategy. These files were no longer required and their removal simplifies our code structure.

* Update package workflow to reference v3.0.1

The GitHub actions workflow has been updated to pull from branch v3.0.1 instead of main. This change affects the commit verification and version setting steps, now using version 3.0.1-preview in the workflow process.

* Update Elsa.Studio packages to version 3.0.1-preview.196

This commit involves updating the versions of `Elsa.Studio`, `Elsa.Studio.Core.BlazorWasm`, and `Elsa.Studio.Login.BlazorWasm` packages in `Elsa.ServerAndStudio.Web.csproj` and `ElsaStudioWebAssembly.csproj` files to 3.0.1-preview.196. This update will incorporate the new changes and improvements included in this newer version.

* Update Elsa.Studio package versions

The Elsa.Studio and Elsa.Studio.Login.BlazorWasm package versions have been updated in the Elsa.Studio.Web project. Both package versions have been upgraded from 3.0.0 to 3.0.1-preview.196.

* Remove unused Workflow models and simplify AddStorageDriver method

The commit removes BackgroundExecutionOutcome and BackgroundExecutionResult models from Elsa.Workflows.Core, as they are no longer in use. Additionally, it simplifies the AddStorageDriver extension method in ModuleExtensions.cs, now it directly adds the service as an implementation of the IStorageDriver interface.

* Remove unnecessary dependencies in DefaultBackgroundActivityInvoker

Dependencies on IBookmarksPersister and IWorkflowStateExtractor have been removed in the DefaultBackgroundActivityInvoker.cs file. Additionally, an unused `using` statement for Elsa.Workflows.Helpers has been eliminated. This commit aims to declutter the code and increase its maintainability by eliminating unnecessary dependencies.

---------

Co-authored-by: Night Wu <lofrank@outlook.com>
Co-authored-by: admin <admin@admin.com>
2024-01-06 17:26:25 +01:00
jdevillard 9175322774
Add a more generic UIHandler to customize how inputAttributes can be handle by UI (#4688)
* add a more generic UIHandler to customize how inputAttributes can be handle by the ui

* Add IPropertyUIHandlerResolver and update PropertyUIHandlerResolver

Introduced a new interface, IPropertyUIHandlerResolver, to resolve UI options for a property. Refactored PropertyUIHandlerResolver to implement this interface and removed the unnecessary partial class structure. Also, cleaned up some unnecessary usings in various files for better code organization.

* Refactor variable name and description in InputDescriptor

The 'uISpecifications' variable in the InputDescriptor model is renamed to 'uiSpecifications' for better readability. Additionally, the associated comment was revised to explain that the dictionary is used by the UI.

* "Refactor codebase for improved organization and cleaner architecture"

The codebase has been significantly refactored, moving several classes to more appropriate namespaces for improved organization and cleaner architecture. This includes shifting UI hint handlers, activities, and memory-related components, amongst others. The changes should improve code readability and maintainability, but as this is a broad refactoring effort, thorough regression testing is advised.

* Add CheckList UIHint with associated handler and provider

This update introduces a new UIHint called CheckList to the Elsa.Workflows.Core. This includes the necessary handler and provider classes. The handler is registered in the WorkflowsFeature.cs, and the CheckList UIHint key has been added to the InputUIHints.cs. Various associated files have been created in both the Elsa.Api.Client and Elsa.Workflows.Core project to support this new UIHint.

---------

Co-authored-by: Jérémie DEVILLARD <jdevillard@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-12-26 18:56:29 +01:00
Sipke Schoorstra 3aedf27727 Update workflow handling and simplify codebase
Refactored the workflow handling logic and simplified numerous components in the codebase. Major changes include the enhancement of the Workflow Runtime handler, altering the broadcast and deliver functions in the Default Workflow Inbox, and the removal of unnecessary details in the Workflow Runtime feature. Also, unnecessary services and requests were removed for better code efficiency and readability.
2023-12-18 16:11:48 +01:00
Sipke Schoorstra ca5692e3bc
Alteration Improvements and fixes (#4518)
* Remove unused NOOP stores

* Add activity execution manager and FindAsync method to store

The notification is used to broadcast changes to SignalR clients.

* Refactor ActiveActivityExectionContexts

Instead, the workflow state extractor is in control of what gets extracted.

* Add missing files for activity execution deleted broadcast

* Update Flowchart to not fault when child faults

* Update Flowchart to support scheduling existing activity instances

* Update namespaces from Services to Stores

* Improve activity cancellation alteration

* Update ScheduleActivity alteration
2023-10-10 12:34:32 +02:00
Sipke Schoorstra 523f301bf7
Feature: Alterations (#4501)
* Implement Alteration types and engine

* Implement Alteration APIs

* Refactor activity work item with support for activity input

* Serialize scheduled activities as part of workflow state

* Add signal to schedule child activity

* Implement stores for plans and jobs

* Add EF Core provider

* Add Retry endpoint

* Keep root context as always active. The root context is the workflow execution context and contains persistent variable state

* Regenerate EF Core migrations

* Handle orphaned activity execution contexts

* Refactor workflow execution factory

* Enable configuring Jint from Program

* Update ModifyVariable alteration with support for type deserialization

* Use ShortGuid for identity

* Cleanup WorkflowServer host
2023-10-07 12:47:36 +02:00
Sipke Schoorstra 03b608410b
Incidents (#4436)
* Implement Incident Strategy interface and resolver

* Add ActivityIncident model

* Add IncidentCount and Incidents properties and update migrations

* Expose incident strategies API endpoint

* Update API client with Incident Strategy models

* Replace Fault with Incidents

* Register fault status when any children have faulted

* Map incident count

* Remove unnecessary AlreadyCompleted result

* Ensure next activities are scheduled only if activity completed normally

* Incident roundtrip and fix workflow instance realtime updates

* Remove unused variable

* Add integration tests
2023-09-14 10:50:02 +02:00
Sipke Schoorstra 62ae3b1f6e
Add ability to push variables to scheduled work (#4306) 2023-08-09 22:35:44 +02:00
Sipke Schoorstra 95cefe631f
Bookmarks & Activity Instances (#4301)
* Add ActivityInstanceId to bookmark model

* Fix Telnyx activity resumption

* Fix missing activity instance ID forwarding to workflow inbox
2023-08-09 13:02:20 +02:00
Sipke Schoorstra c9ca4a121d
Workflow Inbox (#4292)
* Initial implementation of workflow inbox, submission and delivery

* Implement persistence providers

* Update EF Core migrations

* Rename Services to Stores in Dapper module

* Use default serializer for primitive arrays

* Update MongoDB provider to map BookmarkId to be primary key

* Implement MongoDB provider for activity execution logs

* Use EF Core by default

* Fix private JSON constructor logic
2023-08-06 17:03:36 +02:00
Sipke Schoorstra 0c8bbb935c Implement real time workflow update broadcasting using SignalR 2023-07-25 15:12:27 +02:00
Sipke Schoorstra f71e5c89ce Ensure schedules are deleted when bookmarks are deleted 2023-07-20 13:00:09 +02:00
Sipke Schoorstra 72f4461ca3 Implement cascading deletion 2023-07-20 00:05:43 +02:00
Sipke Schoorstra 9671d85013 Cleanup namespaces 2023-07-09 14:02:41 +02:00
Sipke Schoorstra fe2a897a30 Restructure models in Workflows.Core 2023-07-09 13:12:13 +02:00
Sipke Schoorstra a45b7e5fb2
Implement Onboarding sample (#3944)
* Add Parallel activity

* Update polymorphic converter to handle null values

* Add support for converting ExpandoObject into target type

* Update models in Webhooks module

* Add Onboarding sample projects

* Handle null values in converter

* Add error logging to webhook invoker

* Allow activity resumption callbacks to be static

* Handle $ref when serializing expando objects
2023-04-23 20:03:52 +02:00
Sipke Schoorstra c3ed7e7865
Improve JSON serializationfor workflow state (#3913)
* Incremental work on WorkflowStateSerializer

* Update Flow HTTP Request activity to default to Done outcome

* Fix reference handling

* Fix polymorphic object converter

* Add comments

* Add support for registering type alias during DI setup via options

* Fix trigger and bookmark payload persistence

* Update sample model

* Add support for custom dictionary types

* Split general serializer options provider into focused serializer services

* Formatting

* Update migrations
2023-04-17 00:11:58 +02:00
Sipke Schoorstra 17406c1db3 Implement activity cancellation 2023-04-13 20:05:14 +02:00
Sipke Schoorstra 7a09944732
Refactor scheduling and background activity execution (#3884)
* Refactor background activity execution

* Refactor scheduler

* Remove Jobs module

* Add XML comments
2023-04-09 23:35:32 +02:00
Sipke Schoorstra 1a713b4619
Rename "services" to "contracts" and "implementations" to "services" (#3758) 2023-03-03 22:47:23 +01:00
Sipke Schoorstra 431db1c353
Refactor store APIs (#3740)
* Add array support for input/output arguments

* Add alias support for nullable types

* Refactor workflow definition store API

* Fix array and ui hint setting of input/output definition

* Refactor workflow instance store API

* Add XML comments

* Refactor trigger and bookmark store API
2023-02-27 14:14:50 +01:00
Sipke Schoorstra 750c2094cc
RunTask activity (#3594)
* Implement RunTask activity
2023-01-04 20:36:38 +01:00
gurkanguran 4272d5a49c
Implemented workflow sink module (#3571)
* Implemented workflow sink module

* Sequentual process of saving sinks in single consumer

* Refactor

* Log errors during sink export

* Removed redundant classes

* Use in process messaging in bundle

* Fixed the issue with duplicated ResultVariable key on serializing

* Moved workflowsink modules out of persistence projects

* Moved migrations out of old projects

Co-authored-by: Gürkan Güran <grkngrn@gmail.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-01-01 18:02:38 +01:00
Sipke Schoorstra 8cc8d1bcbf
Telnyx integration (#3453)
* Incremental work on Telnyx integration

* Incremental port of Telnyx activities

* Fix polymorphic JSON serialization for webhook data

* Fix IsFirstPass logic

* Fix trigger base to prevent activity auto completion

* Temporarily disable persistent snapshotting provider for testing

* Fix namespaces

* Add method to register activities in bulk using marker type

* Fix Telnyx refit client setup

* Register all Telnyx activities by default

* Incremental work on webhook driven resumption and correlation

* Incremental work on post-bookmark signaling

* Fix bookmark resolution

* Include correlation ID with bookmark

* Fix Telnyx resumption input

* Replace bookmark persisted signal with handler

* Fix SpeakText completion

* Add flow outcomes to SpeakText activity

* Update GatherUsingSpeak and GatherUsingaudio

* Refactor "is first pass" concept

* Fix WebhookEvent activity

* Add HTTP absolute URL provider

* Formatting and small changes

* Various API enhancements
2022-11-19 22:30:43 +01:00
Sipke Schoorstra 31942ae96f
Implement async workflow state exporter for DB workflow instance (#3291)
* Incremental work on in-process message consumers

* Implement asynchronous workflow state DB exporter

* Fixes
2022-09-07 17:28:36 +02:00
Sipke Schoorstra b3bae39bfe
Default Runtime & Persistence (#3287)
* Delete DefaultWorkflowInvoker.cs

* Incremental work

* Refactor runtime

* Implement workflow state store + bookmark store

* Cleanup

* Update migrations

* Move store abstractions to management and runtime projects

* Consolidate persistence providers into single package

* Generate migrations

* Regenerate migrations

* Fix DI of memory stores

* Update migrations

* Refactor definition reference due to unserializable record

* Update web server bundle to use EF core by default

* Update Bookmark.cs

Fixes System Text Json deserialization
2022-09-04 17:04:14 +02:00
Sipke Schoorstra e2ffafc736
Refactor proto actor runtime with snapshot support (#3283)
* Initial working grain

* Incremental work on actor bookmark management & resuming wokflows

* Upgrade Proto Actor package references

* Fix lifetime of actors

* Implement bookmark and resumption

* Update Program.cs

* Update bookmark event publishing

* Refactor proto actor features API

* Implement snapshotting

* Update snapshotting

* Add in-memory provider and use it by default

* Update server bundle to use proto actor sqlite persistence by default
2022-09-02 13:18:53 +02:00
Sipke Schoorstra fccf4737e2
V3 modular configuration (#3070)
* Restructuring (WIP)

* Fix warnings

* Move DSL to modules

* Update namespaces

* Make design factory base reusable

* Update service registrations

* Remove unused namespaces and rename ElsaDbContext

* Incremental work on migrations

* Generate WorkflowsDbContext SQLite migrations

* Rename ServiceCollectionExtensions

* Update DbContextOptionsBuilderExtensions.cs

* Restructure

* Configuration refactorings (WIP)

* Incremental work on configurator API

* Incremental work on configuration API

* Incremental work on configuration API

* Fix workflow runtime options

* Move standard stream providers

* Implement configurator dependencies

* Cleanup configurator API

* Update configurator dependencies

* Cleanup

* Update Label API endpoints

* Fix eager configuration issue
2022-05-26 12:47:31 +02:00