Commit graph

195 commits

Author SHA1 Message Date
Sipke Schoorstra 3aedf27727 Update workflow handling and simplify codebase
Refactored the workflow handling logic and simplified numerous components in the codebase. Major changes include the enhancement of the Workflow Runtime handler, altering the broadcast and deliver functions in the Default Workflow Inbox, and the removal of unnecessary details in the Workflow Runtime feature. Also, unnecessary services and requests were removed for better code efficiency and readability.
2023-12-18 16:11:48 +01:00
Sipke Schoorstra cc24c5c759 Add bookmarks processing to DefaultBackgroundActivityInvoker
The code addition includes two dependencies, IBookmarksPersister and IWorkflowStateExtractor, in the DefaultBackgroundActivityInvoker service. With these changes, during the workflow execution phase, bookmarks are captured and processed for any state changes. These modifications provide more control and efficiency over handling workflow states during the system's background operations.
2023-12-18 16:05:51 +01:00
Sipke Schoorstra f575d38dcc Add Bookmark Persister and Bookmark Updater
The code introduces BookmarkPersister for processing bookmarks as part of the workflow runtime. This refactoring improves readability and maintainability by separating the bookmark persisting logic from the PersistBookmarkMiddleware class. A new class, BookmarkUpdater, is also added to handle updating of bookmarks. Changes also include adjustments to several other classes for improved functionality and code consistency.
2023-12-18 16:05:12 +01:00
Sipke Schoorstra 6db3cc99b5 Update BulkDispatchWorkflows activity
The BulkDispatchWorkflows.cs activity in Elsa.Workflows.Runtime module has been updated with some key changes: Added detailed description to the input element, introduced the dispatchedInstancesCount to accurately track the number of instances dispatched, and removed some unnecessary white spaces for improved readability.
2023-12-17 21:45:43 +01:00
Sipke Schoorstra 916256e9f6 Change ActivityKind from Job to Task in BulkDispatchWorkflows
The ActivityKind property in BulkDispatchWorkflows class has been changed from Job to Task. This updates the type of activity associated with the BulkDispatchWorkflows process within the Elsa workflow runtime module.
2023-12-17 21:26:58 +01:00
Sipke Schoorstra fc878a4b30 Refactor background activities scheduling
Background activities scheduling was moved to its own middleware where the actual scheduling happens instead of in the invoking middleware. Removed ScheduleBackgroundActivities.cs handler and added ScheduleBackgroundActivitiesMiddleware.cs instead. As a part of this change, restructured BackgroundActivityInvokerMiddleware to BackgroundActivityCollectorMiddleware, and updated all relevant references.

Minor modifications to a few other files include changes in their package dependencies and activity kind settings.
2023-12-17 21:19:15 +01:00
Sipke Schoorstra 65cb2640e3 Implement batch processing in BulkDispatchWorkflows
The changes introduce a batch processing mechanism in the BulkDispatchWorkflows activity. Previously, each workflow was dispatched individually. Now, the process groups the workflows in batches, processes each batch concurrently, and handles errors. This change improves performance, particularly for large data sources.
2023-12-16 11:45:31 +01:00
Sipke Schoorstra 6ff6cc6826 Refactor expression descriptors and remove unnecessary async calls
This commit refactors how expression descriptors are created and retrieved across Elsa project. The asynchronous methods have been simplified to synchronous ones removing unnecessary `ValueTask` returns. Additionally, `IExpressionDescriptorRegistryPopulator` has been removed resulting in less complexity and better performance.
2023-12-10 21:43:24 +01:00
Sipke Schoorstra 683f0ee6d8
Add Bulk Dispatch Workflow activity (#4683)
* Initial version of BulkDispatchWorkflow

* Update comments

* Refactor options models for running workflows and add Properties

* Fix resumption

* Add ability to provide arguments when evaluating expressions

* Update args syntax for JS

* Fix ProtoActor Runtime Properties mapping

* Use default constructor

* Provide completed workflow instance ID as input to BulkDispatchWorkflows

* Implement ChildFaulted port

* Switch to Publish to make Memory service bus work

* Update input evaluation and activity output retrieval

Refined the input evaluation process to only consider inputs with AutoEvaluate set to true. Added a comment to clarify the purpose of the GetActivitiesWithOutputs method. Removed the unnecessary check for AutoEvaluate from the EvaluateInputPropertyAsync method.
2023-12-06 22:32:08 +01:00
Sipke Schoorstra bec8cbc322
Target .NET 8 (#4660) 2023-11-27 11:26:16 +01:00
Sipke Schoorstra c10e1b4232 Update TriggerIndexer to return modifiable lists 2023-11-22 20:47:15 +01:00
Sipke Schoorstra 225a99f39d
Bulk Import + Export of Workflow Definitions and Instances (#4643)
* Implement bulk export of workflow definitions

* Update Import endpoint with support for file uploads

* Update Import endpoint to return imported workflow count

* Implement workflow instance (bulk) export

* Implement workflow state, bookmark, workflow + activity execution export

* Remove globally imported usings
2023-11-22 14:31:48 +01:00
Sipke Schoorstra e4d6ad22c4
Decouple Expression Types (#4622)
* Refactor Expressions model

* Enable expression type filtering

* Implement dynamic enablement of satelite dependencies
2023-11-14 20:44:42 +01:00
Sipke Schoorstra 02ec4c2391
Include NodeId for unique addressability within graph (#4575)
* Leverage ActivityNode for graph querying

* Make workflow definition activity Root port non-browsable

* Update Join activity

* Make NodeId first-class citizen
2023-10-25 22:46:06 +02:00
Sipke Schoorstra c9e2b11dfe Fix RunTask activity
This fixes the issue where its bookmark would include the activity instance ID, which causes the task completion reporter to miss the bookmark, since the activity instance ID is not included. Including the activity instance ID is not necessary for locating the correct activity instance because the generated Task ID is unique for each activity run.

Fixes #4511
2023-10-15 11:35:38 +02:00
Sipke Schoorstra 4f50850a1e
Capture activity state (#4530) 2023-10-13 10:34:21 +02:00
Sipke Schoorstra ca5692e3bc
Alteration Improvements and fixes (#4518)
* Remove unused NOOP stores

* Add activity execution manager and FindAsync method to store

The notification is used to broadcast changes to SignalR clients.

* Refactor ActiveActivityExectionContexts

Instead, the workflow state extractor is in control of what gets extracted.

* Add missing files for activity execution deleted broadcast

* Update Flowchart to not fault when child faults

* Update Flowchart to support scheduling existing activity instances

* Update namespaces from Services to Stores

* Improve activity cancellation alteration

* Update ScheduleActivity alteration
2023-10-10 12:34:32 +02:00
Sipke Schoorstra bddf522e55
Activity Preconditions (#4507)
* Add CanExecuteAsync precondition method

* Rename GetInput to GetWorkflowInput to disambiguate between input sent to workflow and input sent to indiviudal activities.
2023-10-07 14:22:51 +02:00
Sipke Schoorstra 523f301bf7
Feature: Alterations (#4501)
* Implement Alteration types and engine

* Implement Alteration APIs

* Refactor activity work item with support for activity input

* Serialize scheduled activities as part of workflow state

* Add signal to schedule child activity

* Implement stores for plans and jobs

* Add EF Core provider

* Add Retry endpoint

* Keep root context as always active. The root context is the workflow execution context and contains persistent variable state

* Regenerate EF Core migrations

* Handle orphaned activity execution contexts

* Refactor workflow execution factory

* Enable configuring Jint from Program

* Update ModifyVariable alteration with support for type deserialization

* Use ShortGuid for identity

* Cleanup WorkflowServer host
2023-10-07 12:47:36 +02:00
Sipke Schoorstra 939c403381 Default to memory stores
Issue #4467
2023-09-24 20:22:31 +02:00
Sipke Schoorstra a201672584 FlowSendHttpRequest improvements 2023-09-22 11:13:23 +02:00
Sipke Schoorstra 841628011a Add metadata to control non-serialized output 2023-09-20 22:55:58 +02:00
Sipke Schoorstra 89f94a2261
File Uploads (#4457)
* Initial support for receiving files via HTTP Endpoint activity

* Implement HTTP endpoint request timeout

* Implement request time out handling

* Implement request size validation

* Formatting

* Implement application and system level cancellation tokens

* Create CancellationTokens struct

* Cleanup HttpEndpoint

* Implement file size validation

* Implement file extension validation

* Implement mime type validation

* Update usages of cancellation tokens
2023-09-19 22:42:53 +02:00
Sipke Schoorstra fbdd555dd9 Implement workflow-level exception handling
Fixes #4451
2023-09-18 20:52:53 +02:00
Sipke Schoorstra bb2fa55d0e
WriteHttpFileResponse Activity (#4445)
* Incremental work on WriteFileHttpResponse activity

* BREAKING Schema change - Increase bookmark flexibility

* Implement zipped file response
2023-09-16 11:06:56 +02:00
Sipke Schoorstra 03b608410b
Incidents (#4436)
* Implement Incident Strategy interface and resolver

* Add ActivityIncident model

* Add IncidentCount and Incidents properties and update migrations

* Expose incident strategies API endpoint

* Update API client with Incident Strategy models

* Replace Fault with Incidents

* Register fault status when any children have faulted

* Map incident count

* Remove unnecessary AlreadyCompleted result

* Ensure next activities are scheduled only if activity completed normally

* Incident roundtrip and fix workflow instance realtime updates

* Remove unused variable

* Add integration tests
2023-09-14 10:50:02 +02:00
Sipke Schoorstra 287442ca7e Update middleware to also inspect memory block for output values 2023-09-11 21:58:27 +02:00
Sipke Schoorstra cbf609760e Ensure BookmarksDeleting and BookmarksDeleted events are published
Fixes #4389
2023-09-10 16:39:42 +02:00
happtim a3494b43d8
replace DistributedLock package with DistributedLock.FileSystem (#4399) 2023-09-09 12:44:46 +02:00
Sipke Schoorstra 0d05acb98d Ensure workflow state is imported into runtime before and after
This fixes an issue when running activities in the background
2023-09-07 22:28:45 +02:00
Sipke Schoorstra 4b94828780 Include WorkflowExecutionContext with UpdateBookmarksRequest 2023-09-07 22:28:08 +02:00
Sipke Schoorstra 30d48351e2 Update namespace of notification 2023-09-07 22:27:27 +02:00
Sipke Schoorstra 4fe7518976 Add initial support for persisting workflow input 2023-08-31 21:04:19 +02:00
Sipke Schoorstra 45d194125d
Remove WorkflowStateStore (#4374)
* Remove WorkflowStateStore and use WorkflowInstanceStore instead

* Delete RunningWorkflows actor

* Cleanup
2023-08-30 15:42:16 +02:00
Abhijit d9fd79a307
Upgraded nupkg DistributedLock to v2.3.3 (#4360)
DistributedLock v2.3.2 was indirectly referencing System.Drawing.Common v5.0.0 nupkg which has one critical vulnerability.
2023-08-29 12:46:32 +02:00
Sipke Schoorstra 4d37c2a1da
Fix Default Workflow Runtime Race condition (#4362)
* Refactor Telnyx activities to use instance ID for correlating events

* Ensure SetVariable sets value in correct scope

* Fix race condition between new instances that are being resumed

* Fix tests
2023-08-24 21:41:45 +02:00
Sipke Schoorstra a30ac939ee
Refactor safe serializer (#4361) 2023-08-24 13:15:28 +02:00
Sipke Schoorstra 48b2e29b00
Refactor activity state serializer (#4357) 2023-08-23 21:57:10 +02:00
Sipke Schoorstra 73482a8e4c
Enhanced Logging for Activity Outcomes and Outputs (#4352)
* Add result to logs

* Include outcomes and outputs with activity execution log

* Include outputs with completed activity events
2023-08-23 13:34:30 +02:00
Sipke Schoorstra 10533131e3 Add fault information to journal 2023-08-21 22:45:56 +02:00
Sipke Schoorstra 0413cbebf6 Fix duplicate workflow inbox message effect 2023-08-21 21:17:40 +02:00
Sipke Schoorstra e73d618b1a Remove IsHandled filter when delivering messages 2023-08-21 10:18:12 +02:00
Sipke Schoorstra e43e36b157 Fix hashing 2023-08-17 20:27:35 +02:00
Sipke Schoorstra 8c3f2658e8 Include hash in trigger comparer
This ensures triggers are updated when anything changes
2023-08-16 13:44:03 +02:00
Sipke Schoorstra f4ae684edd
Implement workflow inbox cleanup service (#4327) 2023-08-13 12:35:15 +02:00
Sipke Schoorstra af34276d60
Delete ActivityExecutionRecords when deleting workflow instances (#4324)
* Delete ActivityExecutionRecords when deleting workflow instances

* Enable Hangfire for testing, code formatting
2023-08-12 22:22:40 +02:00
Sipke Schoorstra b829a0acef Update Event Trigger endpoint to resume HTTP bookmarks 2023-08-12 21:58:15 +02:00
Sipke Schoorstra e2e861af05 Use FireAndForget strategy instead of Background worker
For some reason, the background worker causes duplicate handling of notifications
2023-08-12 21:09:07 +02:00
Sipke Schoorstra 6b68bd8f6d Prevent WorkflowInbox from triggering new workflows
when targeting a specific workflow instance
2023-08-12 21:08:14 +02:00
Sipke Schoorstra 7c99037de2
EF Core NoTracking (#4323)
* Use NoTracking for EF Core

* Use Activity ID as Node ID for modern tooling

* Fix DispatchWorkflow activity

* Update tests and remove redundant ToolVersion
2023-08-12 16:41:16 +02:00