Commit graph

1554 commits

Author SHA1 Message Date
Marius Vasile Vușcan 7df058dd2a
Improvements 2025-01-14 12:23:21 +02:00
Marius Vasile Vușcan f27563a93f
Added DeleteVariablesAsync method for the workflow context 2025-01-13 16:48:24 +02:00
Sipke Schoorstra fb70022e87 Add Variable parameter to StorageDriverContext
Updated `StorageDriverContext` to include a `Variable` parameter, ensuring more precise context handling for variable-related operations. Adjusted relevant method calls to pass the required `Variable` argument where necessary.
2024-12-18 15:37:38 +01:00
Sipke Schoorstra 8f8f8579a4 Add support for waiting for child workflows in ExecuteWorkflow
This update introduces the ability to optionally wait for child workflows to complete before finishing the ExecuteWorkflow activity. A bookmark mechanism is used to resume the activity when the child workflow completes. Additionally, a new handler was added to manage the resumption of these workflows upon completion events.
2024-12-18 15:05:46 +01:00
Sipke Schoorstra a2f76b8eba
Add output serialization and deserialization support (#6152)
Introduced `Output` message and related data serialization/deserialization methods in `ProtoOutputExtensions`. Updated `WorkflowExecutionResult` and impacted methods to include `Output` handling. Enhanced workflow runtime logic to manage and integrate output data.
2024-11-26 15:59:25 +01:00
Raymond den Haan 4cf797263f Update log level and comment for orphaned subscription removal 2024-11-26 09:49:27 +01:00
Raymond den Haan d49456b5b7 Update warning log message for orphaned subscriptions
Revised the log message to prevent people seeing it as an error and making it more clear this can be expected behavior.
2024-11-25 12:51:56 +01:00
Sipke Schoorstra a6bd386005
Add functionality to execute nested workflows (#6137)
Introduced a new `ExecuteWorkflow` activity that allows executing nested workflows. Added `ExecuteWorkflowResult` model to handle results and integrated component tests to verify nested workflow execution. Enhanced `IWorkflowBuilder` to support fluent methods for adding outputs.
2024-11-21 19:18:47 +01:00
Sipke Schoorstra 492b16bd24
Filter workflow summaries by latest version (#6132)
Ensure that only the latest version of each workflow definition is processed by grouping and selecting the highest version. This change helps prevent redundant updates and ensures data consistency.
2024-11-20 14:44:57 +01:00
Sipke Schoorstra c47157fd9f
Fix workflow definition finder (#6124)
Fixes that the workflow reference updater would collect child workflows from child workflows, instead of collecting only immediate children.
2024-11-18 19:05:36 +01:00
Sipke Schoorstra a0aa420a12
Keep Published Workflows Immutable by Updating Drafts Instead (#6106)
* Add workflow graph network builder and child workflow finder

Introduce IWorkflowGraphNetworkBuilder and IChildWorkflowFinder interfaces with their implementations to build and manage workflow graph networks. Additionally, register new services and handlers to support these functionalities in the Workflow Management feature.

* Remove unused workflow graph network builder

Deleted the unused `IWorkflowGraphNetworkBuilder` interface, its implementation, and related files. Updated affected classes to replace dependency on the removed builder with direct methods. Enhanced affected workflows tracking during publication.

* Optimize workflow update logic to skip redundant checks

Added condition to ensure only relevant activities are processed by verifying their version IDs. This reduces unnecessary computations and ensures the system skips already up-to-date workflow activities.

* Add workflow reference updater implementation

Introduced IWorkflowReferenceUpdater interface to manage workflow references updates. Implemented the corresponding service and integrated it into existing event handling logic. This improves maintainability and reduces code duplication by isolating the reference update logic.

* Set workflow activity version during update

Added a missing assignment of the workflow activity version in the `UpdateWorkflowDefinition` method. This ensures that the `Version` field is properly set when workflow definitions are updated.

* Remove UpdateReferencesInConsumingWorkflows method

Eliminated the UpdateReferencesInConsumingWorkflows method from IWorkflowDefinitionPublisher and its implementation from WorkflowDefinitionPublisher. The responsibility of updating workflow references is now transitioned to IWorkflowReferenceUpdater used in the `UpdateReferences` endpoint.

* Remove WorkflowGraphNetwork and WorkflowGraphNode models

These models were deleted as they are no longer needed in the system. The removal helps in reducing code clutter and improves maintainability by eliminating unused components.

* Refactor parameter naming in IWorkflowReferenceUpdater

Renamed 'definition' to 'referencedDefinition' for clarity in the IWorkflowReferenceUpdater contract and its implementation. This change makes it explicit that the parameter refers to the workflow definition being referenced, ensuring the code is more understandable and maintainable.

* Update WorkflowReferenceUpdater for clarity and accuracy

Renamed variables and updated comments for better clarity. The term "matchingWorkflowGraphs" was changed to "consumingWorkflowGraphs" to more accurately describe its purpose. Adjusted comment to clarify the materialization process of the draft version.

* Add IsReadonly filter to WorkflowDefinitionFilter

This ensures that only non-readonly workflow definitions are fetched. It enhances the accuracy of workflow updates by filtering out readonly versions.
2024-11-08 18:19:42 +01:00
Sipke Schoorstra 6ed6c8a30e Merge remote-tracking branch 'origin/patch/3.2.x' into blueberry 2024-11-05 15:46:02 +01:00
Sipke Schoorstra 91b7e0c800 Add authorization to WorkflowInstanceHub
This commit decorates the WorkflowInstanceHub class with the [Authorize] attribute to ensure that only authorized users can connect to the SignalR hub. This change improves the security of the workflow event notification system.
2024-10-31 19:53:26 +01:00
Sipke Schoorstra fefe36f49a
Include $schema when exporting workflow definition (#6056)
* Refactor AdditionalConvertersConfigurator constructor

Simplify AdditionalConvertersConfigurator by using constructor injection directly in the class declaration. Also, annotate the class with [UsedImplicitly] to improve code clarity and maintainability.

* Update workflow serialization to be asynchronous

Refactor the workflow definition serialization to use asynchronous operations, improving efficiency and handling cancellation tokens. This change ensures the schema is correctly included in the serialized output.
2024-10-25 14:51:27 +02:00
Sipke Schoorstra 52516aa0ff
Fix Bulk Dispatch Missing Input (#6052)
* Remove outdated sample workflows and activities

Deleted CompositeExample, SampleWorkflow, and SlowActivity classes as they are no longer needed. These deletions help clean up the codebase and maintain relevance in the code structure.

* Handle null WorkflowBuilderType in ClrWorkflowMaterializer

Introduce a fallback NotFoundWorkflowbuilder class when WorkflowBuilderType is null. This ensures the MaterializeAsync method functions even if the provided context lacks a specific workflow builder type.

* Add merging of itemDictionary to input

This change ensures that the itemDictionary's contents are also merged into the input dictionary, preventing possible data loss. It builds on existing logic by adding an additional merge operation to incorporate all necessary data.

* Add new workflows to BulkDispatchWorkflows and modify tests

Introduce `FruitWorkflow` and `MixFruitsWorkflow` for bulk dispatch scenarios. Refactor `BulkDispatchWorkflowsTests` to include new tests and rename signals for clarity. Optimize `BulkDispatchWorkflows` activity by removing redundant dictionary merge.
2024-10-23 16:06:28 +02:00
Sipke Schoorstra a0001d3891
Enhance error handling in PolymorphicObjectConverter (#6051)
Add null checks for `referenceResolver` in cases where `RefPropertyName` and `IdPropertyName` are being processed. This prevents potential null reference exceptions and ensures safer deserialization of objects.
2024-10-23 12:12:18 +02:00
Sipke Schoorstra b46d432579
Add JsonNode BSON serializer (#6043)
Introduce JsonNodeBsonConverter for BSON serialization of JsonNode objects. Update MongoDbFeature to register the new serializer, enhancing JSON handling within MongoDB serialization context.
2024-10-19 11:56:50 +02:00
Sipke Schoorstra 1905918247 Update System.Text.Json package reference to use property variable
Switched from a hardcoded version to a property variable `$(SystemTextJsonVersion)` for the `System.Text.Json` package reference in multiple project files. This change centralizes version management and ensures consistency across all projects.
2024-10-12 12:27:54 +02:00
rosca-sabina 090a3fd1b0
Register MongoDb client as singleton (#6013)
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-10-12 12:21:39 +02:00
rosca-sabina 1b4ce452ee
Add MongoDB collection naming strategy (#6005) 2024-10-12 12:16:09 +02:00
Sipke Schoorstra 83238e9931
Update System.Text.Json version reference (#6019)
Replaced hardcoded System.Text.Json version 8.0.4 with a variable $(SystemTextJsonVersion) across multiple project files. This improves maintainability and consistency when updating the package version.
2024-10-11 14:35:24 +02:00
Sipke Schoorstra d2d4b6ee4a Merge remote-tracking branch 'origin/patch/3.2.x' into blueberry 2024-10-11 10:30:52 +02:00
Sipke Schoorstra 11f388338a
Add endpoints for activity execution records and summaries (#6018)
* Add endpoints for activity execution records and summaries

Created new classes and endpoints to handle activity execution records and summaries retrieval. Added related models and updated relevant interfaces.

* Rename endpoint classes for clarity

Updated endpoint classes in `ActivityExecutions` and `ActivityExecutionSummaries` to use the name `Endpoint` instead of `List`. This improves readability and consistency in the codebase.
2024-10-11 10:29:40 +02:00
Sipke Schoorstra 96d82c4210 Refactor deserialization method for entity properties.
Replaced `Deserialize` with `DeserializePayload` to align with updated deserialization strategies, ensuring consistency across the codebase and improving future maintainability. This change affects how 'SerializedProperties' are handled within `ActivityExecutionLogStore`.
2024-10-10 11:40:59 +02:00
Sipke Schoorstra 494d6e7f43 Refactor polymorphic JSON deserialization.
Simplify the handling of `JsonNode` parsing and improve type-checking. Added a guard clause in `ReadType` method to handle non-object JSON token types, ensuring safer type parsing.
2024-10-10 10:18:27 +02:00
Sipke Schoorstra 3f83a2fb51 Remove unused 'Elsa.Expressions.Services' import
Eliminates an unnecessary using directive from PolymorphicObjectConverterFactory.cs to keep the code clean and reduce potential confusion. This change has no impact on functionality.
2024-10-09 23:19:47 +02:00
Sipke Schoorstra 02f1b56eb1 Add tracing for trigger activities in workflow execution
Enhanced the OpenTelemetry tracing middleware to include detailed information about the triggered activity within the workflow execution. This change adds tags for the activity ID, name, and type to provide better observability and debugging insights.
2024-10-07 10:15:35 +02:00
Sipke Schoorstra 078f34fd79 Remove obsolete tag settings and correct tag schemas
Eliminated redundant `correlationId` tag setting. Adjusted `workflowInstance` and `activityInstance` tag schemas for better clarity. Simplified the processors list in `otel-collector-config.yaml`.
2024-10-05 14:12:00 +02:00
Sipke Schoorstra 54624aea77 Refactor tracing to enhance span details
Renamed variables from 'activity' to 'span' for clarity and consistency. Added additional span tags for start and end times, status updates, and detailed error information for improved traceability and debugging.
2024-10-05 13:48:59 +02:00
Sipke Schoorstra 458181c5ce Refactor activity events with status tags helper method
Replaced inline tag creation with a helper method to streamline code and ensure consistency in status tags for activity events. This change also includes a minor comment for clarity on null activity handling.
2024-10-05 13:39:58 +02:00
Sipke Schoorstra 2aa312850d Refactor incident tagging logic in tracing middleware
Reorganized the logic for setting tags when incidents are present in `OpenTelemetryTracingWorkflowExecutionMiddleware`. The new structure ensures consistent tag setting for error status and incidents, improving code readability and maintainability.
2024-10-05 11:19:02 +02:00
Sipke Schoorstra 679a0515c8 Add TypeJsonConverter and enhance JSON serialization
Introduced a TypeJsonConverter to handle Type serialization and deserialization. Updated dependency injection and helper methods to support customized JSON serializer options, improving flexibility and configurability.
2024-10-03 18:50:05 +02:00
Sipke Schoorstra 3baacc83b3 Change WorkflowVersion property in WorkflowExecutionLogRecord
Modified the WorkflowVersion property from init-only to settable. This adjustment allows for updating the workflow version post-initialization, providing greater flexibility in handling workflow instances.
2024-09-27 17:55:47 +02:00
Sipke Schoorstra 750b4804fe Add GetOptions method to ISafeSerializer interface
This new method retrieves the JSON serializer options, enhancing flexibility in how JSON data is handled. It provides an additional way to customize serialization settings contextually.
2024-09-27 13:10:22 +02:00
Sipke Schoorstra 7d45db355d Refactor serializer options retrieval method name.
Renamed GetPayloadSerializerOptions to GetOptions for simplicity and updated method accessibility. This change ensures that the method name is more intuitive and aligns with common naming conventions. Additionally, it introduces a new public GetOptions method in the IPayloadSerializer interface.
2024-09-27 13:07:08 +02:00
Sipke Schoorstra 2e1ed995da Add synchronous serialization methods and mark async as obsolete
Implemented synchronous methods for serialization and deserialization while marking the asynchronous methods as obsolete across various storage and serialization services. This includes updates to handle serialization within the Save and Load methods, enhancing performance by avoiding unnecessary Task usage.
2024-09-27 12:59:59 +02:00
Sipke Schoorstra 0492a7be43 Refactor to use primary constructor in Get class
Refactored the `Get` class to utilize a primary constructor for dependency injection of `IWorkflowExecutionLogStore`. This change simplifies the class definition and removes the need for a private field.
2024-09-26 14:09:28 +02:00
Sipke Schoorstra df827b0fda Update Datadog config, add incident error serialization
Enhanced the Datadog tracing configuration to include priority sampling, rate limits, and sampling rules in the docker-compose-datadog.yml. Updated the agent image version and improved error message serialization in OpenTelemetry middleware to use custom JSON serializer options. Adjusted Dockerfile path to correct bundle location.
2024-09-24 00:27:51 +02:00
Sipke Schoorstra adf36927dc Update Docker and k8s configs, enhance OTEL tracing
Updated `docker-compose-datadog.yml` with new environment variables and updated image versions, enhancing trace sampling and service autodiscovery. Added Kubernetes deployment, service, and role files for `elsa-server`, `elsa-studio`, `plant-uml`, `postgres`, and `trace-lens`, improving the project's infrastructure. Enhanced OpenTelemetry middleware for better error handling and detailed tracing of activity and workflow executions.
2024-09-23 23:35:30 +02:00
Sipke Schoorstra 79b5bfc378 Add OpenTelemetry support for tracing workflow execution
This commit introduces the Elsa.OpenTelemetry module to provide OpenTelemetry sources for tracing workflow and activity execution. Additionally, it updates the program to include default workflow and activity execution pipelines with tracing middleware, and improves pipeline builder extensions to support middleware insertion.
2024-09-23 11:35:43 +02:00
Sipke Schoorstra 53172cded9 Refactor IdentityGraphService constructor and add logging.
Replaced private fields with constructor parameters in IdentityGraphService and introduced a logger. Added logging to handle cases where activity descriptors are not found, improving debugging and maintainability.
2024-09-20 15:59:35 +02:00
Sipke Schoorstra f9f8f05670 Improve exception messages for activity descriptor lookups
Enhanced exception messages to include activity type and version for better debugging. This provides clearer context when activity descriptors or inputs are not found.
2024-09-20 15:51:31 +02:00
Sipke Schoorstra a827ab241b
Fix Azure Service Bus module DI issues (#5975)
* Switch to Azure Service Bus for message brokering

Updated the message broker from in-memory to Azure Service Bus to enhance scalability. Added new configuration settings and dependencies to support Azure Service Bus integration, including scoped service initialization and queue/topic management.

* Refactor StartWorkers for dependency injection

Updated StartWorkers class to use constructor parameter injection and service scope factory for retrieving services. This refactor eliminates the need for field-level dependency storage, enhancing testability and adherence to the dependency injection principles. Additionally, annotated the class with [UsedImplicitly] to eliminate unused code warnings.

* Refactor Worker to use IServiceScopeFactory for dependency resolution

Refactored the Worker class to use IServiceScopeFactory for DI instead of directly injecting IWorkflowInbox. This enhances the flexibility and lifecycle management of the dependencies.

* Add Azure Service Bus integration toggle

Introduced a new configuration constant `useAzureServiceBus` to control the optional integration with Azure Service Bus. Implemented conditional logic to bind Azure Service Bus options from configuration if the flag is enabled.

* Add IComposite interface and implement Setup method

Introduced a new IComposite interface and added a default Setup method to the Composite class. Updated the ActivityFactory to call the Setup method when an activity is an instance of IComposite. The CompositeExample class demonstrates the use of the new interface and method.

* Remove AzureServiceBus configuration setup

The AzureServiceBus configuration setup was removed from the Program.cs file. This change aims to streamline configuration and remove unused or unnecessary setup, ensuring the code remains clean and maintainable.

* Switch MassTransit broker to in-memory

Changed the MassTransit broker from AzureServiceBus to Memory in the application configuration. This adjustment aims to simplify deployment and reduce dependencies in the current environment.
2024-09-19 20:48:09 +02:00
Sipke Schoorstra 7c31332529
Fix variable serialization (#5974)
* Improve dispatched workflow input handling

Addressed input handling in dispatch messages by adding `SerializedInput` property. Also removed initialization logic and moved input deserialization to a helper method, ensuring compatibility with both new and deprecated input property formats.

* Update workflow Docker images and version tags

Changed Docker image tags from v3-2-0-rc3 to v3-2-1-preview across multiple GitHub workflows. Updated the VERSION environment variable in packages.yml to reflect the new versioning scheme. These changes ensure consistency with the new preview release.

* Update versioning to include 'preview' in package workflow

Modified the workflow to append 'preview' to the version number for non-tagged builds. This ensures clearer differentiation between stable and non-stable versions in the CI pipeline.

* Add WorkflowInstanceStorageDriver for workflow variable storage

Introduced a new storage driver, WorkflowInstanceStorageDriver, to store workflow variables directly in the workflow state. Updated relevant classes and methods to incorporate this new storage driver, ensuring seamless read/write/delete operations and extending support for it throughout the codebase.

* Refactor object conversion and update variable retrieval.

Switched from JsonObject to JsonNode for object conversion and corrected a typo in the summary comment. Changed the return type of GetVariablesDictionary method and updated its implementation to use VariablesDictionary.

* Rename 'input' to 'serializedInput' in DispatchWorkflowDefinition.

This change clarifies that the input provided to the workflow should be serialized. It enhances the readability and accuracy of the code documentation, ensuring that developers understand the expected format of the input parameter.

* Add priority and deprecation attributes to storage drivers

Introduced a priority attribute to the `IStorageDriver` interface and implemented it in various storage drivers. Additionally, marked `WorkflowStorageDriver` as deprecated and reordered storage driver listing based on priority.

* Switch MassTransit broker to in-memory and refactor converter

Changed MassTransit broker from AzureServiceBus to in-memory for improved performance in development environment. Simplified PolymorphicObjectConverterFactory by removing redundant constructor and dependencies. Removed unused folder from the project file.
2024-09-19 16:18:56 +02:00
rosca-sabina 18b0d2e70a
Add alteration logs to workflow execution logs (#5897) (#5911) 2024-08-30 12:02:41 +02:00
Raymond den Haan 302292255c Introduce UniqueKeyConstraintViolationException
Refactor DataProcessingException to remove unique key logic and create a dedicated UniqueKeyConstraintViolationException. Update DbExceptionTransformer to use the new exception for unique key violations, improving code clarity and separation of concerns.
2024-08-20 12:37:48 +02:00
Sipke Schoorstra 8a417ca765
Remove ExpandoObject support from JintJavaScriptEvaluator (#5913)
Removed the `ConverterHelper` and `ExpandoObjectConverter` classes, along with their integration in `JintJavaScriptEvaluator`. Updated unit tests to skip unsupported functionality and added references to related issues for context.
2024-08-19 09:13:45 +02:00
Sipke Schoorstra edf365b519
Add new HTTP workflow tests and enhance middleware (#5908)
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
2024-08-16 20:15:02 +02:00
Sipke Schoorstra 0a76741755 Update TaskReporter and RunTask activity
Replaced IWorkflowDispatcher in TaskReporter with IWorkflowInbox to streamline task reporting. This fixes race conditions when the workflow is attempted to be resumed before it committed its bookmarks.
2024-08-14 18:40:04 +02:00
Sipke Schoorstra 5f11026ee7
Add JS functions for encoding/decoding byte arrays from and to strings (#5902)
* Add handlers and improve JavaScript engine configuration

Added several new handlers for configuring the JavaScript engine with common types, functions, variable accessors, and input/output accessors. Refactored the JintJavaScriptEvaluator for better clarity and modularity by breaking down configuration steps into separate methods. Also replaced `IsInsideCompositeActivity` with `IsContainedWithinCompositeActivity` for better semantic consistency.

* Refactor to use primary constructor syntax for TypeDefinitionProviders

Updated CommonTypeDefinitionProvider, VariableTypeDefinitionProvider, and ActivityOutputFunctionsDefinitionProvider to use primary constructors for dependency injection. This change reduces redundancy and simplifies the code structure for better readability and maintainability.

* Add byte conversion functions to JavaScript engine

Introduced functions for converting bytes to/from strings and Base64. Updated CommonFunctionsDefinitionProvider and ConfigureEngineWithCommonFunctions to incorporate these new functions. This enhances the JavaScript engine's ability to handle byte array manipulations.

* Add tests for JavaScript byte and string conversions.

Introduce integration tests to verify byte array to string, string to byte array, byte array to Base64, and Base64 to byte array conversions using JavaScript functions. Ensure accurate transformation of data within different encoding scenarios.

* Add meaningful summaries to Jint engine configuration handlers

Updated comment summaries in four handler classes to provide clear and concise descriptions of their purpose. This helps improve code readability and understanding for future developers.

* Rename and merge variable and input/output handlers

Merged the variable accessor logic into the input/output handler and renamed the class to reflect its broader functionality. This consolidation ensures the accessors are registered in the right order.

* Add string base64 conversion functions

Introduced `stringToBase64` and `stringFromBase64` functions to handle base64 encoding and decoding of strings. Updated corresponding provider, handler, and added tests to ensure functionality.
2024-08-14 17:43:35 +02:00