Commit graph

5221 commits

Author SHA1 Message Date
Sipke Schoorstra a0001d3891
Enhance error handling in PolymorphicObjectConverter (#6051)
Add null checks for `referenceResolver` in cases where `RefPropertyName` and `IdPropertyName` are being processed. This prevents potential null reference exceptions and ensures safer deserialization of objects.
2024-10-23 12:12:18 +02:00
Sipke Schoorstra 61f9e2a59a Add version override for System.Text.Json
This change overrides the version of System.Text.Json due to vulnerabilities found in dependencies referencing older versions. This ensures better security and reliability in performance tests.
2024-10-18 12:39:16 +02:00
Sipke Schoorstra 299f165978 Refactor project paths in solution files
Updated the paths for multiple projects in the solution files to better organize modules under a common directory structure. Modified project references in `ElsaX.Web.Server.csproj` to match the new directory structure. Added a label to an item group for clarity in `Elsa.Server.Web.csproj`.
2024-10-14 09:48:32 +02:00
Sipke Schoorstra 83238e9931
Update System.Text.Json version reference (#6019)
Replaced hardcoded System.Text.Json version 8.0.4 with a variable $(SystemTextJsonVersion) across multiple project files. This improves maintainability and consistency when updating the package version.
2024-10-11 14:35:24 +02:00
Sipke Schoorstra d2d4b6ee4a Merge remote-tracking branch 'origin/patch/3.2.x' into blueberry 2024-10-11 10:30:52 +02:00
Sipke Schoorstra 11f388338a
Add endpoints for activity execution records and summaries (#6018)
* Add endpoints for activity execution records and summaries

Created new classes and endpoints to handle activity execution records and summaries retrieval. Added related models and updated relevant interfaces.

* Rename endpoint classes for clarity

Updated endpoint classes in `ActivityExecutions` and `ActivityExecutionSummaries` to use the name `Endpoint` instead of `List`. This improves readability and consistency in the codebase.
2024-10-11 10:29:40 +02:00
Sipke Schoorstra 96d82c4210 Refactor deserialization method for entity properties.
Replaced `Deserialize` with `DeserializePayload` to align with updated deserialization strategies, ensuring consistency across the codebase and improving future maintainability. This change affects how 'SerializedProperties' are handled within `ActivityExecutionLogStore`.
2024-10-10 11:40:59 +02:00
Sipke Schoorstra 494d6e7f43 Refactor polymorphic JSON deserialization.
Simplify the handling of `JsonNode` parsing and improve type-checking. Added a guard clause in `ReadType` method to handle non-object JSON token types, ensuring safer type parsing.
2024-10-10 10:18:27 +02:00
Sipke Schoorstra 3f83a2fb51 Remove unused 'Elsa.Expressions.Services' import
Eliminates an unnecessary using directive from PolymorphicObjectConverterFactory.cs to keep the code clean and reduce potential confusion. This change has no impact on functionality.
2024-10-09 23:19:47 +02:00
Sipke Schoorstra 6cefe6ea23 Update package versions in Directory.Packages.props
Bumped versions for various Microsoft and third-party packages to improve compatibility and take advantage of recent bug fixes and performance improvements. Specific changes include updates to AspNetCore, EntityFrameworkCore, and Polly packages among others.
2024-10-09 09:44:07 +02:00
Sipke Schoorstra 02f1b56eb1 Add tracing for trigger activities in workflow execution
Enhanced the OpenTelemetry tracing middleware to include detailed information about the triggered activity within the workflow execution. This change adds tags for the activity ID, name, and type to provide better observability and debugging insights.
2024-10-07 10:15:35 +02:00
Sipke Schoorstra 078f34fd79 Remove obsolete tag settings and correct tag schemas
Eliminated redundant `correlationId` tag setting. Adjusted `workflowInstance` and `activityInstance` tag schemas for better clarity. Simplified the processors list in `otel-collector-config.yaml`.
2024-10-05 14:12:00 +02:00
Sipke Schoorstra 54624aea77 Refactor tracing to enhance span details
Renamed variables from 'activity' to 'span' for clarity and consistency. Added additional span tags for start and end times, status updates, and detailed error information for improved traceability and debugging.
2024-10-05 13:48:59 +02:00
Sipke Schoorstra 458181c5ce Refactor activity events with status tags helper method
Replaced inline tag creation with a helper method to streamline code and ensure consistency in status tags for activity events. This change also includes a minor comment for clarity on null activity handling.
2024-10-05 13:39:58 +02:00
Sipke Schoorstra 2aa312850d Refactor incident tagging logic in tracing middleware
Reorganized the logic for setting tags when incidents are present in `OpenTelemetryTracingWorkflowExecutionMiddleware`. The new structure ensures consistent tag setting for error status and incidents, improving code readability and maintainability.
2024-10-05 11:19:02 +02:00
Sipke Schoorstra 41e1062148 Enhance JSON deserialization to include service provider
Updated ReadAsJsonAsync method to accept an IServiceProvider parameter for improved JSON deserialization. Modified corresponding tests and helper classes to utilize the service provider for more flexible JSON serialization options.
2024-10-03 19:12:00 +02:00
Sipke Schoorstra 679a0515c8 Add TypeJsonConverter and enhance JSON serialization
Introduced a TypeJsonConverter to handle Type serialization and deserialization. Updated dependency injection and helper methods to support customized JSON serializer options, improving flexibility and configurability.
2024-10-03 18:50:05 +02:00
Sipke Schoorstra 86826a89dc Add new Elsa logo image
Introduce a new Elsa logo image to the design assets. This image will be used in the custom 2x design section.
2024-10-03 11:16:14 +02:00
Sipke Schoorstra fdb8061324 Update package versions to latest stable releases
Upgraded various packages to their latest stable versions to ensure compatibility and take advantage of new features and bug fixes. Notable upgrades include Microsoft.EntityFrameworkCore, Microsoft.Data.Sqlite, and Polly.
2024-09-30 14:31:51 +02:00
Sipke Schoorstra 3baacc83b3 Change WorkflowVersion property in WorkflowExecutionLogRecord
Modified the WorkflowVersion property from init-only to settable. This adjustment allows for updating the workflow version post-initialization, providing greater flexibility in handling workflow instances.
2024-09-27 17:55:47 +02:00
Sipke Schoorstra 750b4804fe Add GetOptions method to ISafeSerializer interface
This new method retrieves the JSON serializer options, enhancing flexibility in how JSON data is handled. It provides an additional way to customize serialization settings contextually.
2024-09-27 13:10:22 +02:00
Sipke Schoorstra 7d45db355d Refactor serializer options retrieval method name.
Renamed GetPayloadSerializerOptions to GetOptions for simplicity and updated method accessibility. This change ensures that the method name is more intuitive and aligns with common naming conventions. Additionally, it introduces a new public GetOptions method in the IPayloadSerializer interface.
2024-09-27 13:07:08 +02:00
Sipke Schoorstra 2e1ed995da Add synchronous serialization methods and mark async as obsolete
Implemented synchronous methods for serialization and deserialization while marking the asynchronous methods as obsolete across various storage and serialization services. This includes updates to handle serialization within the Save and Load methods, enhancing performance by avoiding unnecessary Task usage.
2024-09-27 12:59:59 +02:00
Sipke Schoorstra 7609b0e7ca Add factory method to PagedListResponse
This update introduces a static factory method `From` in the `PagedListResponse` class. It allows for the creation of `PagedListResponse` instances directly from `Page` objects, streamlining object instantiation.
2024-09-26 21:00:10 +02:00
Sipke Schoorstra 973489c961 Remove unused usings in HungryWorkflow.cs
This commit cleans up the HungryWorkflow.cs file by removing unused using directives. This improves code readability and reduces unnecessary dependencies.
2024-09-26 20:59:59 +02:00
Sipke Schoorstra 0492a7be43 Refactor to use primary constructor in Get class
Refactored the `Get` class to utilize a primary constructor for dependency injection of `IWorkflowExecutionLogStore`. This change simplifies the class definition and removes the need for a private field.
2024-09-26 14:09:28 +02:00
Sipke Schoorstra a62de77900 Update OpenTelemetry configurations and remove unused modules
Revised otel-collector and docker-compose configurations to streamline tracing and metrics collection, replacing Datadog connectors with OpenTelemetry. Removed unused scraping modules and unnecessary dependencies to improve setup efficiency and resource utilization.
2024-09-25 14:12:00 +02:00
Sipke Schoorstra 758b60767b Add OpenTelemetry auto-instrumentation support
Introduce OpenTelemetry configuration and dependencies for observability. Replace DataDog tracer with OpenTelemetry auto-instrumentation in the Dockerfile. Adjust project dependencies and log levels to incorporate OpenTelemetry without disrupting existing functionality.
2024-09-25 00:49:21 +02:00
Sipke Schoorstra df827b0fda Update Datadog config, add incident error serialization
Enhanced the Datadog tracing configuration to include priority sampling, rate limits, and sampling rules in the docker-compose-datadog.yml. Updated the agent image version and improved error message serialization in OpenTelemetry middleware to use custom JSON serializer options. Adjusted Dockerfile path to correct bundle location.
2024-09-24 00:27:51 +02:00
Sipke Schoorstra adf36927dc Update Docker and k8s configs, enhance OTEL tracing
Updated `docker-compose-datadog.yml` with new environment variables and updated image versions, enhancing trace sampling and service autodiscovery. Added Kubernetes deployment, service, and role files for `elsa-server`, `elsa-studio`, `plant-uml`, `postgres`, and `trace-lens`, improving the project's infrastructure. Enhanced OpenTelemetry middleware for better error handling and detailed tracing of activity and workflow executions.
2024-09-23 23:35:30 +02:00
Sipke Schoorstra 79b5bfc378 Add OpenTelemetry support for tracing workflow execution
This commit introduces the Elsa.OpenTelemetry module to provide OpenTelemetry sources for tracing workflow and activity execution. Additionally, it updates the program to include default workflow and activity execution pipelines with tracing middleware, and improves pipeline builder extensions to support middleware insertion.
2024-09-23 11:35:43 +02:00
Sipke Schoorstra 53172cded9 Refactor IdentityGraphService constructor and add logging.
Replaced private fields with constructor parameters in IdentityGraphService and introduced a logger. Added logging to handle cases where activity descriptors are not found, improving debugging and maintainability.
2024-09-20 15:59:35 +02:00
Sipke Schoorstra f9f8f05670 Improve exception messages for activity descriptor lookups
Enhanced exception messages to include activity type and version for better debugging. This provides clearer context when activity descriptors or inputs are not found.
2024-09-20 15:51:31 +02:00
Sipke Schoorstra a827ab241b
Fix Azure Service Bus module DI issues (#5975)
* Switch to Azure Service Bus for message brokering

Updated the message broker from in-memory to Azure Service Bus to enhance scalability. Added new configuration settings and dependencies to support Azure Service Bus integration, including scoped service initialization and queue/topic management.

* Refactor StartWorkers for dependency injection

Updated StartWorkers class to use constructor parameter injection and service scope factory for retrieving services. This refactor eliminates the need for field-level dependency storage, enhancing testability and adherence to the dependency injection principles. Additionally, annotated the class with [UsedImplicitly] to eliminate unused code warnings.

* Refactor Worker to use IServiceScopeFactory for dependency resolution

Refactored the Worker class to use IServiceScopeFactory for DI instead of directly injecting IWorkflowInbox. This enhances the flexibility and lifecycle management of the dependencies.

* Add Azure Service Bus integration toggle

Introduced a new configuration constant `useAzureServiceBus` to control the optional integration with Azure Service Bus. Implemented conditional logic to bind Azure Service Bus options from configuration if the flag is enabled.

* Add IComposite interface and implement Setup method

Introduced a new IComposite interface and added a default Setup method to the Composite class. Updated the ActivityFactory to call the Setup method when an activity is an instance of IComposite. The CompositeExample class demonstrates the use of the new interface and method.

* Remove AzureServiceBus configuration setup

The AzureServiceBus configuration setup was removed from the Program.cs file. This change aims to streamline configuration and remove unused or unnecessary setup, ensuring the code remains clean and maintainable.

* Switch MassTransit broker to in-memory

Changed the MassTransit broker from AzureServiceBus to Memory in the application configuration. This adjustment aims to simplify deployment and reduce dependencies in the current environment.
2024-09-19 20:48:09 +02:00
Sipke Schoorstra 7c31332529
Fix variable serialization (#5974)
* Improve dispatched workflow input handling

Addressed input handling in dispatch messages by adding `SerializedInput` property. Also removed initialization logic and moved input deserialization to a helper method, ensuring compatibility with both new and deprecated input property formats.

* Update workflow Docker images and version tags

Changed Docker image tags from v3-2-0-rc3 to v3-2-1-preview across multiple GitHub workflows. Updated the VERSION environment variable in packages.yml to reflect the new versioning scheme. These changes ensure consistency with the new preview release.

* Update versioning to include 'preview' in package workflow

Modified the workflow to append 'preview' to the version number for non-tagged builds. This ensures clearer differentiation between stable and non-stable versions in the CI pipeline.

* Add WorkflowInstanceStorageDriver for workflow variable storage

Introduced a new storage driver, WorkflowInstanceStorageDriver, to store workflow variables directly in the workflow state. Updated relevant classes and methods to incorporate this new storage driver, ensuring seamless read/write/delete operations and extending support for it throughout the codebase.

* Refactor object conversion and update variable retrieval.

Switched from JsonObject to JsonNode for object conversion and corrected a typo in the summary comment. Changed the return type of GetVariablesDictionary method and updated its implementation to use VariablesDictionary.

* Rename 'input' to 'serializedInput' in DispatchWorkflowDefinition.

This change clarifies that the input provided to the workflow should be serialized. It enhances the readability and accuracy of the code documentation, ensuring that developers understand the expected format of the input parameter.

* Add priority and deprecation attributes to storage drivers

Introduced a priority attribute to the `IStorageDriver` interface and implemented it in various storage drivers. Additionally, marked `WorkflowStorageDriver` as deprecated and reordered storage driver listing based on priority.

* Switch MassTransit broker to in-memory and refactor converter

Changed MassTransit broker from AzureServiceBus to in-memory for improved performance in development environment. Simplified PolymorphicObjectConverterFactory by removing redundant constructor and dependencies. Removed unused folder from the project file.
2024-09-19 16:18:56 +02:00
rosca-sabina 18b0d2e70a
Add alteration logs to workflow execution logs (#5897) (#5911) 2024-08-30 12:02:41 +02:00
raymonddenhaan 9365bb9a20
Merge pull request #5917 from elsa-workflows/enh/uk_ex
Introduce UniqueKeyConstraintViolationException
2024-08-20 13:44:58 +02:00
Raymond den Haan 302292255c Introduce UniqueKeyConstraintViolationException
Refactor DataProcessingException to remove unique key logic and create a dedicated UniqueKeyConstraintViolationException. Update DbExceptionTransformer to use the new exception for unique key violations, improving code clarity and separation of concerns.
2024-08-20 12:37:48 +02:00
Sipke Schoorstra 8a417ca765
Remove ExpandoObject support from JintJavaScriptEvaluator (#5913)
Removed the `ConverterHelper` and `ExpandoObjectConverter` classes, along with their integration in `JintJavaScriptEvaluator`. Updated unit tests to skip unsupported functionality and added references to related issues for context.
2024-08-19 09:13:45 +02:00
Sipke Schoorstra 8917bd9d0c Merge branch 'bug/5879' into patch/3.2.x 2024-08-16 21:50:36 +02:00
Sipke Schoorstra 2ad3287dba Update version to 3.2.0-rc6 in GitHub Actions workflow
Changed the fallback version in the GitHub Actions packages workflow from 3.2.0-rc5 to 3.2.0-rc6. This ensures that the workflows use the updated release candidate version for deployments.
2024-08-16 21:49:52 +02:00
Sipke Schoorstra edf365b519
Add new HTTP workflow tests and enhance middleware (#5908)
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
2024-08-16 20:15:02 +02:00
Sipke Schoorstra 83d0e14248 Add new HTTP workflow tests and enhance middleware
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
2024-08-16 13:16:47 +02:00
Sipke Schoorstra 0a76741755 Update TaskReporter and RunTask activity
Replaced IWorkflowDispatcher in TaskReporter with IWorkflowInbox to streamline task reporting. This fixes race conditions when the workflow is attempted to be resumed before it committed its bookmarks.
2024-08-14 18:40:04 +02:00
Sipke Schoorstra 5f11026ee7
Add JS functions for encoding/decoding byte arrays from and to strings (#5902)
* Add handlers and improve JavaScript engine configuration

Added several new handlers for configuring the JavaScript engine with common types, functions, variable accessors, and input/output accessors. Refactored the JintJavaScriptEvaluator for better clarity and modularity by breaking down configuration steps into separate methods. Also replaced `IsInsideCompositeActivity` with `IsContainedWithinCompositeActivity` for better semantic consistency.

* Refactor to use primary constructor syntax for TypeDefinitionProviders

Updated CommonTypeDefinitionProvider, VariableTypeDefinitionProvider, and ActivityOutputFunctionsDefinitionProvider to use primary constructors for dependency injection. This change reduces redundancy and simplifies the code structure for better readability and maintainability.

* Add byte conversion functions to JavaScript engine

Introduced functions for converting bytes to/from strings and Base64. Updated CommonFunctionsDefinitionProvider and ConfigureEngineWithCommonFunctions to incorporate these new functions. This enhances the JavaScript engine's ability to handle byte array manipulations.

* Add tests for JavaScript byte and string conversions.

Introduce integration tests to verify byte array to string, string to byte array, byte array to Base64, and Base64 to byte array conversions using JavaScript functions. Ensure accurate transformation of data within different encoding scenarios.

* Add meaningful summaries to Jint engine configuration handlers

Updated comment summaries in four handler classes to provide clear and concise descriptions of their purpose. This helps improve code readability and understanding for future developers.

* Rename and merge variable and input/output handlers

Merged the variable accessor logic into the input/output handler and renamed the class to reflect its broader functionality. This consolidation ensures the accessors are registered in the right order.

* Add string base64 conversion functions

Introduced `stringToBase64` and `stringFromBase64` functions to handle base64 encoding and decoding of strings. Updated corresponding provider, handler, and added tests to ensure functionality.
2024-08-14 17:43:35 +02:00
raymonddenhaan fbf3418544
Merge pull request #5896 from elsa-workflows/bug/5895
Add ExpandoObjectConverter for JavaScript object conversion
2024-08-14 09:42:10 +02:00
Sipke Schoorstra 71d56e1665 Add ExpandoObjectConverter for JavaScript object conversion
Introduced `ConverterHelper` and `ExpandoObjectConverter` to handle conversions from .NET dynamic objects to JavaScript objects. Updated integration tests to validate the conversion and sorting functionalities.

Fixes #5895
2024-08-13 19:58:18 +02:00
raymonddenhaan 0a9d4102c9
Merge pull request #5894 from elsa-workflows/fix/rename_ex_helpers
rename DbExceptionHandlers to better reflect their intent.
2024-08-13 15:12:27 +02:00
Raymond den Haan aa28329e47 rename DbExceptionHandlers to better reflect their intent. 2024-08-13 14:29:25 +02:00
Sipke Schoorstra c8669d1bc1
Add optional refresh parameter (#5876)
* Add optional refresh parameter

Renamed several `RefreshDescriptors` methods to `RefreshDescriptorsAsync` to reflect their asynchronous nature consistently. Added an optional refresh parameter in the `ListActivityDescriptorsRequest` class and updated related endpoints to handle this parameter, triggering a registry refresh if necessary.

* Update version to 3.2.0-rc5 in GitHub Actions

Modified the `packages.yml` workflow to change the version from `3.2.0-rc4` to `3.2.0-rc5` for non-tagged releases. This ensures version consistency with the new release candidate.
2024-08-12 09:42:25 +02:00