From 7c313325296adcf15efde0a41bcef03e82ef635b Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 19 Sep 2024 16:18:56 +0200 Subject: [PATCH 01/31] Fix variable serialization (#5974) * Improve dispatched workflow input handling Addressed input handling in dispatch messages by adding `SerializedInput` property. Also removed initialization logic and moved input deserialization to a helper method, ensuring compatibility with both new and deprecated input property formats. * Update workflow Docker images and version tags Changed Docker image tags from v3-2-0-rc3 to v3-2-1-preview across multiple GitHub workflows. Updated the VERSION environment variable in packages.yml to reflect the new versioning scheme. These changes ensure consistency with the new preview release. * Update versioning to include 'preview' in package workflow Modified the workflow to append 'preview' to the version number for non-tagged builds. This ensures clearer differentiation between stable and non-stable versions in the CI pipeline. * Add WorkflowInstanceStorageDriver for workflow variable storage Introduced a new storage driver, WorkflowInstanceStorageDriver, to store workflow variables directly in the workflow state. Updated relevant classes and methods to incorporate this new storage driver, ensuring seamless read/write/delete operations and extending support for it throughout the codebase. * Refactor object conversion and update variable retrieval. Switched from JsonObject to JsonNode for object conversion and corrected a typo in the summary comment. Changed the return type of GetVariablesDictionary method and updated its implementation to use VariablesDictionary. * Rename 'input' to 'serializedInput' in DispatchWorkflowDefinition. This change clarifies that the input provided to the workflow should be serialized. It enhances the readability and accuracy of the code documentation, ensuring that developers understand the expected format of the input parameter. * Add priority and deprecation attributes to storage drivers Introduced a priority attribute to the `IStorageDriver` interface and implemented it in various storage drivers. Additionally, marked `WorkflowStorageDriver` as deprecated and reordered storage driver listing based on priority. * Switch MassTransit broker to in-memory and refactor converter Changed MassTransit broker from AzureServiceBus to in-memory for improved performance in development environment. Simplified PolymorphicObjectConverterFactory by removing redundant constructor and dependencies. Removed unused folder from the project file. --- .github/workflows/elsa-server-and-studio.yml | 2 +- .github/workflows/elsa-server.yml | 2 +- .github/workflows/elsa-studio.yml | 2 +- .github/workflows/packages.yml | 2 +- Elsa.sln.DotSettings | 1 + .../Models/StorageDriverDescriptor.cs | 2 +- .../Helpers/ObjectConverter.cs | 6 +- .../Features/AzureServiceBusFeature.cs | 8 +++ .../Features/RabbitMqServiceBusFeature.cs | 9 ++- .../DispatchWorkflowRequestConsumer.cs | 46 ++++++++------- .../Messages/DispatchResumeWorkflows.cs | 4 ++ .../DispatchTriggerWorkflowsRequest.cs | 4 ++ .../Messages/DispatchWorkflowDefinition.cs | 15 +++-- .../Messages/DispatchWorkflowInstance.cs | 4 ++ .../Services/MassTransitWorkflowDispatcher.cs | 10 +++- .../Endpoints/StorageDrivers/List/Endpoint.cs | 6 +- .../Endpoints/StorageDrivers/List/Models.cs | 2 +- .../Activities/ParallelForEachT.cs | 4 +- .../Contracts/IStorageDriver.cs | 5 ++ .../ExpressionExecutionContextExtensions.cs | 2 +- .../Extensions/VariableExtensions.cs | 8 +-- .../Features/WorkflowsFeature.cs | 1 + .../PolymorphicObjectConverterFactory.cs | 4 +- .../Services/MemoryStorageDriver.cs | 2 + .../Services/WorkflowInstanceStorageDriver.cs | 58 +++++++++++++++++++ .../Services/WorkflowStateExtractor.cs | 2 +- .../Services/WorkflowStorageDriver.cs | 4 ++ .../Activities/BulkDispatchWorkflows.cs | 2 +- .../DefaultBackgroundActivityInvoker.cs | 2 +- .../Variables/CountdownWorkflowTests.cs | 6 +- 30 files changed, 171 insertions(+), 54 deletions(-) create mode 100644 src/modules/Elsa.Workflows.Core/Services/WorkflowInstanceStorageDriver.cs diff --git a/.github/workflows/elsa-server-and-studio.yml b/.github/workflows/elsa-server-and-studio.yml index 46417543a..8f58dbcc2 100644 --- a/.github/workflows/elsa-server-and-studio.yml +++ b/.github/workflows/elsa-server-and-studio.yml @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-server-and-studio-v3-2-0-rc3 + elsaworkflows/elsa-server-and-studio-v3-2-1-preview flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/elsa-server.yml b/.github/workflows/elsa-server.yml index 7c50ce0c8..82a92a31b 100644 --- a/.github/workflows/elsa-server.yml +++ b/.github/workflows/elsa-server.yml @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-server-v3-2-0-rc3 + elsaworkflows/elsa-server-v3-2-1-preview flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/elsa-studio.yml b/.github/workflows/elsa-studio.yml index 35c8eb51a..a8673a0dd 100644 --- a/.github/workflows/elsa-studio.yml +++ b/.github/workflows/elsa-studio.yml @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-studio-v3-2-0-rc3 + elsaworkflows/elsa-studio-v3-2-1-preview flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/packages.yml b/.github/workflows/packages.yml index 549e11d6e..c4461693c 100644 --- a/.github/workflows/packages.yml +++ b/.github/workflows/packages.yml @@ -62,7 +62,7 @@ jobs: TAG_NAME=${TAG_NAME#refs/tags/} # remove the refs/tags/ prefix echo "VERSION=${TAG_NAME}" >> $GITHUB_ENV else - echo "VERSION=3.2.0-rc6.${{github.run_number}}" >> $GITHUB_ENV + echo "VERSION=3.2.1-preview.${{github.run_number}}" >> $GITHUB_ENV fi - name: Set up JDK 17 uses: actions/setup-java@v2 diff --git a/Elsa.sln.DotSettings b/Elsa.sln.DotSettings index b291186ee..37f0ea98b 100644 --- a/Elsa.sln.DotSettings +++ b/Elsa.sln.DotSettings @@ -14,6 +14,7 @@ True True True + True True True True diff --git a/src/clients/Elsa.Api.Client/Resources/StorageDrivers/Models/StorageDriverDescriptor.cs b/src/clients/Elsa.Api.Client/Resources/StorageDrivers/Models/StorageDriverDescriptor.cs index e0bab6929..835893b38 100644 --- a/src/clients/Elsa.Api.Client/Resources/StorageDrivers/Models/StorageDriverDescriptor.cs +++ b/src/clients/Elsa.Api.Client/Resources/StorageDrivers/Models/StorageDriverDescriptor.cs @@ -5,4 +5,4 @@ namespace Elsa.Api.Client.Resources.StorageDrivers.Models; /// /// The type name of the storage driver. /// The display name of the storage driver. -public record StorageDriverDescriptor(string TypeName, string DisplayName); \ No newline at end of file +public record StorageDriverDescriptor(string TypeName, string DisplayName, double Priority = 0, bool Deprecated = false); \ No newline at end of file diff --git a/src/modules/Elsa.Expressions/Helpers/ObjectConverter.cs b/src/modules/Elsa.Expressions/Helpers/ObjectConverter.cs index a66d344e6..26a1a01bf 100644 --- a/src/modules/Elsa.Expressions/Helpers/ObjectConverter.cs +++ b/src/modules/Elsa.Expressions/Helpers/ObjectConverter.cs @@ -99,13 +99,13 @@ public static class ObjectConverter return jsonElement.Deserialize(targetType, serializerOptions); } - if (value is JsonObject jsonObject) + if (value is JsonNode jsonObject) { return underlyingTargetType switch { { } t when t == typeof(string) => jsonObject.ToString(), { } t when t != typeof(object) => jsonObject.Deserialize(targetType, serializerOptions), - _ => jsonObject, + _ => jsonObject }; } @@ -240,7 +240,7 @@ public static class ObjectConverter } /// - /// Returns true if the specified type is date-like type, false otherwise. + /// Returns true if the specified type is a date-like type, false otherwise. /// private static bool IsDateType(Type type) { diff --git a/src/modules/Elsa.MassTransit.AzureServiceBus/Features/AzureServiceBusFeature.cs b/src/modules/Elsa.MassTransit.AzureServiceBus/Features/AzureServiceBusFeature.cs index 0fc93e992..3ba3cc824 100644 --- a/src/modules/Elsa.MassTransit.AzureServiceBus/Features/AzureServiceBusFeature.cs +++ b/src/modules/Elsa.MassTransit.AzureServiceBus/Features/AzureServiceBusFeature.cs @@ -1,4 +1,5 @@ using Azure.Messaging.ServiceBus.Administration; +using Elsa.Common.Contracts; using Elsa.Extensions; using Elsa.Features.Abstractions; using Elsa.Features.Attributes; @@ -121,6 +122,13 @@ public class AzureServiceBusFeature : FeatureBase } configurator.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("Elsa", false)); + + configurator.ConfigureJsonSerializerOptions(serializerOptions => + { + var serializer = context.GetRequiredService(); + serializer.ApplyOptions(serializerOptions); + return serializerOptions; + }); }); }; }); diff --git a/src/modules/Elsa.MassTransit.RabbitMq/Features/RabbitMqServiceBusFeature.cs b/src/modules/Elsa.MassTransit.RabbitMq/Features/RabbitMqServiceBusFeature.cs index f8b443fdf..3359fffb0 100644 --- a/src/modules/Elsa.MassTransit.RabbitMq/Features/RabbitMqServiceBusFeature.cs +++ b/src/modules/Elsa.MassTransit.RabbitMq/Features/RabbitMqServiceBusFeature.cs @@ -1,10 +1,10 @@ +using Elsa.Common.Contracts; using Elsa.Extensions; using Elsa.Features.Abstractions; using Elsa.Features.Attributes; using Elsa.Features.Services; using Elsa.Hosting.Management.Contracts; using Elsa.Hosting.Management.Features; -using Elsa.MassTransit.Consumers; using Elsa.MassTransit.Extensions; using Elsa.MassTransit.Features; using Elsa.MassTransit.Options; @@ -88,6 +88,13 @@ public class RabbitMqServiceBusFeature : FeatureBase } configurator.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter("Elsa", false)); + + configurator.ConfigureJsonSerializerOptions(serializerOptions => + { + var serializer = context.GetRequiredService(); + serializer.ApplyOptions(serializerOptions); + return serializerOptions; + }); }); }; }); diff --git a/src/modules/Elsa.MassTransit/Consumers/DispatchWorkflowRequestConsumer.cs b/src/modules/Elsa.MassTransit/Consumers/DispatchWorkflowRequestConsumer.cs index dbcd4c2db..511a5d5ee 100644 --- a/src/modules/Elsa.MassTransit/Consumers/DispatchWorkflowRequestConsumer.cs +++ b/src/modules/Elsa.MassTransit/Consumers/DispatchWorkflowRequestConsumer.cs @@ -1,5 +1,5 @@ using Elsa.MassTransit.Messages; -using Elsa.Workflows.Management.Contracts; +using Elsa.Workflows.Contracts; using Elsa.Workflows.Runtime.Contracts; using Elsa.Workflows.Runtime.Options; using Elsa.Workflows.Runtime.Parameters; @@ -12,22 +12,12 @@ namespace Elsa.MassTransit.Consumers; /// A consumer of various dispatch message types to asynchronously execute workflows. /// [UsedImplicitly] -public class DispatchWorkflowRequestConsumer : +public class DispatchWorkflowRequestConsumer(IWorkflowRuntime workflowRuntime, IPayloadSerializer jsonSerializer) : IConsumer, IConsumer, IConsumer, IConsumer { - private readonly IWorkflowRuntime _workflowRuntime; - - /// - /// Initializes a new instance of the class. - /// - public DispatchWorkflowRequestConsumer(IWorkflowRuntime workflowRuntime, IWorkflowInstanceManager workflowInstanceManager) - { - _workflowRuntime = workflowRuntime; - } - /// public async Task Consume(ConsumeContext context) { @@ -42,6 +32,7 @@ public class DispatchWorkflowRequestConsumer : { var message = context.Message; var cancellationToken = context.CancellationToken; + var input = message.Input ?? DeserializeInput(message.SerializedInput); var options = new ResumeWorkflowRuntimeParams { @@ -51,12 +42,12 @@ public class DispatchWorkflowRequestConsumer : ActivityNodeId = message.ActivityNodeId, ActivityInstanceId = message.ActivityInstanceId, ActivityHash = message.ActivityHash, - Input = message.Input, + Input = input, Properties = message.Properties, CancellationTokens = cancellationToken }; - await _workflowRuntime.ResumeWorkflowAsync(message.InstanceId, options); + await workflowRuntime.ResumeWorkflowAsync(message.InstanceId, options); } /// @@ -64,16 +55,17 @@ public class DispatchWorkflowRequestConsumer : { var message = context.Message; var cancellationToken = context.CancellationToken; + var input = message.Input ?? DeserializeInput(message.SerializedInput); var options = new TriggerWorkflowsOptions { CorrelationId = message.CorrelationId, WorkflowInstanceId = message.WorkflowInstanceId, ActivityInstanceId = message.ActivityInstanceId, - Input = message.Input, + Input = input, Properties = message.Properties, CancellationTokens = cancellationToken }; - await _workflowRuntime.TriggerWorkflowsAsync(message.ActivityTypeName, message.BookmarkPayload, options); + await workflowRuntime.TriggerWorkflowsAsync(message.ActivityTypeName, message.BookmarkPayload, options); } /// @@ -81,29 +73,31 @@ public class DispatchWorkflowRequestConsumer : { var message = context.Message; var cancellationToken = context.CancellationToken; + var input = message.Input ?? DeserializeInput(message.SerializedInput); var options = new TriggerWorkflowsOptions { CorrelationId = message.CorrelationId, WorkflowInstanceId = message.WorkflowInstanceId, - Input = message.Input, + Input = input, Properties = message.Properties, CancellationTokens = cancellationToken }; - await _workflowRuntime.ResumeWorkflowsAsync(message.ActivityTypeName, message.BookmarkPayload, options); + await workflowRuntime.ResumeWorkflowsAsync(message.ActivityTypeName, message.BookmarkPayload, options); } - + private async Task DispatchNewWorkflowInstanceAsync(DispatchWorkflowDefinition message, CancellationToken cancellationToken) { if (string.IsNullOrWhiteSpace(message.DefinitionId)) throw new ArgumentException("The definition ID is required when dispatching a workflow definition."); if (message.VersionOptions == null) throw new ArgumentException("The version options are required when dispatching a workflow definition."); + var input = message.Input ?? DeserializeInput(message.SerializedInput); var options = new StartWorkflowRuntimeParams { ParentWorkflowInstanceId = message.ParentWorkflowInstanceId, CorrelationId = message.CorrelationId, - Input = message.Input, + Input = input, Properties = message.Properties, VersionOptions = message.VersionOptions.Value, TriggerActivityId = message.TriggerActivityId, @@ -111,7 +105,7 @@ public class DispatchWorkflowRequestConsumer : CancellationTokens = cancellationToken }; - await _workflowRuntime.TryStartWorkflowAsync(message.DefinitionId, options); + await workflowRuntime.TryStartWorkflowAsync(message.DefinitionId, options); } private async Task DispatchExistingWorkflowInstanceAsync(DispatchWorkflowDefinition message, CancellationToken cancellationToken) @@ -126,6 +120,14 @@ public class DispatchWorkflowRequestConsumer : CancellationTokens = cancellationToken }; - await _workflowRuntime.StartWorkflowAsync(message.InstanceId, options); + await workflowRuntime.StartWorkflowAsync(message.InstanceId, options); + } + + private IDictionary? DeserializeInput(string? json) + { + if (string.IsNullOrWhiteSpace(json)) + return null; + + return jsonSerializer.Deserialize>(json); } } \ No newline at end of file diff --git a/src/modules/Elsa.MassTransit/Messages/DispatchResumeWorkflows.cs b/src/modules/Elsa.MassTransit/Messages/DispatchResumeWorkflows.cs index 9d059d277..1ec3b68de 100644 --- a/src/modules/Elsa.MassTransit/Messages/DispatchResumeWorkflows.cs +++ b/src/modules/Elsa.MassTransit/Messages/DispatchResumeWorkflows.cs @@ -14,6 +14,10 @@ public class DispatchResumeWorkflows(string activityTypeName, object bookmarkPay public string? CorrelationId { get; set; } public string? WorkflowInstanceId { get; set; } public string? ActivityInstanceId { get; set; } + + [Obsolete("This property is no longer used and will be removed in a future version. Use the SerializedInput property instead.")] public IDictionary? Input { get; set; } + + public string? SerializedInput { get; set; } public IDictionary? Properties { get; set; } } \ No newline at end of file diff --git a/src/modules/Elsa.MassTransit/Messages/DispatchTriggerWorkflowsRequest.cs b/src/modules/Elsa.MassTransit/Messages/DispatchTriggerWorkflowsRequest.cs index 36a140b13..71456ecca 100644 --- a/src/modules/Elsa.MassTransit/Messages/DispatchTriggerWorkflowsRequest.cs +++ b/src/modules/Elsa.MassTransit/Messages/DispatchTriggerWorkflowsRequest.cs @@ -14,6 +14,10 @@ public class DispatchTriggerWorkflows(string activityTypeName, object bookmarkPa public string? CorrelationId { get; set; } public string? WorkflowInstanceId { get; set; } public string? ActivityInstanceId { get; set; } + + [Obsolete("This property is no longer used and will be removed in a future version. Use the SerializedInput property instead.")] public IDictionary? Input { get; set; } + + public string? SerializedInput { get; set; } public IDictionary? Properties { get; set; } } \ No newline at end of file diff --git a/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowDefinition.cs b/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowDefinition.cs index 22f6348b0..a5fc979ac 100644 --- a/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowDefinition.cs +++ b/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowDefinition.cs @@ -1,4 +1,3 @@ -using System.Text.Json.Serialization; using Elsa.Common.Models; namespace Elsa.MassTransit.Messages; @@ -27,7 +26,7 @@ public record DispatchWorkflowDefinition /// The ID of the workflow definition to dispatch. /// The version options to use when dispatching the workflow definition. /// The ID of the parent workflow instance. - /// Any input to pass to the workflow. + /// Any input to pass to the workflow. /// Any properties to attach to the workflow. /// A correlation ID to associate the workflow with. /// The ID to use when creating an instance of the workflow to dispatch. @@ -36,7 +35,7 @@ public record DispatchWorkflowDefinition string? definitionId, VersionOptions? versionOptions, string? parentWorkflowInstanceId, - IDictionary? input, + string? serializedInput, IDictionary? properties, string? correlationId, string? instanceId, @@ -47,7 +46,7 @@ public record DispatchWorkflowDefinition DefinitionId = definitionId, VersionOptions = versionOptions, ParentWorkflowInstanceId = parentWorkflowInstanceId, - Input = input, + SerializedInput = serializedInput, Properties = properties, CorrelationId = correlationId, InstanceId = instanceId, @@ -64,8 +63,14 @@ public record DispatchWorkflowDefinition /// The ID of the parent workflow instance. public string? ParentWorkflowInstanceId { get; init; } + /// Deprecated. Use the property instead. + [Obsolete("This property is no longer used and will be removed in a future version. Use the SerializedInput property instead.")] + public IDictionary? Input { get; set; } + + /// /// Any input to pass to the workflow. - public IDictionary? Input { get; init; } + /// + public string? SerializedInput { get; set; } /// Any properties to attach to the workflow. public IDictionary? Properties { get; init; } diff --git a/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowInstance.cs b/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowInstance.cs index 212d70a89..04bc4935c 100644 --- a/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowInstance.cs +++ b/src/modules/Elsa.MassTransit/Messages/DispatchWorkflowInstance.cs @@ -8,7 +8,11 @@ public class DispatchWorkflowInstance(string instanceId) public string? ActivityNodeId { get; set; } public string? ActivityInstanceId { get; set; } public string? ActivityHash { get; set; } + + [Obsolete("This property is no longer used and will be removed in a future version. Use the SerializedInput property instead.")] public IDictionary? Input { get; set; } + + public string? SerializedInput { get; set; } public IDictionary? Properties { get; set; } public string? CorrelationId { get; set; } } \ No newline at end of file diff --git a/src/modules/Elsa.MassTransit/Services/MassTransitWorkflowDispatcher.cs b/src/modules/Elsa.MassTransit/Services/MassTransitWorkflowDispatcher.cs index a3870d3d3..edf665e58 100644 --- a/src/modules/Elsa.MassTransit/Services/MassTransitWorkflowDispatcher.cs +++ b/src/modules/Elsa.MassTransit/Services/MassTransitWorkflowDispatcher.cs @@ -11,7 +11,6 @@ using Elsa.Workflows.Runtime.Models; using Elsa.Workflows.Runtime.Requests; using Elsa.Workflows.Runtime.Responses; using MassTransit; -using Medallion.Threading; using Microsoft.Extensions.Logging; namespace Elsa.MassTransit.Services; @@ -27,6 +26,7 @@ public class MassTransitWorkflowDispatcher( IBookmarkHasher bookmarkHasher, ITriggerStore triggerStore, IBookmarkStore bookmarkStore, + IPayloadSerializer jsonSerializer, ILogger logger) : IWorkflowDispatcher { @@ -57,6 +57,7 @@ public class MassTransitWorkflowDispatcher( public async Task DispatchAsync(DispatchWorkflowInstanceRequest request, DispatchWorkflowOptions? options = default, CancellationToken cancellationToken = default) { var sendEndpoint = await GetSendEndpointAsync(options); + var serializedInput = SerializeInput(request.Input); await sendEndpoint.Send(new DispatchWorkflowInstance(request.InstanceId) { @@ -66,7 +67,7 @@ public class MassTransitWorkflowDispatcher( ActivityInstanceId = request.ActivityInstanceId, ActivityHash = request.ActivityHash, CorrelationId = request.CorrelationId, - Input = request.Input + SerializedInput = serializedInput, }, cancellationToken); return DispatchWorkflowResponse.Success(); } @@ -180,4 +181,9 @@ public class MassTransitWorkflowDispatcher( var sendEndpoint = await bus.GetSendEndpoint(new Uri($"queue:{endpointName}")); return sendEndpoint; } + + private string? SerializeInput(object? input) + { + return input != null ? jsonSerializer.Serialize(input) : null; + } } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Endpoint.cs index a38030ea5..207501e21 100644 --- a/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Endpoint.cs +++ b/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Endpoint.cs @@ -31,7 +31,7 @@ public class List : ElsaEndpointWithoutRequest public override Task ExecuteAsync(CancellationToken ct) { var drivers = _registry.List(); - var descriptors = drivers.Select(FromDriver).ToList(); + var descriptors = drivers.Select(FromDriver).OrderByDescending(x => x.Priority).ToList(); var response = new Response(descriptors); return Task.FromResult(response); @@ -40,7 +40,9 @@ public class List : ElsaEndpointWithoutRequest private static StorageDriverDescriptor FromDriver(IStorageDriver driver) { var type = driver.GetType(); + var deprecated = type.GetCustomAttribute() != null; var displayName = type.GetCustomAttribute()?.Name ?? type.GetCustomAttribute()?.DisplayName ?? type.Name.Replace("StorageDriver", ""); - return new StorageDriverDescriptor(type.GetSimpleAssemblyQualifiedName(), displayName); + var priority = driver.Priority; + return new StorageDriverDescriptor(type.GetSimpleAssemblyQualifiedName(), displayName, priority, deprecated); } } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Models.cs b/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Models.cs index 127a1ce68..c52464164 100644 --- a/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Models.cs +++ b/src/modules/Elsa.Workflows.Api/Endpoints/StorageDrivers/List/Models.cs @@ -5,4 +5,4 @@ public class Response(ICollection items) public ICollection Items { get; set; } = items; } -public record StorageDriverDescriptor(string TypeName, string DisplayName); \ No newline at end of file +public record StorageDriverDescriptor(string TypeName, string DisplayName, double Priority = 0, bool Deprecated = false); \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Activities/ParallelForEachT.cs b/src/modules/Elsa.Workflows.Core/Activities/ParallelForEachT.cs index d5d8fcd63..3c3ab86da 100644 --- a/src/modules/Elsa.Workflows.Core/Activities/ParallelForEachT.cs +++ b/src/modules/Elsa.Workflows.Core/Activities/ParallelForEachT.cs @@ -53,10 +53,10 @@ public class ParallelForEach : Activity var currentValueVariable = new Variable("CurrentValue", item) { // TODO: This should be configurable, because this won't work for e.g. file streams and other non-serializable types. - StorageDriverType = typeof(WorkflowStorageDriver) + StorageDriverType = typeof(WorkflowInstanceStorageDriver) }; - var currentIndexVariable = new Variable("CurrentIndex", currentIndex++) { StorageDriverType = typeof(WorkflowStorageDriver) }; + var currentIndexVariable = new Variable("CurrentIndex", currentIndex++) { StorageDriverType = typeof(WorkflowInstanceStorageDriver) }; var variables = new List { currentValueVariable, currentIndexVariable }; // Schedule a body of work for each item. diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IStorageDriver.cs b/src/modules/Elsa.Workflows.Core/Contracts/IStorageDriver.cs index d17ab4735..50ca0eec2 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/IStorageDriver.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/IStorageDriver.cs @@ -5,6 +5,11 @@ namespace Elsa.Workflows.Contracts; /// public interface IStorageDriver { + /// + /// The priority of the storage driver. Drivers with higher priority are used before drivers with lower priority. + /// + double Priority { get; } + /// /// Writes a value to the storage driver. /// diff --git a/src/modules/Elsa.Workflows.Core/Extensions/ExpressionExecutionContextExtensions.cs b/src/modules/Elsa.Workflows.Core/Extensions/ExpressionExecutionContextExtensions.cs index 5759d88d7..7bb7d2f97 100644 --- a/src/modules/Elsa.Workflows.Core/Extensions/ExpressionExecutionContextExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Extensions/ExpressionExecutionContextExtensions.cs @@ -141,7 +141,7 @@ public static class ExpressionExecutionContextExtensions var variable = new Variable(name, value) { - StorageDriverType = storageDriverType ?? typeof(WorkflowStorageDriver) + StorageDriverType = storageDriverType ?? typeof(WorkflowInstanceStorageDriver) }; // Find the first parent context that has a variable container. diff --git a/src/modules/Elsa.Workflows.Core/Extensions/VariableExtensions.cs b/src/modules/Elsa.Workflows.Core/Extensions/VariableExtensions.cs index d40876542..85f242ceb 100644 --- a/src/modules/Elsa.Workflows.Core/Extensions/VariableExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Extensions/VariableExtensions.cs @@ -31,14 +31,14 @@ public static class VariableExtensions new ExpandoObjectConverterFactory()); /// - /// Configures the variable to use the . + /// Configures the variable to use the . /// - public static Variable WithWorkflowStorage(this Variable variable) => variable.WithStorage(); + public static Variable WithWorkflowStorage(this Variable variable) => variable.WithStorage(); /// - /// Configures the variable to use the . + /// Configures the variable to use the . /// - public static Variable WithWorkflowStorage(this Variable variable) => (Variable)variable.WithStorage(); + public static Variable WithWorkflowStorage(this Variable variable) => (Variable)variable.WithStorage(); /// /// Configures the variable to use the . diff --git a/src/modules/Elsa.Workflows.Core/Features/WorkflowsFeature.cs b/src/modules/Elsa.Workflows.Core/Features/WorkflowsFeature.cs index 1dc698cf0..1026cb7c2 100644 --- a/src/modules/Elsa.Workflows.Core/Features/WorkflowsFeature.cs +++ b/src/modules/Elsa.Workflows.Core/Features/WorkflowsFeature.cs @@ -171,6 +171,7 @@ public class WorkflowsFeature : FeatureBase // Storage drivers. .AddScoped() .AddStorageDriver() + .AddStorageDriver() .AddStorageDriver() // Serialization. diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs index d50c29b88..448fc16be 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs @@ -2,6 +2,7 @@ using System.Dynamic; using System.Text.Json; using System.Text.Json.Serialization; using Elsa.Expressions.Contracts; +using Elsa.Expressions.Services; namespace Elsa.Workflows.Serialization.Converters; @@ -16,7 +17,8 @@ public class PolymorphicObjectConverterFactory(IWellKnownTypeRegistry wellKnownT var canConvert = typeToConvert.IsClass && typeToConvert == typeof(object) || typeToConvert == typeof(ExpandoObject) - || typeToConvert == typeof(Dictionary); + || typeToConvert == typeof(Dictionary) + || typeToConvert == typeof(IDictionary); return canConvert; } diff --git a/src/modules/Elsa.Workflows.Core/Services/MemoryStorageDriver.cs b/src/modules/Elsa.Workflows.Core/Services/MemoryStorageDriver.cs index 485e0715f..e04642ddf 100644 --- a/src/modules/Elsa.Workflows.Core/Services/MemoryStorageDriver.cs +++ b/src/modules/Elsa.Workflows.Core/Services/MemoryStorageDriver.cs @@ -11,6 +11,8 @@ public class MemoryStorageDriver : IStorageDriver { private readonly IDictionary _dictionary = new Dictionary(); + public double Priority => 0; + /// public ValueTask WriteAsync(string id, object value, StorageDriverContext context) { diff --git a/src/modules/Elsa.Workflows.Core/Services/WorkflowInstanceStorageDriver.cs b/src/modules/Elsa.Workflows.Core/Services/WorkflowInstanceStorageDriver.cs new file mode 100644 index 000000000..b6e7fd7ae --- /dev/null +++ b/src/modules/Elsa.Workflows.Core/Services/WorkflowInstanceStorageDriver.cs @@ -0,0 +1,58 @@ +using System.ComponentModel.DataAnnotations; +using System.Text.Json; +using System.Text.Json.Nodes; +using Elsa.Extensions; +using Elsa.Workflows.Contracts; +using JetBrains.Annotations; + +namespace Elsa.Workflows.Services; + +/// A storage driver that stores objects in the workflow state itself. +[Display(Name = "Workflow Instance")] +[UsedImplicitly] +public class WorkflowInstanceStorageDriver : IStorageDriver +{ + /// The key used to store the variables in the workflow state. + public const string VariablesDictionaryStateKey = "Variables"; + + /// + public double Priority => 1; + + /// + public ValueTask WriteAsync(string id, object value, StorageDriverContext context) + { + UpdateVariablesDictionary(context, dictionary => + { + var node = JsonSerializer.SerializeToNode(value); + dictionary[id] = node; + }); + return ValueTask.CompletedTask; + } + + /// + public ValueTask ReadAsync(string id, StorageDriverContext context) + { + var dictionary = GetVariablesDictionary(context); + var node = dictionary.GetValueOrDefault(id); + return new(node); + } + + /// + public ValueTask DeleteAsync(string id, StorageDriverContext context) + { + UpdateVariablesDictionary(context, dictionary => dictionary.Remove(id)); + return ValueTask.CompletedTask; + } + + private VariablesDictionary GetVariablesDictionary(StorageDriverContext context) => context.ExecutionContext.Properties.GetOrAdd(VariablesDictionaryStateKey, () => new VariablesDictionary()); + private void SetVariablesDictionary(StorageDriverContext context, VariablesDictionary dictionary) => context.ExecutionContext.Properties[VariablesDictionaryStateKey] = dictionary; + + private void UpdateVariablesDictionary(StorageDriverContext context, Action update) + { + var dictionary = GetVariablesDictionary(context); + update(dictionary); + SetVariablesDictionary(context, dictionary); + } +} + +public class VariablesDictionary : Dictionary; \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Services/WorkflowStateExtractor.cs b/src/modules/Elsa.Workflows.Core/Services/WorkflowStateExtractor.cs index 009079217..26c70baf8 100644 --- a/src/modules/Elsa.Workflows.Core/Services/WorkflowStateExtractor.cs +++ b/src/modules/Elsa.Workflows.Core/Services/WorkflowStateExtractor.cs @@ -71,7 +71,7 @@ public class WorkflowStateExtractor : IWorkflowStateExtractor private IDictionary GetPersistableInput(WorkflowExecutionContext workflowExecutionContext) { // TODO: This is a temporary solution. We need to find a better way to handle this. - var persistableInput = workflowExecutionContext.Workflow.Inputs.Where(x => x.StorageDriverType == typeof(WorkflowStorageDriver)).ToList(); + var persistableInput = workflowExecutionContext.Workflow.Inputs.Where(x => x.StorageDriverType == typeof(WorkflowStorageDriver) || x.StorageDriverType == typeof(WorkflowInstanceStorageDriver)).ToList(); var input = workflowExecutionContext.Input; var filteredInput = new Dictionary(); diff --git a/src/modules/Elsa.Workflows.Core/Services/WorkflowStorageDriver.cs b/src/modules/Elsa.Workflows.Core/Services/WorkflowStorageDriver.cs index 7cbf7ac8d..b2ee7e8cf 100644 --- a/src/modules/Elsa.Workflows.Core/Services/WorkflowStorageDriver.cs +++ b/src/modules/Elsa.Workflows.Core/Services/WorkflowStorageDriver.cs @@ -8,6 +8,7 @@ namespace Elsa.Workflows.Services; /// A storage driver that stores objects in the workflow state itself. /// [Display(Name = "Workflow")] +[Obsolete("This is no longer used and will be removed in a future version. Use the WorkflowInstanceStorageDriver instead.")] public class WorkflowStorageDriver : IStorageDriver { /// @@ -15,6 +16,9 @@ public class WorkflowStorageDriver : IStorageDriver /// public const string VariablesDictionaryStateKey = "PersistentVariablesDictionary"; + /// + public double Priority => -1; + /// public ValueTask WriteAsync(string id, object value, StorageDriverContext context) { diff --git a/src/modules/Elsa.Workflows.Runtime/Activities/BulkDispatchWorkflows.cs b/src/modules/Elsa.Workflows.Runtime/Activities/BulkDispatchWorkflows.cs index b04f1b296..06eb5306a 100644 --- a/src/modules/Elsa.Workflows.Runtime/Activities/BulkDispatchWorkflows.cs +++ b/src/modules/Elsa.Workflows.Runtime/Activities/BulkDispatchWorkflows.cs @@ -237,7 +237,7 @@ public class BulkDispatchWorkflows : Activity var childInstanceId = new Variable("ChildInstanceId", workflowInstanceId) { - StorageDriverType = typeof(WorkflowStorageDriver) + StorageDriverType = typeof(WorkflowInstanceStorageDriver) }; var variables = new List diff --git a/src/modules/Elsa.Workflows.Runtime/Services/DefaultBackgroundActivityInvoker.cs b/src/modules/Elsa.Workflows.Runtime/Services/DefaultBackgroundActivityInvoker.cs index 40e5518b2..1141abbbd 100644 --- a/src/modules/Elsa.Workflows.Runtime/Services/DefaultBackgroundActivityInvoker.cs +++ b/src/modules/Elsa.Workflows.Runtime/Services/DefaultBackgroundActivityInvoker.cs @@ -97,7 +97,7 @@ public class DefaultBackgroundActivityInvoker : IBackgroundActivityInvoker var driver = variableMetadata?.StorageDriverType; // We only capture output written to the workflow itself. Other drivers like blob storage, etc. will be ignored since the foreground context will be loading those. - if (driver != typeof(WorkflowStorageDriver)) + if (driver != typeof(WorkflowStorageDriver) && driver != typeof(WorkflowInstanceStorageDriver)) continue; var outputValue = activityExecutionContext.Get(memoryBlockReference); diff --git a/test/component/Elsa.Workflows.ComponentTests/Scenarios/Variables/CountdownWorkflowTests.cs b/test/component/Elsa.Workflows.ComponentTests/Scenarios/Variables/CountdownWorkflowTests.cs index b0f3f7b9b..a0067ea35 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Scenarios/Variables/CountdownWorkflowTests.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Scenarios/Variables/CountdownWorkflowTests.cs @@ -48,6 +48,8 @@ public class CountdownWorkflowTests(App app) : AppComponentTest(app) } } - private IDictionary GetVariablesDictionary(ActivityExecutionContextState context) => - context.Properties.GetOrAdd(WorkflowStorageDriver.VariablesDictionaryStateKey, () => new Dictionary()); + private VariablesDictionary GetVariablesDictionary(ActivityExecutionContextState context) + { + return context.Properties.GetOrAdd(WorkflowInstanceStorageDriver.VariablesDictionaryStateKey, () => new VariablesDictionary()); + } } \ No newline at end of file From a827ab241b03825168a57503a51ab1360097c795 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 19 Sep 2024 20:48:09 +0200 Subject: [PATCH 02/31] Fix Azure Service Bus module DI issues (#5975) * Switch to Azure Service Bus for message brokering Updated the message broker from in-memory to Azure Service Bus to enhance scalability. Added new configuration settings and dependencies to support Azure Service Bus integration, including scoped service initialization and queue/topic management. * Refactor StartWorkers for dependency injection Updated StartWorkers class to use constructor parameter injection and service scope factory for retrieving services. This refactor eliminates the need for field-level dependency storage, enhancing testability and adherence to the dependency injection principles. Additionally, annotated the class with [UsedImplicitly] to eliminate unused code warnings. * Refactor Worker to use IServiceScopeFactory for dependency resolution Refactored the Worker class to use IServiceScopeFactory for DI instead of directly injecting IWorkflowInbox. This enhances the flexibility and lifecycle management of the dependencies. * Add Azure Service Bus integration toggle Introduced a new configuration constant `useAzureServiceBus` to control the optional integration with Azure Service Bus. Implemented conditional logic to bind Azure Service Bus options from configuration if the flag is enabled. * Add IComposite interface and implement Setup method Introduced a new IComposite interface and added a default Setup method to the Composite class. Updated the ActivityFactory to call the Setup method when an activity is an instance of IComposite. The CompositeExample class demonstrates the use of the new interface and method. * Remove AzureServiceBus configuration setup The AzureServiceBus configuration setup was removed from the Program.cs file. This change aims to streamline configuration and remove unused or unnecessary setup, ensuring the code remains clean and maintainable. * Switch MassTransit broker to in-memory Changed the MassTransit broker from AzureServiceBus to Memory in the application configuration. This adjustment aims to simplify deployment and reduce dependencies in the current environment. --- .../Activities/CompositeExample.cs | 40 +++++++++++++++++++ .../Elsa.Server.Web/Elsa.Server.Web.csproj | 1 + src/bundles/Elsa.Server.Web/Program.cs | 4 ++ src/bundles/Elsa.Server.Web/appsettings.json | 13 ++++++ .../Features/AzureServiceBusFeature.cs | 2 +- .../CreateQueuesTopicsAndSubscriptions.cs | 18 +++++---- .../HostedServices/StartWorkers.cs | 28 +++++-------- .../Elsa.AzureServiceBus/Services/Worker.cs | 11 +++-- .../Activities/Composite.cs | 6 ++- .../Contracts/IComposite.cs | 6 +++ .../Services/ActivityFactory.cs | 3 ++ 11 files changed, 100 insertions(+), 32 deletions(-) create mode 100644 src/bundles/Elsa.Server.Web/Activities/CompositeExample.cs create mode 100644 src/modules/Elsa.Workflows.Core/Contracts/IComposite.cs diff --git a/src/bundles/Elsa.Server.Web/Activities/CompositeExample.cs b/src/bundles/Elsa.Server.Web/Activities/CompositeExample.cs new file mode 100644 index 000000000..7eb7a335c --- /dev/null +++ b/src/bundles/Elsa.Server.Web/Activities/CompositeExample.cs @@ -0,0 +1,40 @@ +using Elsa.AzureServiceBus.Activities; +using Elsa.Extensions; +using Elsa.Workflows.Activities; +using Elsa.Workflows.Attributes; +using Elsa.Workflows.Models; + +namespace Elsa.Server.Web.Activities; + +[Activity("Elsa", "Example")] +public class CompositeExample : Composite +{ + /// + /// The name of the queue or topic to read from. + /// + [Input(Description = "The name of the queue or topic to read from.")] + public Input QueueOrTopic { get; set; } = default!; + + private MessageReceived _messageReceived = default!; + + /// + public override void Setup() + { + _messageReceived = new MessageReceived + { + QueueOrTopic = QueueOrTopic, + CanStartWorkflow = true + }; + + var writeLine = new WriteLine("Hello World!"); + + Root = new Sequence + { + Activities = + { + _messageReceived, + writeLine + } + }; + } +} \ No newline at end of file diff --git a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj index 15636b9b6..71cb87c96 100644 --- a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj +++ b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj @@ -8,6 +8,7 @@ + diff --git a/src/bundles/Elsa.Server.Web/Program.cs b/src/bundles/Elsa.Server.Web/Program.cs index c4f259015..33328dace 100644 --- a/src/bundles/Elsa.Server.Web/Program.cs +++ b/src/bundles/Elsa.Server.Web/Program.cs @@ -49,6 +49,7 @@ const bool useMemoryStores = false; const bool useCaching = true; const bool useReadOnlyMode = false; const bool useSignalR = true; +const bool useAzureServiceBus = false; const DistributedCachingTransport distributedCachingTransport = DistributedCachingTransport.MassTransit; const MassTransitBroker useMassTransitBroker = MassTransitBroker.Memory; @@ -327,6 +328,9 @@ services elsa.UseRealTimeWorkflows(); } + if (useAzureServiceBus) + elsa.UseAzureServiceBus(asb => asb.AzureServiceBusOptions += options => configuration.GetSection("AzureServiceBus").Bind(options)); + if (useMassTransit) { elsa.UseMassTransit(massTransit => diff --git a/src/bundles/Elsa.Server.Web/appsettings.json b/src/bundles/Elsa.Server.Web/appsettings.json index cf7b67665..cf664177e 100644 --- a/src/bundles/Elsa.Server.Web/appsettings.json +++ b/src/bundles/Elsa.Server.Web/appsettings.json @@ -125,5 +125,18 @@ "def say_hello_world(): return greet('World')" ] } + }, + "AzureServiceBus": { + "ConnectionStringOrName": "AzureServiceBus", + "Queues": [ + { + "name": "order-created" + }, + { + "name": "order-completed" + } + ], + "Topics": [], + "Subscriptions": [] } } diff --git a/src/modules/Elsa.AzureServiceBus/Features/AzureServiceBusFeature.cs b/src/modules/Elsa.AzureServiceBus/Features/AzureServiceBusFeature.cs index b575f9cf1..79f0e49a0 100644 --- a/src/modules/Elsa.AzureServiceBus/Features/AzureServiceBusFeature.cs +++ b/src/modules/Elsa.AzureServiceBus/Features/AzureServiceBusFeature.cs @@ -71,7 +71,7 @@ public class AzureServiceBusFeature : FeatureBase .AddSingleton(ServiceBusClientFactory) .AddSingleton() .AddSingleton() - .AddTransient(); + .AddScoped(); // Definition providers. Services diff --git a/src/modules/Elsa.AzureServiceBus/HostedServices/CreateQueuesTopicsAndSubscriptions.cs b/src/modules/Elsa.AzureServiceBus/HostedServices/CreateQueuesTopicsAndSubscriptions.cs index b6285f044..3e488421e 100644 --- a/src/modules/Elsa.AzureServiceBus/HostedServices/CreateQueuesTopicsAndSubscriptions.cs +++ b/src/modules/Elsa.AzureServiceBus/HostedServices/CreateQueuesTopicsAndSubscriptions.cs @@ -1,4 +1,6 @@ using Elsa.AzureServiceBus.Contracts; +using JetBrains.Annotations; +using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Hosting; namespace Elsa.AzureServiceBus.HostedServices; @@ -6,16 +8,16 @@ namespace Elsa.AzureServiceBus.HostedServices; /// /// A blocking hosted service that creates queues, topics and subscriptions. /// -public class CreateQueuesTopicsAndSubscriptions : IHostedService +[UsedImplicitly] +public class CreateQueuesTopicsAndSubscriptions(IServiceScopeFactory scopeFactory) : IHostedService { - private readonly IServiceBusInitializer _serviceBusInitializer; - /// - /// Constructor. - /// - public CreateQueuesTopicsAndSubscriptions(IServiceBusInitializer serviceBusInitializer) => _serviceBusInitializer = serviceBusInitializer; - /// - public Task StartAsync(CancellationToken cancellationToken) => _serviceBusInitializer.InitializeAsync(cancellationToken); + public async Task StartAsync(CancellationToken cancellationToken) + { + await using var scope = scopeFactory.CreateAsyncScope(); + var initializer = scope.ServiceProvider.GetRequiredService(); + await initializer.InitializeAsync(cancellationToken); + } /// public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask; diff --git a/src/modules/Elsa.AzureServiceBus/HostedServices/StartWorkers.cs b/src/modules/Elsa.AzureServiceBus/HostedServices/StartWorkers.cs index e7fa20ab4..cd9c20118 100644 --- a/src/modules/Elsa.AzureServiceBus/HostedServices/StartWorkers.cs +++ b/src/modules/Elsa.AzureServiceBus/HostedServices/StartWorkers.cs @@ -5,6 +5,8 @@ using Elsa.Extensions; using Elsa.Workflows.Helpers; using Elsa.Workflows.Runtime.Contracts; using Elsa.Workflows.Runtime.Filters; +using JetBrains.Annotations; +using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Hosting; namespace Elsa.AzureServiceBus.HostedServices; @@ -12,30 +14,20 @@ namespace Elsa.AzureServiceBus.HostedServices; /// /// Creates workers for each trigger & bookmark in response to updated workflow trigger indexes and bookmarks. /// -public class StartWorkers : IHostedService +[UsedImplicitly] +public class StartWorkers(IWorkerManager workerManager, IServiceScopeFactory scopeFactory) : IHostedService { - private readonly ITriggerStore _triggerStore; - private readonly IBookmarkStore _bookmarkStore; - private readonly IWorkerManager _workerManager; - - /// - /// Constructor. - /// - public StartWorkers(ITriggerStore triggerStore, IBookmarkStore bookmarkStore, IWorkerManager workerManager) - { - _triggerStore = triggerStore; - _bookmarkStore = bookmarkStore; - _workerManager = workerManager; - } - /// public async Task StartAsync(CancellationToken cancellationToken) { + using var scope = scopeFactory.CreateScope(); + var triggerStore = scope.ServiceProvider.GetRequiredService(); + var bookmarkStore = scope.ServiceProvider.GetRequiredService(); var activityType = ActivityTypeNameHelper.GenerateTypeName(); var triggerFilter = new TriggerFilter { Name = activityType}; - var triggers = (await _triggerStore.FindManyAsync(triggerFilter, cancellationToken)).Select(x => x.GetPayload()).ToList(); + var triggers = (await triggerStore.FindManyAsync(triggerFilter, cancellationToken)).Select(x => x.GetPayload()).ToList(); var bookmarkFilter = new BookmarkFilter { ActivityTypeName = activityType }; - var bookmarks = (await _bookmarkStore.FindManyAsync(bookmarkFilter, cancellationToken)).Select(x => x.GetPayload()).ToList(); + var bookmarks = (await bookmarkStore.FindManyAsync(bookmarkFilter, cancellationToken)).Select(x => x.GetPayload()).ToList(); var payloads = triggers.Concat(bookmarks).ToList(); await EnsureWorkersAsync(payloads, cancellationToken); @@ -46,6 +38,6 @@ public class StartWorkers : IHostedService private async Task EnsureWorkersAsync(IEnumerable payloads, CancellationToken cancellationToken) { - foreach (var payload in payloads) await _workerManager.EnsureWorkerAsync(payload.QueueOrTopic, payload.Subscription, cancellationToken); + foreach (var payload in payloads) await workerManager.EnsureWorkerAsync(payload.QueueOrTopic, payload.Subscription, cancellationToken); } } \ No newline at end of file diff --git a/src/modules/Elsa.AzureServiceBus/Services/Worker.cs b/src/modules/Elsa.AzureServiceBus/Services/Worker.cs index 7cb40eccd..561b4feae 100644 --- a/src/modules/Elsa.AzureServiceBus/Services/Worker.cs +++ b/src/modules/Elsa.AzureServiceBus/Services/Worker.cs @@ -4,6 +4,7 @@ using Elsa.AzureServiceBus.Models; using Elsa.Workflows.Helpers; using Elsa.Workflows.Runtime.Contracts; using Elsa.Workflows.Runtime.Models; +using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; namespace Elsa.AzureServiceBus.Services; @@ -15,18 +16,18 @@ namespace Elsa.AzureServiceBus.Services; public class Worker : IAsyncDisposable { private readonly ServiceBusProcessor _processor; - private readonly IWorkflowInbox _workflowInbox; + private readonly IServiceScopeFactory _scopeFactory; private readonly ILogger _logger; private int _refCount = 1; /// /// Initializes a new instance of the class. /// - public Worker(string queueOrTopic, string? subscription, ServiceBusClient client, IWorkflowInbox workflowInbox, ILogger logger) + public Worker(string queueOrTopic, string? subscription, ServiceBusClient client, IServiceScopeFactory scopeFactory, ILogger logger) { QueueOrTopic = queueOrTopic; Subscription = subscription == "" ? default : subscription; - _workflowInbox = workflowInbox; + _scopeFactory = scopeFactory; _logger = logger; var options = new ServiceBusProcessorOptions(); @@ -98,8 +99,10 @@ public class Worker : IAsyncDisposable var messageModel = CreateMessageModel(message); var input = new Dictionary { [MessageReceived.InputKey] = messageModel }; var activityTypeName = ActivityTypeNameHelper.GenerateTypeName(); + await using var scope = _scopeFactory.CreateAsyncScope(); + var workflowInbox = scope.ServiceProvider.GetRequiredService(); - var results = await _workflowInbox.SubmitAsync(new NewWorkflowInboxMessage + var results = await workflowInbox.SubmitAsync(new NewWorkflowInboxMessage { ActivityTypeName = activityTypeName, BookmarkPayload = payload, diff --git a/src/modules/Elsa.Workflows.Core/Activities/Composite.cs b/src/modules/Elsa.Workflows.Core/Activities/Composite.cs index f702057a8..6661adb95 100644 --- a/src/modules/Elsa.Workflows.Core/Activities/Composite.cs +++ b/src/modules/Elsa.Workflows.Core/Activities/Composite.cs @@ -17,7 +17,7 @@ namespace Elsa.Workflows.Activities; /// Represents a composite activity that has a single activity. Like a workflow, but without workflow-level properties. /// [PublicAPI] -public abstract class Composite : Activity, IVariableContainer +public abstract class Composite : Activity, IVariableContainer, IComposite { /// protected Composite(string? source = default, int? line = default) : base(source, line) @@ -172,6 +172,10 @@ public abstract class Composite : Activity, IVariableContainer /// Creates a new activity. /// protected static SetVariable SetVariable(Variable variable, Variable value, [CallerFilePath] string? source = default, [CallerLineNumber] int? line = default) => new(variable, value, source, line); + + public virtual void Setup() + { + } } /// diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IComposite.cs b/src/modules/Elsa.Workflows.Core/Contracts/IComposite.cs new file mode 100644 index 000000000..6953b59ef --- /dev/null +++ b/src/modules/Elsa.Workflows.Core/Contracts/IComposite.cs @@ -0,0 +1,6 @@ +namespace Elsa.Workflows.Contracts; + +public interface IComposite +{ + void Setup(); +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Services/ActivityFactory.cs b/src/modules/Elsa.Workflows.Core/Services/ActivityFactory.cs index 16c612135..5806a6221 100644 --- a/src/modules/Elsa.Workflows.Core/Services/ActivityFactory.cs +++ b/src/modules/Elsa.Workflows.Core/Services/ActivityFactory.cs @@ -20,6 +20,9 @@ public class ActivityFactory : IActivityFactory var activityElement = context.Element; var activityDescriptor = context.ActivityDescriptor; var activity = (IActivity)context.Element.Deserialize(type, context.SerializerOptions)!; + var composite = activity as IComposite; + + composite?.Setup(); ReadSyntheticInputs(activityDescriptor, activity, activityElement, context.SerializerOptions); ReadSyntheticOutputs(activityDescriptor, activity, activityElement); From f9f8f05670533340c7026b220bf5838bf26f9280 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 20 Sep 2024 15:51:31 +0200 Subject: [PATCH 03/31] Improve exception messages for activity descriptor lookups Enhanced exception messages to include activity type and version for better debugging. This provides clearer context when activity descriptors or inputs are not found. --- .../Extensions/ActivityExecutionContextExtensions.cs | 4 ++-- .../Elsa.Workflows.Core/Services/IdentityGraphService.cs | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs b/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs index ae6f936d1..5c18306fc 100644 --- a/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs @@ -142,11 +142,11 @@ public static class ActivityExecutionContextExtensions { var activity = context.Activity; var activityRegistryLookup = context.GetRequiredService(); - var activityDescriptor = await activityRegistryLookup.FindAsync(activity.Type) ?? throw new Exception("Activity descriptor not found"); + var activityDescriptor = await activityRegistryLookup.FindAsync(activity.Type) ?? throw new Exception($"Activity descriptor \"{activity.Type}\" not found"); var inputDescriptor = activityDescriptor.GetWrappedInputPropertyDescriptor(activity, inputName); if (inputDescriptor == null) - throw new Exception($"No input with name {inputName} could be found"); + throw new Exception($"No input with name \"{inputName}\" could be found"); return await EvaluateInputPropertyAsync(context, activityDescriptor, inputDescriptor); } diff --git a/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs b/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs index e46cfd17c..c3802b3f1 100644 --- a/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs +++ b/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs @@ -56,7 +56,7 @@ public class IdentityGraphService : IIdentityGraphService /// public async Task AssignInputOutputsAsync(IActivity activity) { - var activityDescriptor = await _activityRegistryLookup.FindAsync(activity.Type, activity.Version) ?? throw new Exception("Activity descriptor not found"); + var activityDescriptor = await _activityRegistryLookup.FindAsync(activity.Type, activity.Version) ?? throw new Exception($"Activity descriptor \"{activity.Type}\" with version \"{activity.Version}\" not found"); var inputDictionary = activityDescriptor.GetWrappedInputProperties(activity); foreach (var (inputName, input) in inputDictionary) From 53172cded9c6f902a652ae762209bccd0e4612bb Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 20 Sep 2024 15:59:35 +0200 Subject: [PATCH 04/31] Refactor IdentityGraphService constructor and add logging. Replaced private fields with constructor parameters in IdentityGraphService and introduced a logger. Added logging to handle cases where activity descriptors are not found, improving debugging and maintainability. --- .../Services/IdentityGraphService.cs | 26 ++++++++----------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs b/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs index c3802b3f1..14d259421 100644 --- a/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs +++ b/src/modules/Elsa.Workflows.Core/Services/IdentityGraphService.cs @@ -3,24 +3,13 @@ using Elsa.Workflows.Activities; using Elsa.Workflows.Contracts; using Elsa.Workflows.Models; using Humanizer; +using Microsoft.Extensions.Logging; namespace Elsa.Workflows.Services; /// -public class IdentityGraphService : IIdentityGraphService +public class IdentityGraphService(IActivityVisitor activityVisitor, IActivityRegistryLookupService activityRegistryLookup, ILogger logger) : IIdentityGraphService { - private readonly IActivityVisitor _activityVisitor; - private readonly IActivityRegistryLookupService _activityRegistryLookup; - - /// - /// Constructor. - /// - public IdentityGraphService(IActivityVisitor activityVisitor, IActivityRegistryLookupService activityRegistryLookup) - { - _activityVisitor = activityVisitor; - _activityRegistryLookup = activityRegistryLookup; - } - /// public async Task AssignIdentitiesAsync(Workflow workflow, CancellationToken cancellationToken = default) { @@ -30,7 +19,7 @@ public class IdentityGraphService : IIdentityGraphService /// public async Task AssignIdentitiesAsync(IActivity root, CancellationToken cancellationToken = default) { - var graph = await _activityVisitor.VisitAsync(root, cancellationToken); + var graph = await activityVisitor.VisitAsync(root, cancellationToken); await AssignIdentitiesAsync(graph); } @@ -56,7 +45,14 @@ public class IdentityGraphService : IIdentityGraphService /// public async Task AssignInputOutputsAsync(IActivity activity) { - var activityDescriptor = await _activityRegistryLookup.FindAsync(activity.Type, activity.Version) ?? throw new Exception($"Activity descriptor \"{activity.Type}\" with version \"{activity.Version}\" not found"); + var activityDescriptor = await activityRegistryLookup.FindAsync(activity.Type, activity.Version); + + if (activityDescriptor == null!) + { + logger.LogWarning("Activity descriptor not found for activity type {ActivityType}. Skipping identity assignment", activity.Type); + return; + } + var inputDictionary = activityDescriptor.GetWrappedInputProperties(activity); foreach (var (inputName, input) in inputDictionary) From 79b5bfc378762af2429988d17891328325de7600 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Mon, 23 Sep 2024 11:35:43 +0200 Subject: [PATCH 05/31] Add OpenTelemetry support for tracing workflow execution This commit introduces the Elsa.OpenTelemetry module to provide OpenTelemetry sources for tracing workflow and activity execution. Additionally, it updates the program to include default workflow and activity execution pipelines with tracing middleware, and improves pipeline builder extensions to support middleware insertion. --- .github/workflows/elsa-server-and-studio.yml | 4 +- .github/workflows/elsa-server.yml | 4 +- .github/workflows/elsa-studio.yml | 4 +- .github/workflows/packages.yml | 15 +----- Elsa.sln | 7 +++ .../Elsa.Server.Web/Elsa.Server.Web.csproj | 1 + src/bundles/Elsa.Server.Web/Program.cs | 6 +++ .../Elsa.OpenTelemetry.csproj | 14 ++++++ .../Features/OpenTelemetryFeature.cs | 11 +++++ .../Elsa.OpenTelemetry/FodyWeavers.xml | 3 ++ .../Helpers/OpenTelemetryHelpers.cs | 8 +++ ...metryTracingActivityExecutionMiddleware.cs | 42 ++++++++++++++++ ...metryTracingWorkflowExecutionMiddleware.cs | 49 +++++++++++++++++++ .../IActivityExecutionPipelineBuilder.cs | 3 ++ .../IWorkflowExecutionPipelineBuilder.cs | 3 ++ .../ActivityExecutionMiddlewareExtensions.cs | 26 ++++++++-- ...ctivityExecutionPipelinePipelineBuilder.cs | 7 +++ .../WorkflowExecutionMiddlewareExtensions.cs | 22 ++++----- .../WorkflowExecutionPipelineBuilder.cs | 8 +++ .../PipelineWorkflowsFeatureExtensions.cs | 35 +++++++++++++ .../Extensions/WorkflowsFeatureExtensions.cs | 1 + 21 files changed, 238 insertions(+), 35 deletions(-) create mode 100644 src/modules/Elsa.OpenTelemetry/Elsa.OpenTelemetry.csproj create mode 100644 src/modules/Elsa.OpenTelemetry/Features/OpenTelemetryFeature.cs create mode 100644 src/modules/Elsa.OpenTelemetry/FodyWeavers.xml create mode 100644 src/modules/Elsa.OpenTelemetry/Helpers/OpenTelemetryHelpers.cs create mode 100644 src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs create mode 100644 src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs create mode 100644 src/modules/Elsa.Workflows.Runtime/Extensions/PipelineWorkflowsFeatureExtensions.cs diff --git a/.github/workflows/elsa-server-and-studio.yml b/.github/workflows/elsa-server-and-studio.yml index 8f58dbcc2..cbca2daf3 100644 --- a/.github/workflows/elsa-server-and-studio.yml +++ b/.github/workflows/elsa-server-and-studio.yml @@ -3,7 +3,7 @@ on: workflow_dispatch: push: branches: - - patch/3.2.x + - blueberry jobs: push_to_registry: @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-server-and-studio-v3-2-1-preview + elsaworkflows/elsa-server-and-studio-v3-2-1-blueberry flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/elsa-server.yml b/.github/workflows/elsa-server.yml index 82a92a31b..e305d5072 100644 --- a/.github/workflows/elsa-server.yml +++ b/.github/workflows/elsa-server.yml @@ -3,7 +3,7 @@ on: workflow_dispatch: push: branches: - - patch/3.2.x + - blueberry jobs: push_to_registry: @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-server-v3-2-1-preview + elsaworkflows/elsa-server-v3-2-1-blueberry flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/elsa-studio.yml b/.github/workflows/elsa-studio.yml index a8673a0dd..a275af27d 100644 --- a/.github/workflows/elsa-studio.yml +++ b/.github/workflows/elsa-studio.yml @@ -3,7 +3,7 @@ on: workflow_dispatch: push: branches: - - patch/3.2.x + - blueberry jobs: push_to_registry: @@ -29,7 +29,7 @@ jobs: with: # list of Docker images to use as base name for tags images: | - elsaworkflows/elsa-studio-v3-2-1-preview + elsaworkflows/elsa-studio-v3-2-1-blueberry flavor: | latest=true # generate Docker tags based on the following events/attributes diff --git a/.github/workflows/packages.yml b/.github/workflows/packages.yml index c4461693c..2869399e3 100644 --- a/.github/workflows/packages.yml +++ b/.github/workflows/packages.yml @@ -3,18 +3,7 @@ on: workflow_dispatch: push: branches: - - 'main' - - 'feature/*' - - 'feat/*' - - 'issue/*' - - 'bug/*' - - 'enhancement/*' - - 'enh/*' - - 'patch/*' - - 'fix/*' - - 'perf/*' - - 'hotfix/*' - - 'chore/*' + - 'blueberry' release: types: [ prereleased, published ] env: @@ -62,7 +51,7 @@ jobs: TAG_NAME=${TAG_NAME#refs/tags/} # remove the refs/tags/ prefix echo "VERSION=${TAG_NAME}" >> $GITHUB_ENV else - echo "VERSION=3.2.1-preview.${{github.run_number}}" >> $GITHUB_ENV + echo "VERSION=3.2.1-blueberry.${{github.run_number}}" >> $GITHUB_ENV fi - name: Set up JDK 17 uses: actions/setup-java@v2 diff --git a/Elsa.sln b/Elsa.sln index 3d9bca0c0..58f882779 100644 --- a/Elsa.sln +++ b/Elsa.sln @@ -355,6 +355,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elsa.Workflows.PerformanceT EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elsa.MongoDb.UnitTests", "test\unit\Elsa.MongoDb.UnitTests\Elsa.MongoDb.UnitTests.csproj", "{56CAA9F2-1882-4EFA-BAC0-9C3D804553F1}" EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elsa.OpenTelemetry", "src\modules\Elsa.OpenTelemetry\Elsa.OpenTelemetry.csproj", "{25880971-403E-4872-93A1-D33089E07C91}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -844,6 +846,10 @@ Global {56CAA9F2-1882-4EFA-BAC0-9C3D804553F1}.Debug|Any CPU.Build.0 = Debug|Any CPU {56CAA9F2-1882-4EFA-BAC0-9C3D804553F1}.Release|Any CPU.ActiveCfg = Release|Any CPU {56CAA9F2-1882-4EFA-BAC0-9C3D804553F1}.Release|Any CPU.Build.0 = Release|Any CPU + {25880971-403E-4872-93A1-D33089E07C91}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {25880971-403E-4872-93A1-D33089E07C91}.Debug|Any CPU.Build.0 = Debug|Any CPU + {25880971-403E-4872-93A1-D33089E07C91}.Release|Any CPU.ActiveCfg = Release|Any CPU + {25880971-403E-4872-93A1-D33089E07C91}.Release|Any CPU.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE @@ -991,6 +997,7 @@ Global {CBB515F3-A0EF-43B5-A907-FD4E652DD66E} = {90031D64-CA0F-46D0-9AF4-8DC023A5FFCD} {90CD37A9-C866-4D90-A3B1-8C87F53B845E} = {CBB515F3-A0EF-43B5-A907-FD4E652DD66E} {56CAA9F2-1882-4EFA-BAC0-9C3D804553F1} = {18453B51-25EB-4317-A4B3-B10518252E92} + {25880971-403E-4872-93A1-D33089E07C91} = {5BA4A8FA-F7F4-45B3-AEC8-8886D35AAC79} EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {D4B5CEAA-7D70-4FCB-A68E-B03FBE5E0E5E} diff --git a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj index 71cb87c96..ff8b01a9d 100644 --- a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj +++ b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj @@ -12,6 +12,7 @@ + diff --git a/src/bundles/Elsa.Server.Web/Program.cs b/src/bundles/Elsa.Server.Web/Program.cs index 33328dace..d72e04aca 100644 --- a/src/bundles/Elsa.Server.Web/Program.cs +++ b/src/bundles/Elsa.Server.Web/Program.cs @@ -20,6 +20,7 @@ using Elsa.MongoDb.Modules.Alterations; using Elsa.MongoDb.Modules.Identity; using Elsa.MongoDb.Modules.Management; using Elsa.MongoDb.Modules.Runtime; +using Elsa.OpenTelemetry.Middleware; using Elsa.Server.Web; using Elsa.Workflows; using Elsa.Workflows.Management.Compression; @@ -131,6 +132,11 @@ services identity.UseConfigurationBasedRoleProvider(options => identitySection.Bind(options)); }) .UseDefaultAuthentication() + .UseWorkflows(workflows => + { + workflows.WithDefaultWorkflowExecutionPipeline(pipeline => pipeline.UseWorkflowExecutionTracing()); + workflows.WithDefaultActivityExecutionPipeline(pipeline => pipeline.UseActivityExecutionTracing()); + }) .UseWorkflowManagement(management => { if (useMongoDb) diff --git a/src/modules/Elsa.OpenTelemetry/Elsa.OpenTelemetry.csproj b/src/modules/Elsa.OpenTelemetry/Elsa.OpenTelemetry.csproj new file mode 100644 index 000000000..dc06e309b --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/Elsa.OpenTelemetry.csproj @@ -0,0 +1,14 @@ + + + + + Provides OpenTelemetry sources for tracing workflow and activity execution. + + elsa module open-telemetry + + + + + + + \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/Features/OpenTelemetryFeature.cs b/src/modules/Elsa.OpenTelemetry/Features/OpenTelemetryFeature.cs new file mode 100644 index 000000000..67201c44d --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/Features/OpenTelemetryFeature.cs @@ -0,0 +1,11 @@ +using Elsa.Features.Abstractions; +using Elsa.Features.Services; + +namespace Elsa.OpenTelemetry.Features; + +public class OpenTelemetryFeature(IModule module) : FeatureBase(module) +{ + public override void Configure() + { + } +} \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/FodyWeavers.xml b/src/modules/Elsa.OpenTelemetry/FodyWeavers.xml new file mode 100644 index 000000000..00e1d9a1c --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/FodyWeavers.xml @@ -0,0 +1,3 @@ + + + \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/Helpers/OpenTelemetryHelpers.cs b/src/modules/Elsa.OpenTelemetry/Helpers/OpenTelemetryHelpers.cs new file mode 100644 index 000000000..302659456 --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/Helpers/OpenTelemetryHelpers.cs @@ -0,0 +1,8 @@ +using System.Diagnostics; + +namespace Elsa.OpenTelemetry.Helpers; + +public class ElsaOpenTelemetry +{ + public static readonly ActivitySource ActivitySource = new("Elsa.Workflows"); +} \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs new file mode 100644 index 000000000..cd389bdb7 --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs @@ -0,0 +1,42 @@ +using System.Diagnostics; +using Elsa.OpenTelemetry.Helpers; +using Elsa.Workflows; +using Elsa.Workflows.Contracts; +using Elsa.Workflows.Pipelines.ActivityExecution; +using Elsa.Workflows.Pipelines.WorkflowExecution; +using JetBrains.Annotations; +using Activity = System.Diagnostics.Activity; +using ActivityKind = System.Diagnostics.ActivityKind; + +namespace Elsa.OpenTelemetry.Middleware; + +/// +[UsedImplicitly] +public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareDelegate next) : IActivityExecutionMiddleware +{ + /// + public async ValueTask InvokeAsync(ActivityExecutionContext context) + { + var activity = context.Activity; + using var span = ElsaOpenTelemetry.ActivitySource.StartActivity($"ActivityExecution {context.ActivityDescriptor.TypeName}", ActivityKind.Internal, Activity.Current?.Context ?? default); + span?.AddTag("activity.nodeId", activity.NodeId); + span?.AddTag("activity.type", activity.Type); + span?.AddTag("activity.name", activity.Name); + span?.AddTag("activityInstance.id", context.Id); + span?.AddTag("activityInstance.originalStatus", context.Status.ToString()); + span?.AddEvent(new ActivityEvent("Executing")); + await next(context); + span?.AddEvent(new ActivityEvent("Executed")); + span?.AddTag("activityInstance.newStatus", context.Status.ToString()); + } +} + +/// +/// Contains extension methods for . +/// +[UsedImplicitly] +public static class OpenTelemetryTracingActivityExecutionMiddlewareExtensions +{ + /// Installs the component in the workflow execution pipeline. + public static IActivityExecutionPipelineBuilder UseActivityExecutionTracing(this IActivityExecutionPipelineBuilder pipelineBuilder) => pipelineBuilder.Insert(0); +} diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs new file mode 100644 index 000000000..ae6d67572 --- /dev/null +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -0,0 +1,49 @@ +using System.Diagnostics; +using Elsa.OpenTelemetry.Helpers; +using Elsa.Workflows; +using Elsa.Workflows.Contracts; +using Elsa.Workflows.Pipelines.WorkflowExecution; +using JetBrains.Annotations; +using Activity = System.Diagnostics.Activity; +using ActivityKind = System.Diagnostics.ActivityKind; + +namespace Elsa.OpenTelemetry.Middleware; + +/// +/// Middleware that traces workflow execution using OpenTelemetry. +/// +[UsedImplicitly] +public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareDelegate next) : WorkflowExecutionMiddleware(next) +{ + /// + public override async ValueTask InvokeAsync(WorkflowExecutionContext context) + { + var workflowInstanceId = context.Id; + var workflow = context.Workflow; + using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity($"WorkflowExecution {workflow.WorkflowMetadata.Name}", ActivityKind.Internal, Activity.Current?.Context ?? default); + + if(!string.IsNullOrWhiteSpace(context.CorrelationId)) + activity?.AddTag("correlationId", context.CorrelationId); + + activity?.AddTag("workflowInstance.id", workflowInstanceId); + activity?.AddTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); + activity?.AddTag("workflowDefinition.version", workflow.Identity.Version); + activity?.AddTag("workflowInstance.originalStatus", context.Status.ToString()); + activity?.AddTag("workflowInstance.originalSubStatus", context.SubStatus.ToString()); + activity?.AddEvent(new ActivityEvent("Executing")); + await Next(context); + activity?.AddEvent(new ActivityEvent("Executed")); + activity?.AddTag("workflowInstance.newStatus", context.Status.ToString()); + activity?.AddTag("workflowInstance.newSubStatus", context.SubStatus.ToString()); + } +} + +/// +/// Contains extension methods for . +/// +[UsedImplicitly] +public static class OpenTelemetryWorkflowExecutionMiddlewareExtensions +{ + /// Installs the component in the workflow execution pipeline. + public static IWorkflowExecutionPipelineBuilder UseWorkflowExecutionTracing(this IWorkflowExecutionPipelineBuilder pipelineBuilder) => pipelineBuilder.Insert(0); +} diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IActivityExecutionPipelineBuilder.cs b/src/modules/Elsa.Workflows.Core/Contracts/IActivityExecutionPipelineBuilder.cs index 7c7968451..6cf6c926d 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/IActivityExecutionPipelineBuilder.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/IActivityExecutionPipelineBuilder.cs @@ -19,6 +19,9 @@ public interface IActivityExecutionPipelineBuilder /// The current . IActivityExecutionPipelineBuilder Use(Func middleware); + /// Inserts the middleware component at the specified index. + IActivityExecutionPipelineBuilder Insert(int index, Func middleware); + /// /// Constructs the final delegate that invokes each installed middleware component. /// diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowExecutionPipelineBuilder.cs b/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowExecutionPipelineBuilder.cs index a2f37c9b0..04ce5ff62 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowExecutionPipelineBuilder.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowExecutionPipelineBuilder.cs @@ -36,6 +36,9 @@ public interface IWorkflowExecutionPipelineBuilder /// Clears the current pipeline. /// IWorkflowExecutionPipelineBuilder Reset(); + + /// Inserts the middleware component at the specified index. + IWorkflowExecutionPipelineBuilder Insert(int index, Func middleware); /// /// Replaces the middleware component at the specified index with the specified delegate. diff --git a/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionMiddlewareExtensions.cs b/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionMiddlewareExtensions.cs index 487b41340..c42152859 100644 --- a/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionMiddlewareExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionMiddlewareExtensions.cs @@ -1,3 +1,4 @@ +using System.Diagnostics.CodeAnalysis; using Elsa.Workflows.Contracts; using Microsoft.Extensions.DependencyInjection; @@ -5,16 +6,33 @@ namespace Elsa.Workflows.Pipelines.ActivityExecution; public static class ActivityExecutionMiddlewareExtensions { - public static IActivityExecutionPipelineBuilder UseMiddleware(this IActivityExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IActivityExecutionMiddleware + public static IActivityExecutionPipelineBuilder UseMiddleware<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>(this IActivityExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IActivityExecutionMiddleware + { + var delegateFactory = CreateMiddlewareDelegateFactory(pipelineBuilder, args); + return pipelineBuilder.Use(delegateFactory); + } + + public static IActivityExecutionPipelineBuilder Insert<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)]TMiddleware>(this IActivityExecutionPipelineBuilder pipelineBuilder, int index, params object[] args) where TMiddleware : IActivityExecutionMiddleware + { + var delegateFactory = CreateMiddlewareDelegateFactory(pipelineBuilder, args); + return pipelineBuilder.Insert(index, delegateFactory); + } + + /// Creates a middleware delegate for the specified middleware component. + public static Func CreateMiddlewareDelegateFactory<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( + this IActivityExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IActivityExecutionMiddleware { var middleware = typeof(TMiddleware); - return pipelineBuilder.Use(next => + return next => { var invokeMethod = MiddlewareHelpers.GetInvokeMethod(middleware); - var ctorArgs = new[] { next }.Concat(args).Select(x => x!).ToArray(); + var ctorArgs = new[] + { + next + }.Concat(args).Select(x => x).ToArray(); var instance = ActivatorUtilities.CreateInstance(pipelineBuilder.ServiceProvider, middleware, ctorArgs); return (ActivityMiddlewareDelegate)invokeMethod.CreateDelegate(typeof(ActivityMiddlewareDelegate), instance); - }); + }; } } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionPipelinePipelineBuilder.cs b/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionPipelinePipelineBuilder.cs index 971ebfbca..b64289f03 100644 --- a/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionPipelinePipelineBuilder.cs +++ b/src/modules/Elsa.Workflows.Core/Pipelines/ActivityExecution/ActivityExecutionPipelinePipelineBuilder.cs @@ -25,6 +25,13 @@ public class ActivityExecutionPipelinePipelineBuilder : IActivityExecutionPipeli return this; } + /// + public IActivityExecutionPipelineBuilder Insert(int index, Func middleware) + { + _components.Insert(index, middleware); + return this; + } + /// public ActivityMiddlewareDelegate Build() { diff --git a/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionMiddlewareExtensions.cs b/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionMiddlewareExtensions.cs index 4e7221c11..4007c8a16 100644 --- a/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionMiddlewareExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionMiddlewareExtensions.cs @@ -4,44 +4,42 @@ using Microsoft.Extensions.DependencyInjection; namespace Elsa.Workflows.Pipelines.WorkflowExecution; -/// /// Provides extensions to that adds support for installing components. -/// public static class WorkflowExecutionMiddlewareExtensions { - /// /// Installs the specified middleware component into the pipeline being built. - /// public static IWorkflowExecutionPipelineBuilder UseMiddleware<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( this IWorkflowExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IWorkflowExecutionMiddleware { var delegateFactory = CreateMiddlewareDelegateFactory(pipelineBuilder, args); return pipelineBuilder.Use(delegateFactory); } + + /// Installs the specified middleware component into the pipeline being built. + public static IWorkflowExecutionPipelineBuilder Insert<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( + this IWorkflowExecutionPipelineBuilder pipelineBuilder, int index, params object[] args) where TMiddleware : IWorkflowExecutionMiddleware + { + var delegateFactory = CreateMiddlewareDelegateFactory(pipelineBuilder, args); + return pipelineBuilder.Insert(index, delegateFactory); + } - /// /// Replaces the terminal middleware component with the specified middleware component. - /// public static IWorkflowExecutionPipelineBuilder ReplaceTerminal<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( this IWorkflowExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IWorkflowExecutionMiddleware { var index = pipelineBuilder.Components.Count() - 1; return pipelineBuilder.Replace(index, args); } - - /// + /// Replaces the middleware component at the specified index with the specified middleware component. - /// public static IWorkflowExecutionPipelineBuilder Replace<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( this IWorkflowExecutionPipelineBuilder pipelineBuilder, int index, params object[] args) where TMiddleware : IWorkflowExecutionMiddleware { var delegateFactory = CreateMiddlewareDelegateFactory(pipelineBuilder, args); return pipelineBuilder.Replace(index, delegateFactory); } - - /// + /// Creates a middleware delegate for the specified middleware component. - /// public static Func CreateMiddlewareDelegateFactory<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicConstructors)] TMiddleware>( this IWorkflowExecutionPipelineBuilder pipelineBuilder, params object[] args) where TMiddleware : IWorkflowExecutionMiddleware { diff --git a/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionPipelineBuilder.cs b/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionPipelineBuilder.cs index c61c6c4e5..940461e8f 100644 --- a/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionPipelineBuilder.cs +++ b/src/modules/Elsa.Workflows.Core/Pipelines/WorkflowExecution/WorkflowExecutionPipelineBuilder.cs @@ -54,6 +54,14 @@ public class WorkflowExecutionPipelineBuilder : IWorkflowExecutionPipelineBuilde return this; } + /// + public IWorkflowExecutionPipelineBuilder Insert(int index, Func middleware) + { + _components.Insert(index, middleware); + return this; + } + + /// public IWorkflowExecutionPipelineBuilder Replace(int index, Func middleware) { _components[index] = middleware; diff --git a/src/modules/Elsa.Workflows.Runtime/Extensions/PipelineWorkflowsFeatureExtensions.cs b/src/modules/Elsa.Workflows.Runtime/Extensions/PipelineWorkflowsFeatureExtensions.cs new file mode 100644 index 000000000..b68a86779 --- /dev/null +++ b/src/modules/Elsa.Workflows.Runtime/Extensions/PipelineWorkflowsFeatureExtensions.cs @@ -0,0 +1,35 @@ +using Elsa.Workflows.Contracts; +using Elsa.Workflows.Features; +using Elsa.Workflows.Middleware.Activities; + +// ReSharper disable once CheckNamespace +namespace Elsa.Extensions; + +/// Adds an extension method to the that installs a default workflow runtime execution pipeline. +public static class PipelineWorkflowsFeatureExtensions +{ + /// Installs a default workflow runtime execution pipeline. + public static WorkflowsFeature WithDefaultWorkflowExecutionPipeline(this WorkflowsFeature workflowsFeature, Action? configurePipeline = null) + { + return workflowsFeature.WithWorkflowExecutionPipeline(pipeline => + { + pipeline.UseDefaultPipeline(); + configurePipeline?.Invoke(pipeline); + }); + } + + /// Installs an activity invoker that can run activities asynchronously in the background. + public static WorkflowsFeature WithDefaultActivityExecutionPipeline(this WorkflowsFeature workflowsFeature, Action? configurePipeline = null) + { + return workflowsFeature.WithActivityExecutionPipeline(pipeline => + { + pipeline + .UseExceptionHandling() + .UseExecutionLogging() + .UseNotifications() + .UseBackgroundActivityInvoker(); + + configurePipeline?.Invoke(pipeline); + }); + } +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Runtime/Extensions/WorkflowsFeatureExtensions.cs b/src/modules/Elsa.Workflows.Runtime/Extensions/WorkflowsFeatureExtensions.cs index 9d8bda420..7031ca4a1 100644 --- a/src/modules/Elsa.Workflows.Runtime/Extensions/WorkflowsFeatureExtensions.cs +++ b/src/modules/Elsa.Workflows.Runtime/Extensions/WorkflowsFeatureExtensions.cs @@ -6,6 +6,7 @@ using Elsa.Workflows.Middleware.Activities; // ReSharper disable once CheckNamespace using Elsa.Workflows.Features; +// ReSharper disable once CheckNamespace namespace Elsa.Extensions; /// From adf36927dcd19a4d8f579eb1b875c27e59cfe8d6 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Mon, 23 Sep 2024 23:35:30 +0200 Subject: [PATCH 06/31] Update Docker and k8s configs, enhance OTEL tracing Updated `docker-compose-datadog.yml` with new environment variables and updated image versions, enhancing trace sampling and service autodiscovery. Added Kubernetes deployment, service, and role files for `elsa-server`, `elsa-studio`, `plant-uml`, `postgres`, and `trace-lens`, improving the project's infrastructure. Enhanced OpenTelemetry middleware for better error handling and detailed tracing of activity and workflow executions. --- Elsa.sln | 43 +++++++++++++ docker/ElsaServer-Datadog.Dockerfile | 4 +- docker/docker-compose-datadog.yml | 63 ++++++++++++++++-- scripts/k8s/elsa-server/deployment.yaml | 60 +++++++++++++++++ scripts/k8s/elsa-server/role-binding.yaml | 11 ++++ scripts/k8s/elsa-server/role.yaml | 14 ++++ scripts/k8s/elsa-server/service-account.yml | 5 ++ scripts/k8s/elsa-server/service.yaml | 12 ++++ scripts/k8s/elsa-studio/deployment.yaml | 25 ++++++++ scripts/k8s/elsa-studio/service.yaml | 12 ++++ scripts/k8s/plant-uml/deployment.yaml | 20 ++++++ scripts/k8s/postgres/config-map.yaml | 21 ++++++ scripts/k8s/postgres/deployment.yaml | 41 ++++++++++++ scripts/k8s/postgres/service.yml | 13 ++++ scripts/k8s/trace-lens/deployment.yaml | 26 ++++++++ scripts/k8s/trace-lens/service.yaml | 25 ++++++++ ...metryTracingActivityExecutionMiddleware.cs | 54 ++++++++++++---- ...metryTracingWorkflowExecutionMiddleware.cs | 64 ++++++++++++++----- 18 files changed, 478 insertions(+), 35 deletions(-) create mode 100644 scripts/k8s/elsa-server/deployment.yaml create mode 100644 scripts/k8s/elsa-server/role-binding.yaml create mode 100644 scripts/k8s/elsa-server/role.yaml create mode 100644 scripts/k8s/elsa-server/service-account.yml create mode 100644 scripts/k8s/elsa-server/service.yaml create mode 100644 scripts/k8s/elsa-studio/deployment.yaml create mode 100644 scripts/k8s/elsa-studio/service.yaml create mode 100644 scripts/k8s/plant-uml/deployment.yaml create mode 100644 scripts/k8s/postgres/config-map.yaml create mode 100644 scripts/k8s/postgres/deployment.yaml create mode 100644 scripts/k8s/postgres/service.yml create mode 100644 scripts/k8s/trace-lens/deployment.yaml create mode 100644 scripts/k8s/trace-lens/service.yaml diff --git a/Elsa.sln b/Elsa.sln index 58f882779..8e4c08bb0 100644 --- a/Elsa.sln +++ b/Elsa.sln @@ -357,6 +357,43 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elsa.MongoDb.UnitTests", "t EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Elsa.OpenTelemetry", "src\modules\Elsa.OpenTelemetry\Elsa.OpenTelemetry.csproj", "{25880971-403E-4872-93A1-D33089E07C91}" EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "scripts", "scripts", "{FE1AFEC0-7C63-4EF8-8E24-D9703590A778}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "k8s", "k8s", "{924BA0AE-46B1-40B5-992C-CFB4EB14EDB0}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "elsa-server", "elsa-server", "{A2743EC6-7117-46E2-8AC1-8CDD52C8AAEF}" + ProjectSection(SolutionItems) = preProject + scripts\k8s\elsa-server\deployment.yaml = scripts\k8s\elsa-server\deployment.yaml + scripts\k8s\elsa-server\role-binding.yaml = scripts\k8s\elsa-server\role-binding.yaml + scripts\k8s\elsa-server\role.yaml = scripts\k8s\elsa-server\role.yaml + scripts\k8s\elsa-server\service-account.yml = scripts\k8s\elsa-server\service-account.yml + scripts\k8s\elsa-server\service.yaml = scripts\k8s\elsa-server\service.yaml + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "elsa-studio", "elsa-studio", "{A782913B-142F-4EA8-A2C7-82679CD2ABC1}" + ProjectSection(SolutionItems) = preProject + scripts\k8s\elsa-studio\deployment.yaml = scripts\k8s\elsa-studio\deployment.yaml + scripts\k8s\elsa-studio\service.yaml = scripts\k8s\elsa-studio\service.yaml + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "plant-uml", "plant-uml", "{BFB87B50-3FE9-4AB1-8F18-96BE667C953D}" + ProjectSection(SolutionItems) = preProject + scripts\k8s\plant-uml\deployment.yaml = scripts\k8s\plant-uml\deployment.yaml + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "postgres", "postgres", "{399C61BC-A51B-4B57-AA32-46BC24D4AE1C}" + ProjectSection(SolutionItems) = preProject + scripts\k8s\postgres\config-map.yaml = scripts\k8s\postgres\config-map.yaml + scripts\k8s\postgres\deployment.yaml = scripts\k8s\postgres\deployment.yaml + scripts\k8s\postgres\service.yml = scripts\k8s\postgres\service.yml + EndProjectSection +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "trace-lens", "trace-lens", "{2FDF67FD-2BD3-4F73-9A24-DA255641ED25}" + ProjectSection(SolutionItems) = preProject + scripts\k8s\trace-lens\deployment.yaml = scripts\k8s\trace-lens\deployment.yaml + scripts\k8s\trace-lens\service.yaml = scripts\k8s\trace-lens\service.yaml + EndProjectSection +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -998,6 +1035,12 @@ Global {90CD37A9-C866-4D90-A3B1-8C87F53B845E} = {CBB515F3-A0EF-43B5-A907-FD4E652DD66E} {56CAA9F2-1882-4EFA-BAC0-9C3D804553F1} = {18453B51-25EB-4317-A4B3-B10518252E92} {25880971-403E-4872-93A1-D33089E07C91} = {5BA4A8FA-F7F4-45B3-AEC8-8886D35AAC79} + {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} = {FE1AFEC0-7C63-4EF8-8E24-D9703590A778} + {A2743EC6-7117-46E2-8AC1-8CDD52C8AAEF} = {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} + {A782913B-142F-4EA8-A2C7-82679CD2ABC1} = {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} + {BFB87B50-3FE9-4AB1-8F18-96BE667C953D} = {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} + {399C61BC-A51B-4B57-AA32-46BC24D4AE1C} = {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} + {2FDF67FD-2BD3-4F73-9A24-DA255641ED25} = {924BA0AE-46B1-40B5-992C-CFB4EB14EDB0} EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {D4B5CEAA-7D70-4FCB-A68E-B03FBE5E0E5E} diff --git a/docker/ElsaServer-Datadog.Dockerfile b/docker/ElsaServer-Datadog.Dockerfile index bd02ed1e8..15e799a1a 100644 --- a/docker/ElsaServer-Datadog.Dockerfile +++ b/docker/ElsaServer-Datadog.Dockerfile @@ -18,7 +18,7 @@ COPY *.props ./ RUN dotnet restore "./src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj" # build and publish (UseAppHost=false creates platform independent binaries). -WORKDIR /source/src/bundles/Elsa.Server.Web +WORKDIR /source/src/apps/Elsa.Server.Web RUN dotnet build "Elsa.Server.Web.csproj" -c Release -o /app/build RUN dotnet publish "Elsa.Server.Web.csproj" -c Release -o /app/publish /p:UseAppHost=false --no-restore -f net8.0 @@ -36,7 +36,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ rm -rf /var/lib/apt/lists/* # Set PYTHONNET_PYDLL environment variable -ENV PYTHONNET_PYDLL /usr/lib/aarch64-linux-gnu/libpython3.11.so +ENV PYTHONNET_PYDLL=/usr/lib/aarch64-linux-gnu/libpython3.11.so # Copy the tracer from build target COPY --from=build /tmp/datadog-dotnet-apm.deb /tmp/datadog-dotnet-apm.deb diff --git a/docker/docker-compose-datadog.yml b/docker/docker-compose-datadog.yml index 36fb7952f..d92ad76ed 100644 --- a/docker/docker-compose-datadog.yml +++ b/docker/docker-compose-datadog.yml @@ -12,7 +12,7 @@ services: - postgres-data:/var/lib/postgresql/data ports: - "5432:5432" - + cockroachdb: image: cockroachdb/cockroach:v22.1.0 command: start-single-node --insecure @@ -23,19 +23,20 @@ services: - cockroachdb-data:/cockroach/cockroach-data environment: - COCKROACH_DATABASE=elsa - + rabbitmq: image: "rabbitmq:3-management" ports: - "15672:15672" - "5672:5672" - + redis: image: redis:latest ports: - "127.0.0.1:6379:6379" elsa-server: + pull_policy: always build: context: ../. dockerfile: ./docker/ElsaServer-Datadog.Dockerfile @@ -46,6 +47,11 @@ services: - datadog-agent environment: DD_AGENT_HOST: datadog-agent + DD_ENV: development + DD_TRACE_DEBUG: true + DD_TRACE_OTEL_ENABLED: true + DD_SERVICE: "elsa-server-local" + DD_VERSION: "3.2.1-blueberry" ASPNETCORE_ENVIRONMENT: Development PYTHONNET_PYDLL: /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11 PYTHONNET_RUNTIME: coreclr @@ -55,11 +61,12 @@ services: DISTRIBUTEDLOCKPROVIDER: "Postgres" ports: - "13000:8080" - + elsa-studio: + pull_policy: always build: context: ../. - dockerfile: ./docker/ElsaStudio-Datadog.Dockerfile + dockerfile: ./docker/ElsaStudio.Dockerfile environment: ASPNETCORE_ENVIRONMENT: Development ELSASERVER__URL: "http://localhost:13000/elsa/api" @@ -67,16 +74,60 @@ services: - "14000:8080" datadog-agent: - image: datadog/agent:7 + image: datadog/agent:7.42.0 environment: DD_API_KEY: "YOUR_API_KEY" DD_SITE: "datadoghq.eu" DD_LOGS_ENABLED: "true" DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL: "true" DD_APM_ENABLED: "true" + DD_REMOTE_CONFIGURATION_ENABLED: "true" DD_APM_NON_LOCAL_TRAFFIC: "true" + + # Service autodiscovery DD_AC_INCLUDE: "name:postgres,name:rabbitmq,name:redis,name:elsa-server" DD_AC_EXCLUDE: "name:datadog-agent" + + # Enable priority sampling + DD_TRACE_SAMPLING_PRIORITY: "true" + + # Global rate limiting of traces (number of spans per second) + DD_TRACE_RATE_LIMIT: 100 + + # Global sample rate for all traces (applies to spans that do not match a specific sampling rule) + DD_TRACE_SAMPLE_RATE: 1.0 # Keep 100% of the traces globally (adjust as needed) + + # Sampling rules for controlling sampling of specific services and errors + DD_TRACE_SAMPLING_RULES: > + [ + { + "service": "elsa-server-local", + "name": "WorkflowExecution", + "sample_rate": 0.5 + }, + { + "service": "elsa-server-local", + "name": "WorkflowExecution", + "sample_rate": 1.0, + "condition": {"tags": {"hasIncidents": "true"}} + } + { + "service": "elsa-server-local", + "name": "ActivityExecution", + "sample_rate": 1.0, + "condition": {"tags": {"hasIncidents": "true"}} + }, + { + "service": "elsa-server-local", + "name": "ActivityExecution", + "sample_rate": 0.3 + }, + { + "service": "elsa-server-local", + "sample_rate": 1.0, + "condition": {"error": true} + } + ] volumes: - /var/run/docker.sock:/var/run/docker.sock - /proc/:/host/proc/:ro diff --git a/scripts/k8s/elsa-server/deployment.yaml b/scripts/k8s/elsa-server/deployment.yaml new file mode 100644 index 000000000..22c4f48ef --- /dev/null +++ b/scripts/k8s/elsa-server/deployment.yaml @@ -0,0 +1,60 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: elsa-server-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: elsa-server + template: + metadata: + labels: + app: elsa-server + spec: + serviceAccountName: proto-cluster + containers: + - name: elsa-server + imagePullPolicy: Never + image: elsa-server:latest + ports: + - containerPort: 8080 + env: + - name: ASPNETCORE_ENVIRONMENT + value: Development + - name: "ProtoActor__AdvertisedHost" + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: CONNECTIONSTRINGS__POSTGRESQL + value: "Server=postgres;Username=elsa;Database=elsa;Port=5432;Password=elsa;SSLMode=Prefer;MaxPoolSize=2000;Timeout=60" + - name: COR_ENABLE_PROFILING + value: "1" + - name: COR_PROFILER + value: "{918728DD-259F-4A6A-AC2B-B85E1B658318}" + - name: CORECLR_PROFILER_PATH + value: "$INSTALL_DIR/osx-x64/OpenTelemetry.AutoInstrumentation.Native.dylib" + - name: DOTNET_ADDITIONAL_DEPS + value: "$INSTALL_DIR/AdditionalDeps" + - name: DOTNET_EnableDiagnostics + value: "1" + - name: DOTNET_SHARED_STORE + value: "$INSTALL_DIR/store" + - name: DOTNET_STARTUP_HOOKS + value: "OpenTelemetry.AutoInstrumentation.StartupHook" + - name: OTEL_DOTNET_AUTO_HOME + value: "$INSTALL_DIR" + - name: OTEL_DOTNET_AUTO_LOGS_CONSOLE_EXPORTER_ENABLED + value: "false" + - name: OTEL_DOTNET_AUTO_METRICS_CONSOLE_EXPORTER_ENABLED + value: "false" + - name: OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES + value: "Proto.Actor,Elsa.Workflows" + - name: OTEL_DOTNET_AUTO_TRACES_CONSOLE_EXPORTER_ENABLED + value: "false" + - name: OTEL_EXPORTER_OTLP_ENDPOINT + value: "http://trace-lens-collector:4317" + - name: OTEL_EXPORTER_OTLP_PROTOCOL + value: "grpc" + - name: OTEL_RESOURCE_ATTRIBUTES + value: "service.name=Elsa Server,service.version=3.3.0" \ No newline at end of file diff --git a/scripts/k8s/elsa-server/role-binding.yaml b/scripts/k8s/elsa-server/role-binding.yaml new file mode 100644 index 000000000..222766da6 --- /dev/null +++ b/scripts/k8s/elsa-server/role-binding.yaml @@ -0,0 +1,11 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: proto-cluster +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: proto-cluster +subjects: + - kind: ServiceAccount + name: proto-cluster # this is the service account that should have the role applied \ No newline at end of file diff --git a/scripts/k8s/elsa-server/role.yaml b/scripts/k8s/elsa-server/role.yaml new file mode 100644 index 000000000..51c2bf9ca --- /dev/null +++ b/scripts/k8s/elsa-server/role.yaml @@ -0,0 +1,14 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: proto-cluster +rules: + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + - patch \ No newline at end of file diff --git a/scripts/k8s/elsa-server/service-account.yml b/scripts/k8s/elsa-server/service-account.yml new file mode 100644 index 000000000..b639b7a85 --- /dev/null +++ b/scripts/k8s/elsa-server/service-account.yml @@ -0,0 +1,5 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: proto-cluster + namespace: default # Ensure the namespace matches your deployment \ No newline at end of file diff --git a/scripts/k8s/elsa-server/service.yaml b/scripts/k8s/elsa-server/service.yaml new file mode 100644 index 000000000..cf9ebc193 --- /dev/null +++ b/scripts/k8s/elsa-server/service.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Service +metadata: + name: elsa-server-service +spec: + type: LoadBalancer + sessionAffinity: None + ports: + - port: 8001 + targetPort: 8080 + selector: + app: elsa-server diff --git a/scripts/k8s/elsa-studio/deployment.yaml b/scripts/k8s/elsa-studio/deployment.yaml new file mode 100644 index 000000000..e56d84afb --- /dev/null +++ b/scripts/k8s/elsa-studio/deployment.yaml @@ -0,0 +1,25 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: elsa-studio-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: elsa-studio + template: + metadata: + labels: + app: elsa-studio + spec: + containers: + - name: elsa-studio + imagePullPolicy: Never + image: elsa-studio:latest + ports: + - containerPort: 8080 + env: + - name: ASPNETCORE_ENVIRONMENT + value: Development + - name: "ELSASERVER__URL" + value: "http://localhost:8001/elsa/api" \ No newline at end of file diff --git a/scripts/k8s/elsa-studio/service.yaml b/scripts/k8s/elsa-studio/service.yaml new file mode 100644 index 000000000..067c10fca --- /dev/null +++ b/scripts/k8s/elsa-studio/service.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Service +metadata: + name: elsa-studio-service +spec: + type: LoadBalancer + sessionAffinity: None + ports: + - port: 9001 + targetPort: 8080 + selector: + app: elsa-studio diff --git a/scripts/k8s/plant-uml/deployment.yaml b/scripts/k8s/plant-uml/deployment.yaml new file mode 100644 index 000000000..7c49679c4 --- /dev/null +++ b/scripts/k8s/plant-uml/deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: plant-uml-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: plant-uml + template: + metadata: + labels: + app: plant-uml + spec: + containers: + - name: plant-uml + imagePullPolicy: Always + image: plantuml/plantuml-server:tomcat + ports: + - containerPort: 8080 \ No newline at end of file diff --git a/scripts/k8s/postgres/config-map.yaml b/scripts/k8s/postgres/config-map.yaml new file mode 100644 index 000000000..a2ea928f6 --- /dev/null +++ b/scripts/k8s/postgres/config-map.yaml @@ -0,0 +1,21 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: init-db-config +data: + init-db.sh: | + #!/bin/bash + echo "Starting database initialization" + + psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL + GRANT ALL ON SCHEMA public TO elsa; + GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO elsa; + GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO elsa; + GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN SCHEMA public TO elsa; + CREATE USER tracelens WITH PASSWORD 'tracelenspass'; + CREATE DATABASE tracelens; + GRANT ALL PRIVILEGES ON DATABASE tracelens TO tracelens; + GRANT ALL ON SCHEMA public TO tracelens; + EOSQL + + echo "Database initialization completed" \ No newline at end of file diff --git a/scripts/k8s/postgres/deployment.yaml b/scripts/k8s/postgres/deployment.yaml new file mode 100644 index 000000000..4ec79128b --- /dev/null +++ b/scripts/k8s/postgres/deployment.yaml @@ -0,0 +1,41 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgres-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: postgres + template: + metadata: + labels: + app: postgres + spec: + containers: + - name: postgres + imagePullPolicy: Always + image: postgres:latest + # command: ["-c", "max_connections=2000"] + volumeMounts: + - name: postgres-data + mountPath: /var/lib/postgresql/data + - name: init-db + mountPath: /docker-entrypoint-initdb.d/init-db.sh + subPath: init-db.sh + ports: + - containerPort: 5432 + env: + - name: POSTGRES_USER + value: "elsa" + - name: POSTGRES_PASSWORD + value: "elsa" + - name: POSTGRES_DB + value: "elsa" + + volumes: + - name: postgres-data + emptyDir: {} + - name: init-db + configMap: + name: init-db-config \ No newline at end of file diff --git a/scripts/k8s/postgres/service.yml b/scripts/k8s/postgres/service.yml new file mode 100644 index 000000000..ce3bbc467 --- /dev/null +++ b/scripts/k8s/postgres/service.yml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: postgres + labels: + app: postgres +spec: + type: LoadBalancer + ports: + - port: 5432 + targetPort: 5432 + selector: + app: postgres \ No newline at end of file diff --git a/scripts/k8s/trace-lens/deployment.yaml b/scripts/k8s/trace-lens/deployment.yaml new file mode 100644 index 000000000..2586a152d --- /dev/null +++ b/scripts/k8s/trace-lens/deployment.yaml @@ -0,0 +1,26 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: tracelens-deployment +spec: + replicas: 1 + selector: + matchLabels: + app: trace-lens + template: + metadata: + labels: + app: trace-lens + spec: + containers: + - name: trace-lens + imagePullPolicy: Always + image: docker.io/rogeralsing/tracelens:latest + ports: + - containerPort: 5001 + - containerPort: 4317 + env: + - name: PlantUml__RemoteUrl + value: "" + - name: ConnectionStrings__DefaultConnection + value: "USER ID=tracelens;PASSWORD=tracelenspass;HOST=postgres;PORT=5432;DATABASE=tracelens;POOLING=true;" \ No newline at end of file diff --git a/scripts/k8s/trace-lens/service.yaml b/scripts/k8s/trace-lens/service.yaml new file mode 100644 index 000000000..dd41b53fa --- /dev/null +++ b/scripts/k8s/trace-lens/service.yaml @@ -0,0 +1,25 @@ +apiVersion: v1 +kind: Service +metadata: + name: trace-lens-dashboard +spec: + type: LoadBalancer + selector: + app: trace-lens + ports: + - name: dashboard + port: 7001 + targetPort: 5001 + # OTEL collector port is internal-only, avoid exposing it via LoadBalancer +--- +apiVersion: v1 +kind: Service +metadata: + name: trace-lens-collector +spec: + selector: + app: trace-lens + ports: + - name: otel-collector + port: 4317 + targetPort: 4317 \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs index cd389bdb7..6af239905 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs @@ -1,10 +1,12 @@ using System.Diagnostics; +using Elsa.Common.Contracts; using Elsa.OpenTelemetry.Helpers; using Elsa.Workflows; using Elsa.Workflows.Contracts; using Elsa.Workflows.Pipelines.ActivityExecution; using Elsa.Workflows.Pipelines.WorkflowExecution; using JetBrains.Annotations; +using Newtonsoft.Json; using Activity = System.Diagnostics.Activity; using ActivityKind = System.Diagnostics.ActivityKind; @@ -12,22 +14,52 @@ namespace Elsa.OpenTelemetry.Middleware; /// [UsedImplicitly] -public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareDelegate next) : IActivityExecutionMiddleware +public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareDelegate next, ISystemClock systemClock) : IActivityExecutionMiddleware { /// public async ValueTask InvokeAsync(ActivityExecutionContext context) { var activity = context.Activity; - using var span = ElsaOpenTelemetry.ActivitySource.StartActivity($"ActivityExecution {context.ActivityDescriptor.TypeName}", ActivityKind.Internal, Activity.Current?.Context ?? default); - span?.AddTag("activity.nodeId", activity.NodeId); - span?.AddTag("activity.type", activity.Type); - span?.AddTag("activity.name", activity.Name); - span?.AddTag("activityInstance.id", context.Id); - span?.AddTag("activityInstance.originalStatus", context.Status.ToString()); - span?.AddEvent(new ActivityEvent("Executing")); + using var span = ElsaOpenTelemetry.ActivitySource.StartActivity($"ActivityExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); + + if (span == null) + { + await next(context); + return; + } + + span.SetTag("activity.nodeId", activity.NodeId); + span.SetTag("activity.type", activity.Type); + span.SetTag("activity.name", activity.Name); + span.SetTag("activityInstance.id", context.Id); + + span.AddEvent(new ActivityEvent("Executing", tags: new ActivityTagsCollection(new Dictionary + { + ["activityInstance.status"] = context.Status.ToString(), + }))); + await next(context); - span?.AddEvent(new ActivityEvent("Executed")); - span?.AddTag("activityInstance.newStatus", context.Status.ToString()); + + if (context.Status == ActivityStatus.Faulted) + { + span.AddEvent(new ActivityEvent("Faulted")); + span.SetStatus(ActivityStatusCode.Error); + span.SetTag("error", true); + span.SetTag("hasIncidents", true); + + var errorMessage = string.IsNullOrWhiteSpace(context.Exception?.Message) ? "Unknown error" : context.Exception.Message; + span.SetTag("error.message", errorMessage); + + if (!string.IsNullOrEmpty(context.Exception?.StackTrace)) + span.SetTag("error.stackTrace", context.Exception.StackTrace); + } + else + span.AddEvent(new ActivityEvent("Executed", tags: new ActivityTagsCollection(new Dictionary + { + ["activityInstance.status"] = context.Status.ToString(), + }))); + + span.SetTag("activityExecution.durationMs", (systemClock.UtcNow - span.StartTimeUtc).TotalMilliseconds); } } @@ -39,4 +71,4 @@ public static class OpenTelemetryTracingActivityExecutionMiddlewareExtensions { /// Installs the component in the workflow execution pipeline. public static IActivityExecutionPipelineBuilder UseActivityExecutionTracing(this IActivityExecutionPipelineBuilder pipelineBuilder) => pipelineBuilder.Insert(0); -} +} \ No newline at end of file diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index ae6d67572..43f40f4c5 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -1,4 +1,6 @@ using System.Diagnostics; +using System.Text.Json; +using Elsa.Common.Contracts; using Elsa.OpenTelemetry.Helpers; using Elsa.Workflows; using Elsa.Workflows.Contracts; @@ -13,28 +15,58 @@ namespace Elsa.OpenTelemetry.Middleware; /// Middleware that traces workflow execution using OpenTelemetry. /// [UsedImplicitly] -public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareDelegate next) : WorkflowExecutionMiddleware(next) +public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareDelegate next, ISystemClock systemClock) : WorkflowExecutionMiddleware(next) { /// public override async ValueTask InvokeAsync(WorkflowExecutionContext context) { var workflowInstanceId = context.Id; var workflow = context.Workflow; - using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity($"WorkflowExecution {workflow.WorkflowMetadata.Name}", ActivityKind.Internal, Activity.Current?.Context ?? default); - - if(!string.IsNullOrWhiteSpace(context.CorrelationId)) - activity?.AddTag("correlationId", context.CorrelationId); - - activity?.AddTag("workflowInstance.id", workflowInstanceId); - activity?.AddTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); - activity?.AddTag("workflowDefinition.version", workflow.Identity.Version); - activity?.AddTag("workflowInstance.originalStatus", context.Status.ToString()); - activity?.AddTag("workflowInstance.originalSubStatus", context.SubStatus.ToString()); - activity?.AddEvent(new ActivityEvent("Executing")); + using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity($"WorkflowExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); + + if (activity == null) + { + await Next(context); + return; + } + + if (!string.IsNullOrWhiteSpace(context.CorrelationId)) + activity.SetTag("correlationId", context.CorrelationId); + + activity.SetTag("workflowInstance.id", workflowInstanceId); + activity.SetTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); + activity.SetTag("workflowDefinition.version", workflow.Identity.Version); + activity.SetTag("workflowDefinition.name", workflow.WorkflowMetadata.Name); + activity.AddEvent(new ActivityEvent("Executing", tags: new ActivityTagsCollection(new Dictionary + { + ["workflowInstance.status"] = context.Status.ToString(), + ["workflowInstance.subStatus"] = context.SubStatus.ToString() + }))); await Next(context); - activity?.AddEvent(new ActivityEvent("Executed")); - activity?.AddTag("workflowInstance.newStatus", context.Status.ToString()); - activity?.AddTag("workflowInstance.newSubStatus", context.SubStatus.ToString()); + + if (context.SubStatus == WorkflowSubStatus.Faulted) + { + activity.AddEvent(new ActivityEvent("Faulted")); + activity.SetStatus(ActivityStatusCode.Error); + activity.SetTag("error", true); + activity.SetTag("hasIncidents", true); + + if (context.Incidents.Count > 0) + activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents)); + } + else + { + activity.AddEvent(new ActivityEvent("Executed", tags: new ActivityTagsCollection(new Dictionary + { + ["workflowInstance.status"] = context.Status.ToString(), + ["workflowInstance.subStatus"] = context.SubStatus.ToString() + }))); + } + + if (!string.IsNullOrWhiteSpace(context.CorrelationId)) + activity.SetTag("correlationId", context.CorrelationId); + + activity.SetTag("workflowExecution.durationMs", (systemClock.UtcNow - activity.StartTimeUtc).TotalMilliseconds); } } @@ -46,4 +78,4 @@ public static class OpenTelemetryWorkflowExecutionMiddlewareExtensions { /// Installs the component in the workflow execution pipeline. public static IWorkflowExecutionPipelineBuilder UseWorkflowExecutionTracing(this IWorkflowExecutionPipelineBuilder pipelineBuilder) => pipelineBuilder.Insert(0); -} +} \ No newline at end of file From df827b0fda1da3b5d11cfd37db8d01f0346c95a7 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Tue, 24 Sep 2024 00:27:51 +0200 Subject: [PATCH 07/31] Update Datadog config, add incident error serialization Enhanced the Datadog tracing configuration to include priority sampling, rate limits, and sampling rules in the docker-compose-datadog.yml. Updated the agent image version and improved error message serialization in OpenTelemetry middleware to use custom JSON serializer options. Adjusted Dockerfile path to correct bundle location. --- docker/ElsaServer-Datadog.Dockerfile | 2 +- docker/docker-compose-datadog.yml | 69 ++++++++----------- ...metryTracingWorkflowExecutionMiddleware.cs | 7 +- 3 files changed, 35 insertions(+), 43 deletions(-) diff --git a/docker/ElsaServer-Datadog.Dockerfile b/docker/ElsaServer-Datadog.Dockerfile index 15e799a1a..5d9d9a4b4 100644 --- a/docker/ElsaServer-Datadog.Dockerfile +++ b/docker/ElsaServer-Datadog.Dockerfile @@ -18,7 +18,7 @@ COPY *.props ./ RUN dotnet restore "./src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj" # build and publish (UseAppHost=false creates platform independent binaries). -WORKDIR /source/src/apps/Elsa.Server.Web +WORKDIR /source/src/bundles/Elsa.Server.Web RUN dotnet build "Elsa.Server.Web.csproj" -c Release -o /app/build RUN dotnet publish "Elsa.Server.Web.csproj" -c Release -o /app/publish /p:UseAppHost=false --no-restore -f net8.0 diff --git a/docker/docker-compose-datadog.yml b/docker/docker-compose-datadog.yml index d92ad76ed..5ba620230 100644 --- a/docker/docker-compose-datadog.yml +++ b/docker/docker-compose-datadog.yml @@ -52,6 +52,33 @@ services: DD_TRACE_OTEL_ENABLED: true DD_SERVICE: "elsa-server-local" DD_VERSION: "3.2.1-blueberry" + + # Enable priority sampling + DD_TRACE_SAMPLING_PRIORITY: "true" + + # Global rate limiting of traces (number of spans per second) + DD_TRACE_RATE_LIMIT: 100 + + # Global sample rate for all traces (applies to spans that do not match a specific sampling rule) + DD_TRACE_SAMPLE_RATE: 1.0 # Keep 100% of the traces globally (adjust as needed) + + # Sampling rules for controlling sampling of specific services and errors + DD_TRACE_SAMPLING_RULES: > + [ + { + "service": "elsa-server-local", + "name": "WorkflowExecution", + "sample_rate": 1.0, + "condition": {"tags": {"hasIncidents": "true"}} + }, + { + "service": "elsa-server-local", + "name": "ActivityExecution", + "sample_rate": 1.0, + "condition": {"tags": {"hasIncidents": "true"}} + } + ] + ASPNETCORE_ENVIRONMENT: Development PYTHONNET_PYDLL: /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11 PYTHONNET_RUNTIME: coreclr @@ -74,7 +101,7 @@ services: - "14000:8080" datadog-agent: - image: datadog/agent:7.42.0 + image: datadog/agent:7.57.1 environment: DD_API_KEY: "YOUR_API_KEY" DD_SITE: "datadoghq.eu" @@ -88,46 +115,6 @@ services: DD_AC_INCLUDE: "name:postgres,name:rabbitmq,name:redis,name:elsa-server" DD_AC_EXCLUDE: "name:datadog-agent" - # Enable priority sampling - DD_TRACE_SAMPLING_PRIORITY: "true" - - # Global rate limiting of traces (number of spans per second) - DD_TRACE_RATE_LIMIT: 100 - - # Global sample rate for all traces (applies to spans that do not match a specific sampling rule) - DD_TRACE_SAMPLE_RATE: 1.0 # Keep 100% of the traces globally (adjust as needed) - - # Sampling rules for controlling sampling of specific services and errors - DD_TRACE_SAMPLING_RULES: > - [ - { - "service": "elsa-server-local", - "name": "WorkflowExecution", - "sample_rate": 0.5 - }, - { - "service": "elsa-server-local", - "name": "WorkflowExecution", - "sample_rate": 1.0, - "condition": {"tags": {"hasIncidents": "true"}} - } - { - "service": "elsa-server-local", - "name": "ActivityExecution", - "sample_rate": 1.0, - "condition": {"tags": {"hasIncidents": "true"}} - }, - { - "service": "elsa-server-local", - "name": "ActivityExecution", - "sample_rate": 0.3 - }, - { - "service": "elsa-server-local", - "sample_rate": 1.0, - "condition": {"error": true} - } - ] volumes: - /var/run/docker.sock:/var/run/docker.sock - /proc/:/host/proc/:ro diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index 43f40f4c5..83382d4d9 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -1,10 +1,13 @@ using System.Diagnostics; using System.Text.Json; using Elsa.Common.Contracts; +using Elsa.Expressions.Services; +using Elsa.Extensions; using Elsa.OpenTelemetry.Helpers; using Elsa.Workflows; using Elsa.Workflows.Contracts; using Elsa.Workflows.Pipelines.WorkflowExecution; +using Elsa.Workflows.Serialization.Converters; using JetBrains.Annotations; using Activity = System.Diagnostics.Activity; using ActivityKind = System.Diagnostics.ActivityKind; @@ -17,6 +20,8 @@ namespace Elsa.OpenTelemetry.Middleware; [UsedImplicitly] public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareDelegate next, ISystemClock systemClock) : WorkflowExecutionMiddleware(next) { + private readonly JsonSerializerOptions? _incidentSerializerOptions = new JsonSerializerOptions().WithConverters(new TypeJsonConverter(WellKnownTypeRegistry.CreateDefault())); + /// public override async ValueTask InvokeAsync(WorkflowExecutionContext context) { @@ -52,7 +57,7 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD activity.SetTag("hasIncidents", true); if (context.Incidents.Count > 0) - activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents)); + activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents, _incidentSerializerOptions)); } else { From 758b60767ba5fe2fbe0be3d8e3bea620c42dcc1c Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Wed, 25 Sep 2024 00:49:21 +0200 Subject: [PATCH 08/31] Add OpenTelemetry auto-instrumentation support Introduce OpenTelemetry configuration and dependencies for observability. Replace DataDog tracer with OpenTelemetry auto-instrumentation in the Dockerfile. Adjust project dependencies and log levels to incorporate OpenTelemetry without disrupting existing functionality. --- Directory.Packages.props | 6 +- Elsa.sln | 1 + docker/ElsaServer-Datadog.Dockerfile | 47 ++++++----- docker/docker-compose.yml | 4 +- docker/otel-collector-config.yaml | 79 +++++++++++++++++++ .../Elsa.Server.Web/Elsa.Server.Web.csproj | 3 + src/bundles/Elsa.Server.Web/Program.cs | 6 -- src/bundles/Elsa.Server.Web/appsettings.json | 11 +-- 8 files changed, 115 insertions(+), 42 deletions(-) create mode 100644 docker/otel-collector-config.yaml diff --git a/Directory.Packages.props b/Directory.Packages.props index 9dc0dd505..78ecaa67b 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -37,7 +37,8 @@ - + + @@ -65,6 +66,9 @@ + + + diff --git a/Elsa.sln b/Elsa.sln index 8e4c08bb0..47d308237 100644 --- a/Elsa.sln +++ b/Elsa.sln @@ -130,6 +130,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docker", "docker", "{986E54 docker\docker-compose.yml = docker\docker-compose.yml docker\docker-compose-datadog.yml = docker\docker-compose-datadog.yml docker\ElsaServer-Datadog.Dockerfile = docker\ElsaServer-Datadog.Dockerfile + docker\otel-collector-config.yaml = docker\otel-collector-config.yaml EndProjectSection EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Elsa.Samples.AspNet.RunTaskIntegration", "samples\aspnet\Elsa.Samples.AspNet.RunTaskIntegration\Elsa.Samples.AspNet.RunTaskIntegration.csproj", "{51050209-EC2F-4DC7-8F46-07E22B7811CD}" diff --git a/docker/ElsaServer-Datadog.Dockerfile b/docker/ElsaServer-Datadog.Dockerfile index 5d9d9a4b4..e1ec04332 100644 --- a/docker/ElsaServer-Datadog.Dockerfile +++ b/docker/ElsaServer-Datadog.Dockerfile @@ -4,25 +4,20 @@ FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0-bookworm-slim AS build WORKDIR /source -# Determine the architecture and download the appropriate version of the tracer -RUN ARCH=$(if [ "$(uname -m)" = "x86_64" ]; then echo "amd64"; elif [ "$(uname -m)" = "aarch64" ]; then echo "arm64"; else echo "amd64"; fi) \ - && TRACER_VERSION=$(curl -s https://api.github.com/repos/DataDog/dd-trace-dotnet/releases/latest | grep tag_name | cut -d '"' -f 4 | cut -c2-) \ - && curl -Lo /tmp/datadog-dotnet-apm.deb https://github.com/DataDog/dd-trace-dotnet/releases/download/v${TRACER_VERSION}/datadog-dotnet-apm_${TRACER_VERSION}_${ARCH}.deb - -# copy sources. +# Copy sources. COPY src/. ./src COPY ./NuGet.Config ./ COPY *.props ./ -# restore packages. +# Restore packages. RUN dotnet restore "./src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj" -# build and publish (UseAppHost=false creates platform independent binaries). +# Build and publish (UseAppHost=false creates platform independent binaries). WORKDIR /source/src/bundles/Elsa.Server.Web RUN dotnet build "Elsa.Server.Web.csproj" -c Release -o /app/build RUN dotnet publish "Elsa.Server.Web.csproj" -c Release -o /app/publish /p:UseAppHost=false --no-restore -f net8.0 -# move binaries into smaller base image. +# Move binaries into smaller base image. FROM mcr.microsoft.com/dotnet/aspnet:8.0-bookworm-slim AS base WORKDIR /app COPY --from=build /app/publish ./ @@ -38,21 +33,25 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ # Set PYTHONNET_PYDLL environment variable ENV PYTHONNET_PYDLL=/usr/lib/aarch64-linux-gnu/libpython3.11.so -# Copy the tracer from build target -COPY --from=build /tmp/datadog-dotnet-apm.deb /tmp/datadog-dotnet-apm.deb -# Install the tracer -RUN mkdir -p /opt/datadog \ - && mkdir -p /var/log/datadog \ - && dpkg -i /tmp/datadog-dotnet-apm.deb \ - && rm /tmp/datadog-dotnet-apm.deb - -# Enable the tracer -ENV CORECLR_ENABLE_PROFILING=1 -ENV CORECLR_PROFILER={846F5F1C-F9AE-4B07-969E-05C26BC060D8} -ENV CORECLR_PROFILER_PATH=/opt/datadog/Datadog.Trace.ClrProfiler.Native.so -ENV DD_DOTNET_TRACER_HOME=/opt/datadog -ENV DD_INTEGRATIONS=/opt/datadog/integrations.json +# Install dependencies +RUN apt-get update && apt-get install -y wget unzip curl + +# Set environment variables for OpenTelemetry Auto-Instrumentation +ENV OTEL_DOTNET_AUTO_HOME=/otel +ENV OTEL_LOG_LEVEL="debug" + +# Download and extract OpenTelemetry Auto-Instrumentation +ARG OTEL_VERSION=1.7.0 +RUN mkdir /otel +RUN curl -L -o /otel/otel-dotnet-install.sh https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/releases/download/v${OTEL_VERSION}/otel-dotnet-auto-install.sh +RUN chmod +x /otel/otel-dotnet-install.sh +RUN /bin/bash /otel/otel-dotnet-install.sh + +# Provide necessary permissions for the script to execute +RUN chmod +x /otel/instrument.sh EXPOSE 8080/tcp EXPOSE 443/tcp -ENTRYPOINT ["dotnet", "Elsa.Server.Web.dll"] + +# Instrument the application and start it +ENTRYPOINT ["/bin/bash", "-c", "source /otel/instrument.sh && dotnet Elsa.Server.Web.dll"] \ No newline at end of file diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index 057ba4b50..8f90e776a 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -1,6 +1,4 @@ -version: '3.7' - -services: +services: postgres: image: postgres:latest command: -c 'max_connections=2000' diff --git a/docker/otel-collector-config.yaml b/docker/otel-collector-config.yaml new file mode 100644 index 000000000..5ef2a406c --- /dev/null +++ b/docker/otel-collector-config.yaml @@ -0,0 +1,79 @@ +receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 + hostmetrics: + collection_interval: 10s + scrapers: + paging: + metrics: + system.paging.utilization: + enabled: true + cpu: + metrics: + system.cpu.utilization: + enabled: true + disk: + filesystem: + metrics: + system.filesystem.utilization: + enabled: true + load: + memory: + network: + processes: + docker_stats: + metrics: + container.network.io.usage.rx_packets: + enabled: true + container.network.io.usage.tx_packets: + enabled: true + container.cpu.usage.system: + enabled: true + container.memory.rss: + enabled: true + container.blockio.io_serviced_recursive: + enabled: true + +processors: + batch: + send_batch_max_size: 100 + send_batch_size: 10 + timeout: 1s + +connectors: + datadog/connector: + +exporters: + debug: + verbosity: detailed + datadog: + api: + site: ${env:DD_SITE} + key: ${env:DD_API_KEY} + +service: + pipelines: + metrics: + receivers: [ hostmetrics, otlp, datadog/connector ] + processors: [ batch ] + exporters: [ datadog ] + traces: + receivers: [ otlp ] + processors: [ batch ] + exporters: [ datadog/connector ] + traces/sampling: + # This pipeline has a Datadog connector, a batch processor and a Datadog exporter. + # It receivers all traces from the Datadog connector and sends them to Datadog. + # Add any sampling here, so that the generated trace metrics account for all traces. + receivers: [ datadog/connector ] + # Add any sampling here + processors: [ ] + exporters: [ datadog ] + logs: + receivers: [ otlp ] + processors: [ batch ] + exporters: [ datadog ] \ No newline at end of file diff --git a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj index ff8b01a9d..cfc9fcbb3 100644 --- a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj +++ b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj @@ -51,6 +51,9 @@ + + + diff --git a/src/bundles/Elsa.Server.Web/Program.cs b/src/bundles/Elsa.Server.Web/Program.cs index d72e04aca..d1cb0bcdd 100644 --- a/src/bundles/Elsa.Server.Web/Program.cs +++ b/src/bundles/Elsa.Server.Web/Program.cs @@ -384,7 +384,6 @@ services }); services.Configure(options => options.CacheDuration = TimeSpan.FromDays(1)); - services.AddHealthChecks(); services.AddControllers(); services.AddCors(cors => cors.AddDefaultPolicy(policy => policy.AllowAnyHeader().AllowAnyMethod().AllowAnyOrigin().WithExposedHeaders("*"))); @@ -392,11 +391,6 @@ services.AddCors(cors => cors.AddDefaultPolicy(policy => policy.AllowAnyHeader() // Build the web application. var app = builder.Build(); -// app.UseSimulatedLatency( -// TimeSpan.FromMilliseconds(1000), -// TimeSpan.FromMilliseconds(3000) -// ); - // Configure the pipeline. if (app.Environment.IsDevelopment()) app.UseDeveloperExceptionPage(); diff --git a/src/bundles/Elsa.Server.Web/appsettings.json b/src/bundles/Elsa.Server.Web/appsettings.json index cf664177e..a6d14c54d 100644 --- a/src/bundles/Elsa.Server.Web/appsettings.json +++ b/src/bundles/Elsa.Server.Web/appsettings.json @@ -1,15 +1,10 @@ { "Logging": { "LogLevel": { - "Default": "Warning", - "Elsa": "Warning", - "MassTransit": "Warning", - "Microsoft.Extensions.Http": "Warning", + "Default": "Debug", + "Microsoft": "Warning", "Microsoft.Hosting.Lifetime": "Information", - "Microsoft.EntityFrameworkCore": "Warning", - "Microsoft.AspNetCore": "Warning", - "Quartz": "Warning", - "System.Net.Http": "Warning" + "OpenTelemetry": "Debug" } }, "AllowedHosts": "*", From a62de77900d771f81fbc4a027278531085cf6ada Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Wed, 25 Sep 2024 14:12:00 +0200 Subject: [PATCH 09/31] Update OpenTelemetry configurations and remove unused modules Revised otel-collector and docker-compose configurations to streamline tracing and metrics collection, replacing Datadog connectors with OpenTelemetry. Removed unused scraping modules and unnecessary dependencies to improve setup efficiency and resource utilization. --- docker/docker-compose-datadog.yml | 100 +++++++----------- docker/otel-collector-config.yaml | 66 ++++-------- .../Elsa.Server.Web/Elsa.Server.Web.csproj | 2 - 3 files changed, 59 insertions(+), 109 deletions(-) diff --git a/docker/docker-compose-datadog.yml b/docker/docker-compose-datadog.yml index 5ba620230..7d7bc6ed9 100644 --- a/docker/docker-compose-datadog.yml +++ b/docker/docker-compose-datadog.yml @@ -1,6 +1,4 @@ -version: '3.7' - -services: +services: postgres: image: postgres:latest command: -c 'max_connections=2000' @@ -13,17 +11,6 @@ services: ports: - "5432:5432" - cockroachdb: - image: cockroachdb/cockroach:v22.1.0 - command: start-single-node --insecure - ports: - - "26257:26257" # CockroachDB SQL port - - "8080:8080" # CockroachDB UI port - volumes: - - cockroachdb-data:/cockroach/cockroach-data - environment: - - COCKROACH_DATABASE=elsa - rabbitmq: image: "rabbitmq:3-management" ports: @@ -44,40 +31,22 @@ services: - postgres - rabbitmq - redis - - datadog-agent + - otel-collector environment: - DD_AGENT_HOST: datadog-agent - DD_ENV: development - DD_TRACE_DEBUG: true - DD_TRACE_OTEL_ENABLED: true - DD_SERVICE: "elsa-server-local" - DD_VERSION: "3.2.1-blueberry" - - # Enable priority sampling - DD_TRACE_SAMPLING_PRIORITY: "true" - - # Global rate limiting of traces (number of spans per second) - DD_TRACE_RATE_LIMIT: 100 - - # Global sample rate for all traces (applies to spans that do not match a specific sampling rule) - DD_TRACE_SAMPLE_RATE: 1.0 # Keep 100% of the traces globally (adjust as needed) - - # Sampling rules for controlling sampling of specific services and errors - DD_TRACE_SAMPLING_RULES: > - [ - { - "service": "elsa-server-local", - "name": "WorkflowExecution", - "sample_rate": 1.0, - "condition": {"tags": {"hasIncidents": "true"}} - }, - { - "service": "elsa-server-local", - "name": "ActivityExecution", - "sample_rate": 1.0, - "condition": {"tags": {"hasIncidents": "true"}} - } - ] + # OpenTelemetry environment variables + OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4317" # Point to OpenTelemetry Collector + OTEL_EXPORTER_OTLP_PROTOCOL: "grpc" # Use gRPC for OTLP + OTEL_TRACES_EXPORTER: "otlp" + OTEL_METRICS_EXPORTER: "otlp" + OTEL_LOGS_EXPORTER: "otpl" + OTEL_RESOURCE_ATTRIBUTES: "service.name=elsa-server-local,service.version=3.2.1-blueberry,deployment.environment=development" + OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES: "Elsa.Workflows" + OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED: "true" + OTEL_LOG_LEVEL: "debug" + OTEL_DOTNET_AUTO_RESOURCE_DETECTOR_ENABLED: "true" + OTEL_DOTNET_AUTO_LOGS_CONSOLE_EXPORTER_ENABLED: "true" + OTEL_DOTNET_AUTO_METRICS_CONSOLE_EXPORTER_ENABLED: "true" + OTEL_DOTNET_AUTO_TRACES_CONSOLE_EXPORTER_ENABLED: "true" ASPNETCORE_ENVIRONMENT: Development PYTHONNET_PYDLL: /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11 @@ -100,28 +69,41 @@ services: ports: - "14000:8080" - datadog-agent: - image: datadog/agent:7.57.1 + otel-collector: + image: otel/opentelemetry-collector-contrib:latest + volumes: + - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml + command: [ "--config", "/etc/otel-collector-config.yaml", "--feature-gates", "-component.UseLocalHostAsDefaultHost" ] environment: - DD_API_KEY: "YOUR_API_KEY" + DD_API_KEY: "secret api key" DD_SITE: "datadoghq.eu" + ports: + - "13133:13133" + - "4317:4317" + - "4318:4318" + + datadog-agent: + image: datadog/agent:latest + environment: + DD_API_KEY: "secret api key" + DD_SITE: "datadoghq.eu" + DD_HOSTNAME: "otel-collector" DD_LOGS_ENABLED: "true" + DD_OTLP_CONFIG_LOGS_ENABLED: "true" DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL: "true" DD_APM_ENABLED: "true" - DD_REMOTE_CONFIGURATION_ENABLED: "true" DD_APM_NON_LOCAL_TRAFFIC: "true" - + DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT: 0.0.0.0:4317 # The Datadog Agent expects traces from OpenTelemetry Collector + DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT: 0.0.0.0:4318 + # Service autodiscovery DD_AC_INCLUDE: "name:postgres,name:rabbitmq,name:redis,name:elsa-server" DD_AC_EXCLUDE: "name:datadog-agent" - - volumes: - - /var/run/docker.sock:/var/run/docker.sock - - /proc/:/host/proc/:ro - - /sys/fs/cgroup/:/host/sys/fs/cgroup:ro + ports: - "8126:8126" + - "14317:4317" + - "14318:4318" volumes: - postgres-data: - cockroachdb-data: + postgres-data: \ No newline at end of file diff --git a/docker/otel-collector-config.yaml b/docker/otel-collector-config.yaml index 5ef2a406c..a1aaced1b 100644 --- a/docker/otel-collector-config.yaml +++ b/docker/otel-collector-config.yaml @@ -5,47 +5,25 @@ receivers: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 - hostmetrics: - collection_interval: 10s - scrapers: - paging: - metrics: - system.paging.utilization: - enabled: true - cpu: - metrics: - system.cpu.utilization: - enabled: true - disk: - filesystem: - metrics: - system.filesystem.utilization: - enabled: true - load: - memory: - network: - processes: - docker_stats: - metrics: - container.network.io.usage.rx_packets: - enabled: true - container.network.io.usage.tx_packets: - enabled: true - container.cpu.usage.system: - enabled: true - container.memory.rss: - enabled: true - container.blockio.io_serviced_recursive: - enabled: true processors: batch: send_batch_max_size: 100 - send_batch_size: 10 + send_batch_size: 10 # Increased batch size for efficiency timeout: 1s - -connectors: - datadog/connector: + tail_sampling: + decision_wait: 10s + num_traces: 10000 # Increased from 100 to handle more traces + expected_new_traces_per_sec: 100 # Increased from 10 + decision_cache: + sampled_cache_size: 100000 + policies: [ + { + name: incidents-policy, + type: boolean_attribute, + boolean_attribute: { key: hasIncidents, value: true } + } + ] exporters: debug: @@ -58,22 +36,14 @@ exporters: service: pipelines: metrics: - receivers: [ hostmetrics, otlp, datadog/connector ] + receivers: [ otlp ] processors: [ batch ] exporters: [ datadog ] traces: receivers: [ otlp ] - processors: [ batch ] - exporters: [ datadog/connector ] - traces/sampling: - # This pipeline has a Datadog connector, a batch processor and a Datadog exporter. - # It receivers all traces from the Datadog connector and sends them to Datadog. - # Add any sampling here, so that the generated trace metrics account for all traces. - receivers: [ datadog/connector ] - # Add any sampling here - processors: [ ] - exporters: [ datadog ] + processors: [ batch, tail_sampling ] # Added tail_sampling to the main traces pipeline + exporters: [ debug, datadog ] # Directly exporting to debug and datadog logs: receivers: [ otlp ] processors: [ batch ] - exporters: [ datadog ] \ No newline at end of file + exporters: [ debug, datadog ] diff --git a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj index cfc9fcbb3..bbc68b8ef 100644 --- a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj +++ b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj @@ -52,8 +52,6 @@ - - From 0492a7be4356a481ec052c4e5c191aae69641685 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 26 Sep 2024 14:09:28 +0200 Subject: [PATCH 10/31] Refactor to use primary constructor in Get class Refactored the `Get` class to utilize a primary constructor for dependency injection of `IWorkflowExecutionLogStore`. This change simplifies the class definition and removes the need for a private field. --- .../Journal/GetLastEntry/Endpoint.cs | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Journal/GetLastEntry/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Journal/GetLastEntry/Endpoint.cs index 2a3988c5d..a3f0a637f 100644 --- a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Journal/GetLastEntry/Endpoint.cs +++ b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Journal/GetLastEntry/Endpoint.cs @@ -12,16 +12,8 @@ namespace Elsa.Workflows.Api.Endpoints.WorkflowInstances.Journal.GetLastEntry; /// Return the last log entry for the specified workflow instance and activity ID. /// [PublicAPI] -public class Get : ElsaEndpoint +public class Get(IWorkflowExecutionLogStore store) : ElsaEndpoint { - private readonly IWorkflowExecutionLogStore _store; - - /// - public Get(IWorkflowExecutionLogStore store) - { - _store = store; - } - /// public override void Configure() { @@ -36,7 +28,7 @@ public class Get : ElsaEndpoint { WorkflowInstanceId = request.WorkflowInstanceId, ActivityId = request.ActivityId, - EventNames = new[] { "Started", "Completed", "Faulted" } + EventNames = ["Started", "Completed", "Faulted"] }; var sort = new WorkflowExecutionLogRecordOrder( @@ -44,7 +36,7 @@ public class Get : ElsaEndpoint OrderDirection.Descending ); - var entry = await _store.FindAsync(filter, sort, cancellationToken); + var entry = await store.FindAsync(filter, sort, cancellationToken); if (entry == null) { From 973489c961fe6bcee42ff370002740a2cff34152 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 26 Sep 2024 20:59:59 +0200 Subject: [PATCH 11/31] Remove unused usings in HungryWorkflow.cs This commit cleans up the HungryWorkflow.cs file by removing unused using directives. This improves code readability and reduces unnecessary dependencies. --- .../Workflows/HungryWorkflow.cs | 2 -- 1 file changed, 2 deletions(-) diff --git a/samples/aspnet/Elsa.Samples.AspNet.RunTaskIntegration/Workflows/HungryWorkflow.cs b/samples/aspnet/Elsa.Samples.AspNet.RunTaskIntegration/Workflows/HungryWorkflow.cs index 0e5fd8c3d..2653d9f61 100644 --- a/samples/aspnet/Elsa.Samples.AspNet.RunTaskIntegration/Workflows/HungryWorkflow.cs +++ b/samples/aspnet/Elsa.Samples.AspNet.RunTaskIntegration/Workflows/HungryWorkflow.cs @@ -1,11 +1,9 @@ -using System.Collections.Generic; using Elsa.Http; using Elsa.Workflows; using Elsa.Workflows.Activities; using Elsa.Workflows.Contracts; using Elsa.Workflows.Models; using Elsa.Workflows.Runtime.Activities; -using Microsoft.AspNetCore.Http; namespace Elsa.Samples.AspNet.RunTaskIntegration.Workflows; From 7609b0e7ca27cb33df9f4103480716e698d24240 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 26 Sep 2024 21:00:10 +0200 Subject: [PATCH 12/31] Add factory method to PagedListResponse This update introduces a static factory method `From` in the `PagedListResponse` class. It allows for the creation of `PagedListResponse` instances directly from `Page` objects, streamlining object instantiation. --- src/common/Elsa.Api.Common/Models/PagedListResponse.cs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/common/Elsa.Api.Common/Models/PagedListResponse.cs b/src/common/Elsa.Api.Common/Models/PagedListResponse.cs index 07bfebe9e..06cdb55e9 100644 --- a/src/common/Elsa.Api.Common/Models/PagedListResponse.cs +++ b/src/common/Elsa.Api.Common/Models/PagedListResponse.cs @@ -16,4 +16,6 @@ public record PagedListResponse: LinkedResource public ICollection Items { get; set; } = default!; public long TotalCount { get; set; } + + public static PagedListResponse From(Page page) => new(page); } \ No newline at end of file From 2e1ed995dad0d9dd4ca1c1aa226a02f2b752e05e Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 27 Sep 2024 12:59:59 +0200 Subject: [PATCH 13/31] Add synchronous serialization methods and mark async as obsolete Implemented synchronous methods for serialization and deserialization while marking the asynchronous methods as obsolete across various storage and serialization services. This includes updates to handle serialization within the Save and Load methods, enhancing performance by avoiding unnecessary Task usage. --- src/bundles/Elsa.Server.Web/SampleWorkflow.cs | 41 +++++++++ src/bundles/Elsa.Server.Web/appsettings.json | 2 +- .../DapperActivityExecutionRecordStore.cs | 32 +++---- .../Runtime/ActivityExecutionLogStore.cs | 4 +- .../Runtime/WorkflowExecutionLogStore.cs | 4 +- .../WorkflowInstances/Export/Endpoint.cs | 6 +- .../WorkflowInstances/Import/Endpoint.cs | 4 +- .../Contracts/ISafeSerializer.cs | 28 ++++++ .../Contracts/IWorkflowStateSerializer.cs | 88 +++++++++++++++++-- .../ActivityExecutionContextExtensions.cs | 4 +- .../JsonWorkflowStateSerializer.cs | 74 ++++++++++++---- .../Serializers/SafeSerializer.cs | 36 ++++++-- .../SerializerEncodingTests.cs | 20 +---- 13 files changed, 264 insertions(+), 79 deletions(-) create mode 100644 src/bundles/Elsa.Server.Web/SampleWorkflow.cs diff --git a/src/bundles/Elsa.Server.Web/SampleWorkflow.cs b/src/bundles/Elsa.Server.Web/SampleWorkflow.cs new file mode 100644 index 000000000..44c8a15ca --- /dev/null +++ b/src/bundles/Elsa.Server.Web/SampleWorkflow.cs @@ -0,0 +1,41 @@ +using Elsa.Expressions.Models; +using Elsa.Extensions; +using Elsa.Scheduling.Activities; +using Elsa.Workflows; +using Elsa.Workflows.Activities; +using Elsa.Workflows.Contracts; + +namespace Elsa.Server.Web; + +public class SampleWorkflow : WorkflowBase +{ + protected override void Build(IWorkflowBuilder workflow) + { + // The WithVariable method ensures that the created variable will be added to the Workflow's Variables collection, which is required for persistent variables. + var variable1 = workflow.WithVariable("Foo").WithWorkflowStorage(); + + workflow.Variables = + [ + variable1 + ]; + + workflow.Root = new Sequence + { + Activities = + { + new StartAt(DateTimeOffset.UtcNow + TimeSpan.FromSeconds(5)) + { + CanStartWorkflow = true + }, + new WriteLine(variable1), + new SetVariable + { + Variable = variable1, + Value = new (Literal.From("Bar")) + }, + new Delay(TimeSpan.FromSeconds(1)), + new WriteLine(variable1) + } + }; + } +} \ No newline at end of file diff --git a/src/bundles/Elsa.Server.Web/appsettings.json b/src/bundles/Elsa.Server.Web/appsettings.json index a6d14c54d..549fea290 100644 --- a/src/bundles/Elsa.Server.Web/appsettings.json +++ b/src/bundles/Elsa.Server.Web/appsettings.json @@ -1,7 +1,7 @@ { "Logging": { "LogLevel": { - "Default": "Debug", + "Default": "Warning", "Microsoft": "Warning", "Microsoft.Hosting.Lifetime": "Information", "OpenTelemetry": "Debug" diff --git a/src/modules/Elsa.Dapper/Modules/Runtime/Stores/DapperActivityExecutionRecordStore.cs b/src/modules/Elsa.Dapper/Modules/Runtime/Stores/DapperActivityExecutionRecordStore.cs index 6ba1e6757..aba418fc6 100644 --- a/src/modules/Elsa.Dapper/Modules/Runtime/Stores/DapperActivityExecutionRecordStore.cs +++ b/src/modules/Elsa.Dapper/Modules/Runtime/Stores/DapperActivityExecutionRecordStore.cs @@ -38,14 +38,14 @@ public class DapperActivityExecutionRecordStore : IActivityExecutionStore /// public async Task SaveAsync(ActivityExecutionRecord record, CancellationToken cancellationToken = default) { - var mappedRecord = await Map(record, cancellationToken); + var mappedRecord = Map(record, cancellationToken); await _store.SaveAsync(mappedRecord, PrimaryKeyName, cancellationToken); } /// public async Task SaveManyAsync(IEnumerable records, CancellationToken cancellationToken = default) { - var mappedRecords = await Task.WhenAll(records.Select(async x => await Map(x, cancellationToken))); + var mappedRecords = records.Select(x => Map(x, cancellationToken)); await _store.SaveManyAsync(mappedRecords, PrimaryKeyName, cancellationToken); } @@ -53,21 +53,21 @@ public class DapperActivityExecutionRecordStore : IActivityExecutionStore public async Task FindAsync(ActivityExecutionRecordFilter filter, CancellationToken cancellationToken = default) { var record = await _store.FindAsync(q => ApplyFilter(q, filter), cancellationToken); - return record == null ? null : await MapAsync(record, cancellationToken); + return record == null ? null : Map(record, cancellationToken); } /// public async Task> FindManyAsync(ActivityExecutionRecordFilter filter, ActivityExecutionRecordOrder order, CancellationToken cancellationToken = default) { var records = await _store.FindManyAsync(q => ApplyFilter(q, filter), order.KeySelector.GetPropertyName(), order.Direction, cancellationToken); - return await Task.WhenAll(records.Select(async x => await MapAsync(x, cancellationToken))); + return records.Select( x => Map(x, cancellationToken)).ToList(); } /// public async Task> FindManyAsync(ActivityExecutionRecordFilter filter, CancellationToken cancellationToken = default) { var records = await _store.FindManyAsync(q => ApplyFilter(q, filter), cancellationToken); - return await Task.WhenAll(records.Select(async x => await MapAsync(x, cancellationToken))); + return records.Select( x => Map(x, cancellationToken)).ToList(); } /// @@ -115,7 +115,7 @@ public class DapperActivityExecutionRecordStore : IActivityExecutionStore } } - private async ValueTask Map(ActivityExecutionRecord source, CancellationToken cancellationToken) + private ActivityExecutionRecordRecord Map(ActivityExecutionRecord source, CancellationToken cancellationToken) { return new ActivityExecutionRecordRecord { @@ -130,15 +130,15 @@ public class DapperActivityExecutionRecordStore : IActivityExecutionStore HasBookmarks = source.HasBookmarks, Status = source.Status.ToString(), ActivityTypeVersion = source.ActivityTypeVersion, - SerializedActivityState = source.ActivityState != null ? await _safeSerializer.SerializeAsync(source.ActivityState, cancellationToken) : null, - SerializedPayload = source.Payload != null ? await _safeSerializer.SerializeAsync(source.Payload, cancellationToken) : null, - SerializedOutputs = source.Outputs?.Any() == true ? await _safeSerializer.SerializeAsync(source.Outputs, cancellationToken) : null, + SerializedActivityState = source.ActivityState != null ? _safeSerializer.Serialize(source.ActivityState, cancellationToken) : null, + SerializedPayload = source.Payload != null ? _safeSerializer.Serialize(source.Payload, cancellationToken) : null, + SerializedOutputs = source.Outputs?.Any() == true ? _safeSerializer.Serialize(source.Outputs, cancellationToken) : null, SerializedException = source.Exception != null ? _payloadSerializer.Serialize(source.Exception) : null, - SerializedProperties = source.Properties.Any() ? await _safeSerializer.SerializeAsync(source.Properties, cancellationToken) : null + SerializedProperties = source.Properties.Any() ? _safeSerializer.Serialize(source.Properties, cancellationToken) : null }; } - private async ValueTask MapAsync(ActivityExecutionRecordRecord source, CancellationToken cancellationToken) + private ActivityExecutionRecord Map(ActivityExecutionRecordRecord source, CancellationToken cancellationToken) { return new ActivityExecutionRecord { @@ -153,11 +153,11 @@ public class DapperActivityExecutionRecordStore : IActivityExecutionStore HasBookmarks = source.HasBookmarks, Status = Enum.Parse(source.Status), ActivityTypeVersion = source.ActivityTypeVersion, - ActivityState = source.SerializedActivityState != null ? _payloadSerializer.Deserialize>(source.SerializedActivityState) : default, - Payload = source.SerializedPayload != null ? await _safeSerializer.DeserializeAsync>(source.SerializedPayload, cancellationToken) : default, - Outputs = source.SerializedOutputs != null ? await _safeSerializer.DeserializeAsync>(source.SerializedOutputs, cancellationToken) : default, - Exception = source.SerializedException != null ? _payloadSerializer.Deserialize(source.SerializedException) : default, - Properties = source.SerializedProperties != null ? await _safeSerializer.DeserializeAsync>(source.SerializedProperties, cancellationToken) : default + ActivityState = source.SerializedActivityState != null ? _payloadSerializer.Deserialize>(source.SerializedActivityState) : null, + Payload = source.SerializedPayload != null ? _safeSerializer.Deserialize>(source.SerializedPayload, cancellationToken) : null, + Outputs = source.SerializedOutputs != null ? _safeSerializer.Deserialize>(source.SerializedOutputs, cancellationToken) : null, + Exception = source.SerializedException != null ? _payloadSerializer.Deserialize(source.SerializedException) : null, + Properties = source.SerializedProperties != null ? _safeSerializer.Deserialize>(source.SerializedProperties, cancellationToken) : null }; } diff --git a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs index da49cb3f1..68652db65 100644 --- a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs +++ b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs @@ -83,12 +83,12 @@ public class EFCoreActivityExecutionStore( { entity = entity.SanitizeLogMessage(); var compressionAlgorithm = options.Value.CompressionAlgorithm ?? nameof(None); - var serializedActivityState = entity.ActivityState != null ? await safeSerializer.SerializeAsync(entity.ActivityState, cancellationToken) : null; + var serializedActivityState = entity.ActivityState != null ? safeSerializer.Serialize(entity.ActivityState, cancellationToken) : null; var compressedSerializedActivityState = serializedActivityState != null ? await compressionCodecResolver.Resolve(compressionAlgorithm).CompressAsync(serializedActivityState, cancellationToken) : null; dbContext.Entry(entity).Property("SerializedActivityState").CurrentValue = compressedSerializedActivityState; dbContext.Entry(entity).Property("SerializedActivityStateCompressionAlgorithm").CurrentValue = compressionAlgorithm; - dbContext.Entry(entity).Property("SerializedOutputs").CurrentValue = entity.Outputs?.Any() == true ? await safeSerializer.SerializeAsync(entity.Outputs, cancellationToken) : null; + dbContext.Entry(entity).Property("SerializedOutputs").CurrentValue = entity.Outputs?.Any() == true ? safeSerializer.Serialize(entity.Outputs, cancellationToken) : null; dbContext.Entry(entity).Property("SerializedProperties").CurrentValue = entity.Properties.Any() ? payloadSerializer.Serialize(entity.Properties) : null; dbContext.Entry(entity).Property("SerializedException").CurrentValue = entity.Exception != null ? payloadSerializer.Serialize(entity.Exception) : null; dbContext.Entry(entity).Property("SerializedPayload").CurrentValue = entity.Payload?.Any() == true ? payloadSerializer.Serialize(entity.Payload) : null; diff --git a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/WorkflowExecutionLogStore.cs b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/WorkflowExecutionLogStore.cs index 5ff56801b..6c6aa563b 100644 --- a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/WorkflowExecutionLogStore.cs +++ b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/WorkflowExecutionLogStore.cs @@ -87,8 +87,8 @@ public class EFCoreWorkflowExecutionLogStore : IWorkflowExecutionLogStore private async ValueTask OnSaveAsync(RuntimeElsaDbContext dbContext, WorkflowExecutionLogRecord entity, CancellationToken cancellationToken) { entity = entity.SanitizeLogMessage(); - dbContext.Entry(entity).Property("SerializedActivityState").CurrentValue = entity.ActivityState?.Any() == true ? await _safeSerializer.SerializeAsync(entity.ActivityState, cancellationToken) : default; - dbContext.Entry(entity).Property("SerializedPayload").CurrentValue = entity.Payload != null ? await _safeSerializer.SerializeAsync(entity.Payload, cancellationToken) : default; + dbContext.Entry(entity).Property("SerializedActivityState").CurrentValue = entity.ActivityState?.Any() == true ? _safeSerializer.Serialize(entity.ActivityState, cancellationToken) : null; + dbContext.Entry(entity).Property("SerializedPayload").CurrentValue = entity.Payload != null ? _safeSerializer.Serialize(entity.Payload, cancellationToken) : null; } private async ValueTask OnLoadAsync(RuntimeElsaDbContext dbContext, WorkflowExecutionLogRecord? entity, CancellationToken cancellationToken) diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Export/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Export/Endpoint.cs index 5a925a207..4ce168a0b 100644 --- a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Export/Endpoint.cs +++ b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Export/Endpoint.cs @@ -121,10 +121,10 @@ internal class Export : ElsaEndpointWithMapper var executionLogRecords = request.IncludeWorkflowExecutionLog ? await LoadWorkflowExecutionLogRecordsAsync(workflowState.Id, cancellationToken) : default; var activityExecutionLogRecords = request.IncludeActivityExecutionLog ? await LoadActivityExecutionLogRecordsAsync(workflowState.Id, cancellationToken) : default; var bookmarks = request.IncludeBookmarks ? await LoadBookmarksAsync(workflowState.Id, cancellationToken) : null; - var workflowStateElement = await _workflowStateSerializer.SerializeToElementAsync(workflowState, cancellationToken); + var workflowStateElement = _workflowStateSerializer.SerializeToElement(workflowState, cancellationToken); var bookmarksElement = bookmarks != null ? SerializeBookmarks(bookmarks) : default(JsonElement?); - var executionLogRecordsElement = executionLogRecords != null ? await _safeSerializer.SerializeToElementAsync(executionLogRecords, cancellationToken) : default(JsonElement?); - var activityExecutionLogRecordsElement = activityExecutionLogRecords != null ? await _safeSerializer.SerializeToElementAsync(activityExecutionLogRecords, cancellationToken) : default(JsonElement?); + var executionLogRecordsElement = executionLogRecords != null ? _safeSerializer.SerializeToElement(executionLogRecords, cancellationToken) : default(JsonElement?); + var activityExecutionLogRecordsElement = activityExecutionLogRecords != null ? _safeSerializer.SerializeToElement(activityExecutionLogRecords, cancellationToken) : default(JsonElement?); var model = new ExportedWorkflowState(workflowStateElement, bookmarksElement, activityExecutionLogRecordsElement, executionLogRecordsElement); return model; } diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Import/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Import/Endpoint.cs index 1a8170769..295a577df 100644 --- a/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Import/Endpoint.cs +++ b/src/modules/Elsa.Workflows.Api/Endpoints/WorkflowInstances/Import/Endpoint.cs @@ -137,13 +137,13 @@ internal class Import : ElsaEndpointWithoutRequest if (model.ActivityExecutionRecords != null) { - var activityExecutionRecords = await _safeSerializer.DeserializeAsync>(model.ActivityExecutionRecords.Value, cancellationToken); + var activityExecutionRecords = _safeSerializer.Deserialize>(model.ActivityExecutionRecords.Value, cancellationToken); await _activityExecutionStore.SaveManyAsync(activityExecutionRecords, cancellationToken); } if (model.WorkflowExecutionLogRecords != null) { - var workflowExecutionLogRecords = await _safeSerializer.DeserializeAsync>(model.WorkflowExecutionLogRecords.Value, cancellationToken); + var workflowExecutionLogRecords = _safeSerializer.Deserialize>(model.WorkflowExecutionLogRecords.Value, cancellationToken); await _workflowExecutionLogStore.SaveManyAsync(workflowExecutionLogRecords, cancellationToken); } } diff --git a/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs b/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs index e241694d3..79f18e7b2 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs @@ -11,24 +11,52 @@ public interface ISafeSerializer /// /// Serializes the specified state. /// + [Obsolete("Use the non-async Serialize instead.")] [RequiresUnreferencedCode("The type T may be trimmed.")] ValueTask SerializeAsync(object? value, CancellationToken cancellationToken = default); /// /// Serializes the specified state to a object. /// + [Obsolete("Use the non-async SerializeToElement instead.")] [RequiresUnreferencedCode("The type T may be trimmed.")] ValueTask SerializeToElementAsync(object? value, CancellationToken cancellationToken = default); /// /// Deserializes the specified state. /// + [Obsolete("Use the non-async Deserialize instead.")] [RequiresUnreferencedCode("The type T may be trimmed.")] ValueTask DeserializeAsync(string json, CancellationToken cancellationToken = default); /// /// Deserializes the specified state. /// + [Obsolete("Use the non-async Deserialize instead.")] [RequiresUnreferencedCode("The type T may be trimmed.")] ValueTask DeserializeAsync(JsonElement element, CancellationToken cancellationToken = default); + + /// + /// Serializes the specified state. + /// + [RequiresUnreferencedCode("The type T may be trimmed.")] + string Serialize(object? value, CancellationToken cancellationToken = default); + + /// + /// Serializes the specified state to a object. + /// + [RequiresUnreferencedCode("The type T may be trimmed.")] + JsonElement SerializeToElement(object? value, CancellationToken cancellationToken = default); + + /// + /// Deserializes the specified state. + /// + [RequiresUnreferencedCode("The type T may be trimmed.")] + T Deserialize(string json, CancellationToken cancellationToken = default); + + /// + /// Deserializes the specified state. + /// + [RequiresUnreferencedCode("The type T may be trimmed.")] + T Deserialize(JsonElement element, CancellationToken cancellationToken = default); } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowStateSerializer.cs b/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowStateSerializer.cs index 5a113bef3..89325b77d 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowStateSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/IWorkflowStateSerializer.cs @@ -16,8 +16,9 @@ public interface IWorkflowStateSerializer /// The cancellation token. /// The serialized workflow state. [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version Serialize instead.")] Task SerializeAsync(WorkflowState workflowState, CancellationToken cancellationToken = default); - + /// /// Serializes the specified workflow state. /// @@ -25,8 +26,18 @@ public interface IWorkflowStateSerializer /// The cancellation token. /// The serialized workflow state. [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + string Serialize(WorkflowState workflowState, CancellationToken cancellationToken = default); + + /// + /// Serializes the specified workflow state. + /// + /// The workflow state to serialize. + /// The cancellation token. + /// The serialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version SerializeToUtfBytes instead.")] Task SerializeToUtfBytesAsync(WorkflowState workflowState, CancellationToken cancellationToken = default); - + /// /// Serializes the specified workflow state. /// @@ -34,8 +45,18 @@ public interface IWorkflowStateSerializer /// The cancellation token. /// The serialized workflow state. [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + byte[] SerializeToUtfBytes(WorkflowState workflowState, CancellationToken cancellationToken = default); + + /// + /// Serializes the specified workflow state. + /// + /// The workflow state to serialize. + /// The cancellation token. + /// The serialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version SerializeToElement instead.")] Task SerializeToElementAsync(WorkflowState workflowState, CancellationToken cancellationToken = default); - + /// /// Serializes the specified workflow state. /// @@ -43,32 +64,81 @@ public interface IWorkflowStateSerializer /// The cancellation token. /// The serialized workflow state. [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + JsonElement SerializeToElement(WorkflowState workflowState, CancellationToken cancellationToken = default); + + /// + /// Serializes the specified workflow state. + /// + /// The workflow state to serialize. + /// The cancellation token. + /// The serialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version Serialize instead.")] Task SerializeAsync(object workflowState, CancellationToken cancellationToken = default); - + + /// + /// Serializes the specified workflow state. + /// + /// The workflow state to serialize. + /// The cancellation token. + /// The serialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + string Serialize(object workflowState, CancellationToken cancellationToken = default); + /// /// Deserializes the specified serialized state. /// /// The serialized state. /// The cancellation token. /// The deserialized workflow state. - [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] Task DeserializeAsync(string serializedState, CancellationToken cancellationToken = default); - + /// /// Deserializes the specified serialized state. /// /// The serialized state. /// The cancellation token. /// The deserialized workflow state. - [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + WorkflowState Deserialize(string serializedState, CancellationToken cancellationToken = default); + + /// + /// Deserializes the specified serialized state. + /// + /// The serialized state. + /// The cancellation token. + /// The deserialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] Task DeserializeAsync(JsonElement serializedState, CancellationToken cancellationToken = default); - + /// /// Deserializes the specified serialized state. /// /// The serialized state. /// The cancellation token. /// The deserialized workflow state. - [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + WorkflowState Deserialize(JsonElement serializedState, CancellationToken cancellationToken = default); + + /// + /// Deserializes the specified serialized state. + /// + /// The serialized state. + /// The cancellation token. + /// The deserialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] Task DeserializeAsync(string serializedState, CancellationToken cancellationToken = default); + + /// + /// Deserializes the specified serialized state. + /// + /// The serialized state. + /// The cancellation token. + /// The deserialized workflow state. + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + T Deserialize(string serializedState, CancellationToken cancellationToken = default); } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs b/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs index 5c18306fc..9c7f5c2c4 100644 --- a/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs +++ b/src/modules/Elsa.Workflows.Core/Extensions/ActivityExecutionContextExtensions.cs @@ -253,7 +253,7 @@ public static class ActivityExecutionContextExtensions // Serializing the value ensures we store a copy of the value and not a reference to the input, which may change over time. if (inputDescriptor.IsSerializable != false) { - var serializedValue = await context.GetRequiredService().SerializeToElementAsync(value); + var serializedValue = context.GetRequiredService().SerializeToElement(value); context.ActivityState[inputDescriptor.Name] = serializedValue; } @@ -409,7 +409,7 @@ public static class ActivityExecutionContextExtensions if (outputValue == null!) continue; - var serializedOutputValue = await serializer.SerializeAsync(outputValue, cancellationToken); + var serializedOutputValue = serializer.Serialize(outputValue, cancellationToken); context.JournalData[outputName] = serializedOutputValue; } diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonWorkflowStateSerializer.cs b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonWorkflowStateSerializer.cs index 3d71dd61c..341f2ff5f 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonWorkflowStateSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonWorkflowStateSerializer.cs @@ -31,62 +31,100 @@ public class JsonWorkflowStateSerializer : ConfigurableSerializer, IWorkflowStat /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version Serialize instead.")] public Task SerializeAsync(WorkflowState workflowState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return Task.FromResult(JsonSerializer.Serialize(workflowState, options)); + return Task.FromResult(Serialize(workflowState, cancellationToken)); } /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version SerializeToUtfBytes instead.")] public Task SerializeToUtfBytesAsync(WorkflowState workflowState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return Task.FromResult(JsonSerializer.SerializeToUtf8Bytes(workflowState, options)); + return Task.FromResult(SerializeToUtfBytes(workflowState, cancellationToken)); } /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version SerializeToElement instead.")] public Task SerializeToElementAsync(WorkflowState workflowState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return Task.FromResult(JsonSerializer.SerializeToElement(workflowState, options)); + return Task.FromResult(SerializeToElement(workflowState, cancellationToken)); } /// - [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The serialization process may require access to the type.")] + [Obsolete("Use the non-async version Serialize instead.")] public Task SerializeAsync(object workflowState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - var json = JsonSerializer.Serialize(workflowState, workflowState.GetType(), options); - return Task.FromResult(json); + return Task.FromResult(Serialize(workflowState, cancellationToken)); } /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] public Task DeserializeAsync(string serializedState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - var workflowState = JsonSerializer.Deserialize(serializedState, options)!; - return Task.FromResult(workflowState); + return Task.FromResult(Deserialize(serializedState, cancellationToken)); } /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] public Task DeserializeAsync(JsonElement serializedState, CancellationToken cancellationToken = default) { - var options = GetOptions(); - var workflowState = serializedState.Deserialize(options)!; - return Task.FromResult(workflowState); + return Task.FromResult(Deserialize(serializedState, cancellationToken)); } /// [RequiresUnreferencedCode("The type 'T' may be trimmed from the output. The deserialization process may require access to the type.")] + [Obsolete("Use the non-async version Deserialize instead.")] public Task DeserializeAsync(string serializedState, CancellationToken cancellationToken = default) + { + return Task.FromResult(Deserialize(serializedState, cancellationToken)); + } + + public string Serialize(WorkflowState workflowState, CancellationToken cancellationToken = default) { var options = GetOptions(); - var workflowState = JsonSerializer.Deserialize(serializedState, options)!; - return Task.FromResult(workflowState); + return JsonSerializer.Serialize(workflowState, options); + } + + public byte[] SerializeToUtfBytes(WorkflowState workflowState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.SerializeToUtf8Bytes(workflowState, options); + } + + public JsonElement SerializeToElement(WorkflowState workflowState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.SerializeToElement(workflowState, options); + } + + public string Serialize(object workflowState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.Serialize(workflowState, workflowState.GetType(), options); + } + + public WorkflowState Deserialize(string serializedState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.Deserialize(serializedState, options)!; + } + + public WorkflowState Deserialize(JsonElement serializedState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return serializedState.Deserialize(options)!; + } + + public T Deserialize(string serializedState, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.Deserialize(serializedState, options)!; } /// diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/SafeSerializer.cs b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/SafeSerializer.cs index a9437dba0..6a868c337 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/SafeSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/SafeSerializer.cs @@ -22,39 +22,59 @@ public class SafeSerializer : ConfigurableSerializer, ISafeSerializer [RequiresUnreferencedCode("The type T may be trimmed.")] public ValueTask SerializeAsync(object? value, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return ValueTask.FromResult(JsonSerializer.Serialize(value, options)); + return ValueTask.FromResult(Serialize(value, cancellationToken)); } /// [RequiresUnreferencedCode("The type T may be trimmed.")] public ValueTask SerializeToElementAsync(object? value, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return new(JsonSerializer.SerializeToElement(value, options)); + return new(SerializeToElement(value, cancellationToken)); } /// [RequiresUnreferencedCode("The type T may be trimmed.")] public ValueTask DeserializeAsync(string json, CancellationToken cancellationToken = default) { - var options = GetOptions(); - return new(JsonSerializer.Deserialize(json, options)!); + return new(Deserialize(json, cancellationToken)); } /// [RequiresUnreferencedCode("The type T may be trimmed.")] public ValueTask DeserializeAsync(JsonElement element, CancellationToken cancellationToken = default) + { + return new(Deserialize(element, cancellationToken)); + } + + public string Serialize(object? value, CancellationToken cancellationToken = default) { var options = GetOptions(); - return new(element.Deserialize(options)!); + return JsonSerializer.Serialize(value, options); + } + + public JsonElement SerializeToElement(object? value, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.SerializeToElement(value, options); + } + + public T Deserialize(string json, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return JsonSerializer.Deserialize(json, options)!; + } + + public T Deserialize(JsonElement element, CancellationToken cancellationToken = default) + { + var options = GetOptions(); + return element.Deserialize(options)!; } /// protected override void AddConverters(JsonSerializerOptions options) { var expressionDescriptorRegistry = ServiceProvider.GetRequiredService(); - + options.Converters.Add(new JsonStringEnumConverter(JsonNamingPolicy.CamelCase)); options.Converters.Add(new TypeJsonConverter(WellKnownTypeRegistry.CreateDefault())); options.Converters.Add(new SafeValueConverterFactory()); diff --git a/test/unit/Elsa.Workflows.Core.UnitTests/SerializerEncodingTests.cs b/test/unit/Elsa.Workflows.Core.UnitTests/SerializerEncodingTests.cs index 803cb7f9b..2e9901668 100644 --- a/test/unit/Elsa.Workflows.Core.UnitTests/SerializerEncodingTests.cs +++ b/test/unit/Elsa.Workflows.Core.UnitTests/SerializerEncodingTests.cs @@ -32,17 +32,17 @@ public class SerializerUnicodeEncodingTests(ITestOutputHelper testOutputHelper) } [Fact] - public async Task TestSafeSerializer() + public void TestSafeSerializer() { var serializer = _serviceProvider.GetRequiredService(); - await TestSerializerAsync(input => serializer.SerializeAsync(input).AsTask()); + TestSerializer(input => serializer.Serialize(input)); } [Fact] - public async Task TestWorkflowStateSerializer() + public void TestWorkflowStateSerializer() { var serializer = _serviceProvider.GetRequiredService(); - await TestSerializerAsync(input => serializer.SerializeAsync(input)); + TestSerializer(input => serializer.Serialize(input)); } private void TestSerializer(Func serialize) @@ -56,18 +56,6 @@ public class SerializerUnicodeEncodingTests(ITestOutputHelper testOutputHelper) var serializedStringValue = GetSerializedTextValue(serializedJson); Assert.Equal(unicodeString, serializedStringValue); } - - private async Task TestSerializerAsync(Func> serialize) - { - var unicodeString = UnicodeRangeGenerator.GenerateUnicodeString(); - var anonymousObject = new - { - Text = unicodeString - }; - var serializedJson = await serialize(anonymousObject); - var serializedStringValue = GetSerializedTextValue(serializedJson); - Assert.Equal(unicodeString, serializedStringValue); - } private string GetSerializedTextValue(string serializedJson) { From 7d45db355d88c24c8ab2170ff753df323a6fa77c Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 27 Sep 2024 13:07:08 +0200 Subject: [PATCH 14/31] Refactor serializer options retrieval method name. Renamed GetPayloadSerializerOptions to GetOptions for simplicity and updated method accessibility. This change ensures that the method name is more intuitive and aligns with common naming conventions. Additionally, it introduces a new public GetOptions method in the IPayloadSerializer interface. --- .../Contracts/IPayloadSerializer.cs | 5 +++++ .../Serializers/JsonPayloadSerializer.cs | 11 ++++++----- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/src/modules/Elsa.Workflows.Core/Contracts/IPayloadSerializer.cs b/src/modules/Elsa.Workflows.Core/Contracts/IPayloadSerializer.cs index 5768049ea..762765bf4 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/IPayloadSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/IPayloadSerializer.cs @@ -48,4 +48,9 @@ public interface IPayloadSerializer /// The serialized state. /// The deserialized state. T Deserialize(JsonElement serializedData); + + /// + /// Gets the JSON serializer options. + /// + JsonSerializerOptions GetOptions(); } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonPayloadSerializer.cs b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonPayloadSerializer.cs index cba5c7c01..820893889 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonPayloadSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Serializers/JsonPayloadSerializer.cs @@ -25,14 +25,14 @@ public class JsonPayloadSerializer : IPayloadSerializer /// public string Serialize(object payload) { - var options = GetPayloadSerializerOptions(); + var options = GetOptions(); return JsonSerializer.Serialize(payload, options); } /// public JsonElement SerializeToElement(object payload) { - var options = GetPayloadSerializerOptions(); + var options = GetOptions(); return JsonSerializer.SerializeToElement(payload, options); } @@ -51,18 +51,19 @@ public class JsonPayloadSerializer : IPayloadSerializer /// public T Deserialize(string payload) { - var options = GetPayloadSerializerOptions(); + var options = GetOptions(); return JsonSerializer.Deserialize(payload, options)!; } /// public T Deserialize(JsonElement payload) { - var options = GetPayloadSerializerOptions(); + var options = GetOptions(); return payload.Deserialize(options)!; } - private JsonSerializerOptions GetPayloadSerializerOptions() + /// + public JsonSerializerOptions GetOptions() { var options = new JsonSerializerOptions { From 750b4804fe09b1daa4b26824b0261a4eac43f20f Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 27 Sep 2024 13:10:22 +0200 Subject: [PATCH 15/31] Add GetOptions method to ISafeSerializer interface This new method retrieves the JSON serializer options, enhancing flexibility in how JSON data is handled. It provides an additional way to customize serialization settings contextually. --- src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs b/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs index 79f18e7b2..1bf90ac24 100644 --- a/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs +++ b/src/modules/Elsa.Workflows.Core/Contracts/ISafeSerializer.cs @@ -59,4 +59,9 @@ public interface ISafeSerializer /// [RequiresUnreferencedCode("The type T may be trimmed.")] T Deserialize(JsonElement element, CancellationToken cancellationToken = default); + + /// + /// Gets the JSON serializer options. + /// + JsonSerializerOptions GetOptions(); } \ No newline at end of file From 3baacc83b3266c6dac46faf8a67789056a425a35 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 27 Sep 2024 17:55:47 +0200 Subject: [PATCH 16/31] Change WorkflowVersion property in WorkflowExecutionLogRecord Modified the WorkflowVersion property from init-only to settable. This adjustment allows for updating the workflow version post-initialization, providing greater flexibility in handling workflow instances. --- .../Entities/WorkflowExecutionLogRecord.cs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/modules/Elsa.Workflows.Runtime/Entities/WorkflowExecutionLogRecord.cs b/src/modules/Elsa.Workflows.Runtime/Entities/WorkflowExecutionLogRecord.cs index 11194aeb4..ecea3665b 100644 --- a/src/modules/Elsa.Workflows.Runtime/Entities/WorkflowExecutionLogRecord.cs +++ b/src/modules/Elsa.Workflows.Runtime/Entities/WorkflowExecutionLogRecord.cs @@ -26,7 +26,7 @@ public class WorkflowExecutionLogRecord : Entity, ILogRecord /// /// The version of the workflow definition. /// - public int WorkflowVersion { get; init; } + public int WorkflowVersion { get; set; } /// /// The ID of the activity instance. From fdb80613247aeba9b881876466e2d5dbf2e55f0a Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Mon, 30 Sep 2024 14:31:51 +0200 Subject: [PATCH 17/31] Update package versions to latest stable releases Upgraded various packages to their latest stable versions to ensure compatibility and take advantage of new features and bug fixes. Notable upgrades include Microsoft.EntityFrameworkCore, Microsoft.Data.Sqlite, and Polly. --- Directory.Packages.props | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/Directory.Packages.props b/Directory.Packages.props index 9dc0dd505..ab8552aff 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -138,17 +138,17 @@ - - - - - - - - - - - + + + + + + + + + + + @@ -156,14 +156,14 @@ - + - - - + + + From 86826a89dc8edbc2e7a3cd4e96427a61b324bef4 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 3 Oct 2024 11:16:14 +0200 Subject: [PATCH 18/31] Add new Elsa logo image Introduce a new Elsa logo image to the design assets. This image will be used in the custom 2x design section. --- design/custom/2x/elsa-logo.png | Bin 0 -> 10238 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 design/custom/2x/elsa-logo.png diff --git a/design/custom/2x/elsa-logo.png b/design/custom/2x/elsa-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..baa1319c08eb46aa2fbd21c219e03228b0fe27d2 GIT binary patch literal 10238 zcmbVy30zFw|Nm{JnzT|AGL0#TwyC5cTBn)PDwQHNb4SIr@5s}lX&Wt*2t`ev$I?Pk z+LNfp(xOF+NRhN6MAGv=_xAAme!s8pfBSj4=ALst=d-*&@6S2+-XqD*#zI_FK@@@@ zaVo{k0fP9k$hA@!^oS{_r+^=k0LuPg2wJ0zTzpgq!UqTv{Nm&265?XLheY@H)$oM< zX$*}B-vA&DK|6Ox1bEVqGD2`PhL?|@A%3=|0gv;64e`!;)>_s9rVMW%%CR8E-eWe7 z^kYZq1~7j2F5J!t5>VjF2=T;4_%i*1NfCy4R4)nKBi)*K94ZoW)DUls7{s|)+u=<8 zgBUmi4Q+M0mX@beUi2=?w>z+U@-_j zOmZ+I|6>od4DsF}Aps;!&G7JWjc^?e{~#|-Z36=XO)a7(k*E$N)Pp1aLOdhX{en0C z(qP62rU&^1g!uUT;Si0UH2=^LLp< zQT>8LJpJels+l1kM6KcD1Cw;Mw-L4UJ$2RT$UnL|G={pTwk}Z}Heh%XY5F=eEuF)^ z=9~G`Ly_G6nh*bf&bJNn0rlm{{2%KHi_!%VNR)heN_WgBZqM0ImNALTLi`HIZ6ejzsf6 zgG9&vrEY(TgT(;3{k;-_#_zR^;RhD5Ah1%s_HyWhApA|LnXzNU!1R!9V(#FFq5L;Up0e3ovR>6X=l zWi7q%Js)y+OdEbeh5O8;Yin zSN0C2gx5b-gK(QQ!Y{wFsBWM8=2`i?CKYD8s`fHQYA;Gd$-{e`cm0*Pf_Ke9?LI6K zrm3Z`0D0topGmWMW=veTk)K`mmJb>aE>w0Rsq{povnLb3+QL+MJA6Hy&P@mV}H_ob;QiI*1m)TEXXCKbGVIMkd{C?RT1F0hr@+n3coT%PNWJp% zH0KlCwqFhh;eu&d^`=DTX$Z2D)7}}R$-5T5cvmDBismHIwZbyl6V93k4nPp$&W#L{ z>3A7?KBzKd6;J+vmP)rf1O;@3Js%~7pXexDEW7+kM9N}wO#yTdmPf;x8ZCD12l zq~gjUNS+og$Fuu`4>HOf+Ehbr)@#fTF@~V@Ll=*B#dZcN0qbtY<1g{b^!D<&Lk9#Q z2v$=N6o8@^&FS2gJe)iP>E4HXImTKLq-vpy?5;<2pJicUfyPMg>+CQHN+ciSl(yul zL(qWby0@h}sJy=5>8;X-@DMcpXRp+oP?6?7MXwBCl;47r!8UxJdUW@wl>mgp{^gK? z$t6J0m6nJ?Wz$U}5JU~nVV64)WsmVg={x6&GueLYnr zgo061kaqcIG9G)pWc|r80nl1EW}%EtHspis1YL_^)vtn3o=aP|I(BXs3_;0^vNgP0 z$$U`s+1_eGlNtaZyU3APM1!E{lg>KmudCDV-zN{jb3~k$_8E9edq@U?m`@gOaRMJC ztc0L#o=hA9&6S?VyNn>{mWYLL7Mr?35rX<+UD$nwiEz%x*QYNHD(v%^ zBTif{+D`?$BFei*O78o<{0|=lc{r>$TQcoi%=~-Jp?5>NT>g%ygwa|FX21yl%gApb z;aT@ji*t;L9nC$dF7GRUE?Lyxz2SMp4Zy59^Tkt~EKe+jfD%fY^L6LpJ>S!d4n)K0nBnFu@j=x=+Po9z^@AR9l6m)Xx0;R>$)VyfZ3{2J zi5EJj{=yN5B?we{jx`UY+0OFB4Jz~DqMvbr8*JJq9)6BpD?*5aAvbB6oH~7df5vnc zJ5{1(LWm;FgBC8AzW!Mic#w6Bt=R33*!1Xd6>q9N+47Cd39NcEE#-S1(S0k*K_u6f zLgvkBkHiMh5y5*0PEXh}4xVF+#PR|E`6N$^R-UMXWt~4qemUc)&yT2|82Lu~`0%p% znkZ?^vdlut?&r#*m_`U0P9$eNygZ525I`B?+P{C(F3GNAG5HXG_0sT{-BnAPt|SJI zR8x!qZh4dNm7GK8*hOjK4J7Z!@knl>&)%B6cS4F=s)5~q_hxk-g0BbBTgSIZ32RZd znmwALaTu|Dhr{1S!bkfE@C$anaf5Y;xZE>=e+T?aV%YXm^!?3=(6>Hz1pi6Dr+>lEy0BimfLl8?g&@6aFZ6XJCC#f^XCrbyzj z(Z?^s-fluW6Dh(Sn%YkRM#ZVC2>s5uvy$3c2S&!W2z_B5bD+D6h)vs1`+#yY`r3PYm0m zS=oVTY^SmkB!*v)?pkpJPu`x6F~5vdgwfjvjd=vq&P{0AzUe6Q0`BZEB1UtlAI9fH z2pchN=ECWiyaO5;>;aNhFQ!chG!asbK0x4zE}Q*wiqI6d0*NaZj`#p6?wkolNf*~r zw@SWsUo~7=3Czp4ngh@gBma7V0pKV=^*^s=8|WxXStK0fiiRP;xaCpA%C8F|fFvMX z{Lc%AMWRGU5sBXy#6x(!Jc>yCzLp(CM^WY@sX;CPG;s3Y-TxJe9dy_5L?Yb%Y1p8C zoFAlN*nJbW=l0wOA31q|WzhNkcx-6kkDqCk_dvw5kJ}2~QCQWe^LgKnu?3Y)4+2=A zc>~_HxPvRk4tfLc9?_>Qw(+i=xBcV&E;$9DWh8M!KY=bQk#}BMZXIZIm)pff7VTBT zK&|&dWvuIdG%5M6&JveZ{%JvO$~-HUx(tA>!jgjCfP~f-b3&Ax;#PvZ6-#qo-_u2q zM0)QiH+g6PS>b>TjMsS-&$mO*eXN?Vl5SQ@B7|;9eJ{lEJd7sN&al&*N__GJg-C78 zL1R96fs+r(0=50}1XDg$1R}e(pE(Kei?1J)W1qDVmqwnziO9FPx3&n0HhSSOYj!m$w$v~ao z*dkZAiAolV&+8KNQ$hT_)`Y|A8Gb%|c6r;w0Y)gHYwbj1qZc908&PAlNrcxCh$J{h zkVIW|`75w%hGxWdAC=YXgvfaBQ^w*H5J&{U2s$m=BOSvwNXJNuBaX-Nab9Of zAU-`y+;H-x6=9v9QOP!A;s@y-oatrf|{e>jZ&nTO`bov}l;%B1^jcMv0%p$23`I_CCs( z=&=vMEbm#@-qqn8j7txqj96yr-LLNCBro76__x)ntlr_faGz*J>c%J+G1_ z;)i=GAh0+51OXWFAYCjSla68-#K(y_eRP{_A_wS38l)QgJrc@5I7{dD6*oa2YV&Ka z%X9HP+r8-TI)fG2V7AA&lIz%k?v|!k*{(l78P0Rqq{Q0K&mL>svG4ZX{xJ&}{#>O@ zGx4h1`C+)eaJDfisX+xBN)_&u|LBywnTx23L#^vShKi*XgKdB-eyo~S{lRA zrtxqJ>N2Y|SAaw%3aP!v_irW8A8?Fy2b{HTtUn9fqQy)I#lgBjt!VOUtT2ALJ z*sSeRRGfF8jS`knFg`8mm2kzl=-#{xr<0%OB6oVnZ1tdd&dSo=MTmL!YiD&P5LlF) zan_=cQwSTS4V#nJtI$K6CXijuNnKQu{Y_~n1sS6s)aB+i#}|izY_W@6ZJ)z0ZkS&% zt8TX-%^CrZA=_$$LtM8>_Z*|Jrax!AzqPvx9E3J>0 zT`QIakGp%{yx>|ArO#r7@~eFRjQF08al^OXj5`Grwqhqk6_TRt`%V?MbR6HD!G2L~ zhIy)1e-A9MU@UU+#;m->m!Er5vh|Y(@o8m1XZ0 zJL3L8qOaP7D4oe}9h|Mb?R`nOACy$w0gt}vjO_#5wsf$MZY(P+`}A_@XIlL%S!pdw zNA|HLAzteNBJ*-Of^e(QK!u?RkqETZ%(~Yzf1F|h?i*`#eEzzU*J?-X4Ib~1(5T61 zMPQ3F(fD-f<@do?OJ7Ir8_rS4WKQS$m*w6O&SS+lv9;Y4mXjkp_1&R}z|!mC(!5px zT#Cu%sS}d|`pz?^bzl*YKWTFH$L86QwsA7^Q@yNrMo(M$?I$%1yqcLV*Wwd;PT; zZgfw|wYsBoKUnDe^A~rNdboNoeNpgzVWNbsAZx9&KN`+py!~0x;6))fa-@<{96hh8 z%W^UjU~j5A{pb~Iuu#OoeI~|})pitCiPUEc*_oD!;3KnZ0KQLpRfCSG^1##Hp2@8R zM+u_ydrw>i2$#M$P@T~G^cQcRILDq($4YdEj@REy7O(t|J_qmU*a^K`b^QwBwFGK& zz!XVqc41#QSf9y?BAKfS!sd>KD`UmM#-B2EHMIWw*hMjey4?WWajR{z2{(Fh*rya$ zkC-{)6|9b(6zNh~dqj`Ye2rlGZen;NQ4CnQejH!s>#;E?)AqYvmCkqYMCYw-MjbN3! zvilvP$>}k9tM~=nh0d?MftyT@eHz@S$h-Oye-i#PAOZ_Q+rK)gnUkN8J}txXJWhDd z=X$eYX-@3YSKzH~_S?5_ttuP;hW}Cj#LAz>sTYDUp9TFc2YMv3%pa6 zoD&?h?QsPe$nYF#uE^zh<&u*)`AhiX{Qaftr3;HLJwRZ))HhF9x4Fj$d|FD;G{&n&YL^c;m^K2#<;nZ)+Y7mpA~KO{J&@ z<93aSVkWoGVFW5%`>_ra-<5y?O8oxatDO=okDG5i%Ih$ui6We^xA{q?$wf-#ZmP0u zPNmD8c%|OADruj!P!5>U+A6(Csd-mX&*`v4aw;d^y!?CQ8g28wu2hgCA6+N?3v#Dl z5aK*s$|V8q%%;g!t_Ok_j{7zIE%mC!O;duA9HMemJc$Z~|COXES9EZZ0*O%$Hz0Y3z(bK)}KRMvR9!u6AH4 z0*yE(l=Ub+@D(DBQCbUCA@)x(q3PnPZx4By1IuF1(wI{}7EEU&lOR!2JdnJuT)KF) zO*mjQN6Z%4Mkp-DGcK#P8O#1D6Xe;3ytWcdk65y|gl`kI+*q-SF4iey1gxiD2!8u0 zOF}k*D_DP`y^cgag@lg}yp?%O{Ech4RAJam5E3ghAIiGKRwxXeVg72CU%6!X0~7|* z9=H%enKX$x>j)Yk8|wS_?~l|}9B}vW-Z46FTQZ>_1zaLDnt6IVzB_(C-}l7V)EcMR z?=N@XtPPq_*S`S(Q`+2eLwMkUT*KF=JuCix`$b|1Yz-6^%t?{c)VSVbl)2d0x1soD z|M#mB#qJ2)98a#(D6j4At`GN{u6^%HTH(~NFnYOoaY6sFpBds*kWEe;twfmoa42p4 z!ne}?%nem@NO+qG+RjaP?79!WhM{mK7G*F4hM%%F{4DI3L!55yjhrsO ztPvSV8*D0Hn&G7>{@`hLA40sw4_aYn9j?Y)yp(gK@+=ZJkuox@7OK!!v_xC{ar0UO z$~WQmtgNj0`KX`y^R+IWTjURYe|_{Hb4R#1S>qz6;kLph6Ocj4>)YEKK8$eXe5)Qb z89G(G5J5bksCPy>sFZTvPRIe|&5oprpi}uIq9VJ6PH-K+o3OB|wDfYz{=aLh{$+9R};Y=jn5xF~ZQMY5IgsMu& z%S;MOI@ICn_Vm6alF%z>y?yn>euXg#-$6^DYGT>QLo=mu@5;H7notf}ji!%MOga|q z$2X&uw3+DNFxSQfJrPUsa&iY5s3+;Pf?cNTmqRR9c29g@^iLyelY3(L37;IC{1bJT7N=ixw{{?Vrlx^ zhNX{(URuZt6RJZGnEx|CsIX>cqWC&k(FWhND@HX}P%jS=0v{C@TA(Q7^?9Z(OFu{3 zUk1v>bXeoTL>A@C_mr!tQ4}0zNE$`s6R4fy9?g@}7tLs5G{O(?z!7!|$tp)O>e=dGE!GBq8U7<;ko) zOfkt6+Y%b$BT%MBS(mrQJXO&2@ePy=PC7^MinkC2=@_X(Y}VY^MgJQQ5#CZ*i9MLB zMaf9gX?ms>NvJ#n-HM;vBVizf-KkBkG2P>)n)5*V$^XmfFHKecV=~Oi6US)^SM3zNGWhTWF?^54`lXoY6WGv=E9 zSnP}~kzUR_M_@xrR!yW|s!^SZvA1?eDPi#jn_i<|jy?mUN}OQn z88zx5;!N$NpK~72%-fc3V`9-^l$K@Y(i`ftt|F2M2OVrx z)+1%7RTs11p%WnTE zvF|dnwI$wB((chIxJ{+7OzcqltH)+U=D+*0M^*_t$(1^E3sq=Mm=F|`W_#=;6zqMp zVVy8l7ooZocwGhu_ z2!~q%ZNQG`O>>M;%G`Ok3BC!h`xtX#Y$DFp_!=_w%wKf-RMG6$9(P{$r z^1gLJe6K0;z((C+-+|q&$TQdbBEOc(s%9(I*mG4|BUKS{UCD4co$BQ1hVOY!XA-b7 zyYeP%BE>iM@^1n8r2KH&YO)p6c_Zg>olR*5oPExI^sMF7$A<#l>6%DrZJ8K5S?-v{18-e_=r5u4g&u* zY1`IsY=t&Yj4UnikhzNBd!yX zfQ*9Xw7&Mf7AU8%N=Ly#cN2V2!|p1qDkucy4ejZ9%U5p6ngd~Y>sd^ML=PyNwtxd- zhG;Vde8dEY+-hyzoz!R9@aII9rW_Q_FG#<_@niuA-mYwJRYpYYSRT4~>=9ky?jbFO zOz;U4x<|?Ox71R|Lhg+gd~XzIVKR;Rplk7Q)(x-Rl(2@}*F}H;EUz;0*{{76R+k$e zB!5A5x4eJX@w$DS&aROgHUd!e5{c40@YTjY7yc}J>P8ocGKRnX3ZdYt&1hl@z`**h zh04H!qJKI3xwZM!G|=dm{fBn%*YJo*1Z(<`M0dOpq?&6ZIy(as3M{t$RRmxZ)tGs4 z@!9bmn|b5=d$jfnKz0t(4;_!pP7;)XNb~dUvm*i!{Ddi5F4s>br!hmoYYRktp@6mh zxjwfR_6`?J{ z#6Xh6tJ~&Q&BEnt_ns1fjJ8smRh_n{s>%yNOuBb(BWGeOg$Wn;s-IUxCF;ANcx$8Do(-+1U{k53#WIoaC iYR$e~e(_w=(uT0Tr>=)e$MyhzLR51bvl0`J*#8AowwK8O literal 0 HcmV?d00001 From 679a0515c87fbbd912e6375998d76a3b01930d8c Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 3 Oct 2024 18:50:05 +0200 Subject: [PATCH 19/31] Add TypeJsonConverter and enhance JSON serialization Introduced a TypeJsonConverter to handle Type serialization and deserialization. Updated dependency injection and helper methods to support customized JSON serializer options, improving flexibility and configurability. --- .../Converters/TypeJsonConverter.cs | 56 +++++++++++++++++++ .../DependencyInjectionExtensions.cs | 6 +- .../Helpers/RefitSettingsHelper.cs | 19 +++---- .../Options/ElsaClientBuilderOptions.cs | 6 ++ .../Converters/TypeJsonConverter.cs | 2 +- 5 files changed, 73 insertions(+), 16 deletions(-) create mode 100644 src/clients/Elsa.Api.Client/Converters/TypeJsonConverter.cs diff --git a/src/clients/Elsa.Api.Client/Converters/TypeJsonConverter.cs b/src/clients/Elsa.Api.Client/Converters/TypeJsonConverter.cs new file mode 100644 index 000000000..60602c7bc --- /dev/null +++ b/src/clients/Elsa.Api.Client/Converters/TypeJsonConverter.cs @@ -0,0 +1,56 @@ +using System.Text.Json; +using System.Text.Json.Serialization; +using Elsa.Api.Client.Extensions; +using JetBrains.Annotations; + +namespace Elsa.Api.Client.Converters; + +/// +/// Converts objects to and from their assembly-qualified name. +/// +[UsedImplicitly] +public class TypeJsonConverter : JsonConverter +{ + /// + public override bool CanConvert(Type typeToConvert) + { + return typeToConvert == typeof(Type) || typeToConvert.FullName == "System.RuntimeType"; + } + + /// + public override Type? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + var typeName = reader.GetString()!; + + // Handle collection types. + if (typeName.EndsWith("[]")) + { + var elementTypeName = typeName[..^"[]".Length]; + var elementType = Type.GetType(elementTypeName)!; + return typeof(List<>).MakeGenericType(elementType); + } + + return Type.GetType(typeName); + } + + /// + public override void Write(Utf8JsonWriter writer, Type value, JsonSerializerOptions options) + { + // Handle collection types. + if (value is { IsGenericType: true, GenericTypeArguments.Length: 1 }) + { + var elementType = value.GenericTypeArguments.First(); + var typedEnumerable = typeof(IEnumerable<>).MakeGenericType(elementType); + + if (typedEnumerable.IsAssignableFrom(value)) + { + var elementTypeName = value.GetSimpleAssemblyQualifiedName(); + JsonSerializer.Serialize(writer, $"{elementTypeName}[]", options); + return; + } + } + + var typeName = value.GetSimpleAssemblyQualifiedName(); + JsonSerializer.Serialize(writer, typeName, options); + } +} \ No newline at end of file diff --git a/src/clients/Elsa.Api.Client/Extensions/DependencyInjectionExtensions.cs b/src/clients/Elsa.Api.Client/Extensions/DependencyInjectionExtensions.cs index 1dec341fd..e7bed4507 100644 --- a/src/clients/Elsa.Api.Client/Extensions/DependencyInjectionExtensions.cs +++ b/src/clients/Elsa.Api.Client/Extensions/DependencyInjectionExtensions.cs @@ -124,7 +124,7 @@ public static class DependencyInjectionExtensions /// An options object that can be used to configure the HTTP client builder. public static IServiceCollection AddApi(this IServiceCollection services, Type apiType, ElsaClientBuilderOptions? httpClientBuilderOptions = default) { - var builder = services.AddRefitClient(apiType, _ => CreateRefitSettings(), apiType.Name).ConfigureHttpClient(ConfigureElsaApiHttpClient); + var builder = services.AddRefitClient(apiType, sp => CreateRefitSettings(sp, httpClientBuilderOptions?.ConfigureJsonSerializerOptions), apiType.Name).ConfigureHttpClient(ConfigureElsaApiHttpClient); httpClientBuilderOptions?.ConfigureHttpClientBuilder(builder); httpClientBuilderOptions?.ConfigureRetryPolicy?.Invoke(builder); return services; @@ -139,7 +139,7 @@ public static class DependencyInjectionExtensions public static void AddApiWithoutRetryPolicy(this IServiceCollection services, ElsaClientBuilderOptions? httpClientBuilderOptions = default) where T : class { var builder = services - .AddRefitClient(_ => CreateRefitSettings(), typeof(T).Name) + .AddRefitClient(sp => CreateRefitSettings(sp), typeof(T).Name) .ConfigureHttpClient(ConfigureElsaApiHttpClient); httpClientBuilderOptions?.ConfigureHttpClientBuilder(builder); } @@ -156,7 +156,7 @@ public static class DependencyInjectionExtensions /// Creates an API client for the specified API type. public static T CreateApi(this IServiceProvider serviceProvider, HttpClient httpClient) where T : class { - return RestService.For(httpClient, CreateRefitSettings()); + return RestService.For(httpClient, CreateRefitSettings(serviceProvider)); } private static void ConfigureElsaApiHttpClient(IServiceProvider serviceProvider, HttpClient httpClient) diff --git a/src/clients/Elsa.Api.Client/Helpers/RefitSettingsHelper.cs b/src/clients/Elsa.Api.Client/Helpers/RefitSettingsHelper.cs index ec47e8710..8a87220a3 100644 --- a/src/clients/Elsa.Api.Client/Helpers/RefitSettingsHelper.cs +++ b/src/clients/Elsa.Api.Client/Helpers/RefitSettingsHelper.cs @@ -10,17 +10,12 @@ namespace Elsa.Api.Client; /// public static class RefitSettingsHelper { - private static JsonSerializerOptions? _jsonSerializerOptions; - /// /// Creates a instance configured for Elsa. /// - public static RefitSettings CreateRefitSettings() + public static RefitSettings CreateRefitSettings(IServiceProvider serviceProvider, Action? configureJsonSerializerOptions = null) { - var settings = new RefitSettings - { - ContentSerializer = new SystemTextJsonContentSerializer(CreateJsonSerializerOptions()) - }; + var settings = new RefitSettings { ContentSerializer = new SystemTextJsonContentSerializer(CreateJsonSerializerOptions(serviceProvider, configureJsonSerializerOptions)) }; return settings; } @@ -28,11 +23,8 @@ public static class RefitSettingsHelper /// /// Creates a instance configured for Elsa. /// - public static JsonSerializerOptions CreateJsonSerializerOptions() + public static JsonSerializerOptions CreateJsonSerializerOptions(IServiceProvider serviceProvider, Action? configureJsonSerializerOptions = null) { - if (_jsonSerializerOptions != null) - return _jsonSerializerOptions; - var options = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase, @@ -40,7 +32,10 @@ public static class RefitSettingsHelper options.Converters.Add(new JsonStringEnumConverter()); options.Converters.Add(new VersionOptionsJsonConverter()); + options.Converters.Add(new TypeJsonConverter()); - return _jsonSerializerOptions = options; + configureJsonSerializerOptions?.Invoke(serviceProvider, options); + + return options; } } \ No newline at end of file diff --git a/src/clients/Elsa.Api.Client/Options/ElsaClientBuilderOptions.cs b/src/clients/Elsa.Api.Client/Options/ElsaClientBuilderOptions.cs index a7312eb60..52b0b1b77 100644 --- a/src/clients/Elsa.Api.Client/Options/ElsaClientBuilderOptions.cs +++ b/src/clients/Elsa.Api.Client/Options/ElsaClientBuilderOptions.cs @@ -1,3 +1,4 @@ +using System.Text.Json; using Elsa.Api.Client.HttpMessageHandlers; using Microsoft.Extensions.DependencyInjection; using Polly; @@ -39,4 +40,9 @@ public class ElsaClientBuilderOptions /// Gets or sets a delegate that can be used to configure the retry policy. /// public Action? ConfigureRetryPolicy { get; set; } = builder => builder.AddTransientHttpErrorPolicy(p => p.WaitAndRetryAsync(3, attempt => TimeSpan.FromSeconds(Math.Pow(2, attempt)))); + + /// + /// Gets or sets a delegate that can be used to configure the JSON serializer options. + /// + public Action? ConfigureJsonSerializerOptions { get; set; } } \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Converters/TypeJsonConverter.cs b/src/modules/Elsa.Workflows.Core/Serialization/Converters/TypeJsonConverter.cs index 3442a9c11..dee636927 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Converters/TypeJsonConverter.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Converters/TypeJsonConverter.cs @@ -9,7 +9,7 @@ namespace Elsa.Workflows.Serialization.Converters; /// /// Serializes objects to a simple alias representing the type. /// -[PublicAPI] +[UsedImplicitly] public class TypeJsonConverter : JsonConverter { private readonly IWellKnownTypeRegistry _wellKnownTypeRegistry; From 41e10621484641b9ff35ec78eefbe5bc5a1c2188 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 3 Oct 2024 19:12:00 +0200 Subject: [PATCH 20/31] Enhance JSON deserialization to include service provider Updated ReadAsJsonAsync method to accept an IServiceProvider parameter for improved JSON deserialization. Modified corresponding tests and helper classes to utilize the service provider for more flexible JSON serialization options. --- .../Helpers/Extensions/HttpResponseMessageExtensions.cs | 5 +++-- .../Helpers/Fixtures/WorkflowServer.cs | 2 +- .../Scenarios/BasicWorkflows/HelloWorldTests.cs | 2 +- .../Scenarios/LogPersistenceModes/InputOutputLoggingTests.cs | 2 +- .../Scenarios/WorkflowCompletion/WorkflowCompletionTests.cs | 2 +- 5 files changed, 7 insertions(+), 6 deletions(-) diff --git a/test/component/Elsa.Workflows.ComponentTests/Helpers/Extensions/HttpResponseMessageExtensions.cs b/test/component/Elsa.Workflows.ComponentTests/Helpers/Extensions/HttpResponseMessageExtensions.cs index 3d50e634d..bc4ffff80 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Helpers/Extensions/HttpResponseMessageExtensions.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Helpers/Extensions/HttpResponseMessageExtensions.cs @@ -1,14 +1,15 @@ using System.Text.Json; using Elsa.Api.Client; +// ReSharper disable once CheckNamespace namespace Elsa.Workflows.ComponentTests; public static class HttpResponseMessageExtensions { - public static async Task ReadAsJsonAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) + public static async Task ReadAsJsonAsync(this HttpResponseMessage response, IServiceProvider serviceProvider, CancellationToken cancellationToken = default) { var json = await response.Content.ReadAsStringAsync(cancellationToken); - var options = RefitSettingsHelper.CreateJsonSerializerOptions(); + var options = RefitSettingsHelper.CreateJsonSerializerOptions(serviceProvider); return JsonSerializer.Deserialize(json, options)!; } } \ No newline at end of file diff --git a/test/component/Elsa.Workflows.ComponentTests/Helpers/Fixtures/WorkflowServer.cs b/test/component/Elsa.Workflows.ComponentTests/Helpers/Fixtures/WorkflowServer.cs index a026a35a4..9f9f99897 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Helpers/Fixtures/WorkflowServer.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Helpers/Fixtures/WorkflowServer.cs @@ -31,7 +31,7 @@ public class WorkflowServer(Infrastructure infrastructure, string url) : WebAppl var client = CreateClient(); client.BaseAddress = new Uri(client.BaseAddress!, "/elsa/api"); client.Timeout = TimeSpan.FromMinutes(1); - return RestService.For(client, CreateRefitSettings()); + return RestService.For(client, CreateRefitSettings(Services)); } public HttpClient CreateHttpWorkflowClient() diff --git a/test/component/Elsa.Workflows.ComponentTests/Scenarios/BasicWorkflows/HelloWorldTests.cs b/test/component/Elsa.Workflows.ComponentTests/Scenarios/BasicWorkflows/HelloWorldTests.cs index 7b8da577b..f2446ee40 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Scenarios/BasicWorkflows/HelloWorldTests.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Scenarios/BasicWorkflows/HelloWorldTests.cs @@ -11,7 +11,7 @@ public class HelloWorldTests(App app) : AppComponentTest(app) { var client = WorkflowServer.CreateApiClient(); using var response = await client.ExecuteAsync("1590068018aa4f0a"); - var model = await response.ReadAsJsonAsync(); + var model = await response.ReadAsJsonAsync(WorkflowServer.Services); Assert.Equal(HttpStatusCode.OK, response.StatusCode); Assert.Equal(WorkflowSubStatus.Finished, model.WorkflowState.SubStatus); } diff --git a/test/component/Elsa.Workflows.ComponentTests/Scenarios/LogPersistenceModes/InputOutputLoggingTests.cs b/test/component/Elsa.Workflows.ComponentTests/Scenarios/LogPersistenceModes/InputOutputLoggingTests.cs index 883e19a60..d3e7abc56 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Scenarios/LogPersistenceModes/InputOutputLoggingTests.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Scenarios/LogPersistenceModes/InputOutputLoggingTests.cs @@ -63,7 +63,7 @@ public class InputOutputLoggingTests(App app) : AppComponentTest(app) { var client = WorkflowServer.CreateApiClient(); using var response = await client.ExecuteAsync(workflowDefinitionId); - var model = await response.ReadAsJsonAsync(); + var model = await response.ReadAsJsonAsync(WorkflowServer.Services); return model.WorkflowState; } diff --git a/test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowCompletion/WorkflowCompletionTests.cs b/test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowCompletion/WorkflowCompletionTests.cs index 1618746ca..171fe196c 100644 --- a/test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowCompletion/WorkflowCompletionTests.cs +++ b/test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowCompletion/WorkflowCompletionTests.cs @@ -13,7 +13,7 @@ public class WorkflowCompletionTests(App app) : AppComponentTest(app) { var client = WorkflowServer.CreateApiClient(); using var response = await client.ExecuteAsync(workflowDefinitionId); - var model = await response.ReadAsJsonAsync(); + var model = await response.ReadAsJsonAsync(WorkflowServer.Services); Assert.Equal(HttpStatusCode.OK, response.StatusCode); Assert.Equal(WorkflowSubStatus.Finished, model.WorkflowState.SubStatus); } From 2aa312850db574e9200346572cd30f45372c5bc6 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Sat, 5 Oct 2024 11:19:02 +0200 Subject: [PATCH 21/31] Refactor incident tagging logic in tracing middleware Reorganized the logic for setting tags when incidents are present in `OpenTelemetryTracingWorkflowExecutionMiddleware`. The new structure ensures consistent tag setting for error status and incidents, improving code readability and maintainability. --- ...nTelemetryTracingWorkflowExecutionMiddleware.cs | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index 83382d4d9..e113c77ec 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -54,10 +54,6 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD activity.AddEvent(new ActivityEvent("Faulted")); activity.SetStatus(ActivityStatusCode.Error); activity.SetTag("error", true); - activity.SetTag("hasIncidents", true); - - if (context.Incidents.Count > 0) - activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents, _incidentSerializerOptions)); } else { @@ -68,6 +64,16 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD }))); } + if(context.Incidents.Any()) + { + activity.SetStatus(ActivityStatusCode.Error); + activity.SetTag("hasIncidents", true); + activity.SetTag("error", true); + + if (context.Incidents.Count > 0) + activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents, _incidentSerializerOptions)); + } + if (!string.IsNullOrWhiteSpace(context.CorrelationId)) activity.SetTag("correlationId", context.CorrelationId); From 458181c5ceb2ffdc8e1c277610414bda4074cf73 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Sat, 5 Oct 2024 13:39:58 +0200 Subject: [PATCH 22/31] Refactor activity events with status tags helper method Replaced inline tag creation with a helper method to streamline code and ensure consistency in status tags for activity events. This change also includes a minor comment for clarity on null activity handling. --- ...metryTracingWorkflowExecutionMiddleware.cs | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index e113c77ec..1fd7f8ad7 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -27,9 +27,9 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD { var workflowInstanceId = context.Id; var workflow = context.Workflow; - using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity($"WorkflowExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); + using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity("WorkflowExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); - if (activity == null) + if (activity == null) // No listener is registered. { await Next(context); return; @@ -42,26 +42,18 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD activity.SetTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); activity.SetTag("workflowDefinition.version", workflow.Identity.Version); activity.SetTag("workflowDefinition.name", workflow.WorkflowMetadata.Name); - activity.AddEvent(new ActivityEvent("Executing", tags: new ActivityTagsCollection(new Dictionary - { - ["workflowInstance.status"] = context.Status.ToString(), - ["workflowInstance.subStatus"] = context.SubStatus.ToString() - }))); + activity.AddEvent(new ActivityEvent("Executing", tags: CreateStatusTags(context))); await Next(context); if (context.SubStatus == WorkflowSubStatus.Faulted) { - activity.AddEvent(new ActivityEvent("Faulted")); + activity.AddEvent(new ActivityEvent("Faulted", tags: CreateStatusTags(context))); activity.SetStatus(ActivityStatusCode.Error); activity.SetTag("error", true); } else { - activity.AddEvent(new ActivityEvent("Executed", tags: new ActivityTagsCollection(new Dictionary - { - ["workflowInstance.status"] = context.Status.ToString(), - ["workflowInstance.subStatus"] = context.SubStatus.ToString() - }))); + activity.AddEvent(new ActivityEvent("Executed", tags: CreateStatusTags(context))); } if(context.Incidents.Any()) @@ -79,6 +71,15 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD activity.SetTag("workflowExecution.durationMs", (systemClock.UtcNow - activity.StartTimeUtc).TotalMilliseconds); } + + private ActivityTagsCollection CreateStatusTags(WorkflowExecutionContext context) + { + return new ActivityTagsCollection(new Dictionary + { + ["workflowInstance.status"] = context.Status.ToString(), + ["workflowInstance.subStatus"] = context.SubStatus.ToString() + }); + } } /// From 54624aea77b1a03d824cf589ac4817c265c59051 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Sat, 5 Oct 2024 13:48:59 +0200 Subject: [PATCH 23/31] Refactor tracing to enhance span details Renamed variables from 'activity' to 'span' for clarity and consistency. Added additional span tags for start and end times, status updates, and detailed error information for improved traceability and debugging. --- ...metryTracingActivityExecutionMiddleware.cs | 28 ++++++++----- ...metryTracingWorkflowExecutionMiddleware.cs | 40 ++++++++++--------- 2 files changed, 40 insertions(+), 28 deletions(-) diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs index 6af239905..b09895521 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs @@ -32,17 +32,15 @@ public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareD span.SetTag("activity.type", activity.Type); span.SetTag("activity.name", activity.Name); span.SetTag("activityInstance.id", context.Id); + span.SetTag("activityExecution.startTimeUtc", span.StartTimeUtc); - span.AddEvent(new ActivityEvent("Executing", tags: new ActivityTagsCollection(new Dictionary - { - ["activityInstance.status"] = context.Status.ToString(), - }))); + span.AddEvent(new ActivityEvent("Executing", tags: CreateStatusTags(context))); await next(context); if (context.Status == ActivityStatus.Faulted) { - span.AddEvent(new ActivityEvent("Faulted")); + span.AddEvent(new ActivityEvent("Faulted", tags: CreateStatusTags(context))); span.SetStatus(ActivityStatusCode.Error); span.SetTag("error", true); span.SetTag("hasIncidents", true); @@ -54,12 +52,22 @@ public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareD span.SetTag("error.stackTrace", context.Exception.StackTrace); } else - span.AddEvent(new ActivityEvent("Executed", tags: new ActivityTagsCollection(new Dictionary - { - ["activityInstance.status"] = context.Status.ToString(), - }))); + { + span.AddEvent(new ActivityEvent("Executed", tags: CreateStatusTags(context))); + span.SetStatus(ActivityStatusCode.Ok); + } - span.SetTag("activityExecution.durationMs", (systemClock.UtcNow - span.StartTimeUtc).TotalMilliseconds); + var now = systemClock.UtcNow; + span.SetTag("activityExecution.endTimeUtc", now); + span.SetTag("activityExecution.durationMs", (now - span.StartTimeUtc).TotalMilliseconds); + } + + private ActivityTagsCollection CreateStatusTags(ActivityExecutionContext context) + { + return new ActivityTagsCollection(new Dictionary + { + ["activityInstance.status"] = context.Status.ToString() + }); } } diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index 1fd7f8ad7..56e2ef17d 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -27,49 +27,53 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD { var workflowInstanceId = context.Id; var workflow = context.Workflow; - using var activity = ElsaOpenTelemetry.ActivitySource.StartActivity("WorkflowExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); + using var span = ElsaOpenTelemetry.ActivitySource.StartActivity("WorkflowExecution", ActivityKind.Internal, Activity.Current?.Context ?? default); - if (activity == null) // No listener is registered. + if (span == null) // No listener is registered. { await Next(context); return; } if (!string.IsNullOrWhiteSpace(context.CorrelationId)) - activity.SetTag("correlationId", context.CorrelationId); + span.SetTag("correlationId", context.CorrelationId); - activity.SetTag("workflowInstance.id", workflowInstanceId); - activity.SetTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); - activity.SetTag("workflowDefinition.version", workflow.Identity.Version); - activity.SetTag("workflowDefinition.name", workflow.WorkflowMetadata.Name); - activity.AddEvent(new ActivityEvent("Executing", tags: CreateStatusTags(context))); + span.SetTag("workflowInstance.id", workflowInstanceId); + span.SetTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); + span.SetTag("workflowDefinition.version", workflow.Identity.Version); + span.SetTag("workflowDefinition.name", workflow.WorkflowMetadata.Name); + span.SetTag("workflowExecution.startTimeUtc", span.StartTimeUtc); + span.AddEvent(new ActivityEvent("Executing", tags: CreateStatusTags(context))); await Next(context); if (context.SubStatus == WorkflowSubStatus.Faulted) { - activity.AddEvent(new ActivityEvent("Faulted", tags: CreateStatusTags(context))); - activity.SetStatus(ActivityStatusCode.Error); - activity.SetTag("error", true); + span.AddEvent(new ActivityEvent("Faulted", tags: CreateStatusTags(context))); + span.SetStatus(ActivityStatusCode.Error); + span.SetTag("error", true); } else { - activity.AddEvent(new ActivityEvent("Executed", tags: CreateStatusTags(context))); + span.AddEvent(new ActivityEvent("Executed", tags: CreateStatusTags(context))); + span.SetStatus(ActivityStatusCode.Ok); } if(context.Incidents.Any()) { - activity.SetStatus(ActivityStatusCode.Error); - activity.SetTag("hasIncidents", true); - activity.SetTag("error", true); + span.SetStatus(ActivityStatusCode.Error); + span.SetTag("hasIncidents", true); + span.SetTag("error", true); if (context.Incidents.Count > 0) - activity.SetTag("error.message", JsonSerializer.Serialize(context.Incidents, _incidentSerializerOptions)); + span.SetTag("error.message", JsonSerializer.Serialize(context.Incidents, _incidentSerializerOptions)); } if (!string.IsNullOrWhiteSpace(context.CorrelationId)) - activity.SetTag("correlationId", context.CorrelationId); + span.SetTag("correlationId", context.CorrelationId); - activity.SetTag("workflowExecution.durationMs", (systemClock.UtcNow - activity.StartTimeUtc).TotalMilliseconds); + var now = systemClock.UtcNow; + span.SetTag("workflowExecution.endTimeUtc", now); + span.SetTag("workflowExecution.durationMs", (now - span.StartTimeUtc).TotalMilliseconds); } private ActivityTagsCollection CreateStatusTags(WorkflowExecutionContext context) From 078f34fd799ba8c5a26863fa9506edf72a42d479 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Sat, 5 Oct 2024 14:12:00 +0200 Subject: [PATCH 24/31] Remove obsolete tag settings and correct tag schemas Eliminated redundant `correlationId` tag setting. Adjusted `workflowInstance` and `activityInstance` tag schemas for better clarity. Simplified the processors list in `otel-collector-config.yaml`. --- docker/otel-collector-config.yaml | 2 +- .../OpenTelemetryTracingActivityExecutionMiddleware.cs | 2 +- .../OpenTelemetryTracingWorkflowExecutionMiddleware.cs | 7 ++----- 3 files changed, 4 insertions(+), 7 deletions(-) diff --git a/docker/otel-collector-config.yaml b/docker/otel-collector-config.yaml index a1aaced1b..6cf8cefaf 100644 --- a/docker/otel-collector-config.yaml +++ b/docker/otel-collector-config.yaml @@ -41,7 +41,7 @@ service: exporters: [ datadog ] traces: receivers: [ otlp ] - processors: [ batch, tail_sampling ] # Added tail_sampling to the main traces pipeline + processors: [ batch ] # Added tail_sampling to the main traces pipeline exporters: [ debug, datadog ] # Directly exporting to debug and datadog logs: receivers: [ otlp ] diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs index b09895521..043a11092 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingActivityExecutionMiddleware.cs @@ -43,7 +43,7 @@ public class OpenTelemetryTracingActivityExecutionMiddleware(ActivityMiddlewareD span.AddEvent(new ActivityEvent("Faulted", tags: CreateStatusTags(context))); span.SetStatus(ActivityStatusCode.Error); span.SetTag("error", true); - span.SetTag("hasIncidents", true); + span.SetTag("activityInstance.hasIncidents", true); var errorMessage = string.IsNullOrWhiteSpace(context.Exception?.Message) ? "Unknown error" : context.Exception.Message; span.SetTag("error.message", errorMessage); diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index 56e2ef17d..f374f1a78 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -35,9 +35,6 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD return; } - if (!string.IsNullOrWhiteSpace(context.CorrelationId)) - span.SetTag("correlationId", context.CorrelationId); - span.SetTag("workflowInstance.id", workflowInstanceId); span.SetTag("workflowDefinition.definitionId", workflow.Identity.DefinitionId); span.SetTag("workflowDefinition.version", workflow.Identity.Version); @@ -61,7 +58,7 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD if(context.Incidents.Any()) { span.SetStatus(ActivityStatusCode.Error); - span.SetTag("hasIncidents", true); + span.SetTag("workflowInstance.hasIncidents", true); span.SetTag("error", true); if (context.Incidents.Count > 0) @@ -69,7 +66,7 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD } if (!string.IsNullOrWhiteSpace(context.CorrelationId)) - span.SetTag("correlationId", context.CorrelationId); + span.SetTag("workflowInstance.correlationId", context.CorrelationId); var now = systemClock.UtcNow; span.SetTag("workflowExecution.endTimeUtc", now); From 02f1b56eb14250c5a642b02304227ae25939871a Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Mon, 7 Oct 2024 10:15:35 +0200 Subject: [PATCH 25/31] Add tracing for trigger activities in workflow execution Enhanced the OpenTelemetry tracing middleware to include detailed information about the triggered activity within the workflow execution. This change adds tags for the activity ID, name, and type to provide better observability and debugging insights. --- .../OpenTelemetryTracingWorkflowExecutionMiddleware.cs | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs index f374f1a78..f5d20b7c5 100644 --- a/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs +++ b/src/modules/Elsa.OpenTelemetry/Middleware/OpenTelemetryTracingWorkflowExecutionMiddleware.cs @@ -40,6 +40,15 @@ public class OpenTelemetryTracingWorkflowExecutionMiddleware(WorkflowMiddlewareD span.SetTag("workflowDefinition.version", workflow.Identity.Version); span.SetTag("workflowDefinition.name", workflow.WorkflowMetadata.Name); span.SetTag("workflowExecution.startTimeUtc", span.StartTimeUtc); + + if(context.TriggerActivityId != null) + { + var activity = context.FindActivityById(context.TriggerActivityId) ?? throw new Exception($"Trigger activity with ID {context.TriggerActivityId} not found. This should not happen."); + span.SetTag("workflowExecution.trigger.activityId", activity.Id); + span.SetTag("workflowExecution.trigger.activityName", activity.Name); + span.SetTag("workflowExecution.trigger.activityType", activity.Type); + } + span.AddEvent(new ActivityEvent("Executing", tags: CreateStatusTags(context))); await Next(context); From 6cefe6ea234c7bdd41f81466c70b7fac47f15cf8 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Wed, 9 Oct 2024 09:44:07 +0200 Subject: [PATCH 26/31] Update package versions in Directory.Packages.props Bumped versions for various Microsoft and third-party packages to improve compatibility and take advantage of recent bug fixes and performance improvements. Specific changes include updates to AspNetCore, EntityFrameworkCore, and Polly packages among others. --- Directory.Packages.props | 46 ++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/Directory.Packages.props b/Directory.Packages.props index 78ecaa67b..5626e149c 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -142,33 +142,33 @@ - - - - - - - - - - - + + + + + + + + + + + - + - - - - - - - + + + + + + + - - - + + + - + \ No newline at end of file From 3f83a2fb5112c183bc19a45bcc1c5fa7f160b6af Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Wed, 9 Oct 2024 23:19:47 +0200 Subject: [PATCH 27/31] Remove unused 'Elsa.Expressions.Services' import Eliminates an unnecessary using directive from PolymorphicObjectConverterFactory.cs to keep the code clean and reduce potential confusion. This change has no impact on functionality. --- .../Converters/PolymorphicObjectConverterFactory.cs | 1 - 1 file changed, 1 deletion(-) diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs index 448fc16be..fc94427d4 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverterFactory.cs @@ -2,7 +2,6 @@ using System.Dynamic; using System.Text.Json; using System.Text.Json.Serialization; using Elsa.Expressions.Contracts; -using Elsa.Expressions.Services; namespace Elsa.Workflows.Serialization.Converters; From 494d6e7f43ac8aa375a6125c8a06ac4189a01302 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 10 Oct 2024 10:18:27 +0200 Subject: [PATCH 28/31] Refactor polymorphic JSON deserialization. Simplify the handling of `JsonNode` parsing and improve type-checking. Added a guard clause in `ReadType` method to handle non-object JSON token types, ensuring safer type parsing. --- .../Converters/PolymorphicObjectConverter.cs | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverter.cs b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverter.cs index 49eeeb940..f8121b24d 100644 --- a/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverter.cs +++ b/src/modules/Elsa.Workflows.Core/Serialization/Converters/PolymorphicObjectConverter.cs @@ -93,9 +93,9 @@ public class PolymorphicObjectConverter(IWellKnownTypeRegistry wellKnownTypeRegi if (isDictionary) { // Remove the _type property name from the JSON, if any. - var parsedModel = (JsonObject)JsonNode.Parse(ref reader)!; - parsedModel.Remove(TypePropertyName); - return parsedModel.Deserialize(targetType, newOptions)!; + var parsedNode = JsonNode.Parse(ref reader)!; + if (parsedNode is JsonObject parsedModel) parsedModel.Remove(TypePropertyName); + return parsedNode.Deserialize(targetType, newOptions)!; } var isCollection = typeof(ICollection).IsAssignableFrom(targetType); @@ -247,7 +247,7 @@ public class PolymorphicObjectConverter(IWellKnownTypeRegistry wellKnownTypeRegi { var typeOptions = newOptions.Clone(); typeOptions.Converters.RemoveWhere(c => c.GetType() != typeof(TypeJsonConverter)); - + if (typeOptions.Converters.Any()) { var typeValue = JsonSerializer.Serialize(type, typeOptions).Trim('"'); @@ -265,6 +265,9 @@ public class PolymorphicObjectConverter(IWellKnownTypeRegistry wellKnownTypeRegi private Type? ReadType(Utf8JsonReader reader) { + if (reader.TokenType != JsonTokenType.StartObject) + return null; + reader.Read(); // Move to the first token inside the object. string? typeName = null; From 96d82c42105b24ebc2eb745dbb56c7a58ebd7518 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Thu, 10 Oct 2024 11:40:59 +0200 Subject: [PATCH 29/31] Refactor deserialization method for entity properties. Replaced `Deserialize` with `DeserializePayload` to align with updated deserialization strategies, ensuring consistency across the codebase and improving future maintainability. This change affects how 'SerializedProperties' are handled within `ActivityExecutionLogStore`. --- .../Modules/Runtime/ActivityExecutionLogStore.cs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs index 68652db65..9ee9078d7 100644 --- a/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs +++ b/src/modules/Elsa.EntityFrameworkCore/Modules/Runtime/ActivityExecutionLogStore.cs @@ -102,7 +102,7 @@ public class EFCoreActivityExecutionStore( entity.ActivityState = await DeserializeActivityState(dbContext, entity, cancellationToken); entity.Outputs = Deserialize>(dbContext, entity, "SerializedOutputs"); - entity.Properties = Deserialize?>(dbContext, entity, "SerializedProperties") ?? new Dictionary(); + entity.Properties = DeserializePayload?>(dbContext, entity, "SerializedProperties") ?? new Dictionary(); entity.Exception = DeserializePayload(dbContext, entity, "SerializedException"); entity.Payload = DeserializePayload>(dbContext, entity, "SerializedPayload"); } From 11f388338a0a9557c2e66575441c8658ce4fdbc7 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 11 Oct 2024 10:29:40 +0200 Subject: [PATCH 30/31] Add endpoints for activity execution records and summaries (#6018) * Add endpoints for activity execution records and summaries Created new classes and endpoints to handle activity execution records and summaries retrieval. Added related models and updated relevant interfaces. * Rename endpoint classes for clarity Updated endpoint classes in `ActivityExecutions` and `ActivityExecutionSummaries` to use the name `Endpoint` instead of `List`. This improves readability and consistency in the codebase. --- .../Contracts/IActivityExecutionsApi.cs | 18 ++++ .../Models/ActivityExecutionRecordSummary.cs | 82 +++++++++++++++++++ .../ListSummaries/Endpoint.cs | 38 +++++++++ .../ListSummaries/Models.cs | 22 +++++ .../ActivityExecutions/Get/Endpoint.cs | 41 ++++++++++ .../ActivityExecutionRecordSummary.cs | 6 +- 6 files changed, 204 insertions(+), 3 deletions(-) create mode 100644 src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Models/ActivityExecutionRecordSummary.cs create mode 100644 src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Endpoint.cs create mode 100644 src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Models.cs create mode 100644 src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutions/Get/Endpoint.cs diff --git a/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Contracts/IActivityExecutionsApi.cs b/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Contracts/IActivityExecutionsApi.cs index f7ae99343..2238c8162 100644 --- a/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Contracts/IActivityExecutionsApi.cs +++ b/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Contracts/IActivityExecutionsApi.cs @@ -27,4 +27,22 @@ public interface IActivityExecutionsApi /// The response containing a list of activity executions. [Get("/activity-executions/list")] Task> ListAsync(ListActivityExecutionsRequest request, CancellationToken cancellationToken = default); + + /// + /// Lists activity execution summaries for a given activity in a workflow instance. + /// + /// The request. + /// An optional cancellation token. + /// The response containing a list of activity execution summaries. + [Get("/activity-execution-summaries/list")] + Task> ListSummariesAsync(ListActivityExecutionsRequest request, CancellationToken cancellationToken = default); + + /// + /// Gets a single activity execution by ID. + /// + /// The ID of the activity execution. + /// An optional cancellation token. + /// The activity execution. + [Get("/activity-executions/{id}")] + Task GetAsync(string id, CancellationToken cancellationToken = default); } \ No newline at end of file diff --git a/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Models/ActivityExecutionRecordSummary.cs b/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Models/ActivityExecutionRecordSummary.cs new file mode 100644 index 000000000..a224c62de --- /dev/null +++ b/src/clients/Elsa.Api.Client/Resources/ActivityExecutions/Models/ActivityExecutionRecordSummary.cs @@ -0,0 +1,82 @@ +using System.Linq.Expressions; +using Elsa.Api.Client.Resources.WorkflowInstances.Models; +using Elsa.Api.Client.Shared.Models; + +namespace Elsa.Api.Client.Resources.ActivityExecutions.Models; + +/// +/// Represents a summarized view of a single activity execution of an activity instance. +/// +public class ActivityExecutionRecordSummary : Entity +{ + /// + /// Gets or sets the workflow instance ID. + /// + public string WorkflowInstanceId { get; set; } = default!; + + /// + /// Gets or sets the activity ID. + /// + public string ActivityId { get; set; } = default!; + + /// + /// Gets or sets the activity node ID. + /// + public string ActivityNodeId { get; set; } = default!; + + /// + /// The type of the activity. + /// + public string ActivityType { get; set; } = default!; + + /// + /// The version of the activity type. + /// + public int ActivityTypeVersion { get; set; } + + /// + /// The name of the activity. + /// + public string? ActivityName { get; set; } + + /// + /// Gets or sets the time at which the activity execution began. + /// + public DateTimeOffset StartedAt { get; set; } + + /// + /// Gets or sets whether the activity has any bookmarks. + /// + public bool HasBookmarks { get; set; } + + /// + /// Gets or sets the status of the activity. + /// + public ActivityStatus Status { get; set; } + + /// + /// Gets or sets the time at which the activity execution completed. + /// + public DateTimeOffset? CompletedAt { get; set; } + + /// + /// Returns a summary view of the specified . + /// + public static ActivityExecutionRecordSummary FromRecord(ActivityExecutionRecord record) + { + return new ActivityExecutionRecordSummary + { + Id = record.Id, + WorkflowInstanceId = record.WorkflowInstanceId, + ActivityId = record.ActivityId, + ActivityNodeId = record.ActivityNodeId, + ActivityType = record.ActivityType, + ActivityTypeVersion = record.ActivityTypeVersion, + ActivityName = record.ActivityName, + StartedAt = record.StartedAt, + HasBookmarks = record.HasBookmarks, + Status = record.Status, + CompletedAt = record.CompletedAt + }; + } +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Endpoint.cs new file mode 100644 index 000000000..55638298e --- /dev/null +++ b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Endpoint.cs @@ -0,0 +1,38 @@ +using Elsa.Abstractions; +using Elsa.Common.Entities; +using Elsa.Models; +using Elsa.Workflows.Runtime.Contracts; +using Elsa.Workflows.Runtime.Entities; +using Elsa.Workflows.Runtime.Filters; +using Elsa.Workflows.Runtime.OrderDefinitions; +using JetBrains.Annotations; + +namespace Elsa.Workflows.Api.Endpoints.ActivityExecutionSummaries.ListSummaries; + +/// +/// Lists a summary view of the executions for a given activity. +/// +[PublicAPI] +internal class Endpoint(IActivityExecutionStore store) : ElsaEndpoint> +{ + /// + public override void Configure() + { + Get("/activity-execution-summaries/list"); + ConfigurePermissions("read:activity-execution"); + } + + /// + public override async Task> ExecuteAsync(Request request, CancellationToken cancellationToken) + { + var filter = new ActivityExecutionRecordFilter + { + WorkflowInstanceId = request.WorkflowInstanceId, + ActivityNodeId = request.ActivityNodeId, + Completed = request.Completed + }; + var order = new ActivityExecutionRecordOrder(x => x.StartedAt, OrderDirection.Ascending); + var records = (await store.FindManySummariesAsync(filter, order, cancellationToken)).ToList(); + return new ListResponse(records); + } +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Models.cs b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Models.cs new file mode 100644 index 000000000..b90456857 --- /dev/null +++ b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutionSummaries/ListSummaries/Models.cs @@ -0,0 +1,22 @@ +namespace Elsa.Workflows.Api.Endpoints.ActivityExecutionSummaries.ListSummaries; + +/// +/// A request for a list of activity execution log record summaries for a given activity in a workflow instance. +/// +internal class Request +{ + /// + /// The ID of the workflow instance to get the execution log for. + /// + public string WorkflowInstanceId { get; set; } = default!; + + /// + /// The node ID of the activity to get the execution record for. + /// + public string ActivityNodeId { get; set; } = default!; + + /// + /// Whether to include completed activity execution records. If not specified, all activity execution records will be included. + /// + public bool? Completed { get; set; } +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutions/Get/Endpoint.cs b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutions/Get/Endpoint.cs new file mode 100644 index 000000000..8828ebb8b --- /dev/null +++ b/src/modules/Elsa.Workflows.Api/Endpoints/ActivityExecutions/Get/Endpoint.cs @@ -0,0 +1,41 @@ +using Elsa.Abstractions; +using Elsa.Workflows.Runtime.Contracts; +using Elsa.Workflows.Runtime.Entities; +using Elsa.Workflows.Runtime.Filters; +using JetBrains.Annotations; + +namespace Elsa.Workflows.Api.Endpoints.ActivityExecutions.Get; + +/// +/// Gets an individual execution for a given activity. +/// +[PublicAPI] +internal class Endpoint(IActivityExecutionStore store) : ElsaEndpointWithoutRequest +{ + /// + public override void Configure() + { + Get("/activity-executions/{id}"); + ConfigurePermissions("read:activity-execution"); + } + + /// + public override async Task HandleAsync(CancellationToken cancellationToken) + { + var id = Route("id"); + var filter = new ActivityExecutionRecordFilter + { + Id = id + }; + + var record = await store.FindAsync(filter, cancellationToken); + + if (record == null) + { + await SendNotFoundAsync(cancellationToken); + return; + } + + await SendOkAsync(record, cancellationToken); + } +} \ No newline at end of file diff --git a/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecordSummary.cs b/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecordSummary.cs index ae26ba8fe..9b13941cb 100644 --- a/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecordSummary.cs +++ b/src/modules/Elsa.Workflows.Runtime/Entities/ActivityExecutionRecordSummary.cs @@ -42,7 +42,7 @@ public class ActivityExecutionRecordSummary : Entity /// /// Gets or sets the time at which the activity execution began. /// - public DateTimeOffset StartedAt { get; set; } = default!; + public DateTimeOffset StartedAt { get; set; } /// /// Gets or sets whether the activity has any bookmarks. @@ -60,7 +60,7 @@ public class ActivityExecutionRecordSummary : Entity public DateTimeOffset? CompletedAt { get; set; } /// - /// Returns a summary view of the specified . + /// Returns a summary view of the specified . /// public static ActivityExecutionRecordSummary FromRecord(ActivityExecutionRecord record) { @@ -81,7 +81,7 @@ public class ActivityExecutionRecordSummary : Entity } /// - /// Returns a summary view of the specified . + /// Returns a summary view of the specified . /// public static Expression> FromRecordExpression() { From 83238e9931597e592bdb6a0b631d206955f1b367 Mon Sep 17 00:00:00 2001 From: Sipke Schoorstra Date: Fri, 11 Oct 2024 14:35:24 +0200 Subject: [PATCH 31/31] Update System.Text.Json version reference (#6019) Replaced hardcoded System.Text.Json version 8.0.4 with a variable $(SystemTextJsonVersion) across multiple project files. This improves maintainability and consistency when updating the package version. --- Directory.Build.props | 1 + Directory.Packages.props | 2 +- src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj | 2 +- .../Elsa.ServerAndStudio.Web/Elsa.ServerAndStudio.Web.csproj | 2 +- src/bundles/Elsa.Studio.Web/Elsa.Studio.Web.csproj | 2 +- src/clients/Elsa.Api.Client/Elsa.Api.Client.csproj | 2 +- .../Elsa.EntityFrameworkCore.Common.csproj | 2 +- .../Elsa.EntityFrameworkCore.MySql.csproj | 2 +- .../Elsa.EntityFrameworkCore.PostgreSql.csproj | 2 +- .../Elsa.EntityFrameworkCore.SqlServer.csproj | 2 +- .../Elsa.EntityFrameworkCore.Sqlite.csproj | 2 +- .../Elsa.EntityFrameworkCore/Elsa.EntityFrameworkCore.csproj | 2 +- src/modules/Elsa.FileStorage/Elsa.FileStorage.csproj | 2 +- src/modules/Elsa.Http/Elsa.Http.csproj | 2 +- src/modules/Elsa.MongoDb/Elsa.MongoDb.csproj | 2 +- .../Elsa.Quartz.EntityFrameworkCore.MySql.csproj | 2 +- .../Elsa.Quartz.EntityFrameworkCore.PostgreSql.csproj | 2 +- .../Elsa.Quartz.EntityFrameworkCore.SqlServer.csproj | 2 +- .../Elsa.Quartz.EntityFrameworkCore.Sqlite.csproj | 2 +- src/modules/Elsa.Telnyx/Elsa.Telnyx.csproj | 2 +- .../Elsa.WorkflowProviders.BlobStorage.csproj | 2 +- src/modules/Elsa.Workflows.Api/Elsa.Workflows.Api.csproj | 2 +- 22 files changed, 22 insertions(+), 21 deletions(-) diff --git a/Directory.Build.props b/Directory.Build.props index c24271ac0..76eef1f08 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -28,5 +28,6 @@ 3.2.0-rc4.473 + 8.0.5 \ No newline at end of file diff --git a/Directory.Packages.props b/Directory.Packages.props index 5626e149c..4db1b43be 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -108,7 +108,7 @@ - + diff --git a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj index bbc68b8ef..eaf5bc04b 100644 --- a/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj +++ b/src/bundles/Elsa.Server.Web/Elsa.Server.Web.csproj @@ -58,7 +58,7 @@ - + diff --git a/src/bundles/Elsa.ServerAndStudio.Web/Elsa.ServerAndStudio.Web.csproj b/src/bundles/Elsa.ServerAndStudio.Web/Elsa.ServerAndStudio.Web.csproj index 23d188639..e2d1c97f8 100644 --- a/src/bundles/Elsa.ServerAndStudio.Web/Elsa.ServerAndStudio.Web.csproj +++ b/src/bundles/Elsa.ServerAndStudio.Web/Elsa.ServerAndStudio.Web.csproj @@ -42,7 +42,7 @@ - + \ No newline at end of file diff --git a/src/bundles/Elsa.Studio.Web/Elsa.Studio.Web.csproj b/src/bundles/Elsa.Studio.Web/Elsa.Studio.Web.csproj index 432ff7c6c..2cb27ad13 100644 --- a/src/bundles/Elsa.Studio.Web/Elsa.Studio.Web.csproj +++ b/src/bundles/Elsa.Studio.Web/Elsa.Studio.Web.csproj @@ -18,7 +18,7 @@ - + diff --git a/src/clients/Elsa.Api.Client/Elsa.Api.Client.csproj b/src/clients/Elsa.Api.Client/Elsa.Api.Client.csproj index 7ce828980..5358e76f3 100644 --- a/src/clients/Elsa.Api.Client/Elsa.Api.Client.csproj +++ b/src/clients/Elsa.Api.Client/Elsa.Api.Client.csproj @@ -19,7 +19,7 @@ - + diff --git a/src/modules/Elsa.EntityFrameworkCore.Common/Elsa.EntityFrameworkCore.Common.csproj b/src/modules/Elsa.EntityFrameworkCore.Common/Elsa.EntityFrameworkCore.Common.csproj index eab7ce6b2..26a175db6 100644 --- a/src/modules/Elsa.EntityFrameworkCore.Common/Elsa.EntityFrameworkCore.Common.csproj +++ b/src/modules/Elsa.EntityFrameworkCore.Common/Elsa.EntityFrameworkCore.Common.csproj @@ -19,7 +19,7 @@ - + diff --git a/src/modules/Elsa.EntityFrameworkCore.MySql/Elsa.EntityFrameworkCore.MySql.csproj b/src/modules/Elsa.EntityFrameworkCore.MySql/Elsa.EntityFrameworkCore.MySql.csproj index 5df8320c1..6ab40ff8f 100644 --- a/src/modules/Elsa.EntityFrameworkCore.MySql/Elsa.EntityFrameworkCore.MySql.csproj +++ b/src/modules/Elsa.EntityFrameworkCore.MySql/Elsa.EntityFrameworkCore.MySql.csproj @@ -14,7 +14,7 @@ - + diff --git a/src/modules/Elsa.EntityFrameworkCore.PostgreSql/Elsa.EntityFrameworkCore.PostgreSql.csproj b/src/modules/Elsa.EntityFrameworkCore.PostgreSql/Elsa.EntityFrameworkCore.PostgreSql.csproj index ce9f15c01..078b0ff95 100644 --- a/src/modules/Elsa.EntityFrameworkCore.PostgreSql/Elsa.EntityFrameworkCore.PostgreSql.csproj +++ b/src/modules/Elsa.EntityFrameworkCore.PostgreSql/Elsa.EntityFrameworkCore.PostgreSql.csproj @@ -14,7 +14,7 @@ - + diff --git a/src/modules/Elsa.EntityFrameworkCore.SqlServer/Elsa.EntityFrameworkCore.SqlServer.csproj b/src/modules/Elsa.EntityFrameworkCore.SqlServer/Elsa.EntityFrameworkCore.SqlServer.csproj index 2a69f65ae..26f28d9a2 100644 --- a/src/modules/Elsa.EntityFrameworkCore.SqlServer/Elsa.EntityFrameworkCore.SqlServer.csproj +++ b/src/modules/Elsa.EntityFrameworkCore.SqlServer/Elsa.EntityFrameworkCore.SqlServer.csproj @@ -21,6 +21,6 @@ - + \ No newline at end of file diff --git a/src/modules/Elsa.EntityFrameworkCore.Sqlite/Elsa.EntityFrameworkCore.Sqlite.csproj b/src/modules/Elsa.EntityFrameworkCore.Sqlite/Elsa.EntityFrameworkCore.Sqlite.csproj index 6580755a3..10feafd59 100644 --- a/src/modules/Elsa.EntityFrameworkCore.Sqlite/Elsa.EntityFrameworkCore.Sqlite.csproj +++ b/src/modules/Elsa.EntityFrameworkCore.Sqlite/Elsa.EntityFrameworkCore.Sqlite.csproj @@ -15,7 +15,7 @@ - + diff --git a/src/modules/Elsa.EntityFrameworkCore/Elsa.EntityFrameworkCore.csproj b/src/modules/Elsa.EntityFrameworkCore/Elsa.EntityFrameworkCore.csproj index 3dd2e4a6a..1a05f4320 100644 --- a/src/modules/Elsa.EntityFrameworkCore/Elsa.EntityFrameworkCore.csproj +++ b/src/modules/Elsa.EntityFrameworkCore/Elsa.EntityFrameworkCore.csproj @@ -13,7 +13,7 @@ - + diff --git a/src/modules/Elsa.FileStorage/Elsa.FileStorage.csproj b/src/modules/Elsa.FileStorage/Elsa.FileStorage.csproj index 0393bc645..648f20efc 100644 --- a/src/modules/Elsa.FileStorage/Elsa.FileStorage.csproj +++ b/src/modules/Elsa.FileStorage/Elsa.FileStorage.csproj @@ -17,7 +17,7 @@ - + diff --git a/src/modules/Elsa.Http/Elsa.Http.csproj b/src/modules/Elsa.Http/Elsa.Http.csproj index 1024756e2..26f35e9b5 100644 --- a/src/modules/Elsa.Http/Elsa.Http.csproj +++ b/src/modules/Elsa.Http/Elsa.Http.csproj @@ -13,7 +13,7 @@ - + diff --git a/src/modules/Elsa.MongoDb/Elsa.MongoDb.csproj b/src/modules/Elsa.MongoDb/Elsa.MongoDb.csproj index 39d38e218..2a84690bf 100644 --- a/src/modules/Elsa.MongoDb/Elsa.MongoDb.csproj +++ b/src/modules/Elsa.MongoDb/Elsa.MongoDb.csproj @@ -16,7 +16,7 @@ - + diff --git a/src/modules/Elsa.Quartz.EntityFrameworkCore.MySql/Elsa.Quartz.EntityFrameworkCore.MySql.csproj b/src/modules/Elsa.Quartz.EntityFrameworkCore.MySql/Elsa.Quartz.EntityFrameworkCore.MySql.csproj index a2e95de32..677393259 100644 --- a/src/modules/Elsa.Quartz.EntityFrameworkCore.MySql/Elsa.Quartz.EntityFrameworkCore.MySql.csproj +++ b/src/modules/Elsa.Quartz.EntityFrameworkCore.MySql/Elsa.Quartz.EntityFrameworkCore.MySql.csproj @@ -16,7 +16,7 @@ - + diff --git a/src/modules/Elsa.Quartz.EntityFrameworkCore.PostgreSql/Elsa.Quartz.EntityFrameworkCore.PostgreSql.csproj b/src/modules/Elsa.Quartz.EntityFrameworkCore.PostgreSql/Elsa.Quartz.EntityFrameworkCore.PostgreSql.csproj index 8cca95901..9b887cfcd 100644 --- a/src/modules/Elsa.Quartz.EntityFrameworkCore.PostgreSql/Elsa.Quartz.EntityFrameworkCore.PostgreSql.csproj +++ b/src/modules/Elsa.Quartz.EntityFrameworkCore.PostgreSql/Elsa.Quartz.EntityFrameworkCore.PostgreSql.csproj @@ -22,7 +22,7 @@ - + diff --git a/src/modules/Elsa.Quartz.EntityFrameworkCore.SqlServer/Elsa.Quartz.EntityFrameworkCore.SqlServer.csproj b/src/modules/Elsa.Quartz.EntityFrameworkCore.SqlServer/Elsa.Quartz.EntityFrameworkCore.SqlServer.csproj index f33bc9af8..6a8f00321 100644 --- a/src/modules/Elsa.Quartz.EntityFrameworkCore.SqlServer/Elsa.Quartz.EntityFrameworkCore.SqlServer.csproj +++ b/src/modules/Elsa.Quartz.EntityFrameworkCore.SqlServer/Elsa.Quartz.EntityFrameworkCore.SqlServer.csproj @@ -24,6 +24,6 @@ - + \ No newline at end of file diff --git a/src/modules/Elsa.Quartz.EntityFrameworkCore.Sqlite/Elsa.Quartz.EntityFrameworkCore.Sqlite.csproj b/src/modules/Elsa.Quartz.EntityFrameworkCore.Sqlite/Elsa.Quartz.EntityFrameworkCore.Sqlite.csproj index 283ff1ffb..1a5ad7e07 100644 --- a/src/modules/Elsa.Quartz.EntityFrameworkCore.Sqlite/Elsa.Quartz.EntityFrameworkCore.Sqlite.csproj +++ b/src/modules/Elsa.Quartz.EntityFrameworkCore.Sqlite/Elsa.Quartz.EntityFrameworkCore.Sqlite.csproj @@ -16,7 +16,7 @@ - + diff --git a/src/modules/Elsa.Telnyx/Elsa.Telnyx.csproj b/src/modules/Elsa.Telnyx/Elsa.Telnyx.csproj index ae23a5915..79b745a66 100644 --- a/src/modules/Elsa.Telnyx/Elsa.Telnyx.csproj +++ b/src/modules/Elsa.Telnyx/Elsa.Telnyx.csproj @@ -24,7 +24,7 @@ - + diff --git a/src/modules/Elsa.WorkflowProviders.BlobStorage/Elsa.WorkflowProviders.BlobStorage.csproj b/src/modules/Elsa.WorkflowProviders.BlobStorage/Elsa.WorkflowProviders.BlobStorage.csproj index c6c83beb6..779e52c49 100644 --- a/src/modules/Elsa.WorkflowProviders.BlobStorage/Elsa.WorkflowProviders.BlobStorage.csproj +++ b/src/modules/Elsa.WorkflowProviders.BlobStorage/Elsa.WorkflowProviders.BlobStorage.csproj @@ -17,6 +17,6 @@ - + diff --git a/src/modules/Elsa.Workflows.Api/Elsa.Workflows.Api.csproj b/src/modules/Elsa.Workflows.Api/Elsa.Workflows.Api.csproj index 95079450f..c41a9e2d3 100644 --- a/src/modules/Elsa.Workflows.Api/Elsa.Workflows.Api.csproj +++ b/src/modules/Elsa.Workflows.Api/Elsa.Workflows.Api.csproj @@ -17,6 +17,6 @@ - +