Commit graph

4462 commits

Author SHA1 Message Date
Sipke Schoorstra 58e4523406 Update persistence feature base classes to include self-type
Refactored multiple persistence feature classes to implement the self-referencing generic type parameter in the `PersistenceFeatureBase` class. This ensures that each feature correctly references its own type, enhancing type safety and overall code maintainability.
2024-08-09 19:20:15 +02:00
Sipke Schoorstra d7c11db280 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-09 18:55:26 +02:00
Sipke Schoorstra ef2dfd3957
Refactor journal data access and improve byte[] handling (#5878)
Switched to dictionary index access for "ResumeAt" in Delay.cs to ensure more efficient updating. Added handling for byte[] in ObjectConverter.cs to correctly deserialize from base64 strings.
2024-08-09 18:20:49 +02:00
rosca-sabina b6acb18dc6
Changed RegisterClassMap to TryRegisterClassMap on SerializedKeyValuePair classmap registration (#5877) 2024-08-09 18:14:09 +02:00
Sipke Schoorstra 38998aa051
Add Orchard Core Integration + Agents Module (#5871)
* Add "integrations" project and update dependencies

A new project named "integrations" has been added to the Elsa solution. Related dependencies have been updated accordingly, with some connections reassigned to the new project. This includes both new connections established and existing connections removed in response to the project addition.

* Stash

* Refactor ActivityDescriber to clean up unused code

Removed commented out code lines related to input options in the `ActivityDescriber` class. This change enhances code readability and maintains the focus on active and necessary functionality. The main logic for getting UI properties remains intact.

* Remove deprecated webhook dispatchers and handlers

Eliminated classes and interfaces related to webhook dispatching and registration from the Elsa.Webhooks module as part of refactoring. Introduced new classes and features for handling webhook events within the Elsa.OrchardCore module. Updated several configuration files and references accordingly.

* Add OrchardCore integration sample and enhance webhooks

Added a new ASP.NET sample for integrating with OrchardCore and set up various related configuration files and project references. Improved the webhooks module by modifying the WebhookEventActivityProvider for better event handling and description management, and enhanced the WebhooksFeature to include the WebhookEventActivityProvider.

* Set activity descriptor description from eventType.DisplayName

The activity descriptor now sets its Description property using the eventType's DisplayName. This ensures that the description is consistent with the event type's display name across the system.

* Rename 'Description' to 'DisplayName' for activity descriptor

This change ensures that 'DisplayName' is assigned the correct value from 'eventType.DisplayName'. It helps avoid any confusion between 'Description' and 'DisplayName' properties in the activity descriptor object.

* Add PayloadType and Constructor logic for Webhooks

Added PayloadType to content item published events in `appsettings.json`. Updated `WebhookEventActivityProvider` to include constructor logic and manage payload types. Enhanced `OrchardCoreFeature` and `WebhooksFeature` with alias definitions for variable types.

* Add helper for webhook activity type names and refactor code to use it

Introduce `WebhookActivityTypeNameHelper` for consistent generation of webhook activity type names. Refactor relevant classes to utilize this helper, simplifying and standardizing the codebase. Adjust other minor settings, such as editor config line length and dependency configurations.

* Refactor and introduce Orchard core integration

Replace `ContentItemPublished` with a new handlers and payload structure. This change includes the creation of various classes and handlers necessary to support the OrchardCore integration, define the content item events, and configure them via app settings. This refactor aids in standardizing the handling of content item publication events through newly introduced classes and configurations.

* Refactor stimulus class name for content publication

Renamed `WebhookEventReceivedStimulus` to `ContentItemPublishedStimulus` to better reflect its purpose in handling content item publication events. Updated all references and imports accordingly to maintain consistency in the codebase.

* Refactor webhook payload handling

Switched from using only `ContentItemPublishedPayload` to incorporating both `WebhookEvent` and `ContentItemPublishedPayload` in the input dictionary. This change ensures more comprehensive data handling for webhook events.

* Add new dependency and correct project reference path

Included Elsa.Webhooks.ActivityProviders in WebhooksFeature to support new webhook activities. Corrected the project reference path in Elsa.Samples.AspNet.OrchardCoreIntegration.csproj to reflect the accurate module directory structure.

* Rename and enhance webhook handling functionality

Renamed `InvokeOrchardActivities` to `InvokeOrchardWebhookEventActivities` for clarity. Also added an `ActivityProvider` to improve webhook event handling.

* Refactor: replace Workflow with WorkflowGraph in context

Updated references from Workflow to WorkflowGraph across various files to reflect the new context structure. This change improves the accuracy of context information and aligns with recent architectural updates.

* Add Semantic Kernel modules and features

Introduced core classes and services to support Semantic Kernel agents, their configuration, and execution within workflows. Added API endpoints for agent execution, integrated kernel configuration settings, and ensured necessary extensions for skill and plugin integration.

* Update ActivityDescriber and AgentActivityProvider

Default `inputAttribute` to null in `GetUIHint` and use it in `AgentActivityProvider`. This change ensures the `GetUIHint` method is called directly, improving code reusability and simplifying input handling.

* Add GraphQL client with token authentication

Introduced a DefaultGraphQlClient to send GraphQL queries, incorporated token-based authentication using AuthenticatingDelegatingHandler, and included support for refresh tokens through DefaultSecurityTokenService. Updated projects and configuration accordingly.

* Rename OrchardWebhooks to OrchardCore

Updated method and related variables to use 'OrchardCore' instead of 'OrchardWebhooks' for consistency and improved clarity. Also added activity attributes for SendGraphQLQuery to better describe its purpose.

* Refactor webhook handling and add ProofreaderResult model

Refactor activities and event handling to support multiple webhook event types beyond content publishing. Introduce new models under OrchardCoreIntegration for ProofreaderResult and ProofreadError, and update existing methods for more generic payload handling.

* Add support for API key configuration and reference

Implemented a new `ApiKeyConfig` class to manage API keys. Updated various components to utilize API key references, enhancing configuration flexibility. Added new options and settings to facilitate these changes.

* Add translation functionality and improve agent activity displays

This commit introduces a new `TranslationResult` model and updates the project configuration to support translation features. Additionally, it adds two new project references and modifies the `appsettings.json` to include translation skills. The `AgentActivityProvider` class now has improved display names for agent activities for better readability.

* Add FactCheck and Tagging functionality

Introduced new models: FactCheckResult, GenerateTagsResult, and IncorrectFact. Integrated JetBrains.Annotations package and extended variable types in workflow management. Updated prompt templates and appsettings for proofreader, fact-checker, and tagger functions.

* Enhance 'Tagger' skill in appsettings.json

Updated the 'GenerateTags' function in the 'Tagger' skill to define the number of tags to generate and to ensure JSON response format. Added a 'Number' input variable and the 'ResponseFormat' property for clarity and precision.

* Add support for patching content items via OrchardCore API

Implement DefaultRestApiClient and IRestApiClient for GET and PATCH requests to the content API. Introduce Activity for patching content items and update app settings to handle TitleResult in JSON format.

* Add variable type aliasing and improve serialization

Introduced extension methods to add variable type and alias in WorkflowManagementFeature. Enhanced JSON serialization in AgentActivity and DefaultRestApiClient. Cleaned up solution by removing old project references and updated appsettings.json for revised prompt templates.

* Remove Proofreader models and update AgentActivity properties

Removed obsolete `ProofreaderResult` and `ProofreadError` classes to streamline the codebase. Updated `AgentActivity` properties to be internal and JSON-ignored to enhance encapsulation and reduce serialization overhead.

* Add localization support for content items in OrchardCore

Implemented a new activity `LocalizeContentItem` to create localized versions of content items based on a provided culture code. Added a corresponding API client method and request model to facilitate this feature, along with necessary format adjustments in the appsettings file.

* Enhance SkillExecutor functionality and update config details

Enabled 'AllowDangerouslySetContent' in SkillExecutor for improved flexibility with input variables. Streamlined the Proofread function prompt template and adjusted various execution settings in `appsettings.json`. Removed explicit MaxTokens limits to allow more dynamic content handling.

* Add attribute to BackgroundActivityStimulus

Added the `ExcludeFromHash` attribute to the `JobId` property in `BackgroundActivityStimulus` class, fixing background activity resumption.

* Rename SemanticKernel to Agents module for better clarity

This commit renames the SemanticKernel module to Agents across the entire project, including all relevant file paths, namespaces, and configuration settings. Major refactors encompass changing file names, class names, and namespaces to reflect the new module name, removing the SemanticKernel integration in favor of the more general Agents framework.

* Update configuration section for SemanticKernel

Changed the configuration section name from "Agents" to "SemanticKernel" in Program.cs and appsettings.json. This ensures the options are correctly bound to the relevant settings in appsettings.json.

* Add new entities and projects for Elsa Agents

Introduced new entities: `AgentDefinition`, `ApiKeyDefinition`, `ModelDefinition`, `PluginDefinition`, `SkillDefinition`. Added new projects: `Elsa.Agents.Management` and `Elsa.Agents.Persistence`. Renamed `ConfigureAgents.cs` to `ConfigureKernel.cs` and updated relevant references.

* Add ActivityHandle to workflow execution endpoint

Extended the endpoint to accept ActivityHandle as part of the workflow execution request. This change improves the ability to specify and manage activity-specific workflows. Additionally, updated `appsettings.json` to remove "Article" from the list of content types.

* Add nullable checks and new skills with functions

Improve null safety in AgentActivity and AgentActivityProvider classes by adding nullable checks. Additionally, update `appsettings.json` with corrected skill names and introduce new skills including ArticleWriter and Author.

* Register Random type in JavaScript engine

Added the Random type registration to both JintJavaScriptEvaluator and CommonTypesDefinitionProvider. This allows JavaScript code to utilize the Random class for generating random numbers.

* Add CreateContentItem activity and API client method

Introduce a new CreateContentItem activity to create and optionally publish content items. Added a new REST API client method to handle content creation, including the necessary request model.

* Update API endpoints and variable name for consistency

Changed API endpoints in DefaultRestApiClient.cs to use 'content-items' instead of 'content' for consistency with the project's URL structure. Also updated a variable name in CreateContentItem.cs for clarity and readability.

* Fix potential NullReferenceException on timer disposal.

Ensure that the timer is not null before calling Dispose in both ScheduledRecurringTask and ScheduledCronTask. This prevents potential NullReferenceExceptions when the timer event is fired.

* Add Image Generator and integrate OpenAI text-to-image feature

Introduced `ImageGenerator` for generating images from text using OpenAI's text-to-image functionality. Updated `KernelFactory` to support `OpenAITextToImage` service, and modified configurations and services to utilize the new image generation capabilities.

* Add UploadMedia activity and WorkflowInstanceStorageDriver

This commit introduces a new UploadMedia activity for uploading files to a media library and a WorkflowInstanceStorageDriver for managing workflow state storage. Additionally, it includes changes in IRestApiClient interface, DefaultRestApiClient implementation, and minor logging enhancements in ScheduledSpecificInstantTask. Unnecessary project references were removed and project files were updated accordingly.

* Mark `WorkflowStorageDriver` as obsolete and update condition.

Added [Obsolete] attribute to `WorkflowStorageDriver` indicating it will be removed in future releases. Modified `BackgroundActivityInvoker` to also recognize `WorkflowInstanceStorageDriver` for output capture.

* Handle JsonNode deserialization in ObjectConverter

Add support for deserializing JsonNode types in ObjectConverter.cs. This ensures proper handling of JsonNode along with JsonElement and JsonObject, enhancing the overall deserialization process.

* Add ResolveTags activity to OrchardCore module

Created a new ResolveTags activity that interacts with the Orchard Core API to resolve or create tags. Implemented the ResolveTagsRequest model and updated the IRestApiClient and DefaultRestApiClient to support this new functionality.

* Refactor project structure and update configuration

Renamed the Elsa.Agents project to Elsa.Agents.Activities and updated all relevant namespaces and file paths. Added a new setting to the appsettings.json to include the language parameter for article generation.

* Update project metadata in multiple csproj files

Revised descriptions and package tags across several projects to ensure consistent terminology related to Agents. Removed unused folder references in Elsa.Agents.Core.csproj.

* Simplify JSON node deserialization logic

Removed redundant JsonObject checks and unified the handling of JsonNode instances. This refactoring streamlines the code and ensures consistency in the deserialization process.
2024-08-08 19:35:43 +02:00
Raymond den Haan 7341f1f93a Add customizable DB exception handlers
Introduced customizable database exception handlers across various modules, enabling specific error handling strategies for different DbContexts. Implemented a generic interface for exception handling and provided specific implementations for PostgreSQL and a No-op handler. Updated the code to utilize these handlers in various persistence features, enhancing error management and modularity.
2024-08-08 16:08:01 +02:00
Sipke Schoorstra e129cbf0e7 Refactor workflow execution and pipeline handling
Removed subStatus parameter from ValidateStatusTransition method to simplify its usage. Also, eliminated redundant UseBackgroundActivities call in WorkflowExecutionPipelineBuilderExtensions to streamline the pipeline configuration.
2024-08-08 08:37:19 +02:00
Sipke Schoorstra d7c16ce53a Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-07 21:25:07 +02:00
raymonddenhaan 1437635903
Merge pull request #5860 from elsa-workflows/enh/engine_ex
Add engine exception handling middleware
2024-08-07 11:37:48 +02:00
Raymond den Haan 4f90ee71d9 Add TODO 2024-08-07 11:33:38 +02:00
Raymond den Haan 9f841c70c6 Fix logger type 2024-08-07 09:48:31 +02:00
Sipke Schoorstra 3c39c02ee3 Fix incorrect serializer reference
Corrected the serializer reference from `_payloadSerializer` to `payloadSerializer` in `WorkflowDefinitionStore.cs`. This change ensures that the correct serializer instance is used when serializing `WorkflowDefinitionState`.
2024-08-06 23:15:44 +02:00
Sipke Schoorstra b2b29a7f4f Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-06 22:47:05 +02:00
Sipke Schoorstra d2fc7fb005
Rework Workflow Context Feature (#5861)
* Refactor workflow context extension structure

Removed unused WorkflowContextWorkflowDefinitionExtensions and renamed several extension and middleware classes for clarity and consistency. Simplified context provider type handling using JsonArray in custom properties.

* Refactor context provider and update email workflow

Refactored `CustomerWorkflowContextProvider` to use constructor parameter directly, removing redundant fields and constructors. Updated comments and list syntax in `CustomerCommunicationsWorkflow`. Improved README.md documentation for workflow contexts and context providers.

* Adjust logging levels and improve workflow logic

Modified `appsettings.json` to reduce the verbosity of logging by adjusting log levels. Updated `CustomerCommunicationsWorkflow` to include a timer activity and simplified the workflow context parameter setting.

* Remove PropertyBag and related functionalities.

Deleted the PropertyBag class and extension methods, and removed all instances where PropertyBag was utilized across various modules. This simplifies the workflow configuration and management by relying solely on CustomProperties for storing metadata.

* Add workflow context management extensions and optimizations

Introduces `WorkflowExecutionContextExtensions` for loading and saving workflow contexts more efficiently. Replaces `JsonArray` usage with `List<object>` for context provider types, simplifying the type management in the workflow execution context. Streamlines `WorkflowContextActivityExecutionMiddleware` by removing redundant service scope creation and enhancing readability.

* Remove redundant timer in CustomerCommunicationsWorkflow

The timer activity that was set to not start the workflow has been removed to streamline the workflow process. Updated the README to include instructions on how to start the CustomerCommunicationsWorkflow using the REST API or Elsa Studio.

* Refactor middleware to streamline context management

Removed dependency on `IServiceScopeFactory` and refactored middleware to directly load and save workflow contexts. Simplified workflow context provider invocation, enhancing code readability and maintainability.

* Remove redundant PropertyBag property

The PropertyBag property was commented out and is no longer needed in the WorkflowDefinition class. This change removes the commented-out code to clean up the class definition.

* Add JsonSerializerOptions for deserialization

Introduce JsonSerializerOptions with camelCase policy and case insensitivity for property names. This ensures consistent deserialization of custom properties in activity workflow context settings.
2024-08-06 22:36:24 +02:00
Raymond den Haan de4a4b3505 Add engine exception handling middleware
Integrate new EngineExceptionHandlingMiddleware into the workflow execution pipeline to catch and log exceptions outside of running workflow activity.
2024-08-06 13:54:02 +02:00
Sipke Schoorstra 596d85f0b9 Switch database provider from PostgreSQL to SQLite
Updated the database provider in the program configuration to use SQLite instead of PostgreSQL. This change simplifies local development setup and reduces dependencies.
2024-07-25 19:14:37 +02:00
Sipke Schoorstra f0f12c7edd Remove unused console log and obsolete project reference
Eliminated an unnecessary console log statement in Program.cs, which was logging the KUBERNETES_SERVICE_HOST value. Additionally, removed an obsolete project reference to "bundles" in the solution file.
2024-07-25 10:48:11 +02:00
Sipke Schoorstra 5ec99e73f8 Refactor to use public ClusterName property
Replaced the private _clusterName variable with a public ClusterName property to allow external configuration of the cluster name. Updated cluster setup configuration to use the new property, ensuring flexibility and maintainability.
2024-07-25 09:07:56 +02:00
Sipke Schoorstra e1bb6a52d2 Remove unnecessary namespace declarations and using statements
This commit consolidates namespaces by removing redundant declarations and unused using statements across multiple files. This helps to streamline the code and improve readability, reducing potential confusion.
2024-07-24 22:51:06 +02:00
Sipke Schoorstra 4f288f499b Remove unused import from WorkflowExecutionBuilderExtensions.cs
Removed the unused `Elsa.Workflows.Contracts` import to clean up the code. This change reduces clutter and potential confusion by ensuring only necessary namespaces are included.
2024-07-24 22:37:27 +02:00
Sipke Schoorstra 19d1da12ec Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-24 22:33:36 +02:00
raymonddenhaan c413f210a9
Add activity execution log sink (#5833)
* Add activity execution log sink

Introduced an interface `IActivityExecutionLogSink` for storing workflow execution log records and implemented it as `StoreActivityExecutionLogSink`. This centralizes and streamlines the handling of persisting execution logs into the workflow's runtime, thus making the process more modular and maintainable. The `PersistActivityExecutionLogMiddleware` has also been refactored to use this sink, replacing the direct usage of the workflow execution log store.

* Refactor logging interfaces and implementations

Replaced specific logging interfaces and implementations with generic ones. The individual interfaces for workflow and activity execution logging have been replaced with a single ILogRecordExtractor and ILogRecordSink interface. Specific implementations have been adjusted to use these new interfaces. This allows for greater flexibility and reuse of logging code.

* Introduce ILogRecordStore interface

The `ILogRecordStore` interface has been introduced to provide a centralized place for handling log records. Both `IActivityExecutionStore` and `IWorkflowExecutionLogStore` have been updated to inherit from this new interface. As a result of this change, the `SaveManyAsync` methods in these two interfaces have been removed to avoid redundancy.
2024-07-24 22:30:14 +02:00
Sipke Schoorstra 631036f404
Proto.Actor implementation for ChangeTokenSignalPublisher (#5817)
* **Refactor ProtoActor modules and integrate new core module**

Removed obsolete proto actor-related files and introduced a new core module under `Elsa.ProtoActor.Core` to centralize ProtoActor functionalities. Updated services and extensions to align with the new core structure, focusing on efficient persistence and actor system configurations.

* Add Proto.Actor-based distributed caching module

Introduces a new module `Elsa.Caching.Distributed.ProtoActor` for Proto.Actor-based distributed caching, including configuration extensions, proto files, and required services. Refactors some existing Proto.Actor-related features and updates Dockerfile and example projects to use the new module.

* Refactor ProtoActor cache handling and virtual actor setup.

Reorganize the distributed caching by introducing LocalCacheVirtualActorProvider and StartLocalCacheActor. Update WorkflowInstanceVirtualActorProvider for better cluster kind handling. Adjust namespaces in Protobuf definitions for consistency.

* Refactor LocalCacheImpl to use IChangeTokenSignalInvoker

Replace IChangeTokenSignaler with IChangeTokenSignalInvoker to align with updated dependency contract. Adjust method call to use InvokeAsync for triggering token signals with cancellation support.

* Refactor ConfigureClusterConfig and config mutation

Change ConfigureClusterConfig from Action to Func for better flexibility. Update clusterConfig and remoteConfig to support reassignment from configuration methods.

* Add ProtoActor support for distributed caching

Introduced ProtoActor as a new distributed caching transport option. Updated the configuration and workflow runtime settings to utilize ProtoActor. Added necessary project reference for Elsa.Caching.Distributed.ProtoActor in the .csproj file.

* Add LocalNodeStrategy and integrate it in actor provider

Introduced `LocalNodeStrategy` to handle member placement on the current node. Integrated the new strategy in `LocalCacheVirtualActorProvider`, ensuring it uses `LocalNodeStrategy` for member management.

* Prevent duplicate member additions based on ID.

Updated the member checking logic to include member IDs. In addition, this change improves the robustness of the member management in `LocalNodeStrategy.cs`.

* Refactor virtual actor configuration into a separate method

Moved virtual actor setup logic from `ProtoActorFeature` to a new `AddVirtualActors` method to improve code readability and reusability. Updated related files to maintain consistency and enhance documentation clarity.

* Refactor LocalCache to use PubSub for change token signals

Replaced direct event stream usage with PubSub in LocalCache implementation. Updated service and hosted service to support PubSub subscription and publishing. Removed obsolete Start and Stop RPC methods from LocalCache service definition.

* Add UsedImplicitly attribute to notification handler

This change introduces the [UsedImplicitly] attribute to the DistributedWorkflowDefinitionNotificationsHandler class. The attribute is intended to prevent any accidental removal by static analysis tools, ensuring the class remains available for dynamic usage scenarios.

* Refactor caching and signal handling mechanisms

Replaced `TriggerChangeTokenSignalConsumer` with `ChangeTokenSignalInvoker` and added new decorators for change token handling. Renamed namespaces and file paths for better consistency and clarity. Updated test files to align with these changes.

* Remove unnecessary interface dependencies from services

Eliminated the ISignalManager and related interfaces to streamline dependency management. Updated services and test components to use concrete implementations directly, reducing complexity and improving maintainability.

* Remove Shared.proto and associated imports

Deleted the Shared.proto file and removed related import statements across multiple files. This cleanup also involved modifying the proto actor provider and project file to exclude references to Shared.proto.

* Simplify namespaces in component test helpers

Consolidated several namespaces into 'Elsa.Workflows.ComponentTests.Helpers' to reduce redundancy and improve maintainability. Removed unnecessary using directives in multiple test files for cleaner and more readable code.

* Refactor imports in component tests

Consolidated various helper imports in component tests by removing redundant specific references and utilizing general 'Elsa.Workflows.ComponentTests.Helpers'. This change simplifies the dependency management and ensures cleaner and more maintainable code.

* Remove redundant state persistence calls

Eliminated multiple calls to PersistStateAsync in WorkflowInstanceImpl.cs as they were unnecessary given that the WorkflowRunner already invokes the commit handler. This change simplifies the workflow execution and cancellation logic by avoiding redundant state persistence operations.

* Add workflowInstanceId to response mapping

Updated methods to include workflowInstanceId in response mapping functions for consistency and clarity. Additionally, fixed project reference paths and added error handling for missing workflow variables in tests.

* Remove unnecessary variable existence check

Removed a redundant check for the existence of the "Workflow1:variable-1" key in the variables dictionary. This streamlines the test and relies on the assumption that the key exists as expected without explicit validation.

* Add Kubernetes deployment and service configurations

Introduced a Deployment and Service configuration for the Kubernetes cluster. Updated Dockerfiles and build script to align with port 8080 configuration and renamed images for consistency. Updated solution file to include new deployment files.

* Add Kubernetes cluster integration

Introduced Kubernetes cluster provider for Proto.Actor and configured the application to use it if running in a Kubernetes environment. Added necessary RBAC roles, role bindings, and service accounts to support Kubernetes integration. Updated deployment configuration and package references to include Proto.Cluster.Kubernetes.

* Refactor deployment configurations and add service support.

Reorganized deployment YAML files into designated subdirectories for elsa-server, postgres, plant-uml, and trace-lens. Introduced new configuration maps, service accounts, roles, and service bindings. Updated .NET environment variables and solution structure to reflect these changes.

* Update service configurations and environment variables

Renamed and split services in trace-lens to isolate the OTEL collector. Updated environment variables in elsa-server to enhance instrumentation, connection strings, and profiling settings. Adjusted OTEL exporter endpoint to match the new service naming.

* Increase deployment replicas to 3

Updated the 'replicas' field in the deployment configuration to enhance the system's availability and load balancing. This change ensures that three instances of 'elsa-server' will be running simultaneously.

* Rename LocalCacheImpl to LocalCache and add logging

Renamed `LocalCacheImpl` class to `LocalCache` to better reflect its purpose. Added a logging statement in `OnReceive` method to log incoming `ProtoTriggerChangeTokenSignal` messages. These changes improve code readability and debugging.

* Disable OTEL console exporters and set session affinity

Disabled console exporters for logs, metrics, and traces in the OTEL configuration to reduce unnecessary console output. Additionally, set session affinity to 'None' in the elsa-server service configuration for load balancing.

* Rename WorkflowInstanceImpl to WorkflowInstance

Updated the class name from WorkflowInstanceImpl to WorkflowInstance for clarity and simplicity. Adjusted all relevant references and instances in the codebase to match the new class name.

* Remove ActivityIncidentStateMapper and Update ProtoBuf Mappings

Removed the unused ActivityIncidentStateMapper class to streamline the codebase. Updated all related ProtoBuf mappings and imports to ensure consistency and remove redundancy across the project.

* Remove duplicate actor spawn verification timeout setting

The code had a redundant setting for actor spawn verification timeout, which was specified twice. This commit removes the duplicate line to ensure cleaner and more maintainable configuration.

* Remove unused imports from Program.cs

Eliminated unnecessary imports for ActivityExecution, WorkflowExecution, and k8s libraries. This cleanup helps reduce the code footprint and may improve compile time.

* Remove debug logging from LocalCache actor

The `Console.WriteLine` statement was removed from the `OnReceive` method in `LocalCache.cs`. This change eliminates unnecessary console output during the token signal handling, improving performance and reducing log clutter.
2024-07-24 22:23:33 +02:00
Marko Lahma f5dc29cdcc
Drop NET 7 support and cleanup NuGet package references (#5828)
* Directory.Packages.props should mandate used versions
* Add NuGet audit mode setting to show any problems during restore
* System.Text.Json should be 8.0.4 which is patched against security vulnerabilities
* Fix build GH Actions generation

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-24 14:30:39 +02:00
Sipke Schoorstra e9f64f49c2 Implement IDisposable for scheduled tasks
Added IDisposable interface to scheduled task classes to ensure proper resource cleanup. Specifically, added dispose methods to release timers and cancellation tokens where applicable.
2024-07-24 14:26:37 +02:00
Sipke Schoorstra 369263f9ca Fix null value handling for AppRole in Program.cs
Updated the application role parsing to provide a default value when the configuration is missing. Removed redundant inbox cleanup options binding, and added a Resharper disable comment for redundant assignment.
2024-07-24 14:21:21 +02:00
Sipke Schoorstra f301da3b12 Switch to RunAsync for all web applications
Updated the Program.cs files across web applications to use `await app.RunAsync()` instead of `app.Run()`. This change ensures proper asynchronous operation and aligns with best practices for hosting .NET applications.
2024-07-24 14:18:04 +02:00
Sipke Schoorstra 3b304c8e08 Add OnStopped method to dispose of _linkedTokenSource
This commit introduces the OnStopped method in the WorkflowInstanceImpl class to properly dispose of _linkedTokenSource. This change ensures that resources are appropriately cleaned up when the actor is stopped, preventing potential memory leaks.
2024-07-24 14:14:50 +02:00
Sipke Schoorstra 95476cac99 Fix project reference path in csproj file
Corrected the path in the ProjectReference for Elsa.csproj to ensure proper module loading. This resolves issues related to incorrect project references and allows the project to build successfully.
2024-07-24 14:12:31 +02:00
Sipke Schoorstra 2115a9b4e9 Update project reference path in Elsa.Testing.Shared.Component
Changed the project reference from the bundles directory to the module directory for Elsa.csproj. This update ensures proper inclusion of the Elsa project within Elsa.Testing.Shared.Component.
2024-07-24 14:12:08 +02:00
Sipke Schoorstra 2cfa8f935c Update project reference path for Elsa module
Corrected the project reference path from 'bundles' to 'modules' to ensure proper linkage and compilation. This change addresses a path configuration error in the project file.
2024-07-24 14:11:29 +02:00
Sipke Schoorstra f96d594f6f Refactor folder paths and update method implementations
Updated Dockerfile paths to reflect 'apps' directory structure, removed redundant XML comments, and modified methods in WorkflowDefinitionExtensions to use FindWorkflowGraphAsync for consistent workflow retrieval.
2024-07-24 14:10:27 +02:00
Sipke Schoorstra aef5e7b311 Add lang attribute to HTML tag
Including the lang attribute with the value "en" improves accessibility and helps search engines better understand the page's content. This small change enhances the overall SEO and user experience for English-speaking users.
2024-07-24 14:05:57 +02:00
Sipke Schoorstra e4a161f40a Set language attribute in _Host.cshtml
Added the `lang` attribute with the value "en" to the `<html>` tag to specify the language of the document. This improves accessibility and enhances SEO by clearly defining the page's primary language.
2024-07-24 14:04:45 +02:00
Sipke Schoorstra ed79494dba Rename Elsa.ProtoActor to Elsa.Workflows.Runtime.ProtoActor 2024-07-22 15:09:25 +02:00
Sipke Schoorstra d3f9d5ef8f Merge branch 'main' of github.com:elsa-workflows/elsa-core 2024-07-22 14:36:43 +02:00
Sipke Schoorstra 70b8582b23
Add OpenTelemetry module (#5810)
* Add OpenTelemetry integration for workflow tracing

Introduced a new module, Elsa.OpenTelemetry, to provide OpenTelemetry sources for tracing workflow and activity execution. Updated various components and pipeline extensions to support OpenTelemetry tracing throughout the workflow execution process.

* Refactor workflow execution pipelines

Deleted `WorkflowsFeatureExtensions` and migrated methods to `PipelineWorkflowsFeatureExtensions` with added configurability. Enhanced `ActivityExecutionMiddlewareExtensions` and `ActivityExecutionPipelinePipelineBuilder` to support middleware insertion. Added comprehensive tracing to `OpenTelemetryTracingWorkflowExecutionMiddleware`.

* Add OpenTelemetry tracing to activity execution

Introduced `OpenTelemetryTracingActivityExecutionMiddleware` to capture tracing information for activity execution within workflows. This middleware logs activity execution start and end events, attaching pertinent activity tags. Added extension method to register this middleware in the workflow execution pipeline.

* Set `PYTHONNET_PYDLL` consistently and update workflow pipelines

Update Dockerfiles to set the `PYTHONNET_PYDLL` environment variable consistently without spaces. Additionally, refactor `Program.cs` to streamline workflow and activity execution pipeline configurations by using the `WithDefaultWorkflowExecutionPipeline` and `WithDefaultActivityExecutionPipeline` methods.

* Enhance OpenTelemetry Tracing Middleware Implementation

Add missing activity tags and events to improve telemetry data. Simplify middleware installation syntax for both workflow and activity execution tracing pipelines, ensuring consistent and clear tracing across modules.
2024-07-22 14:36:33 +02:00
Sipke Schoorstra 4f49252fbf Add dependencies to docker-compose and update README
Updated `docker-compose.yml` to include `depends_on` for `trace-lens`, ensuring it starts after `plantuml-server` and `postgres`. Modified the README to include `Elsa.Workflows` in `OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES` and corrected formatting for `OTEL_RESOURCE_ATTRIBUTES`.
2024-07-22 14:36:02 +02:00
Sipke Schoorstra b4cd4dd465
Replace custom webhooks implementation with WebhooksCore (#5811)
* Replace custom webhooks implementation with WebhooksCore

Replaced the existing custom webhooks registration and dispatching system with the WebhooksCore library. This involved removing all previous custom webhook classes and interfaces, updating project dependencies, and modifying appsettings and program files to use the new configuration. The update simplifies the webhook handling architecture and leverages the features provided by WebhooksCore.

* Hide API key in Datadog Docker compose file

This change replaces the exposed Datadog API key with a placeholder "<HIDDEN>". This enhances security by preventing the API key from being visible in the version control system.
2024-07-21 16:48:38 +02:00
Mohamed Ali 53cb8e75c3
fix workflow context execution middleware and scrip completion (#5802) 2024-07-20 12:35:56 +02:00
Sipke Schoorstra 2a6a73d96e Enable tracing for WorkflowInstanceActor conditionally
Uncommented conditional enabling of tracing for WorkflowInstanceActor. This ensures that tracing is enabled only if the `_enableTracing` flag is set, improving the clarity and functionality of the feature.
2024-07-19 23:49:34 +02:00
Sipke Schoorstra 6924bd0770 Update build paths, add Datadog tracing, and adjust versions
Corrected Dockerfile paths for Elsa.Server and Elsa.Studio, and added Datadog tracing and related environment variables. Included Datadog.Trace package in Elsa.Server.Web project and updated Elsa.Studio package versions to 3.3.0-preview.466.
2024-07-19 23:36:31 +02:00
Sipke Schoorstra 283824fcfd
Enable Proto Actor Tracing for TraceLens (#5800)
* Add database initialization script and update dependencies

Added a script to initialize the 'tracelens' database and modified the Docker setup to include this script. Refactored and improved the ProtoActorFeature class, added OpenTelemetry dependencies, and updated project settings.

* Enable OpenTelemetry integration for Proto.Actor

Added OpenTelemetry environment configuration details to the README and included the Proto.OpenTelemetry package in the project file. Updated the ProtoActorFeature to apply tracing with OpenTelemetry to WorkflowInstanceActor.

* Refactor VariablePersistenceManager to use primary constructor

This refactor simplifies the VariablePersistenceManager by moving the storageDriverManager initialization into the primary constructor. It removes the redundant field and constructor, aligning with the concise nature of modern C# syntax, and ensures consistency in accessing the storageDriverManager throughout the class.

* Remove unused Open Telemetry code from Program.cs

The code for configuring Open Telemetry was commented out but not removed, cluttering the file. This commit cleans up Program.cs by deleting these unused lines, maintaining a cleaner and more readable codebase.

* Add metrics and tracing configurations for ProtoActorFeature

Introduced methods to enable metrics and tracing in ProtoActorFeature. Removed redundant properties and updated the workflow runtime to utilize the new configurations.

* Add Directory.Build.props for shared project settings

Introduce Directory.Build.props to centralize common project settings and dependencies. Consolidate target framework, language version, and package references to reduce duplication. Remove redundant property definitions from Elsa.Server.Web.csproj.

* Move apps from bundles to apps folder and Elsa module to modules folder
2024-07-19 20:23:32 +02:00
Sipke Schoorstra 5586151bf1 Remove IWorkflowExecutionContextStore and related classes
Removed the IWorkflowExecutionContextStore interface and its implementations, along with the associated handlers and registrations. This change simplifies the codebase by eliminating unnecessary components and their dependencies.
2024-07-18 18:57:42 +02:00
Sipke Schoorstra 827680b3c4 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-18 18:25:49 +02:00
Sipke Schoorstra e5ba9a8a9d Refactor scheduling and system clock configurations
Moved specific services from Scoped to Singleton in SchedulingFeature and SystemClockFeature for better performance and consistency. Refactored DefaultWorkflowScheduler to use a constructor with an IScheduler parameter and removed redundant private field. Removed attribute RequiresUnreferencedCode in WorkflowManagementFeature.
2024-07-18 16:00:49 +02:00
Sipke Schoorstra 149f91e538
Update API endpoint for Polling Observer (#5787)
* Rename and refactor journal update endpoint

Replaced `/workflow-instances/{id}/journal/has-updates` endpoint with `/workflow-instances/{id}/updated-at` to simplify API responses. Deleted `HasUpdates` related classes and introduced `GetUpdatedAtResponse` for consistency and clarity. Updated client contracts accordingly.

* Remove HasUpdates endpoint and refactor workflow observer

Deleted the HasUpdates endpoint and refactored related code to use an updated timestamp approach instead. Improved nullable handling in WorkflowInstanceDesigner and ensured proper observer disposal to avoid memory leaks. Updated workflow observer factory and observer implementations to support observer names and enhanced logging.

* Rename updated workflow instance endpoint and handle execution state

Renamed the endpoint from "/updated-at" to "/execution-state" to better reflect its purpose. Updated related response models and documentation to capture workflow execution state details such as status, sub-status, and last updated timestamp.

* Enable SignalR for real-time workflows

Add a flag to use SignalR and activate real-time workflows when enabled. Refactor code to wrap SignalR setup in conditional checks based on the new flag. This enhances the application's interactivity through real-time capabilities.

* Remove obsolete endpoints and rename execution state paths

Deleted the outdated Api1 and DynamicWorkflows endpoints under Elsa.Server.Web. Also, renamed paths related to execution state models and endpoint to remove "Journal" from the namespace for better clarity and organization.
2024-07-18 07:51:46 +02:00
Sipke Schoorstra f448e9520a
Fix reloading logic for workflow definitions (#5781)
* Add reloading logic for workflow definitions

Introduced a new mechanism to handle reloaded workflow definitions. This includes creating a "ReloadedWorkflowDefinition" model, updating the caching logic, and modifying notification handlers to work with the enhanced workflow reloading logic. This ensures workflow definitions are updated and managed correctly when published, retracted, or deleted.

* Improve RefreshActivityRegistry documentation

Updated the XML documentation to clarify that `RefreshActivityRegistry` refreshes the `IActivityRegistry` for `WorkflowDefinitionActivityProvider` whenever workflow definitions are reloaded, instead of when they are published, retracted, or deleted.

* Add 'materializer' to user dictionary

The term 'materializer' has been added to the user dictionary to improve code spelling and naming consistency. This change ensures that 'materializer' is recognized as a correct term in the codebase.

* Organize test files by adding a 'Workflows' directory

Renamed 'http-workflow.json' to indicate it belongs under 'Workflows'. This improves file organization and clarity within the 'WorkflowDefinitionReload' scenario.

* Add TestWorkflowMaterializer and TestWorkflowProvider

Introduce `TestWorkflowMaterializer` for deserializing workflows from `TestWorkflowProvider`. Added integration of these new components in the `ReloadWorkflowTests` and `WorkflowServer`. Also renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests`.

* Rename and expand workflow reload tests

Renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests` to better reflect its purpose and expanded with additional test cases. Added tests to verify workflow and activity registry updates after source provider changes and workflow reloads.

* Update copy settings for workflow test scenarios

Reorganized and added 'CopyToOutputDirectory' settings for JSON files in workflow test scenarios. Ensured all necessary files are correctly included and copied during output directory builds to maintain test consistency.
2024-07-17 09:51:45 +02:00
Raymond den Haan 85c701e5d0 Overwriting System.Formats.Asn1 due to vulnerability 2024-07-16 15:04:46 +02:00
Mohamed Ali a0846effae
bugfixing issues related to new PropertyBag in workflow definition (#5778) 2024-07-16 14:54:20 +02:00