Commit graph

5214 commits

Author SHA1 Message Date
Sipke Schoorstra 369263f9ca Fix null value handling for AppRole in Program.cs
Updated the application role parsing to provide a default value when the configuration is missing. Removed redundant inbox cleanup options binding, and added a Resharper disable comment for redundant assignment.
2024-07-24 14:21:21 +02:00
Sipke Schoorstra f301da3b12 Switch to RunAsync for all web applications
Updated the Program.cs files across web applications to use `await app.RunAsync()` instead of `app.Run()`. This change ensures proper asynchronous operation and aligns with best practices for hosting .NET applications.
2024-07-24 14:18:04 +02:00
Sipke Schoorstra 3b304c8e08 Add OnStopped method to dispose of _linkedTokenSource
This commit introduces the OnStopped method in the WorkflowInstanceImpl class to properly dispose of _linkedTokenSource. This change ensures that resources are appropriately cleaned up when the actor is stopped, preventing potential memory leaks.
2024-07-24 14:14:50 +02:00
Sipke Schoorstra 95476cac99 Fix project reference path in csproj file
Corrected the path in the ProjectReference for Elsa.csproj to ensure proper module loading. This resolves issues related to incorrect project references and allows the project to build successfully.
2024-07-24 14:12:31 +02:00
Sipke Schoorstra 2115a9b4e9 Update project reference path in Elsa.Testing.Shared.Component
Changed the project reference from the bundles directory to the module directory for Elsa.csproj. This update ensures proper inclusion of the Elsa project within Elsa.Testing.Shared.Component.
2024-07-24 14:12:08 +02:00
Sipke Schoorstra 2cfa8f935c Update project reference path for Elsa module
Corrected the project reference path from 'bundles' to 'modules' to ensure proper linkage and compilation. This change addresses a path configuration error in the project file.
2024-07-24 14:11:29 +02:00
Sipke Schoorstra f96d594f6f Refactor folder paths and update method implementations
Updated Dockerfile paths to reflect 'apps' directory structure, removed redundant XML comments, and modified methods in WorkflowDefinitionExtensions to use FindWorkflowGraphAsync for consistent workflow retrieval.
2024-07-24 14:10:27 +02:00
Sipke Schoorstra aef5e7b311 Add lang attribute to HTML tag
Including the lang attribute with the value "en" improves accessibility and helps search engines better understand the page's content. This small change enhances the overall SEO and user experience for English-speaking users.
2024-07-24 14:05:57 +02:00
Sipke Schoorstra e4a161f40a Set language attribute in _Host.cshtml
Added the `lang` attribute with the value "en" to the `<html>` tag to specify the language of the document. This improves accessibility and enhances SEO by clearly defining the page's primary language.
2024-07-24 14:04:45 +02:00
Sipke Schoorstra ed79494dba Rename Elsa.ProtoActor to Elsa.Workflows.Runtime.ProtoActor 2024-07-22 15:09:25 +02:00
Sipke Schoorstra d3f9d5ef8f Merge branch 'main' of github.com:elsa-workflows/elsa-core 2024-07-22 14:36:43 +02:00
Sipke Schoorstra 70b8582b23
Add OpenTelemetry module (#5810)
* Add OpenTelemetry integration for workflow tracing

Introduced a new module, Elsa.OpenTelemetry, to provide OpenTelemetry sources for tracing workflow and activity execution. Updated various components and pipeline extensions to support OpenTelemetry tracing throughout the workflow execution process.

* Refactor workflow execution pipelines

Deleted `WorkflowsFeatureExtensions` and migrated methods to `PipelineWorkflowsFeatureExtensions` with added configurability. Enhanced `ActivityExecutionMiddlewareExtensions` and `ActivityExecutionPipelinePipelineBuilder` to support middleware insertion. Added comprehensive tracing to `OpenTelemetryTracingWorkflowExecutionMiddleware`.

* Add OpenTelemetry tracing to activity execution

Introduced `OpenTelemetryTracingActivityExecutionMiddleware` to capture tracing information for activity execution within workflows. This middleware logs activity execution start and end events, attaching pertinent activity tags. Added extension method to register this middleware in the workflow execution pipeline.

* Set `PYTHONNET_PYDLL` consistently and update workflow pipelines

Update Dockerfiles to set the `PYTHONNET_PYDLL` environment variable consistently without spaces. Additionally, refactor `Program.cs` to streamline workflow and activity execution pipeline configurations by using the `WithDefaultWorkflowExecutionPipeline` and `WithDefaultActivityExecutionPipeline` methods.

* Enhance OpenTelemetry Tracing Middleware Implementation

Add missing activity tags and events to improve telemetry data. Simplify middleware installation syntax for both workflow and activity execution tracing pipelines, ensuring consistent and clear tracing across modules.
2024-07-22 14:36:33 +02:00
Sipke Schoorstra 4f49252fbf Add dependencies to docker-compose and update README
Updated `docker-compose.yml` to include `depends_on` for `trace-lens`, ensuring it starts after `plantuml-server` and `postgres`. Modified the README to include `Elsa.Workflows` in `OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES` and corrected formatting for `OTEL_RESOURCE_ATTRIBUTES`.
2024-07-22 14:36:02 +02:00
Sipke Schoorstra b4cd4dd465
Replace custom webhooks implementation with WebhooksCore (#5811)
* Replace custom webhooks implementation with WebhooksCore

Replaced the existing custom webhooks registration and dispatching system with the WebhooksCore library. This involved removing all previous custom webhook classes and interfaces, updating project dependencies, and modifying appsettings and program files to use the new configuration. The update simplifies the webhook handling architecture and leverages the features provided by WebhooksCore.

* Hide API key in Datadog Docker compose file

This change replaces the exposed Datadog API key with a placeholder "<HIDDEN>". This enhances security by preventing the API key from being visible in the version control system.
2024-07-21 16:48:38 +02:00
Sipke Schoorstra 7c34f22d56 Remove commented OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES
Deleted an unused commented configuration line in docker-compose-datadog.yml to improve readability and maintainability. This change does not affect the existing tracing configuration or application behavior.
2024-07-20 00:05:49 +02:00
Sipke Schoorstra 8acf9b7212 Add pull_policy and environment variables for Datadog
Added `pull_policy: always` to `elsa-server` and `elsa-studio` services in the Docker Compose file to ensure the latest images are always pulled. Updated `elsa-server` environment variables to improve Datadog tracing and monitoring. Adjusted the Dockerfile path for the `elsa-studio` service.
2024-07-19 23:51:46 +02:00
Sipke Schoorstra 2a6a73d96e Enable tracing for WorkflowInstanceActor conditionally
Uncommented conditional enabling of tracing for WorkflowInstanceActor. This ensures that tracing is enabled only if the `_enableTracing` flag is set, improving the clarity and functionality of the feature.
2024-07-19 23:49:34 +02:00
Sipke Schoorstra 6924bd0770 Update build paths, add Datadog tracing, and adjust versions
Corrected Dockerfile paths for Elsa.Server and Elsa.Studio, and added Datadog tracing and related environment variables. Included Datadog.Trace package in Elsa.Server.Web project and updated Elsa.Studio package versions to 3.3.0-preview.466.
2024-07-19 23:36:31 +02:00
Sipke Schoorstra 107d660210 Update Dockerfile paths for consistency
Changed paths from "bundles" to "apps" in multiple Dockerfiles to standardize the directory structure. This ensures all Dockerfiles use a consistent directory naming convention across the project.
2024-07-19 21:01:46 +02:00
Sipke Schoorstra 283824fcfd
Enable Proto Actor Tracing for TraceLens (#5800)
* Add database initialization script and update dependencies

Added a script to initialize the 'tracelens' database and modified the Docker setup to include this script. Refactored and improved the ProtoActorFeature class, added OpenTelemetry dependencies, and updated project settings.

* Enable OpenTelemetry integration for Proto.Actor

Added OpenTelemetry environment configuration details to the README and included the Proto.OpenTelemetry package in the project file. Updated the ProtoActorFeature to apply tracing with OpenTelemetry to WorkflowInstanceActor.

* Refactor VariablePersistenceManager to use primary constructor

This refactor simplifies the VariablePersistenceManager by moving the storageDriverManager initialization into the primary constructor. It removes the redundant field and constructor, aligning with the concise nature of modern C# syntax, and ensures consistency in accessing the storageDriverManager throughout the class.

* Remove unused Open Telemetry code from Program.cs

The code for configuring Open Telemetry was commented out but not removed, cluttering the file. This commit cleans up Program.cs by deleting these unused lines, maintaining a cleaner and more readable codebase.

* Add metrics and tracing configurations for ProtoActorFeature

Introduced methods to enable metrics and tracing in ProtoActorFeature. Removed redundant properties and updated the workflow runtime to utilize the new configurations.

* Add Directory.Build.props for shared project settings

Introduce Directory.Build.props to centralize common project settings and dependencies. Consolidate target framework, language version, and package references to reduce duplication. Remove redundant property definitions from Elsa.Server.Web.csproj.

* Move apps from bundles to apps folder and Elsa module to modules folder
2024-07-19 20:23:32 +02:00
Sipke Schoorstra 5586151bf1 Remove IWorkflowExecutionContextStore and related classes
Removed the IWorkflowExecutionContextStore interface and its implementations, along with the associated handlers and registrations. This change simplifies the codebase by eliminating unnecessary components and their dependencies.
2024-07-18 18:57:42 +02:00
Sipke Schoorstra 827680b3c4 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-18 18:25:49 +02:00
Sipke Schoorstra e5ba9a8a9d Refactor scheduling and system clock configurations
Moved specific services from Scoped to Singleton in SchedulingFeature and SystemClockFeature for better performance and consistency. Refactored DefaultWorkflowScheduler to use a constructor with an IScheduler parameter and removed redundant private field. Removed attribute RequiresUnreferencedCode in WorkflowManagementFeature.
2024-07-18 16:00:49 +02:00
Sipke Schoorstra 68c6623555 Fix Workflow Reload Test and Update Activity Registry Logic
Unskip the test for workflow reload after updating the source provider and fix the issue with activity name formatting using Humanizer library. Ensure activity names are consistent for both workflow versions in the test.
2024-07-18 14:23:39 +02:00
Raymond den Haan 481545813d Skip failing test 2024-07-18 10:34:48 +02:00
Sipke Schoorstra 149f91e538
Update API endpoint for Polling Observer (#5787)
* Rename and refactor journal update endpoint

Replaced `/workflow-instances/{id}/journal/has-updates` endpoint with `/workflow-instances/{id}/updated-at` to simplify API responses. Deleted `HasUpdates` related classes and introduced `GetUpdatedAtResponse` for consistency and clarity. Updated client contracts accordingly.

* Remove HasUpdates endpoint and refactor workflow observer

Deleted the HasUpdates endpoint and refactored related code to use an updated timestamp approach instead. Improved nullable handling in WorkflowInstanceDesigner and ensured proper observer disposal to avoid memory leaks. Updated workflow observer factory and observer implementations to support observer names and enhanced logging.

* Rename updated workflow instance endpoint and handle execution state

Renamed the endpoint from "/updated-at" to "/execution-state" to better reflect its purpose. Updated related response models and documentation to capture workflow execution state details such as status, sub-status, and last updated timestamp.

* Enable SignalR for real-time workflows

Add a flag to use SignalR and activate real-time workflows when enabled. Refactor code to wrap SignalR setup in conditional checks based on the new flag. This enhances the application's interactivity through real-time capabilities.

* Remove obsolete endpoints and rename execution state paths

Deleted the outdated Api1 and DynamicWorkflows endpoints under Elsa.Server.Web. Also, renamed paths related to execution state models and endpoint to remove "Journal" from the namespace for better clarity and organization.
2024-07-18 07:51:46 +02:00
Sipke Schoorstra f448e9520a
Fix reloading logic for workflow definitions (#5781)
* Add reloading logic for workflow definitions

Introduced a new mechanism to handle reloaded workflow definitions. This includes creating a "ReloadedWorkflowDefinition" model, updating the caching logic, and modifying notification handlers to work with the enhanced workflow reloading logic. This ensures workflow definitions are updated and managed correctly when published, retracted, or deleted.

* Improve RefreshActivityRegistry documentation

Updated the XML documentation to clarify that `RefreshActivityRegistry` refreshes the `IActivityRegistry` for `WorkflowDefinitionActivityProvider` whenever workflow definitions are reloaded, instead of when they are published, retracted, or deleted.

* Add 'materializer' to user dictionary

The term 'materializer' has been added to the user dictionary to improve code spelling and naming consistency. This change ensures that 'materializer' is recognized as a correct term in the codebase.

* Organize test files by adding a 'Workflows' directory

Renamed 'http-workflow.json' to indicate it belongs under 'Workflows'. This improves file organization and clarity within the 'WorkflowDefinitionReload' scenario.

* Add TestWorkflowMaterializer and TestWorkflowProvider

Introduce `TestWorkflowMaterializer` for deserializing workflows from `TestWorkflowProvider`. Added integration of these new components in the `ReloadWorkflowTests` and `WorkflowServer`. Also renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests`.

* Rename and expand workflow reload tests

Renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests` to better reflect its purpose and expanded with additional test cases. Added tests to verify workflow and activity registry updates after source provider changes and workflow reloads.

* Update copy settings for workflow test scenarios

Reorganized and added 'CopyToOutputDirectory' settings for JSON files in workflow test scenarios. Ensured all necessary files are correctly included and copied during output directory builds to maintain test consistency.
2024-07-17 09:51:45 +02:00
raymonddenhaan ab8deafcc1
Merge pull request #5779 from elsa-workflows/fix/vuln_formats
Overwriting System.Formats.Asn1 due to vulnerability
2024-07-16 15:07:27 +02:00
Raymond den Haan 85c701e5d0 Overwriting System.Formats.Asn1 due to vulnerability 2024-07-16 15:04:46 +02:00
Mohamed Ali a0846effae
bugfixing issues related to new PropertyBag in workflow definition (#5778) 2024-07-16 14:54:20 +02:00
raymonddenhaan 49c0a29427
Merge pull request #5777 from elsa-workflows/fix/vuln_json
Overwritten System.Text.Json version due to vulnerability
2024-07-16 14:46:31 +02:00
Raymond den Haan a69492d521 Overwritten System.Text.Json version due to vulnerability 2024-07-16 14:31:20 +02:00
Sipke Schoorstra 9fb51c6cd0 Add functions to get workflow definition details
Introduced three new functions: `getWorkflowDefinitionId`, `getWorkflowDefinitionVersionId`, and `getWorkflowDefinitionVersion`. These functions provide easy access to specific workflow definition details within the JavaScript evaluator. Removed redundant constructor in CommonFunctionsDefinitionProvider to streamline the code.
2024-07-16 13:55:23 +02:00
Sipke Schoorstra f63f823e7b Update ElsaStudioVersion to 3.2.0-rc3.443.
This change updates the ElsaStudioVersion property to the latest release candidate. It ensures compatibility with recent updates and bug fixes provided in version 3.2.0-rc3.443.
2024-07-16 09:02:24 +02:00
Sipke Schoorstra 1a0d4714bd
Update activity ID reference in BroadcastWorkflowProgress (#5767)
Replaced the use of ActivityId with ActivityNodeId when retrieving distinct activity IDs. This change ensures that we are using the correct node identifier for logging workflow progress.
2024-07-16 07:58:27 +02:00
Sipke Schoorstra ba2603ebd3
Add Bookmark Queue and Restore Background Activity Execution (#5758)
* Update package versions and add PrivateAssets attributes

Updated multiple package versions to the latest releases and added the `PrivateAssets="All"` attribute to several dependencies to improve project isolation. This ensures that the specified packages will not be propagated as transitive dependencies.

* Fix incorrect serializer and generator references.

Replaced `_payloadSerializer` and `_identityGenerator` with `payloadSerializer` and `identityGenerator` respectively. This resolves potential null reference issues and ensures the correct instances are used during workflow and definition processing.

* Refactor background activity scheduling logic

Moved the state commit outside of the deferred task to ensure workflow state is saved before scheduling activities. This change ensures the workflow instance is updated promptly, preventing potential concurrency issues.

* Add bookmark queue management system

Introduced a comprehensive bookmark queue system to manage and process bookmarks efficiently. This includes entities, stores, filters, processors, and workers for both in-memory and distributed environments. Additionally, added notifications to signal bookmark queue workers and updated related contracts.

* Add state commit handler with various implementations

Introduced ICommitStateHandler for committing workflow state. Provided NoopCommitStateHandler and StoreCommitStateHandler implementations. Updated namespaces and dependencies across the project to incorporate these changes.

* Fix order in CommitAsync method for proper task execution

Reorder `ExecuteDeferredTasksAsync` after `SaveAsync` in `CommitAsync` method to ensure that deferred tasks execute correctly after the state is saved, addressing potential issues with task execution dependencies.

* Remove unused and deprecated middleware and annotations

Deleted unused classes `ExecuteDeferredActivityTasks` and `ScheduleBackgroundActivitiesMiddleware`. Removed unnecessary summary tags and unused usings across multiple files to clean up the codebase.

* Remove unnecessary initial migration files

Deleted initial migration files for alterations, management, and identity contexts. These files are no longer needed and their removal helps keep the repository clean and maintainable.

* Add delay in TriggerBookmarkQueueWorker loop

Introduced a 10-second delay within the while loop of TriggerBookmarkQueueWorker. This change aims to alleviate potential tight loop issues, ensuring better performance and resource management.

* Enable all database providers in migration script

Uncomment the providers array and add previously commented database providers (SqlServer, Sqlite, PostgreSql, Oracle). This ensures compatibility with multiple database systems during the migration process.

* Remove unused Microsoft.Extensions.DependencyInjection import

The import statement for `Microsoft.Extensions.DependencyInjection` was not being utilized and has been removed. This cleanup helps in maintaining cleaner and more readable code without unnecessary imports.

* Add Dapper persistence support for bookmark queue

Introduced a new Dapper-based bookmark queue store and related classes for handling bookmark queue items. Various migrations were added to support the new `BookmarkQueueItems` table with tenant-specific columns. Refactored existing EF Core and in-memory bookmark queue item stores to use a unified interface.

* Add MongoBookmarkQueueStore implementation

Added a MongoDB-based BookmarkQueueStore to handle CRUD operations for bookmark queue items. Updated the workflow runtime persistence feature to include and utilize this new store for managing bookmark queue items.

* Add migration helper for altering columns and update keys

Introduced MigrationHelper to simplify altering DateTime and Boolean columns. Updated keys in KeyValueStore and KeyValueFilter classes to use 'Id' instead of 'Key'. Revised migration scripts to utilize the new helper methods for modifying column types.

* Prevent BookmarkQueueWorker.Stop from cancelling when not running

Previously, the Stop method would always cancel the token source regardless of the worker's state. By checking if the worker is running before cancelling, we avoid unnecessary operations and potential errors related to an already cancelled token source.

* Update database schema and bookmark handling logic

Refactored various database migration scripts to allow nullable `WorkflowInstanceId` fields and added new fields such as `ActivityInstanceId` and `CorrelationId`. Enhanced bookmark queue and bound workflow handling logic to support these new fields, ensuring precise and efficient workflow execution and queuing.

* Refactor BookmarkHash to StimulusHash and introduce ActivityTypeName

Renamed BookmarkHash to StimulusHash across multiple components and added ActivityTypeName to enhance traceability. Updated indices, filters, and entities for this change, ensuring consistent naming and improved functionality.

* Remove unnecessary timeouts in AzureServiceBusTests.

Eliminated redundant timeout parameters in _signalManager.WaitAsync calls to streamline test execution and reduce potential waiting time. This change ensures more efficient and accurate testing synchronization.

* Rename and refactor BookmarkQueueWorkerSignaler

Renamed IBookmarkQueueWorkerSignaler to IBookmarkQueueSignaler across the project for clarity. Updated related classes and methods to reflect this change. Added bookmark queue signaling to ensure new items are processed, and renamed BookmarkQueueStore to EFBookmarkQueueStore for consistency.

* Refactor EF Core stores and add migration field

Refactored EF Core store classes to simplify field usage and constructors. Added "CorrelationId" field and created corresponding index in SQLite and PostgreSQL migration files to support new functionality.

* Update workflow context ID and optimize Task handling

Added `ParentInstanceId` to `BulkDispatchWorkflowsStimulus` for context propagation. Also, reset `TaskCompletionSource` in `BookmarkQueueSignaler` to reduce memory usage and ensure proper task lifecycle management.

* Update async method signature and fix variable usage

Modified `AddAsync` to include the `OnSaveAsync` parameter. Corrected the variable used for `parentInstanceId` and utilized `ActivityTypeNameHelper` for generating type names.

* Reset migrations to 3.2

* Generate 3.3 migrations
2024-07-15 22:37:14 +02:00
raymonddenhaan a96180ed8b
Merge pull request #5751 from elsa-workflows/enh/journal_polling
Added endpoint to poll for journal updates
2024-07-12 08:43:28 +02:00
MariusVuscanNx feb7da4d28
Implemented workflows reload endpoint (#5732)
* Implemented workflows reload endpoint

* Added componenent tests for reload

* Adjusted cache handling

* Made the refresh endpoint return refreshed and not found definitions

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowDefinitionReload/RemoveReloadWorkflowTests.cs

Fix typo

Co-authored-by: raymonddenhaan <155616759+raymonddenhaan@users.noreply.github.com>

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: raymonddenhaan <155616759+raymonddenhaan@users.noreply.github.com>
2024-07-12 08:35:46 +02:00
Raymond den Haan e52d716eaa Used DateTimeOffset instead of DateTime 2024-07-12 08:26:21 +02:00
Nokecy d37e312f7a
feat: Modify the access level of FlowScope type and add Remove to specify the activity method (#5752)
* update hangfire version to 1.8.5

* fix: fix hashSet deserialize error

* fix: replace HashSet to ISet<T>

* Update polymorphic converter and tests

* feat: Modify the access level of FlowScope type and add Remove to specify the activity method

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-11 22:10:57 +02:00
Sipke Schoorstra 1c7a96034c Merge branch 'patch/3.2.x' of github.com:elsa-workflows/elsa-core into patch/3.2.x 2024-07-11 22:09:32 +02:00
Sipke Schoorstra 1c068373fd Refactor API client DI methods for cleaner configuration
Refactor `AddElsaApiKeyClient`, `AddElsaClient`, and related methods to improve readability and maintainability. Introduced `AddDefaultApiClientsUsingApiKey`, `AddDefaultApiClients`, and `AddApiClients` for a more structured approach, consolidating redundant code and ensuring better configurability.
2024-07-11 22:09:13 +02:00
Raymond den Haan bd3797dea5 Fix datetime conversion when sending datetime to API endpoints 2024-07-11 15:18:17 +02:00
raymonddenhaan d777252e77
Merge pull request #5728 from elsa-workflows/enh/log_extractor
Add WorkflowExecutionLogRecordExtractor
2024-07-11 10:56:14 +02:00
Raymond den Haan 99c0797438 Add endpoint for checking for journal updates 2024-07-11 10:34:09 +02:00
Sipke Schoorstra 88f657e940 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-11 10:28:24 +02:00
Sipke Schoorstra 7e29472b87 Merge branch 'patch/3.2.x' of github.com:elsa-workflows/elsa-core into patch/3.2.x 2024-07-10 19:48:10 +02:00
Sipke Schoorstra dc4f904108 Update workflow revert to return summary and location
Enhanced the Revert endpoint to return a summary of the newly created workflow definition and include its location. Adjusted the API contract and endpoint configuration accordingly.
2024-07-10 19:47:50 +02:00
raymonddenhaan c7b5caafdc
Added aliases for all missing types used in UI (#5743) 2024-07-10 18:38:11 +02:00
Sipke Schoorstra 59bc7bc6a2 Add caching modules to project references
Included Elsa.Caching.Distributed.MassTransit and Elsa.Caching.Distributed as project references in Elsa.ServerAndStudio.Web.csproj. This integration aims to enhance caching capabilities in the application.
2024-07-10 17:58:28 +02:00