Commit graph

6237 commits

Author SHA1 Message Date
raymonddenhaan ab8deafcc1
Merge pull request #5779 from elsa-workflows/fix/vuln_formats
Overwriting System.Formats.Asn1 due to vulnerability
2024-07-16 15:07:27 +02:00
Raymond den Haan 85c701e5d0 Overwriting System.Formats.Asn1 due to vulnerability 2024-07-16 15:04:46 +02:00
Mohamed Ali a0846effae
bugfixing issues related to new PropertyBag in workflow definition (#5778) 2024-07-16 14:54:20 +02:00
raymonddenhaan 49c0a29427
Merge pull request #5777 from elsa-workflows/fix/vuln_json
Overwritten System.Text.Json version due to vulnerability
2024-07-16 14:46:31 +02:00
Raymond den Haan a69492d521 Overwritten System.Text.Json version due to vulnerability 2024-07-16 14:31:20 +02:00
Sipke Schoorstra 9fb51c6cd0 Add functions to get workflow definition details
Introduced three new functions: `getWorkflowDefinitionId`, `getWorkflowDefinitionVersionId`, and `getWorkflowDefinitionVersion`. These functions provide easy access to specific workflow definition details within the JavaScript evaluator. Removed redundant constructor in CommonFunctionsDefinitionProvider to streamline the code.
2024-07-16 13:55:23 +02:00
Sipke Schoorstra f63f823e7b Update ElsaStudioVersion to 3.2.0-rc3.443.
This change updates the ElsaStudioVersion property to the latest release candidate. It ensures compatibility with recent updates and bug fixes provided in version 3.2.0-rc3.443.
2024-07-16 09:02:24 +02:00
Sipke Schoorstra 1a0d4714bd
Update activity ID reference in BroadcastWorkflowProgress (#5767)
Replaced the use of ActivityId with ActivityNodeId when retrieving distinct activity IDs. This change ensures that we are using the correct node identifier for logging workflow progress.
2024-07-16 07:58:27 +02:00
Sipke Schoorstra ba2603ebd3
Add Bookmark Queue and Restore Background Activity Execution (#5758)
* Update package versions and add PrivateAssets attributes

Updated multiple package versions to the latest releases and added the `PrivateAssets="All"` attribute to several dependencies to improve project isolation. This ensures that the specified packages will not be propagated as transitive dependencies.

* Fix incorrect serializer and generator references.

Replaced `_payloadSerializer` and `_identityGenerator` with `payloadSerializer` and `identityGenerator` respectively. This resolves potential null reference issues and ensures the correct instances are used during workflow and definition processing.

* Refactor background activity scheduling logic

Moved the state commit outside of the deferred task to ensure workflow state is saved before scheduling activities. This change ensures the workflow instance is updated promptly, preventing potential concurrency issues.

* Add bookmark queue management system

Introduced a comprehensive bookmark queue system to manage and process bookmarks efficiently. This includes entities, stores, filters, processors, and workers for both in-memory and distributed environments. Additionally, added notifications to signal bookmark queue workers and updated related contracts.

* Add state commit handler with various implementations

Introduced ICommitStateHandler for committing workflow state. Provided NoopCommitStateHandler and StoreCommitStateHandler implementations. Updated namespaces and dependencies across the project to incorporate these changes.

* Fix order in CommitAsync method for proper task execution

Reorder `ExecuteDeferredTasksAsync` after `SaveAsync` in `CommitAsync` method to ensure that deferred tasks execute correctly after the state is saved, addressing potential issues with task execution dependencies.

* Remove unused and deprecated middleware and annotations

Deleted unused classes `ExecuteDeferredActivityTasks` and `ScheduleBackgroundActivitiesMiddleware`. Removed unnecessary summary tags and unused usings across multiple files to clean up the codebase.

* Remove unnecessary initial migration files

Deleted initial migration files for alterations, management, and identity contexts. These files are no longer needed and their removal helps keep the repository clean and maintainable.

* Add delay in TriggerBookmarkQueueWorker loop

Introduced a 10-second delay within the while loop of TriggerBookmarkQueueWorker. This change aims to alleviate potential tight loop issues, ensuring better performance and resource management.

* Enable all database providers in migration script

Uncomment the providers array and add previously commented database providers (SqlServer, Sqlite, PostgreSql, Oracle). This ensures compatibility with multiple database systems during the migration process.

* Remove unused Microsoft.Extensions.DependencyInjection import

The import statement for `Microsoft.Extensions.DependencyInjection` was not being utilized and has been removed. This cleanup helps in maintaining cleaner and more readable code without unnecessary imports.

* Add Dapper persistence support for bookmark queue

Introduced a new Dapper-based bookmark queue store and related classes for handling bookmark queue items. Various migrations were added to support the new `BookmarkQueueItems` table with tenant-specific columns. Refactored existing EF Core and in-memory bookmark queue item stores to use a unified interface.

* Add MongoBookmarkQueueStore implementation

Added a MongoDB-based BookmarkQueueStore to handle CRUD operations for bookmark queue items. Updated the workflow runtime persistence feature to include and utilize this new store for managing bookmark queue items.

* Add migration helper for altering columns and update keys

Introduced MigrationHelper to simplify altering DateTime and Boolean columns. Updated keys in KeyValueStore and KeyValueFilter classes to use 'Id' instead of 'Key'. Revised migration scripts to utilize the new helper methods for modifying column types.

* Prevent BookmarkQueueWorker.Stop from cancelling when not running

Previously, the Stop method would always cancel the token source regardless of the worker's state. By checking if the worker is running before cancelling, we avoid unnecessary operations and potential errors related to an already cancelled token source.

* Update database schema and bookmark handling logic

Refactored various database migration scripts to allow nullable `WorkflowInstanceId` fields and added new fields such as `ActivityInstanceId` and `CorrelationId`. Enhanced bookmark queue and bound workflow handling logic to support these new fields, ensuring precise and efficient workflow execution and queuing.

* Refactor BookmarkHash to StimulusHash and introduce ActivityTypeName

Renamed BookmarkHash to StimulusHash across multiple components and added ActivityTypeName to enhance traceability. Updated indices, filters, and entities for this change, ensuring consistent naming and improved functionality.

* Remove unnecessary timeouts in AzureServiceBusTests.

Eliminated redundant timeout parameters in _signalManager.WaitAsync calls to streamline test execution and reduce potential waiting time. This change ensures more efficient and accurate testing synchronization.

* Rename and refactor BookmarkQueueWorkerSignaler

Renamed IBookmarkQueueWorkerSignaler to IBookmarkQueueSignaler across the project for clarity. Updated related classes and methods to reflect this change. Added bookmark queue signaling to ensure new items are processed, and renamed BookmarkQueueStore to EFBookmarkQueueStore for consistency.

* Refactor EF Core stores and add migration field

Refactored EF Core store classes to simplify field usage and constructors. Added "CorrelationId" field and created corresponding index in SQLite and PostgreSQL migration files to support new functionality.

* Update workflow context ID and optimize Task handling

Added `ParentInstanceId` to `BulkDispatchWorkflowsStimulus` for context propagation. Also, reset `TaskCompletionSource` in `BookmarkQueueSignaler` to reduce memory usage and ensure proper task lifecycle management.

* Update async method signature and fix variable usage

Modified `AddAsync` to include the `OnSaveAsync` parameter. Corrected the variable used for `parentInstanceId` and utilized `ActivityTypeNameHelper` for generating type names.

* Reset migrations to 3.2

* Generate 3.3 migrations
2024-07-15 22:37:14 +02:00
raymonddenhaan a96180ed8b
Merge pull request #5751 from elsa-workflows/enh/journal_polling
Added endpoint to poll for journal updates
2024-07-12 08:43:28 +02:00
MariusVuscanNx feb7da4d28
Implemented workflows reload endpoint (#5732)
* Implemented workflows reload endpoint

* Added componenent tests for reload

* Adjusted cache handling

* Made the refresh endpoint return refreshed and not found definitions

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowDefinitionReload/RemoveReloadWorkflowTests.cs

Fix typo

Co-authored-by: raymonddenhaan <155616759+raymonddenhaan@users.noreply.github.com>

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Co-authored-by: raymonddenhaan <155616759+raymonddenhaan@users.noreply.github.com>
2024-07-12 08:35:46 +02:00
Raymond den Haan e52d716eaa Used DateTimeOffset instead of DateTime 2024-07-12 08:26:21 +02:00
Nokecy d37e312f7a
feat: Modify the access level of FlowScope type and add Remove to specify the activity method (#5752)
* update hangfire version to 1.8.5

* fix: fix hashSet deserialize error

* fix: replace HashSet to ISet<T>

* Update polymorphic converter and tests

* feat: Modify the access level of FlowScope type and add Remove to specify the activity method

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-11 22:10:57 +02:00
Sipke Schoorstra 1c7a96034c Merge branch 'patch/3.2.x' of github.com:elsa-workflows/elsa-core into patch/3.2.x 2024-07-11 22:09:32 +02:00
Sipke Schoorstra 1c068373fd Refactor API client DI methods for cleaner configuration
Refactor `AddElsaApiKeyClient`, `AddElsaClient`, and related methods to improve readability and maintainability. Introduced `AddDefaultApiClientsUsingApiKey`, `AddDefaultApiClients`, and `AddApiClients` for a more structured approach, consolidating redundant code and ensuring better configurability.
2024-07-11 22:09:13 +02:00
Raymond den Haan bd3797dea5 Fix datetime conversion when sending datetime to API endpoints 2024-07-11 15:18:17 +02:00
raymonddenhaan d777252e77
Merge pull request #5728 from elsa-workflows/enh/log_extractor
Add WorkflowExecutionLogRecordExtractor
2024-07-11 10:56:14 +02:00
Raymond den Haan 99c0797438 Add endpoint for checking for journal updates 2024-07-11 10:34:09 +02:00
Sipke Schoorstra 88f657e940 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-11 10:28:24 +02:00
Sipke Schoorstra 7e29472b87 Merge branch 'patch/3.2.x' of github.com:elsa-workflows/elsa-core into patch/3.2.x 2024-07-10 19:48:10 +02:00
Sipke Schoorstra dc4f904108 Update workflow revert to return summary and location
Enhanced the Revert endpoint to return a summary of the newly created workflow definition and include its location. Adjusted the API contract and endpoint configuration accordingly.
2024-07-10 19:47:50 +02:00
raymonddenhaan c7b5caafdc
Added aliases for all missing types used in UI (#5743) 2024-07-10 18:38:11 +02:00
Sipke Schoorstra 59bc7bc6a2 Add caching modules to project references
Included Elsa.Caching.Distributed.MassTransit and Elsa.Caching.Distributed as project references in Elsa.ServerAndStudio.Web.csproj. This integration aims to enhance caching capabilities in the application.
2024-07-10 17:58:28 +02:00
Sipke Schoorstra 4d41cd743b Add DistributedCachingTransport enum
Introduced the DistributedCachingTransport enum to define caching transport options. Also cleaned up the code by removing an unnecessary using directive and a commented summary in the MassTransitBroker enum.
2024-07-10 15:32:01 +02:00
Sipke Schoorstra 073598c748 Remove obsolete project reference
Deleted the project reference to Elsa.Server.Web.csproj from Elsa.ServerAndStudio.Web.csproj. This reference is no longer needed and was causing redundancy in the project configuration.
2024-07-10 15:28:50 +02:00
Sipke Schoorstra 90fba5f4bf Update Docker image tags in GitHub workflows
Corrected Docker image tags in workflow files to ensure consistent naming conventions. This update affects elsa-server, elsa-studio, and elsa-server-and-studio YAML files.
2024-07-10 15:09:12 +02:00
Sipke Schoorstra 847d6244ba Update workflows to target patch/3.2.x branch and tag Docker images
Changed the target branch from 'main' to 'patch/3.2.x' in the workflow YAML files. Updated Docker image tags to specific version '3.2.0-rc3' for both elsa-server and elsa-studio.
2024-07-10 15:07:07 +02:00
Sipke Schoorstra 1e95b6f847 Update ElsaStudio package version to use a variable
Switched ElsaStudio package versions to use a property variable for easier management and consistency. Also, adjusted the YAML workflow to tag Docker images with the specific version 3.2.0-rc3.
2024-07-10 15:05:31 +02:00
Sipke Schoorstra e2f2f61400 Update version to 3.2.0-rc3 in GitHub actions workflow
Changed the fallback version in the GitHub actions workflow from 3.2.0-rc2 to 3.2.0-rc3 to ensure the appropriate versioning reflects the latest release candidate. This keeps the versioning consistent and up-to-date.
2024-07-10 13:51:32 +02:00
Sipke Schoorstra dc89caefe8
Add ActivityNodeIds filter to journal requests (#5741)
This commit introduces the ActivityNodeIds filter to the journal request models. This enhancement allows filtering logs based on specific activity node IDs, providing more granular control over workflow instance logs. Additionally, unnecessary summary comments have been removed for clarity.
2024-07-10 11:38:09 +02:00
Sipke Schoorstra 558486920b
Add PropertyBag for non-polymorphic data transfer between server and client (#5735)
* Fix issues with workflow deletion, newline, and DB context.

Retract published workflows before deletion to avoid exceptions. Added missing newline at EOF for consistency. Removed redundant DisposeAsync call in migration service.

* Add PropertyBag for storing custom workflow properties

Replaced `CustomProperties` with `PropertyBag` class for more structured property management across workflows. Adjusted serializations, extensions, and middleware for seamless integration with the new `PropertyBag` structure.
2024-07-10 10:29:45 +02:00
Raymond den Haan 73b00fa607 Added missing service registration 2024-07-09 08:32:35 +02:00
Raymond den Haan f3a8ff5e47 Add WorkflowExecutionLogRecordExtractor
Introduced a new service, WorkflowExecutionLogRecordExtractor, to abstract the logic for extracting workflow execution logs records from the WorkflowExecutionContext. Updated StoreWorkflowExecutionLogSink to use this new service, which simplifies the execution log persistence method. This modification enhances code readability and enables potential reuse of the extraction logic.
2024-07-08 16:13:58 +02:00
Sipke Schoorstra 0816443dd6 Refactor workflow scheduling with deferred task execution
Reorder pipeline steps to ensure deferred tasks are used correctly in `WorkflowExecutionPipelineBuilderExtensions.cs`. Add `workflowInstanceManager` to `ScheduleBackgroundActivitiesMiddleware` and defer background activity scheduling to execute after state is saved for better consistency.
2024-07-07 23:37:58 +02:00
Sipke Schoorstra 2a31f598a7 Add error logging to JobRunnerHostedService
Previously, only job cancellations were logged. Now, in addition to logging job cancellations, the service will also log errors when a job fails, providing better diagnostics and troubleshooting.
2024-07-07 22:49:09 +02:00
Sipke Schoorstra 1c9f6648e4 Refactor ActivityDescriber method order
Reorganized the method order in `ActivityDescriber` to improve readability and logical grouping. Ensured that public methods are listed before private methods, maintaining clean code structure.
2024-07-07 21:41:51 +02:00
Sipke Schoorstra 90dc37ef98 Adjust dictionary to use case-insensitive string comparer
Changed the `_aliasTypeDictionary` to use `StringComparer.OrdinalIgnoreCase`, ensuring that type aliases are not case-sensitive. This enhancement improves the robustness of type look-up operations.
2024-07-07 21:20:20 +02:00
Sipke Schoorstra 42067c035c Refactor PolymorphicObjectConverterFactory constructor
Separated dependency injection in PolymorphicObjectConverterFactory into explicit constructor. Added a default constructor for attribute usage, ensuring WellKnownTypeRegistry is initialized uniformly.
2024-07-07 16:25:26 +02:00
Sipke Schoorstra 469d909ded Add ApiEndpointOptions and refactor imports
Introduce `ApiEndpointOptions` class for API route configuration. Adjust various file imports to use `Elsa.Workflows.Management` and consolidate namespaces to enhance code maintainability and readability. Additionally, mark `HttpActivityOptions.ApiRoutePrefix` as obsolete in favor of `ApiEndpointOptions.RoutePrefix`.
2024-07-07 16:09:50 +02:00
Sipke Schoorstra 27d3a4e68e Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-07 11:45:26 +02:00
Sipke Schoorstra e2db1636ba Merge branch 'main' of github.com:elsa-workflows/elsa-core 2024-07-07 11:34:27 +02:00
Sipke Schoorstra 37d14dd26c Modify properties in ActivityDescriptor and add new workflow file
Changed 'init' to 'set' in ActivityDescriptor properties to enable modification after initialization. Added new GitHub Actions workflow file (pr.yml) to the solution.
2024-07-07 11:34:03 +02:00
Sipke Schoorstra 218f3b46e4 Update grep pattern in release branch check
Changed the grep pattern from 'main' to 'origin/patch/3.2.x' in the release workflow. This ensures the script correctly identifies the relevant branch during the release process.
2024-07-05 20:44:00 +02:00
Sipke Schoorstra 895598fb70 Change MassTransit broker to in-memory
Switched the MassTransit broker configuration from RabbitMQ to in-memory. This change aims to simplify local development and testing by using an easier-to-configure broker.
2024-07-05 19:07:23 +02:00
Sipke Schoorstra 85857f0312 Merge branch 'patch/3.2.x' of github.com:elsa-workflows/elsa-core into patch/3.2.x 2024-07-05 18:38:09 +02:00
Alexander Kislitsyn 038661e7a3
5391 Use database name from options when specified (#5634)
Co-authored-by: raymonddenhaan <155616759+raymonddenhaan@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-05 18:36:49 +02:00
benjamin-hui ab25df077e
ef core oracle provider (#5637)
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-05 18:35:52 +02:00
MariusVuscanNx 6110eb561f
Workflow refresh feature (#5713)
* Refactor workflow definition events handling and dispatch

Removed IDistributedWorkflowDefinitionEventsDispatcher interface and related classes in favor of direct message publishing through MassTransit. Introduced new workflow definition refresh endpoints and services for better modularity and responsiveness. Adjusted caching and notification systems to support the new architecture.

* Add comment

* Added workflow definition refresh component test

* Fixed component tests issue

* Fixed id

* Fixed test

* Rename IsRaisedFromConsumer to IsConsumerExecutionContext

Updated the naming for clarity and consistency across the system. This change affects AmbientConsumerScope, related consumers, and condition checks to improve readability and understanding of the consumer context.

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-07-05 18:35:22 +02:00
Sipke Schoorstra 5c7693b4af Improve comment for workflow execution context creation.
Changed comment to enhance readability and clarity. The modification adds the article "a" to the description, providing a better understanding of the workflow execution context creation process.
2024-07-05 18:20:34 +02:00
Sipke Schoorstra dc81cbb31d Refactor RunAsync method signature in WorkflowRunner.
Updated the RunAsync method in WorkflowRunner to consolidate its parameters onto a single line for improved readability and simplicity. This change does not affect the functionality but enhances code maintainability.
2024-07-05 18:20:11 +02:00