Commit graph

64 commits

Author SHA1 Message Date
Sipke Schoorstra 83238e9931
Update System.Text.Json version reference (#6019)
Replaced hardcoded System.Text.Json version 8.0.4 with a variable $(SystemTextJsonVersion) across multiple project files. This improves maintainability and consistency when updating the package version.
2024-10-11 14:35:24 +02:00
Raymond den Haan a69492d521 Overwritten System.Text.Json version due to vulnerability 2024-07-16 14:31:20 +02:00
Sipke Schoorstra 783e16f3f7 Add V3_2 migration to Elsa modules
This commit adds a new migration, V3_2, to various Elsa modules including MySql, SqlServer, Sqlite and PostgreSql. It includes the creation of new migration files for both alterations and runtime aspects. The commit applies to Elsa's EntityFrameworkCore-based modules and forms part of the database schema evolution process.
2024-06-10 20:54:23 +02:00
Sipke Schoorstra 69aacd654b
Regenerate V3.1 EF Core migrations with custom runtime schema support (#5155)
* Reset EF Core migrations to 3.0

* Add V3_1 migration altering Status column and SerializableWorkflowInstanceIds

This commit adds a new migration file (V3_1) which alters the 'Status' column in the 'AlterationJobs' and 'AlterationPlans' tables to be of type string instead of int. It also renames the 'SerializedWorkflowInstanceIds' to 'SerializedWorkflowInstanceFilter' in the same tables. This update applies across multiple databases (MySQL, SQL Server, SQLite, PostgreSQL).

* Remove migration command from efcore-3.1.sh script

The Entity Framework Core 3.1 migration command line has been removed from the efcore-3.1.sh script. This modification ensures that migration handling relies solely on the 'ef-migration-runtime-schema' function.
2024-03-29 21:15:23 +01:00
raymonddenhaan 4b151ff8af
Feature/cancel dispatched workflows (#5136)
* Made cancel API call dispatch the request

* Added ParentWorkflowInstanceId

* Recursively cancel child workflows

* Made single workflow cancellation include child workflows

* Updated migrations

* Processed comments
2024-03-27 12:07:55 +01:00
Sipke Schoorstra 83dd776233 Add new docker-compose files, new activities and services
Create new docker-compose files for non-data dog and data dog services. New services include Postgres, cockroachdb, rabbitmq, redis, and elsa-server. Along with the services, a new bash script for migration and various new activities and endpoints are also added for the Elsa server. This set of changes broadens the operational environment of Elsa.
2024-03-24 20:28:19 +01:00
Sipke Schoorstra 20d936d3c2
Alterations with Queries (#5032)
* Remove Elsa.Alteration Handlers and implement workflow alteration activities

The Elsa.Alterations.Handlers, Services, and Endpoints have been removed and replaced with Elsa.Alterations.Activities for various steps in executing an alteration plan. These activities include submitting an alteration plan, generating alteration jobs, and dispatching alteration jobs. New models for ActivityFilter, AlterationWorkflowInstanceFilter, and alteration activity summaries are also introduced for better management of alteration plans.

* Add 'PrintMessage' custom activity and update naming rules

A new 'PrintMessage' activity has been added to the custom activities of the ConsoleApp, which simply writes a 'Hello world!' message. On top of this, the naming rules in the code styling settings have been expanded, introducing separate rules for instance and static private fields.

* Remove V3_1 database migration files

The V3_1 database migration files were deleted across all database providers, including MySql, SQL Server, SQLite and PostgreSQL. This is indicative of a rollback or significant revision to the data migration strategy.

* Update product version and modify tables in Elsa DB context

Updated "Elsa" DbSet product version from "7.0.11" to "7.0.14". Additionally, added new properties as well as modified existing ones in tables like "Elsa.KeyValues.Entities.SerializedKeyValuePair" to improve data structure. Also properly ignored certain entities in the Alterations DbContext.

* Add V3_1 migration for Elsa

This commit introduces the V3_1 migration for the Elsa module. Changes involve the introduction of new tables and alteration of existing tables. Specifically, a new 'KeyValuePairs' table was added and several columns on various tables were renamed or altered.

* Remove multiple Elsa.EfCore V3_1 migration files

The commit removes several migration files related to the V3_1 migration in multiple Elsa.EntityFrameworkCore modules including SqlServer, PostgreSQL, and SQLite. These files encompass changes in table structures and alterations.

* Update default value for ArgumentDefinition Type

The default value for the `Type` property in the `ArgumentDefinition` class was updated to `typeof(object)`. The `Type` property of `inputDefinition` in the `WorkflowBuilder` class was also modified to be set as `typeof(T)`. These modifications enable more flexible type handling.

* Refactor SafeSerializer and serialization services

The SafeSerializer service and related serialization services have been comprehensively refactored. This includes the removal of internal notifications, the incorporation of [RequiresUnreferencedCode] attributes, and changes in method signatures. Methods have been simplified to return ValueTasks directly from the JsonSerializer, and the notification sender dependency was removed.

* Add JSON serialization for alterations

Implemented a JSON serializer and a custom converter for the IAlteration interface in the Elsa.Alterations.Core.Serialization namespace. Also, introduced a new IJsonSerializer interface and an implementation for standard JSON serialization in Elsa.Common. These serialization changes enhance modularity and extensibility of JSON operations, and improve handling of IAlteration objects.

* Add "IsSystem" field to Workflow Definitions and Instances

This update introduces a new boolean field "IsSystem" to the Workflow Definition and Workflow Instance models. This field is designed to identify system workflows which aren't meant to be modified by users. Management and Runtime module code, API endpoints, database migrations, and related classes have been updated to include this new field. The MongoDB indices and Dapper migrations have been updated as well, and serialization/deserialization processes for these types have been modified to handle this new attribute.

* Remove outdated database migrations

Deleted database migration files from the MySql, SqlServer, and PostgreSql projects.

* Add V3_1 migration for Elsa.EntityFrameworkCore.Sqlite

This commit introduces the V3_1 migration for the Elsa.EntityFrameworkCore.Sqlite module. It includes changes in Alterations, Runtime, Management, and Identity contexts. These changes involve table alterations, addition of new columns, and creation of new tables. Each context alteration encapsulates relevant changes for specific functionalities of the module.

* Add `IsSystem` property to workflow classes

The `IsSystem` property has been added to several sections of the codebase related to workflows, such as `WorkflowStateMapper.cs`, `ListWorkflowDefinitionsRequest.cs`, `WorkflowDefinitionStore.cs`, and `ListWorkflowInstancesRequest.cs`. This property flag allows us to include or filter system workflows in our queries and results.

* Add 'UsedImplicitly' attribute to Alteration Handlers and Types

The 'UsedImplicitly' attribute from JetBrains.Annotations has been added to all Alteration Handlers and Alteration Types in the Elsa.Alterations module. This change specifies that these classes are being used implicitly and helps tools like ReSharper prevent false unused code detection.

* Update migration script for efcore-3.1

The migration script efcore-3.1.sh has been updated. Module "Alterations" and provider "Sqlite" are added back into the lists while unused elements are removed for cleanup. The script now covers all necessary modules and providers, improving its overall effectiveness.

* Remove PrintMessage class from Elsa.Server.Web bundle

This commit deletes the PrintMessage.cs file that was previously part of the Elsa.Server.Web bundle.

* Refactor MassTransit service configuration

This commit reduces line code length by merging splitted long lines into one, improves maintainability of code. It also updates the Service configuration and removes unused namespaces across files, which help to simplify the codebase. Changes have also been made to switch the MassTransit broker from RabbitMq to memory and adjust the endpoint name formatter configuration.

* Remove extra whitespace in Program.cs

Removed a trailing whitespace on the line declaring the useSqlServer constant in the Program.cs file. No functionality changes were made, simply a minor formatting correction to maintain clean and consistent code.

* Remove AlterationJsonConverter and update other files

The AlterationJsonConverter class has been removed, simplifying the serialization process. The AlterationSerializer.cs file was updated with additional attribute tags, giving more information about serialization expectations. Lastly, the AlterationSerializationOptionConfigurator.cs was simplified by removing unnecessary overrides and the Elsa.Alterations.Core.csproj file was tidied up by removing unused services folder reference.

* Refactor code for improved readability and maintainability

The diff shows changes across multiple files in multiple modules. Primarily, these changes revolve around formatting edits such as removing unnecessary spaces or line breaks and adjusting alignments. A significant change was also made to the MySQL server version in the `DbContextOptionsBuilder.cs` file. These adjustments help improve the readability and maintainability of the code, making it easier to understand and modify in future iterations.

* Remove unnecessary notification files and refactor code

Several files pertaining to workflow serialization notifications were deemed unnecessary and removed. The Endpoint class in the `Elsa.Workflows.Api` module has been refactored for efficient usage of the `IWorkflowDefinitionStore` store, and similarly the `HttpFeature.cs` and `DesignTimeDbContextFactoryBase.cs` files have been slightly refactored for better readability and optimization.

* Refactor HttpFeature class on variable types order

The variable types order within the HttpFeature class in the Elsa Http module has been reorganized for cleaner code. No functional changes have been made - only the order of the types has been rearranged. This refactoring contributes to better readability and code maintenance.

* Add 'issue/*' branch to GitHub workflow triggers

This update includes 'issue/*' branches in the trigger configuration of GitHub workflows. Now, these branches will trigger specified GitHub actions, facilitating more effective issue tracking and resolution.

* Change version option in WorkflowDefinitionManager

The version option in the WorkflowDefinitionManager has been updated. Previously it was set to "LatestAndPublished", but this has been changed to "LatestOrPublished", adjusting the way the manager handles workflow versions.

* Update GitHub Actions to remove duplicate branch pattern

Removed the duplicated 'issue/*' entry from the branch trigger list in packages.yml to prevent redundancy in GitHub Actions workflow triggers. This ensures workflows are triggered correctly and efficiently for branch patterns intended for CI processes.

* Refactor job generation in GenerateAlterationJobs

Refactored the GenerateAlterationJobs activity to enhance code readability and maintainability. Broke down the ExecuteAsync method into smaller, purpose-specific private methods and added the missing `Elsa.Alterations.Core.Models` namespace.

* Refactor consumer concatenation logic in MassTransitFeature

Removed the commented-out code that filtered temporary queues, simplifying the concatenation of consumer types with workflow message consumers.

* Refactor whitespace and improve readability

Removed extraneous whitespaces and reformatted the `IsEmpty` property for better code readability. This enhances the code structure without affecting functionality.

* Refactor MassTransitFeature configuration

Moved transport configuration logic into the dedicated `ConfigureInMemoryTransport` method and removed unused usings to simplify the `MassTransitFeature` class setup. This streamlines setting up MassTransit in-memory transport and improves code maintainability.

---------

Co-authored-by: Raymond den Haan <raymond.den.haan@nexxbiz.io>
Co-authored-by: Marius Vasile Vușcan <marius.vuscan@nexxbiz.io>
2024-03-13 20:34:46 +01:00
Sipke Schoorstra 13304cdb64
Add Properties field to ActivityExecutionRecord and Regenerate V3_1 migrations (#5039) 2024-03-08 08:54:08 +01:00
Raymond den Haan f208c446b4 Added migrations 2024-02-14 09:23:52 +01:00
Raymond den Haan 14993d466f Merge branch 'main' into feature/service-bus-pub-sub 2024-02-12 15:59:46 +01:00
Marko Lahma bc3d6d9bbc Convert to use Central Package Management
* Add GitHubActionsTestLogger
2024-02-04 13:49:54 +02:00
Sipke Schoorstra 0bc46d3f74 Remove migration files for Management module
Deleted all migration files related to the Management module across MySql, SqlServer, Sqlite and PostgreSql data providers. These files included data compression algorithm and data format column additions in the WorkflowInstances table.
2024-02-04 01:24:03 +01:00
Sipke Schoorstra 383eeab751 Add data compression and format fields to WorkflowInstances
This commit introduces two new fields to the WorkflowInstances table: "DataCompressionAlgorithm" and "DataFormat". These changes allow storing additional contextual information about the payloads of workflow instances and enhance future handling and processing of this data.
2024-02-04 00:06:41 +01:00
Sipke Schoorstra 22f72080f3 Add V3_1 migrations for various Elsa DB contexts
This commit includes the creation of migration files for MySql, SqlServer, Sqlite, and PostgreSql contexts in the Elsa project. The files for version V3_1 were automatically generated and are to be implemented with the necessary changes in the Up and Down methods as per the requirements.
2024-02-03 22:39:47 +01:00
Raymond den Haan f1ddea1c95 Add key value store 2024-02-02 11:21:29 +01:00
Marko Lahma 68beb5c61f replace configureawait.props and frameworks.props with Directory.Build.props 2024-01-26 19:04:39 +02:00
Marko Lahma 2f6ecb0b5d Use Directory.Build.props 2024-01-26 18:43:50 +02:00
Sipke Schoorstra bec8cbc322
Target .NET 8 (#4660) 2023-11-27 11:26:16 +01:00
Sipke Schoorstra eb93b3406f Update EF Core to 7.0.13 2023-10-26 15:17:17 +02:00
Sipke Schoorstra 02ec4c2391
Include NodeId for unique addressability within graph (#4575)
* Leverage ActivityNode for graph querying

* Make workflow definition activity Root port non-browsable

* Update Join activity

* Make NodeId first-class citizen
2023-10-25 22:46:06 +02:00
Sipke Schoorstra 8c76360f5d Updated migrations 2023-10-15 14:24:33 +02:00
Sipke Schoorstra 523f301bf7
Feature: Alterations (#4501)
* Implement Alteration types and engine

* Implement Alteration APIs

* Refactor activity work item with support for activity input

* Serialize scheduled activities as part of workflow state

* Add signal to schedule child activity

* Implement stores for plans and jobs

* Add EF Core provider

* Add Retry endpoint

* Keep root context as always active. The root context is the workflow execution context and contains persistent variable state

* Regenerate EF Core migrations

* Handle orphaned activity execution contexts

* Refactor workflow execution factory

* Enable configuring Jint from Program

* Update ModifyVariable alteration with support for type deserialization

* Use ShortGuid for identity

* Cleanup WorkflowServer host
2023-10-07 12:47:36 +02:00
Sipke Schoorstra bb2fa55d0e
WriteHttpFileResponse Activity (#4445)
* Incremental work on WriteFileHttpResponse activity

* BREAKING Schema change - Increase bookmark flexibility

* Implement zipped file response
2023-09-16 11:06:56 +02:00
Sipke Schoorstra 03b608410b
Incidents (#4436)
* Implement Incident Strategy interface and resolver

* Add ActivityIncident model

* Add IncidentCount and Incidents properties and update migrations

* Expose incident strategies API endpoint

* Update API client with Incident Strategy models

* Replace Fault with Incidents

* Register fault status when any children have faulted

* Map incident count

* Remove unnecessary AlreadyCompleted result

* Ensure next activities are scheduled only if activity completed normally

* Incident roundtrip and fix workflow instance realtime updates

* Remove unused variable

* Add integration tests
2023-09-14 10:50:02 +02:00
Sipke Schoorstra 6329a3b805
Use mapping expression for correct EF core projections (#4376) 2023-08-30 18:23:59 +02:00
Sipke Schoorstra 45d194125d
Remove WorkflowStateStore (#4374)
* Remove WorkflowStateStore and use WorkflowInstanceStore instead

* Delete RunningWorkflows actor

* Cleanup
2023-08-30 15:42:16 +02:00
Sipke Schoorstra e2c3ab0efd
Refactor proto files and fix snapshot loading of running workflows grain (#4371) 2023-08-29 23:29:31 +02:00
Sipke Schoorstra a02bacbbde
Support serialization of non(de)serializable values (#4356) 2023-08-23 20:56:12 +02:00
Sipke Schoorstra 73482a8e4c
Enhanced Logging for Activity Outcomes and Outputs (#4352)
* Add result to logs

* Include outcomes and outputs with activity execution log

* Include outputs with completed activity events
2023-08-23 13:34:30 +02:00
Sipke Schoorstra 10533131e3 Add fault information to journal 2023-08-21 22:45:56 +02:00
Sipke Schoorstra 0413cbebf6 Fix duplicate workflow inbox message effect 2023-08-21 21:17:40 +02:00
Sipke Schoorstra 95cefe631f
Bookmarks & Activity Instances (#4301)
* Add ActivityInstanceId to bookmark model

* Fix Telnyx activity resumption

* Fix missing activity instance ID forwarding to workflow inbox
2023-08-09 13:02:20 +02:00
Sipke Schoorstra c9ca4a121d
Workflow Inbox (#4292)
* Initial implementation of workflow inbox, submission and delivery

* Implement persistence providers

* Update EF Core migrations

* Rename Services to Stores in Dapper module

* Use default serializer for primitive arrays

* Update MongoDB provider to map BookmarkId to be primary key

* Implement MongoDB provider for activity execution logs

* Use EF Core by default

* Fix private JSON constructor logic
2023-08-06 17:03:36 +02:00
Sipke Schoorstra a2a2b60d05 Update ActivityExecutionRecord model 2023-07-23 21:05:31 +02:00
Sipke Schoorstra e5e34c821b Add ActivityExecutionRecord entity, persistence and endpoints 2023-07-22 21:05:33 +02:00
Sipke Schoorstra d473f2354b Add ActivityName to execution log, remove FaultedAt instance timestamp 2023-07-21 17:52:52 +02:00
Sipke Schoorstra 72f4461ca3 Implement cascading deletion 2023-07-20 00:05:43 +02:00
Sipke Schoorstra 6ae59f06d6 Include activity version with execution log 2023-07-18 11:20:00 +02:00
Sipke Schoorstra 8c701aa0a7 Implement activity execution log record sequence ordering
This ensures events are always stored in sequence, even if their timestamps are the same.
2023-07-09 12:36:33 +02:00
Sipke Schoorstra 0eea344406 Add ToolVersion field to workflow definition 2023-07-06 15:48:28 +02:00
Sipke Schoorstra a1ce26a35c Explicitly ignore WorkflowState type from EF Core 2023-06-28 23:24:30 +02:00
Sipke Schoorstra 3fccf95413 Rename designtime class 2023-06-28 22:46:45 +02:00
gurkanguran bc2b4e8fde
Mongodb implementation (#4127)
* Implemented MongoDb

* Use main entity models in mongodb

* Fixed object serialization issue

* Fixed issues caused by duplicate collection names

* Optimized count query

* Fixed the issues after merge

* Revert "Fixed the issues after merge"

This reverts commit 5568af60cfc107b8ee68fd4c89fbb02946f8a6a3.

* Use expression helper to solve mongodb query issue

* Handle null values during deserialization

* Moved MongoDb project to persistence folder

* Did some refactor and added type serializer for variable fix

* Modularize mongodb indices creation

* Fixed the build

* Implement variable serializer

* Update SQLite migrations

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-06-28 22:44:51 +02:00
herblinernexxbizz 8c308cc8ee
V3 make workflowdefinition have readonly (#4068)
* add readonly functionality to Elsa

* merge with v3

* remove launch settings

* remove spaces

* Handle merge conflicts

* Exclude non-existent folder from sample project

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-06-12 10:41:38 +02:00
Sipke Schoorstra 6c1cca372c
Dapper persistence provider (#4109)
* Implement Dapper stores
2023-06-06 20:49:14 +02:00
Sipke Schoorstra 0bb2ddcfee
Support sub-workflow dependency handling of JSON workflows (#4098)
* Change "Usable as Activity" setting into checkbox

* Update client model

* Update query to use shadow property

* Update UsableAsActivity usages

* Implement staged provisioning of workflows

* Add tests

* Update migrations
2023-06-03 12:28:10 +02:00
Akash Jadhav cd187f93c1
Schema Name editable in V3 (#4072)
* add ElsaDbContextOptions

* set Schema from elsaDbContextOptions

* file rename

* add options as arg to DbContextOptionsBuilderExtensions

* add options to module extensions in Elsa.EntityFrameworkCore.MySql

* add options to module extensions in Elsa.EntityFrameworkCore.PostgreSql

* add options to module extensions in Elsa.EntityFrameworkCore.Sqlite

* add options to module extensions in Elsa.EntityFrameworkCore.SqlServer

* add validations

* update doc strings
2023-05-28 11:56:49 +02:00
Sipke Schoorstra c8d87cd763
Simplify Proto.Actor bookmarks and triggers (#3917)
* Refactor HTTP route table population

* Update workflow definition EF store to use abstract IWorkflowInstanceStore

This decoupling is necessary, because the application might not be using EF Core, but e.g. Elasticsearch

* Move workflow state persistence to the default workflow runtime feature

* Update JS handler to provide access to variables during trigger indexing

* Configure Proto.Actor runtime with different workflow pipeline

* Add sample using Proto.Actor runtime

* Replace BookmarkGrain with DB based persistence
2023-04-17 15:23:05 +02:00
Sipke Schoorstra c3ed7e7865
Improve JSON serializationfor workflow state (#3913)
* Incremental work on WorkflowStateSerializer

* Update Flow HTTP Request activity to default to Done outcome

* Fix reference handling

* Fix polymorphic object converter

* Add comments

* Add support for registering type alias during DI setup via options

* Fix trigger and bookmark payload persistence

* Update sample model

* Add support for custom dictionary types

* Split general serializer options provider into focused serializer services

* Formatting

* Update migrations
2023-04-17 00:11:58 +02:00
Sipke Schoorstra 30c4e5b1e7
Implement API key authentication (#3834)
* Implement API key storage

* Implement API key validation

* Implement EF Core providers
2023-03-26 23:53:17 +02:00