Commit graph

41 commits

Author SHA1 Message Date
raymonddenhaan 4b151ff8af
Feature/cancel dispatched workflows (#5136)
* Made cancel API call dispatch the request

* Added ParentWorkflowInstanceId

* Recursively cancel child workflows

* Made single workflow cancellation include child workflows

* Updated migrations

* Processed comments
2024-03-27 12:07:55 +01:00
Sipke Schoorstra 83dd776233 Add new docker-compose files, new activities and services
Create new docker-compose files for non-data dog and data dog services. New services include Postgres, cockroachdb, rabbitmq, redis, and elsa-server. Along with the services, a new bash script for migration and various new activities and endpoints are also added for the Elsa server. This set of changes broadens the operational environment of Elsa.
2024-03-24 20:28:19 +01:00
Sipke Schoorstra 20d936d3c2
Alterations with Queries (#5032)
* Remove Elsa.Alteration Handlers and implement workflow alteration activities

The Elsa.Alterations.Handlers, Services, and Endpoints have been removed and replaced with Elsa.Alterations.Activities for various steps in executing an alteration plan. These activities include submitting an alteration plan, generating alteration jobs, and dispatching alteration jobs. New models for ActivityFilter, AlterationWorkflowInstanceFilter, and alteration activity summaries are also introduced for better management of alteration plans.

* Add 'PrintMessage' custom activity and update naming rules

A new 'PrintMessage' activity has been added to the custom activities of the ConsoleApp, which simply writes a 'Hello world!' message. On top of this, the naming rules in the code styling settings have been expanded, introducing separate rules for instance and static private fields.

* Remove V3_1 database migration files

The V3_1 database migration files were deleted across all database providers, including MySql, SQL Server, SQLite and PostgreSQL. This is indicative of a rollback or significant revision to the data migration strategy.

* Update product version and modify tables in Elsa DB context

Updated "Elsa" DbSet product version from "7.0.11" to "7.0.14". Additionally, added new properties as well as modified existing ones in tables like "Elsa.KeyValues.Entities.SerializedKeyValuePair" to improve data structure. Also properly ignored certain entities in the Alterations DbContext.

* Add V3_1 migration for Elsa

This commit introduces the V3_1 migration for the Elsa module. Changes involve the introduction of new tables and alteration of existing tables. Specifically, a new 'KeyValuePairs' table was added and several columns on various tables were renamed or altered.

* Remove multiple Elsa.EfCore V3_1 migration files

The commit removes several migration files related to the V3_1 migration in multiple Elsa.EntityFrameworkCore modules including SqlServer, PostgreSQL, and SQLite. These files encompass changes in table structures and alterations.

* Update default value for ArgumentDefinition Type

The default value for the `Type` property in the `ArgumentDefinition` class was updated to `typeof(object)`. The `Type` property of `inputDefinition` in the `WorkflowBuilder` class was also modified to be set as `typeof(T)`. These modifications enable more flexible type handling.

* Refactor SafeSerializer and serialization services

The SafeSerializer service and related serialization services have been comprehensively refactored. This includes the removal of internal notifications, the incorporation of [RequiresUnreferencedCode] attributes, and changes in method signatures. Methods have been simplified to return ValueTasks directly from the JsonSerializer, and the notification sender dependency was removed.

* Add JSON serialization for alterations

Implemented a JSON serializer and a custom converter for the IAlteration interface in the Elsa.Alterations.Core.Serialization namespace. Also, introduced a new IJsonSerializer interface and an implementation for standard JSON serialization in Elsa.Common. These serialization changes enhance modularity and extensibility of JSON operations, and improve handling of IAlteration objects.

* Add "IsSystem" field to Workflow Definitions and Instances

This update introduces a new boolean field "IsSystem" to the Workflow Definition and Workflow Instance models. This field is designed to identify system workflows which aren't meant to be modified by users. Management and Runtime module code, API endpoints, database migrations, and related classes have been updated to include this new field. The MongoDB indices and Dapper migrations have been updated as well, and serialization/deserialization processes for these types have been modified to handle this new attribute.

* Remove outdated database migrations

Deleted database migration files from the MySql, SqlServer, and PostgreSql projects.

* Add V3_1 migration for Elsa.EntityFrameworkCore.Sqlite

This commit introduces the V3_1 migration for the Elsa.EntityFrameworkCore.Sqlite module. It includes changes in Alterations, Runtime, Management, and Identity contexts. These changes involve table alterations, addition of new columns, and creation of new tables. Each context alteration encapsulates relevant changes for specific functionalities of the module.

* Add `IsSystem` property to workflow classes

The `IsSystem` property has been added to several sections of the codebase related to workflows, such as `WorkflowStateMapper.cs`, `ListWorkflowDefinitionsRequest.cs`, `WorkflowDefinitionStore.cs`, and `ListWorkflowInstancesRequest.cs`. This property flag allows us to include or filter system workflows in our queries and results.

* Add 'UsedImplicitly' attribute to Alteration Handlers and Types

The 'UsedImplicitly' attribute from JetBrains.Annotations has been added to all Alteration Handlers and Alteration Types in the Elsa.Alterations module. This change specifies that these classes are being used implicitly and helps tools like ReSharper prevent false unused code detection.

* Update migration script for efcore-3.1

The migration script efcore-3.1.sh has been updated. Module "Alterations" and provider "Sqlite" are added back into the lists while unused elements are removed for cleanup. The script now covers all necessary modules and providers, improving its overall effectiveness.

* Remove PrintMessage class from Elsa.Server.Web bundle

This commit deletes the PrintMessage.cs file that was previously part of the Elsa.Server.Web bundle.

* Refactor MassTransit service configuration

This commit reduces line code length by merging splitted long lines into one, improves maintainability of code. It also updates the Service configuration and removes unused namespaces across files, which help to simplify the codebase. Changes have also been made to switch the MassTransit broker from RabbitMq to memory and adjust the endpoint name formatter configuration.

* Remove extra whitespace in Program.cs

Removed a trailing whitespace on the line declaring the useSqlServer constant in the Program.cs file. No functionality changes were made, simply a minor formatting correction to maintain clean and consistent code.

* Remove AlterationJsonConverter and update other files

The AlterationJsonConverter class has been removed, simplifying the serialization process. The AlterationSerializer.cs file was updated with additional attribute tags, giving more information about serialization expectations. Lastly, the AlterationSerializationOptionConfigurator.cs was simplified by removing unnecessary overrides and the Elsa.Alterations.Core.csproj file was tidied up by removing unused services folder reference.

* Refactor code for improved readability and maintainability

The diff shows changes across multiple files in multiple modules. Primarily, these changes revolve around formatting edits such as removing unnecessary spaces or line breaks and adjusting alignments. A significant change was also made to the MySQL server version in the `DbContextOptionsBuilder.cs` file. These adjustments help improve the readability and maintainability of the code, making it easier to understand and modify in future iterations.

* Remove unnecessary notification files and refactor code

Several files pertaining to workflow serialization notifications were deemed unnecessary and removed. The Endpoint class in the `Elsa.Workflows.Api` module has been refactored for efficient usage of the `IWorkflowDefinitionStore` store, and similarly the `HttpFeature.cs` and `DesignTimeDbContextFactoryBase.cs` files have been slightly refactored for better readability and optimization.

* Refactor HttpFeature class on variable types order

The variable types order within the HttpFeature class in the Elsa Http module has been reorganized for cleaner code. No functional changes have been made - only the order of the types has been rearranged. This refactoring contributes to better readability and code maintenance.

* Add 'issue/*' branch to GitHub workflow triggers

This update includes 'issue/*' branches in the trigger configuration of GitHub workflows. Now, these branches will trigger specified GitHub actions, facilitating more effective issue tracking and resolution.

* Change version option in WorkflowDefinitionManager

The version option in the WorkflowDefinitionManager has been updated. Previously it was set to "LatestAndPublished", but this has been changed to "LatestOrPublished", adjusting the way the manager handles workflow versions.

* Update GitHub Actions to remove duplicate branch pattern

Removed the duplicated 'issue/*' entry from the branch trigger list in packages.yml to prevent redundancy in GitHub Actions workflow triggers. This ensures workflows are triggered correctly and efficiently for branch patterns intended for CI processes.

* Refactor job generation in GenerateAlterationJobs

Refactored the GenerateAlterationJobs activity to enhance code readability and maintainability. Broke down the ExecuteAsync method into smaller, purpose-specific private methods and added the missing `Elsa.Alterations.Core.Models` namespace.

* Refactor consumer concatenation logic in MassTransitFeature

Removed the commented-out code that filtered temporary queues, simplifying the concatenation of consumer types with workflow message consumers.

* Refactor whitespace and improve readability

Removed extraneous whitespaces and reformatted the `IsEmpty` property for better code readability. This enhances the code structure without affecting functionality.

* Refactor MassTransitFeature configuration

Moved transport configuration logic into the dedicated `ConfigureInMemoryTransport` method and removed unused usings to simplify the `MassTransitFeature` class setup. This streamlines setting up MassTransit in-memory transport and improves code maintainability.

---------

Co-authored-by: Raymond den Haan <raymond.den.haan@nexxbiz.io>
Co-authored-by: Marius Vasile Vușcan <marius.vuscan@nexxbiz.io>
2024-03-13 20:34:46 +01:00
Sipke Schoorstra 13304cdb64
Add Properties field to ActivityExecutionRecord and Regenerate V3_1 migrations (#5039) 2024-03-08 08:54:08 +01:00
Raymond den Haan f208c446b4 Added migrations 2024-02-14 09:23:52 +01:00
Raymond den Haan 14993d466f Merge branch 'main' into feature/service-bus-pub-sub 2024-02-12 15:59:46 +01:00
Sipke Schoorstra 0bc46d3f74 Remove migration files for Management module
Deleted all migration files related to the Management module across MySql, SqlServer, Sqlite and PostgreSql data providers. These files included data compression algorithm and data format column additions in the WorkflowInstances table.
2024-02-04 01:24:03 +01:00
Sipke Schoorstra 383eeab751 Add data compression and format fields to WorkflowInstances
This commit introduces two new fields to the WorkflowInstances table: "DataCompressionAlgorithm" and "DataFormat". These changes allow storing additional contextual information about the payloads of workflow instances and enhance future handling and processing of this data.
2024-02-04 00:06:41 +01:00
Sipke Schoorstra 22f72080f3 Add V3_1 migrations for various Elsa DB contexts
This commit includes the creation of migration files for MySql, SqlServer, Sqlite, and PostgreSql contexts in the Elsa project. The files for version V3_1 were automatically generated and are to be implemented with the necessary changes in the Up and Down methods as per the requirements.
2024-02-03 22:39:47 +01:00
Raymond den Haan f1ddea1c95 Add key value store 2024-02-02 11:21:29 +01:00
Sipke Schoorstra 02ec4c2391
Include NodeId for unique addressability within graph (#4575)
* Leverage ActivityNode for graph querying

* Make workflow definition activity Root port non-browsable

* Update Join activity

* Make NodeId first-class citizen
2023-10-25 22:46:06 +02:00
Sipke Schoorstra 8c76360f5d Updated migrations 2023-10-15 14:24:33 +02:00
Sipke Schoorstra 523f301bf7
Feature: Alterations (#4501)
* Implement Alteration types and engine

* Implement Alteration APIs

* Refactor activity work item with support for activity input

* Serialize scheduled activities as part of workflow state

* Add signal to schedule child activity

* Implement stores for plans and jobs

* Add EF Core provider

* Add Retry endpoint

* Keep root context as always active. The root context is the workflow execution context and contains persistent variable state

* Regenerate EF Core migrations

* Handle orphaned activity execution contexts

* Refactor workflow execution factory

* Enable configuring Jint from Program

* Update ModifyVariable alteration with support for type deserialization

* Use ShortGuid for identity

* Cleanup WorkflowServer host
2023-10-07 12:47:36 +02:00
Sipke Schoorstra bb2fa55d0e
WriteHttpFileResponse Activity (#4445)
* Incremental work on WriteFileHttpResponse activity

* BREAKING Schema change - Increase bookmark flexibility

* Implement zipped file response
2023-09-16 11:06:56 +02:00
Sipke Schoorstra 03b608410b
Incidents (#4436)
* Implement Incident Strategy interface and resolver

* Add ActivityIncident model

* Add IncidentCount and Incidents properties and update migrations

* Expose incident strategies API endpoint

* Update API client with Incident Strategy models

* Replace Fault with Incidents

* Register fault status when any children have faulted

* Map incident count

* Remove unnecessary AlreadyCompleted result

* Ensure next activities are scheduled only if activity completed normally

* Incident roundtrip and fix workflow instance realtime updates

* Remove unused variable

* Add integration tests
2023-09-14 10:50:02 +02:00
Sipke Schoorstra 45d194125d
Remove WorkflowStateStore (#4374)
* Remove WorkflowStateStore and use WorkflowInstanceStore instead

* Delete RunningWorkflows actor

* Cleanup
2023-08-30 15:42:16 +02:00
Sipke Schoorstra 73482a8e4c
Enhanced Logging for Activity Outcomes and Outputs (#4352)
* Add result to logs

* Include outcomes and outputs with activity execution log

* Include outputs with completed activity events
2023-08-23 13:34:30 +02:00
Sipke Schoorstra 10533131e3 Add fault information to journal 2023-08-21 22:45:56 +02:00
Sipke Schoorstra 0413cbebf6 Fix duplicate workflow inbox message effect 2023-08-21 21:17:40 +02:00
Sipke Schoorstra 95cefe631f
Bookmarks & Activity Instances (#4301)
* Add ActivityInstanceId to bookmark model

* Fix Telnyx activity resumption

* Fix missing activity instance ID forwarding to workflow inbox
2023-08-09 13:02:20 +02:00
Sipke Schoorstra c9ca4a121d
Workflow Inbox (#4292)
* Initial implementation of workflow inbox, submission and delivery

* Implement persistence providers

* Update EF Core migrations

* Rename Services to Stores in Dapper module

* Use default serializer for primitive arrays

* Update MongoDB provider to map BookmarkId to be primary key

* Implement MongoDB provider for activity execution logs

* Use EF Core by default

* Fix private JSON constructor logic
2023-08-06 17:03:36 +02:00
Sipke Schoorstra a2a2b60d05 Update ActivityExecutionRecord model 2023-07-23 21:05:31 +02:00
Sipke Schoorstra e5e34c821b Add ActivityExecutionRecord entity, persistence and endpoints 2023-07-22 21:05:33 +02:00
Sipke Schoorstra d473f2354b Add ActivityName to execution log, remove FaultedAt instance timestamp 2023-07-21 17:52:52 +02:00
Sipke Schoorstra 72f4461ca3 Implement cascading deletion 2023-07-20 00:05:43 +02:00
Sipke Schoorstra 6ae59f06d6 Include activity version with execution log 2023-07-18 11:20:00 +02:00
Sipke Schoorstra 8c701aa0a7 Implement activity execution log record sequence ordering
This ensures events are always stored in sequence, even if their timestamps are the same.
2023-07-09 12:36:33 +02:00
Sipke Schoorstra 0eea344406 Add ToolVersion field to workflow definition 2023-07-06 15:48:28 +02:00
Sipke Schoorstra a1ce26a35c Explicitly ignore WorkflowState type from EF Core 2023-06-28 23:24:30 +02:00
gurkanguran bc2b4e8fde
Mongodb implementation (#4127)
* Implemented MongoDb

* Use main entity models in mongodb

* Fixed object serialization issue

* Fixed issues caused by duplicate collection names

* Optimized count query

* Fixed the issues after merge

* Revert "Fixed the issues after merge"

This reverts commit 5568af60cfc107b8ee68fd4c89fbb02946f8a6a3.

* Use expression helper to solve mongodb query issue

* Handle null values during deserialization

* Moved MongoDb project to persistence folder

* Did some refactor and added type serializer for variable fix

* Modularize mongodb indices creation

* Fixed the build

* Implement variable serializer

* Update SQLite migrations

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-06-28 22:44:51 +02:00
herblinernexxbizz 8c308cc8ee
V3 make workflowdefinition have readonly (#4068)
* add readonly functionality to Elsa

* merge with v3

* remove launch settings

* remove spaces

* Handle merge conflicts

* Exclude non-existent folder from sample project

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-06-12 10:41:38 +02:00
Sipke Schoorstra 0bb2ddcfee
Support sub-workflow dependency handling of JSON workflows (#4098)
* Change "Usable as Activity" setting into checkbox

* Update client model

* Update query to use shadow property

* Update UsableAsActivity usages

* Implement staged provisioning of workflows

* Add tests

* Update migrations
2023-06-03 12:28:10 +02:00
Sipke Schoorstra c3ed7e7865
Improve JSON serializationfor workflow state (#3913)
* Incremental work on WorkflowStateSerializer

* Update Flow HTTP Request activity to default to Done outcome

* Fix reference handling

* Fix polymorphic object converter

* Add comments

* Add support for registering type alias during DI setup via options

* Fix trigger and bookmark payload persistence

* Update sample model

* Add support for custom dictionary types

* Split general serializer options provider into focused serializer services

* Formatting

* Update migrations
2023-04-17 00:11:58 +02:00
Sipke Schoorstra 30c4e5b1e7
Implement API key authentication (#3834)
* Implement API key storage

* Implement API key validation

* Implement EF Core providers
2023-03-26 23:53:17 +02:00
Sipke Schoorstra fad27d0fb0
Refactor composite activity input and scoped memory (#3815)
* Incremental work

* Enable control over when to include full compositie root

* Cleanup

* Update WorkflowDefinitionActivity

* Implement input as variables

* Delete unused class

* Remove less common types

* Rename ExpandoObject to JSON

* Implement variable scoping

* Remove parent/child relationship from MemoryRegister

* Add support for sending input to "trigger event" endpoint

* Add support for configuring input storage driver

* Remove unused namespace imports

* Auto-register activity types when running a workflow

* Fix missing variables from containers
2023-03-20 19:55:11 +01:00
Sipke Schoorstra c1dca8fbe2
Fixes for Journal (#3792)
* Include NodeID in execution log and update workflow journal component

* Recreate Runtime migrations
2023-03-14 08:03:44 +01:00
Sipke Schoorstra 1ac64b8197
Store & display evaluated activity state in workflow instance viewer (#3776)
* Implement activity state persistence

* Add sort API

* Implement activity state view

* Update EF Core migrations
2023-03-07 21:40:53 +01:00
gurkanguran 407bfa6913
Use workflow definition as activity (#3654)
* Implemented usage of workflow definitions as activities

* Updated designer

* Removed activity definitions module

* Allow user to mark workflow to be used as activity

* Refactor

* Refactor

* Cleanup csproj

* Cleanup WorkflowDefinitionActivity

* Pascalize typename

* Fix JS intellisense initial load issue

* Add XML comments

* Add Jobs dependency

* Hide Composite root port from designer

* Move finding workflow activities logic to lower level

* Generate unique identities across hierarchy

* Refactor

* Revert "Generate unique identities across hierarchy"

This reverts commit 74f5e7830309ee76af6ab3a000a5e1ec41f1e90c.

* Implement unique node IDs

* Introduce new VersionOptions option

* Remove unused namespaces

---------

Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2023-01-28 00:36:03 +01:00
Sipke Schoorstra 006479c2b7
Finish Azure Service Bus activities (#3617)
* Incremental work on Azure Service Bus

* Update Azure Service Bus activities
2023-01-12 11:59:31 +01:00
Sipke Schoorstra 7a98cdf60d
Normalize extensions namespaces (#3577) 2022-12-31 16:16:43 +01:00
Sipke Schoorstra e6536636d7
Additional samples + fixes and improvements (#3574)
* Incremental work on samples

* Rename HTTP middleware and update samples

* Rename HTTP Activities to Workflows

* Automatically register core activities when adding Elsa

* Fix feature registration bug

* Add default auth feature

* Make activity serializer more flexible

* Bump designer version

* Add workflow server + designer samples

* Simplify EntityFrameworkCore projects

* Update solution

* Add Sqlite package references to ensure the correct version is used

* Update sample project with EF Core

* Update EF extensions

* Add server + designer sample project

* Cleanup AllInOne project and add some XML comments

* Rename ci.yml and add CI for Docker file
2022-12-30 13:11:29 +01:00