Commit graph

340 commits

Author SHA1 Message Date
Sipke Schoorstra 279d55c22c
Merge pull request #6276 from LarsNijholt/fix/no-ref/improve-workflow-instance-definition-filtering
Implemented name filter to workflow instance filter and added case insensitivity to searching definitions and instances by name
2025-01-10 18:39:58 +01:00
Sipke Schoorstra d759624321 Refactor array type handling in variable and JSON converters
Updated the logic to use `MakeArrayType` for array handling instead of generic collection types, ensuring consistency and better alignment with expected type structures. Adjustments were made in both the `VariableDefinitionMapper` and the `TypeJsonConverter`.
2025-01-10 11:40:54 +01:00
Sipke Schoorstra d6c0ba45df Use sliding expiration for cache entries.
Replaced absolute expiration with sliding expiration for cache entries to ensure the cache is kept active during frequent access. This change optimizes cache behavior and improves resource utilization.
2025-01-10 10:08:25 +01:00
Sipke Schoorstra 288097d503 Simplify and optimize workflow instance creation.
Replaced asynchronous instance creation methods with streamlined synchronous alternatives where applicable. Introduced `CreateAndCommitWorkflowInstanceAsync` for combined instantiation and persistence, along with a separate `CreateWorkflowInstance` method for non-committal instantiation. Refactored related code to improve readability, maintainability, and runtime performance.
2025-01-09 19:34:53 +01:00
Lars Nijholt 05f8bc640b Add case insensitivity to the search term of the WorkflowInstanceFilter 2025-01-07 16:00:20 +01:00
Lars Nijholt 788e6f7a0f Add case insensitivity to the name and description of the workflow definition filter 2025-01-07 14:53:21 +01:00
Lars Nijholt e48010e3f9 Use ToLower method for comparison 2025-01-07 14:46:03 +01:00
Lars Nijholt 1c9328597f Add workflow name property to workflow instance filter.
Add the name property to the workflow instance filter to make use of the name property that can be used in the Request class for the list endpoint. This allows us to filter purely on workflow instance name without using the SearchTerm
2025-01-07 14:34:14 +01:00
Sipke Schoorstra 506a36574b Update target framework to .NET 9 and improve null handling
Updated the target framework from .NET 8 to .NET 9 for future compatibility and language improvements. Adjusted null handling in `WorkflowDefinitionImporter` for better readability and correctness. Added a cancellation token to `ReadToEndAsync` for improved async operation control.
2025-01-06 15:35:06 +01:00
Sipke Schoorstra 62df21cbaf Make StringData property nullable in WorkflowDefinition
Updated the `StringData` property to allow null values, improving flexibility and aligning with usage scenarios where the property may not always contain a value. This change ensures better handling of optional data.
2024-12-26 11:52:36 +01:00
Sipke Schoorstra 0992404d96 Add integration test for workflow serialization
Introduce a new integration test to verify workflow serialization functionality, ensuring that newly created workflow definitions can be serialized and deserialized correctly. Adjust default nullability for certain fields in `WorkflowDefinition` to improve consistency and prevent null reference issues. Minor argument update in `New` method of `WorkflowDefinitionPublisher`.
2024-12-26 11:17:54 +01:00
Marius Vasile Vușcan 95e3b15106
Updated to allow string tags 2024-12-11 16:45:23 +02:00
Marius Vasile Vușcan 16122182e2
Introduce the possibility to exclude variable value retrieval for certain storage drivers 2024-12-11 16:45:18 +02:00
Sipke Schoorstra 692937d7d7
Enhance Multitenancy with Runtime Tenant Management and Task Handling (#6173)
* Work in progress: Add DefaultTenantService for tenant management

Introduce `DefaultTenantService` and its corresponding interface `ITenantService` to manage tenant operations such as finding, getting, and listing tenants. Update `MultitenantBackgroundService` to utilize `DefaultTenantService` for handling tenant lifecycle events. This enhancement standardizes tenant operations and improves the maintainability of the multitenancy feature.

* WIP

* Add multitenancy event handlers and task interfaces

Implemented new interfaces IBackgroundTaskStarter and ITaskExecutor to manage task lifecycle events efficiently. Introduced new classes such as RunBackgroundTasks, RunStartupTasks, and StartRecurringTasks for handling tenant activation and deactivation events. Modified TaskExecutor to implement these interfaces and adjusted tenant registration logic to invoke these new handlers.

* Refactor multitenancy and task management services.

Remove background and recurring task runners, and integrate tenant activation and deactivation into the multitenancy feature. Enable multitenancy in the server application and create a new service for tenant activation and deactivation. This refactor simplifies the management of tenant-specific tasks and enhances the modularity of the platform.

* Refactor background service to use startup tasks

Replaced hosted service implementation with startup tasks for executing multi-tenant tasks and EF Core migrations. Introduced `PriorityAttribute` to manage task execution order, ensuring migrations run before other services that require database access. This simplifies tenant activation with an ordered task execution and removes redundant classes.

* Refactor MultitenancyFeature service registrations

Reorganized service registrations for better clarity and maintainability. Changed the registration of some services to use factory delegates for retrieving existing services to ensure correct dependencies. This refactor improves the flexibility of the tenant lifecycle event handling.

* Update V3_3 migration files

* Add tenant management endpoints and enhance tenant handling

Implemented tenant management endpoints including Add, Get, List, and Update. Enhanced tenant handling by introducing configuration and store-based providers, and improved error logging for tenant updates. Adjusted various internal functionalities to better support multitenancy features through different persistence providers.

* Implement tenant deletion endpoint and refactor migration setup.

Introduce a new API endpoint to handle tenant deletions while providing appropriate responses based on successful or unsuccessful attempts. Refactor migration handling by replacing startup tasks with hosted services across various modules to streamline the migration execution process.

* Add and integrate ConfigurationJsonConverter

Introduce a `ConfigurationJsonConverter` to handle JSON serialization and deserialization of `IConfiguration` objects. This change centralizes configuration serialization logic, leading to cleaner and more maintainable code. Updated various parts of the codebase to use the new serialization utility, ensuring a consistent approach throughout the application.

* Refactor JSON conversion and update tenant endpoint.

Removed unused workflow references and streamlined JSON handling in `ConfigurationJsonConverter`. Simplified tenant ID handling by removing `IIdentityGenerator` and setting a default value for `UpdatedTenant.Id`.

* Add logging for cancelled recurring tasks

Integrated ILogger to log information when a recurring task is canceled due to an OperationCanceledException. This change enhances troubleshooting by providing clearer insights into task cancellations and their underlying reasons, improving maintainability and observability of the task execution process.

* Disable multitenancy support and adjust default Tenant ID.

Multitenancy is now disabled by setting 'useMultitenancy' to false in the configuration. Additionally, the default Tenant's ID has been changed from null to an empty string to prevent potential null reference issues.

* Remove MultitenantHostedService abstraction file

The MultitenantHostedService.cs file was removed as it is no longer necessary. Its responsibilities have likely been refactored or integrated into another service, indicating a simplification or restructuring of the multitenancy handling in the codebase.

* Rename PriorityAttribute to OrderAttribute for clarity.

This change improves the clarity of the code by renaming PriorityAttribute to OrderAttribute, reflecting its actual purpose. All occurrences of the attribute in the codebase have been updated accordingly to maintain consistency. This makes the intent of the code more understandable for future maintenance and development.

* Fix message key retrieval in ProduceMessage activity

Update the ProduceMessage activity to use GetOrDefault for retrieving the message key. This change ensures that a null key is used if no explicit key is provided or if the key is empty or whitespace, preventing potential errors during message production.

* Refactor multitenancy and scheduling services.

Removed DefaultTenantContextInitializer interface and class, refactored tenant activation/deactivation to use try-catch logging, and updated tenant context handling to use IDisposable for context push. New activities and workflows added in Elsa.Server.Web, and scheduling services enhanced to schedule jobs with explicit job keys and groups. Also, adjusted configurations to enable multitenancy, providing improved maintainability and flexibility.

* Remove Example1 activities and disable multitenancy

Deleted Example1Activity, Example1Workflow, and FirstActivity classes to clean up unused code and simplify the codebase. Disabled multitenancy by setting useMultitenancy to false, likely to streamline configuration and resource utilization.

* Fix and normalize URL path concatenation.

Ensure that the base URLs in both base path providers consistently end with a forward slash. This normalization prevents potential issues with endpoint routing and path concatenation, improving overall URL construction robustness.
2024-12-03 15:07:55 +01:00
Sipke Schoorstra 08632471e3
Implement Dynamic Expression Evaluation for Log Persistence Mode (#6159)
* Add log persistence configuration and strategy evaluation

Introduced `LogPersistenceConfiguration` to support dynamic strategy and expression-based log persistence evaluations. Updated `Default default values.

* Add LogPersistenceConfiguration and enum support

Introduced LogPersistenceConfiguration class and LogPersistenceEvaluationMode enum in both Elsa.Api.Client and Elsa.Workflows.Runtime. Updated JavaScript services to handle enums correctly and register LogPersistenceMode.

* Rename log persistence key for consistency

Updated the log persistence key from `LogPersistenceStrategyKey` to `LogPersistenceConfigKey` to maintain consistency across the codebase. This change ensures that property access aligns with the updated naming conventions used in the application's configuration.

* Reuse JSON serializer options

* Update log persistence config structure in comments

This commit revises the JSON example in code comments to reflect the updated structure of the log persistence configuration. The changes include updated evaluation modes and strategy types for default, inputs, and outputs sections. These modifications aim to enhance clarity and provide accurate documentation of the expected configuration format.
2024-11-28 12:19:38 +01:00
Sipke Schoorstra 49a7f22543 Merge remote-tracking branch 'origin/blueberry' 2024-11-20 14:45:53 +01:00
Sipke Schoorstra 492b16bd24
Filter workflow summaries by latest version (#6132)
Ensure that only the latest version of each workflow definition is processed by grouping and selecting the highest version. This change helps prevent redundant updates and ensures data consistency.
2024-11-20 14:44:57 +01:00
Sipke Schoorstra c47157fd9f
Fix workflow definition finder (#6124)
Fixes that the workflow reference updater would collect child workflows from child workflows, instead of collecting only immediate children.
2024-11-18 19:05:36 +01:00
Sipke Schoorstra f53e024d25
Add Elsa.Kafka Module for Kafka Integration with Message Sending and Receiving Activities (#6108)
* Add Kafka module with integration and example setup

Introduced the Kafka module providing consumer integration and activities into the project. This includes new classes for consumer handling, configuration, and activities. An example setup using Docker Compose is also added to facilitate development and testing.

* Refactor KafkaTransportMessage to inline Timestamp namespace

Simplify the namespace usage for the Timestamp type within the KafkaTransportMessage record. This change eliminates the need for a separate using directive for Timestamp, enhancing code readability and maintainability.

* Add support for handling Kafka transport messages

This commit introduces the capability to handle and trigger workflows based on Kafka transport messages. It adds a new handler, notifications, and updates the message stimulus to include correlating fields. Additionally, the Kafka consumers are now managed more modularly with updated startup tasks and mediator integration.

* Enable Kafka integration and fix Kafka options naming

Added support for Kafka integration in Elsa.Server.Web by setting up Kafka configurations in appsettings.json and updating Program.cs. Also, renamed `ConsumerConfigs` to `ConsumerDefinitions` in Kafka options for clarity.

* Add consumer definition enumeration and dropdown support

Introduced `IConsumerDefinitionEnumerator` for managing consumer definitions across providers and implemented in `ConsumerDefinitionEnumerator` class. Enhanced `KafkaFeature` to register these services and updated the `MessageReceived` activity to use a dropdown UI hint for consuming definitions. Improved `StartConsumersTask` by refactoring consumer definition retrieval logic.

* Add SendMessage activity and refine Kafka messaging

Introduce a new SendMessage activity for Kafka, enabling message publishing to specific topics. Refine KafkaTransportMessage model by removing headers and timestamp fields. Adjust the StimulusSender logic to streamline the bookmark queuing process and fix key-value pairing in dropdown options. Update appsettings for corrected Kafka bootstrap server and topic configurations.

* Add producer and topic management support

Introduced interfaces and implementations for managing producer and topic definitions along with their respective enumerators and list providers. Updated `SendMessage` activity to include producer selection and refactored consumer definition providers for better consistency.

* Refactor Kafka configuration property names

Renamed Kafka configuration properties for better consistency and readability across the codebase. Updated property names from `ProducerDefinitions` to `Producers`, `ConsumerDefinitions` to `Consumers`, and `TopicDefinitions` to `Topics`. Added missing input attribute in `SendMessage.cs` and registered additional handlers in `KafkaFeature.cs`.

* Add custom serializers for Kafka message handling

Introduced `DefaultSerializers` class for custom serialization and deserialization of Kafka messages. Updated `MessageReceived` and `SendMessage` activities to use these custom serializers, and modified `KafkaOptions` to include them.

* Fix ExpandoObject serialization method parameter

Changed the serialization type from `ExpandoObject` to the actual type of the object to ensure proper serialization. This ensures that derived types are correctly handled during the serialization process.

* Add Producer and Consumer workflows for Kafka

Introduced two new workflows: `ProducerWorkflow` and `ConsumerWorkflow` for handling Kafka messages. Updated `DefaultSerializers` to use camelCase property naming and modified `appsettings.json` to include `topic-2` and format entries.

* Add JSON serialization to log output in ConsumerWorkflow

This change enhances the log output by serializing messages to JSON format before writing them. The addition of System.Text.Json ensures that the message content is presented in a structured and standardized format in logs.

* Add correlation strategies and update Kafka features

Implemented HeaderCorrelationStrategy and NullCorrelationStrategy, and updated KafkaFeature to support customizable correlation strategies. Added correlation ID handling to Kafka transport messages and updated config and handlers accordingly.

* Add tenant accessor to ConsumerDefinitionWorkflowContextProvider

Integrated ITenantAccessor to the provider to support tenant-specific context loading. Updated the constructor and LoadAsync method to retrieve the tenant information and use it for context-specific operations.

* Switch to MySQL and disable Kafka

This commit changes the SQL database provider from SQLite to MySQL and disables Kafka use. It also includes necessary adjustments such as adding MySQL handling in configuration and connection setups, updating `docker-compose` to include MySQL services, and referencing MySQL projects in the `.csproj` file.

* Add UI property handlers to multiple features

This commit introduces various UI property handlers across several features such as Python, JavaScript, CSharp, and Workflow features to enhance user interface property handling. It also updates the property UI handler resolution logic to better manage cases where providers are not available. Furthermore, adjustments were made in the server configuration to switch database providers and enable Kafka.

* Refactor property UI handler retrieval logic

Modified the logic to fetch property UI handlers by preloading them into a list and then filtering. This change improves readability and potentially performance by reducing repetitive service provider calls.

* Incremental work on Kafka workers and predicate evaluation

* Merge BookmarkInvoker with BookmarkResumer

* Register IWorkerManager

* Change lifetime scope of WorkerManager to Singleton

* **Introduce topic subscription handling for Kafka workers**

Added `IWorkerTopicSubscriber` interface and its implementation for managing topic subscriptions. Enhanced workers to bind triggers and bookmarks dynamically based on existing data. Updated several classes and methods to support topic-based subscriptions and headers.

* Refactor trigger matching logic.

Extract trigger matching conditions into `IsMatchAsync` method for reuse. This enhances code maintainability and readability by reducing redundancy. The new `GetTopic` helper method isolates the topic retrieval logic.

* Add Name property to MassTransitActivityTypeProvider

This commit inserts the Name property in the returned object within the MassTransitActivityTypeProvider class. It ensures that the typeName is included, providing a clearer definition of the activity type.

* Add handling for deleted bookmarks and refactor bookmark removal

Added a new event handler for `BookmarksDeleted` to ensure removed bookmarks are processed correctly. Refactored the bookmark removal logic into a helper method to reduce code duplication and streamline the workflow.

* Switch to asynchronous bookmark queue processing

Refactored the `TriggerWorkflows` handler to use `IBookmarkQueue` instead of directly invoking the `IBookmarkResumer`. This change aims to improve scalability by queueing bookmark resumption requests, enabling better load distribution and async processing. Added necessary helpers and configuration options to support this functionality.

* Remove unused IBookmarkResumer dependency

Simplify the constructor by removing the unused IBookmarkResumer dependency. This cleanup reduces potential confusion and improves code maintainability without impacting functionality.

* Add support for local message processing

Introduced an `IsLocal` property to `MessageReceivedStimulus` for determining if the message event is local to a specific workflow instance. Updated `BookmarkBinding` and related handler methods to utilize `CorrelationId` for local event matching. Removed unused `CorrelatingFields` from `MessageReceived` activity.

* Add nullability checks to IWorker retrieval methods

Updated `GetWorker` methods to return nullable `IWorker` to handle cases where a worker might not exist. Modified code to include null checks and conditional operations to prevent potential null reference exceptions when accessing worker methods.

* Add filtering based on activity type name for triggers and bookmarks

This commit introduces filtering for triggers and bookmarks based on the `MessageReceived` activity type name. It also adds an option to mark messages as local in the `SendMessage` activity, where local messages are delivered to the current workflow instance only. These changes help enhance the management and targeted delivery of messages within the workflow framework.

* Add new Kafka topics and clean up producers config

New topics "topic-3" and "topic-4" were added to the Kafka settings. Unused topic references were removed from the producers configuration to simplify and improve clarity.

* Add predicate to KafkaConsumerActivity in ConsumerWorkflow

Introduced a predicate to the KafkaConsumerActivity using JavaScript expressions to filter messages based on OrderId. This ensures only relevant messages are processed in the workflow.
2024-11-18 13:42:54 +01:00
AlmightyLks 153397e63c
Fix all broken summary xml docs (#6123)
* Add proper summary tags back

* Apply suggestions from code review

Co-authored-by: Robin Sue <robinsue@live.de>

* 69 more files fixed

---------

Co-authored-by: Robin Sue <robinsue@live.de>
2024-11-18 13:41:27 +01:00
Sipke Schoorstra 51793d08a6 Refactor variable management methods to use async naming
Updated method names in `IWorkflowInstanceVariableManager` and `WorkflowInstanceVariableManager` to use the `Async` suffix for consistency. This change improves code readability and aligns with async programming conventions.
2024-11-09 17:49:45 +01:00
Sipke Schoorstra 55b9361253
Add services and API endpoints to read and write workflow instance variables (#6107)
* Regenerate 3.3 EF Core migrations

* Refactor variable handling in API responses.

Updated `ApiResponseExtensions` to handle potential null issue flagged by ReSharper. Refactored `Endpoint.cs` in WorkflowInstances to use `ResolvedVariableModel` for better type safety and clarity.

* Add API endpoint to retrieve workflow variables

Introduce a new endpoint to get workflow instance variables in `IWorkflowInstancesApi`. Added `ResolvedVariable` model and adjusted `WorkflowInstances` endpoint to return a list of these variables.

* Add endpoint for updating workflow instance variables

Introduce a new API endpoint to update workflow instance variables using the DefaultWorkflowInstanceVariableWriter. Also creating the necessary interfaces and classes, and renaming existing ones for consistency.

* Add IWorkflowInstanceVariableManager interface and implementation

Introduce the IWorkflowInstanceVariableManager interface and its implementation, enabling variable management for workflow instances. Refactor existing endpoints to utilize this new manager, streamlining the variable retrieval and update processes.
2024-11-09 17:42:08 +01:00
Sipke Schoorstra 560ce08c45 Refactor namespaces and add Scrutor package
Streamlined the used namespaces across multiple files by removing unnecessary imports. Added the Scrutor package to the project dependencies for dependency injection uses.
2024-11-08 22:23:38 +01:00
Sipke Schoorstra e79b8c87d7 Merge remote-tracking branch 'origin/blueberry' 2024-11-08 21:50:55 +01:00
Sipke Schoorstra a0aa420a12
Keep Published Workflows Immutable by Updating Drafts Instead (#6106)
* Add workflow graph network builder and child workflow finder

Introduce IWorkflowGraphNetworkBuilder and IChildWorkflowFinder interfaces with their implementations to build and manage workflow graph networks. Additionally, register new services and handlers to support these functionalities in the Workflow Management feature.

* Remove unused workflow graph network builder

Deleted the unused `IWorkflowGraphNetworkBuilder` interface, its implementation, and related files. Updated affected classes to replace dependency on the removed builder with direct methods. Enhanced affected workflows tracking during publication.

* Optimize workflow update logic to skip redundant checks

Added condition to ensure only relevant activities are processed by verifying their version IDs. This reduces unnecessary computations and ensures the system skips already up-to-date workflow activities.

* Add workflow reference updater implementation

Introduced IWorkflowReferenceUpdater interface to manage workflow references updates. Implemented the corresponding service and integrated it into existing event handling logic. This improves maintainability and reduces code duplication by isolating the reference update logic.

* Set workflow activity version during update

Added a missing assignment of the workflow activity version in the `UpdateWorkflowDefinition` method. This ensures that the `Version` field is properly set when workflow definitions are updated.

* Remove UpdateReferencesInConsumingWorkflows method

Eliminated the UpdateReferencesInConsumingWorkflows method from IWorkflowDefinitionPublisher and its implementation from WorkflowDefinitionPublisher. The responsibility of updating workflow references is now transitioned to IWorkflowReferenceUpdater used in the `UpdateReferences` endpoint.

* Remove WorkflowGraphNetwork and WorkflowGraphNode models

These models were deleted as they are no longer needed in the system. The removal helps in reducing code clutter and improves maintainability by eliminating unused components.

* Refactor parameter naming in IWorkflowReferenceUpdater

Renamed 'definition' to 'referencedDefinition' for clarity in the IWorkflowReferenceUpdater contract and its implementation. This change makes it explicit that the parameter refers to the workflow definition being referenced, ensuring the code is more understandable and maintainable.

* Update WorkflowReferenceUpdater for clarity and accuracy

Renamed variables and updated comments for better clarity. The term "matchingWorkflowGraphs" was changed to "consumingWorkflowGraphs" to more accurately describe its purpose. Adjusted comment to clarify the materialization process of the draft version.

* Add IsReadonly filter to WorkflowDefinitionFilter

This ensures that only non-readonly workflow definitions are fetched. It enhances the accuracy of workflow updates by filtering out readonly versions.
2024-11-08 18:19:42 +01:00
Sipke Schoorstra 939fb95a97
Add multitenancy support for background tasks (#6059)
* Remove initial migrations

Deleted obsolete initial migration files from multiple databases: MySQL, SQL Server, SQLite, and PostgreSQL. This cleanup helps maintain a streamlined and updated migration history.

* Add Document base class and create tenant-specific indices

Introduced a new abstract `Document` base class to unify common properties. Implemented tenant-specific unique indices across multiple collections by including `TenantId` alongside `Id` to ensure uniqueness within tenant scopes.

* Remove outdated migration files

Deleted various migration files under MySql, PostgreSql, Sqlite, and SqlServer directories. This cleanup removes unnecessary schema definitions and helps to streamline the codebase.

* Refactor workflow identity assignment logic

Streamline workflow identity handling to ensure consistent assignment of Id, DefinitionId, and TenantId values. This change integrates tenant prefix and version suffix cleanly, enhancing clarity and maintainability.

* Enable multitenancy support

Added configuration for a new tenant (tenant-1) in appsettings.json and enabled multitenancy feature in Program.cs. This change allows the application to support multiple tenants, with specific configurations for each.

* Refactor route table update to run as startup task

Replaced `UpdateRouteTableHostedService` with `UpdateRouteTableStartupTask` to ensure route table updates are executed during application startup instead of as a hosted service. Updated configuration in `HttpFeature` and adjusted trigger validation logic in `ValidateWorkflowRequestHandler`.

* Add recurring task scheduling and single-node task support.

Introduce `IntervalExpressionType`, recurring task scheduling classes, and `SingleNodeTaskAttribute`. Update `RecurringTasksRunner` to handle schedules and add single-node task logic to `StartupTasksRunner`. Ensure proper namespace changes and configure sample recurring tasks.

* Refactor recurring tasks scheduling system

Replaced existing scheduling classes with a more modular and granular approach. Introduced new classes and interfaces like `ISchedule`, `CronSchedule`, `IntervalSchedule`, and `RecurringTaskScheduleManager`. Updated related methods and code to comply with the new design.

* Refactor background task management

Removed `ExpiredSecretsHostedService` and refactored it into a recurring task. Introduced `TaskExecutor` for shared task execution logic. Updated and renamed feature classes to better represent their purpose, improving task scheduling and execution management.

* Add BackgroundTask abstract class to Elsa.Common module

This new abstract class implements the IBackgroundTask interface with default methods for executing, starting, and stopping tasks asynchronously. It provides a basic framework for background task management in the Elsa.Common module.

* Switch to CreateAsyncScope in DefaultTenantScopeFactory

Updated the CreateScope method to use CreateAsyncScope instead of CreateScope. This change improves asynchronous handling of service scopes within the DefaultTenantScopeFactory class.

* Add tenant handling and move StartWorkers background task

Introduce ITenantAccessor in Worker class for multitenancy support. Rename and relocate StartWorkers service to BackgroundTask, ensuring smoother workflow initialization. Also, update the configuration to support Azure Service Bus connection string.

* Add tenant support and refactor ProtoActor client

Integrated ITenantAccessor in ProtoActorWorkflowClient class to handle multi-tenancy. Refactored methods in the client to support custom headers and added async disposable pattern in various services for proper resource management. Additionally, enabled Azure Service Bus and updated related documentation.

* Add support for custom headers in ProtoActor grain methods

Introduced a T4 template to generate grain methods with custom headers, enabling the use of tenant ID in requests. Updated `ProtoActorWorkflowClient` to employ these methods, removing redundant code and directly utilizing the client for various workflow operations.

* Add tenant middleware to MassTransit configurations

Introduced multitenancy middleware for MassTransit message handling. Added new message type `OrderReceived` and updated RabbitMQ setup in Elsa Server. Applied middleware to configure tenant data on send, publish, and consume operations.

* Add new product workflow and streamline ID handling

Introduced a new `RequestResponseWorkflow` for handling product requests. Simplified ID handling in `WorkflowBuilder` and `ClrWorkflowsProvider` by defaulting to empty strings and adding a version prefix. Enhanced `HttpWorkflowsMiddleware` to correctly parse full request paths.

* Remove redundant files and update configuration

Deleted unused files `Product.cs` and `RequestResponseWorkflow.cs` to clean up the codebase. Updated `Program.cs` configuration: switched MassTransitBroker to Memory and disabled multitenancy.

* Remove MultitenantRecurringTaskService and update AzureServiceBus

Removed `MultitenantRecurringTaskService` and adjusted related code for Azure Service Bus to work without it. This includes removal of tenant accessor dependency from `Worker` and cleanup of service configuration flags in `Program.cs`.

* Increase signal wait timeout to 10000 milliseconds.

Extended the default timeout for signal awaiting methods from 8000 to 10000 milliseconds. This change ensures more flexible and resilient waiting periods, reducing timeout occurrences in scenarios with longer processing times.

* Refactor scheduling service to be a background task

Renamed `CreateSchedulesHostedService` to `CreateSchedulesBackgroundTask` and refactored it to inherit from `BackgroundTask` instead of `BackgroundService`. Simplified the constructor by injecting the required dependencies directly, eliminating the need for a scoped factory.

* Refactor workflow version suffix formatting

Changed the version suffix format from `:v{version}` to `v{version}` and adjusted the ID concatenation accordingly. This improves consistency and readability of workflow IDs.

* Enable multitenancy support in Quartz scheduler

Added `TenantJobListener` to inject tenant context into jobs. Modified `QuartzWorkflowScheduler` to incorporate tenant IDs into job data maps and adjusted the configuration to acknowledge multitenancy settings.

* Remove ConfigureSchedulerHostedService and TenantJobListener

Consolidated tenant resolution logic into JobExecutionExtensions class. Updated ResumeWorkflowJob and RunWorkflowJob to use the new extension method for tenant retrieval. This simplifies the QuartzSchedulerFeature setup by removing the hosted service configuration.

* Refactor HTTP feature and update route table task

Move 'UpdateRouteTableStartupTask' from 'HostedServices' to 'Tasks' and update dependency injection configurations accordingly. Simplify 'DefaultRouteTableUpdater' by removing unnecessary options and tenant-agnostic settings from filters.

* Disable multitenancy in Program.cs

The useMultitenancy flag has been changed from true to false. This update affects the Elsa.Server.Web application configuration.

* Simplify variable usage in HttpWorkflowsMiddleware

Replaced 'fullPath' variable with 'path' to streamline code. This change enhances readability by reducing redundancy and ensures consistency in variable naming throughout the method.

* Enable multitenancy and refactor tenant handling logic

Enable multitenancy in the application and refactor tenant handling logic to use ITenantFinder and ITenantContextInitializer interfaces. Added header constants, updated middleware to use these interfaces, and moved extension methods to the appropriate namespace.

* Add input validation to user registration form

Implemented checks to ensure all required fields are filled and that input data adheres to format requirements. This change reduces errors and enhances form reliability.

* Remove unused import from TenantPrefixHttpEndpointRoutesProvider

This change cleans up the code by removing an unnecessary import statement. It improves code readability and reduces clutter, making future maintenance easier. The functionality remains unchanged.

* Rename filter scope to "tenantPublish" in Probe method

Updated the Probe method in TenantPublishMiddleware.cs to use "tenantPublish" instead of "tenantSend" for better clarity. Ensures consistency with the method's context and aligns with naming conventions.

* Refactor: Remove extraneous whitespace

Eliminate unnecessary whitespace in ProtoActorWorkflowClient.cs for cleaner code. This change helps maintain consistent formatting and improves readability.

* Refactor DefaultRegistriesPopulator for cleaner initialization

Converted constructor to use read-only fields directly, removing unnecessary instance variables. This change simplifies the code by reducing redundancy and making the constructor cleaner.
2024-10-28 19:38:24 +01:00
Sipke Schoorstra ca78f73e0a
Introduce Log Persistence Strategy (#6057)
* Implement log persistence strategy management

Added interfaces, services, and strategies for log persistence. Introduced new endpoint to list available log persistence strategies. Updated configurations and dependency injections accordingly.

* Refactor log record methods to asynchronous

Updated methods for extracting and persisting log records to be asynchronous, enhancing performance and scalability. This change includes modifying interfaces and implementations for better async support in workflow execution logging.

* Remove commented code

* Support nullable values in ActivityState dictionaries

Update ActivityState to support nullable values by changing type to 'IDictionary<string, object?>'. Enhanced DefaultActivityExecutionMapper to handle multiple persistence strategies for logging inputs and outputs.

* Rename ShouldPersistAsync to GetPersistenceModeAsync

Refactor method names for log persistence strategies to improve readability and consistency. Added summary comments for clarification and removed redundant configurations from appsettings.json. Added implicit uses and updated namespaces for better maintainability.

* Refactor activity payload and output retrieval logic

Extract payload and output retrieval into `GetPayload` and `GetOutputs` methods respectively. This modularizes the code for better readability and maintainability, and allows for potential reusability of these methods in other parts of the codebase.

* Add new project reference and update PostgreSQL provider usage

Added a project reference to Elsa.Agents.Persistence.EntityFrameworkCore.PostgreSql in the test project file. Also modified the WorkflowServer setup to specify the assembly in the PostgreSQL provider configuration.

* Add agent persistence to WorkflowServer

Integrated agent support and persistence using PostgreSQL in WorkflowServer. This includes adding necessary project references and configuring agents in the workflow server setup.
2024-10-25 19:41:10 +02:00
Sipke Schoorstra 3208be7b89 Merge remote-tracking branch 'origin/blueberry' 2024-10-23 16:07:32 +02:00
Sipke Schoorstra 52516aa0ff
Fix Bulk Dispatch Missing Input (#6052)
* Remove outdated sample workflows and activities

Deleted CompositeExample, SampleWorkflow, and SlowActivity classes as they are no longer needed. These deletions help clean up the codebase and maintain relevance in the code structure.

* Handle null WorkflowBuilderType in ClrWorkflowMaterializer

Introduce a fallback NotFoundWorkflowbuilder class when WorkflowBuilderType is null. This ensures the MaterializeAsync method functions even if the provided context lacks a specific workflow builder type.

* Add merging of itemDictionary to input

This change ensures that the itemDictionary's contents are also merged into the input dictionary, preventing possible data loss. It builds on existing logic by adding an additional merge operation to incorporate all necessary data.

* Add new workflows to BulkDispatchWorkflows and modify tests

Introduce `FruitWorkflow` and `MixFruitsWorkflow` for bulk dispatch scenarios. Refactor `BulkDispatchWorkflowsTests` to include new tests and rename signals for clarity. Optimize `BulkDispatchWorkflows` activity by removing redundant dictionary merge.
2024-10-23 16:06:28 +02:00
Sipke Schoorstra bdaae64e72 Add base path providers and standardize tenant accessor property
Introduce DefaultHttpEndpointBasePathProvider and TenantPrefixHttpEndpointBasePathProvider to manage HTTP endpoint base paths. Update ITenantAccessor property from CurrentTenant to Tenant for consistency across the codebase.
2024-10-17 19:47:55 +02:00
Sipke Schoorstra b8ab05ce2e Refactor WorkflowInstancesFeature class constructor
Updated the WorkflowInstancesFeature class constructor to remove an unnecessary method override, simplifying the class design. This change enhances code readability and maintains the same functionality.
2024-10-17 08:52:07 +02:00
Sipke Schoorstra 7e7a899bbf
Implement multitenant HTTP routing (#6031)
* Add tenant awareness to bookmark handling and route resolution

Added tenant ID support across various components, including bookmark updates, route resolution, and middleware processing. This ensures that bookmark and route operations can now appropriately handle tenant-specific data, improving the system's multitenancy capabilities.

* Add Multitenant HTTP Routing feature to Tenants module

Introduced a new MultitenantHttpRoutingFeature class to the Elsa.Tenants.AspNetCore module, enhancing the tenant resolution capabilities. Moved RoutePrefixTenantResolver from Elsa.Http to Elsa.Tenants.AspNetCore and updated relevant project references and namespaces accordingly. This refactor improves modularity and separation of concerns between HTTP and tenancy features.

* Refactor route handling and tenant configuration

Removed redundant `RouteTableExtensions` and replaced with new route providers and updaters, enhancing flexibility and modularity. Introduced tenant-specific HTTP endpoint configurations for better customization and configuration management.

* Rename HttpEndpointBookmarkStimulus to HttpEndpointBookmarkPayload

Refactor various classes and methods to reflect the renaming from `HttpEndpointBookmarkStimulus` to `HttpEndpointBookmarkPayload`. Add and configure new extension methods for tenant route handling, update the route provider to support multi-tenancy, and adjust the tenants provider to bind configuration properly.

* Add HeaderTenantResolver and refactor Http namespace.

Introduce HeaderTenantResolver to resolve tenants via HTTP headers. Refactor multiple classes and interfaces to move from the Elsa.Http.Models namespace directly into Elsa.Http for clarity and consistency.

* Add Host-based tenant resolution

Implemented a HostTenantResolver to resolve tenants based on the request's host and updated tenant configurations with host information. Modified the tenant resolver pipeline and added the new host resolver to the service registrations.

* Add tenant-aware caching and accessor support

Enhanced caching by incorporating tenant identifiers into cache keys for more granular cache management. Introduced ITenantAccessor dependencies in various services to retrieve the current tenant information. This ensures that cache entries are correctly isolated per tenant.

* Reorder tenant resolvers for pipeline setup.

Reordered the tenant resolvers in the pipeline to prioritize HostTenantResolver before RoutePrefixTenantResolver. This ensures that tenant resolution is correctly aligned with host-based resolving before checking the route prefix.

* Remove unused imports

This commit eliminates redundant `using` directives across multiple files to streamline the codebase. This cleanup helps improve code readability and maintainability by removing unnecessary dependencies.
2024-10-14 21:27:11 +02:00
Sipke Schoorstra f3961c0af5 Update namespaces and correct merge artefacts 2024-10-11 20:41:49 +02:00
Sipke Schoorstra dd812645c7 Refactor imports to reduce use of Elsa.Common.Contracts
Consolidate imports by replacing Elsa.Common.Contracts with Elsa.Common and Elsa.Common.Multitenancy. This update streamlines import statements across various modules, improving code readability and maintainability.
2024-10-05 18:30:45 +02:00
Sipke Schoorstra 329a79be2e
Secrets API (#5967)
* Add initial implementation for Elsa Secrets modules

This commit introduces new projects for managing secrets within the Elsa framework: `Elsa.Secrets.Api`, `Elsa.Secrets.Core`, and `Elsa.Secrets.Management`. These projects include essential interfaces, models, entities, and endpoints to handle secret storage, retrieval, and management. Specific features include API endpoints for listing secrets, models for secret filtering, and interfaces for encryption key handling.

* Add weavers

* Refactor encryption handling in secrets management

Implemented a new architecture for handling encryption keys and algorithms within the secrets management system. Replaced old encryption key entities and related interfaces with a more modular and extensible approach. Added new services and models to improve encryption and decryption processes, enhancing maintainability and scalability.

* Add IEncryptor interface for encryption functionality

Introduced the IEncryptor interface to standardize encryption operations within the Elsa.Secrets.Management module. This interface includes the EncryptAsync method to handle encryption using a specified key ID and value.

* Add dependency on SecretsFeature and configure secrets provider

Integrate the SecretsFeature dependency and configure a secrets provider within the SecretsManagementFeature class. This adds the StoreSecretProvider to the service collection and ensures the secrets provider is correctly set up. Also, rename method from WithSecretsProvider to UseSecretsProvider for clarity.

* Add EF Core and SQLite support for Secrets module

Introduced Entity Framework Core and SQLite support for the Secrets module, including migration files, EF Core configurations, context factory, and store implementation. Added necessary extensions and configuration code to integrate with the existing API and features. Included updates to the main web application to utilize the new persistence providers.

* Update Microsoft.SemanticKernel to version 1.18.2

Upgrade Microsoft.SemanticKernel package to the latest version to ensure compatibility and new features. Remove unused Elsa.Agents.Persistence using directive from Program.cs for code cleanliness.

* Update workflows and add Agents module

Changed workflow branch targets from `main` to `feature/secrets`. Updated Docker image tags and added the `Agents` module to the Elsa Studio WebAssembly project.

* Enable agent activities in workflow configuration

This change introduces the `.UseAgentActivities()` method in the workflow configuration, enhancing the workflow capabilities. By doing so, it ensures that agent activities are appropriately integrated and available for use in the application.

* Add EF Core migrations for MySQL and SQL Server

Added Entity Framework Core migrations and related configurations to support MySQL and SQL Server for the Agents Persistence module. These changes include new migration files, context factories, and project configurations.

* Fix migration assembly reference and update method syntax

Changed the migration assembly reference in SqlServerProvidersExtensions. Updated method syntax in WorkflowManagementFeature to use array shorthand format.

* Add secret management functionalities

Introduced secret management services with CRUD operations, notifications, and bulk actions. Added unique name generation and validation for secrets, and implemented corresponding API endpoints.

* Enhance Secret Management Feature

Added Elsa.Extensions import and updated MemorySecretStore registration to use the AddMemoryStore method with Secret. This improves code modularity and adheres to the updated registration method conventions.

* Remove encryption services and update migration

Removed multiple files related to encryption services and their dependencies, including encryption algorithms and key providers. Also updated a migration script to reflect schema changes, removing specific columns and constraints.

* Implement versioning and retrieval for secrets management

Added "IsLatest" flag and cloning mechanism for secrets to support versioning. Introduced a new API endpoint for fetching decrypted secret input models. Refactored encryption and decryption logic to handle empty values gracefully.

* Add secret management functionalities

Introduced services and interfaces for secret name generation, validation, and updating. Updated secret handling to include expiration metadata. Refactored methods in ISecretManager to streamline secret creation and update processes.

* Remove DisableSyntaxSelection class and references

Deleted the DisableSyntaxSelection class and its references from various files. This includes removing its registration as a Scoped service and associated usage in the `RunJavaScript` activity.

* Add new migration for secrets and update DefaultSecretManager

Re-created migration files for V3_3 to include the ExpiresIn column. Updated DefaultSecretManager to utilize identityGenerator for generating Id and SecretId, and added additional fields like CreatedAt, UpdatedAt, and IsLatest.
2024-09-15 17:12:13 -07:00
Sipke Schoorstra 2d62a373e0
Add Agents Module with Semantic Kernel Integration (#5937)
* Remove `ServiceProfileConfig` and `SkillConfig`

Refactored the codebase to eliminate `ServiceProfileConfig` and `SkillConfig` in favor of simplifying configurations. Updated related entities and settings to consistently use `ServiceConfig` and `AgentConfig`. Refined function execution to enhance overall system coherence and maintainability.

* Refactor agent management and plugin system

This update eliminates AgentManager and related classes, introducing new management interfaces and memory-based stores. The plugin system is refactored with better separation of concerns and the addition of provider interfaces.

* Add new Agents features and services

Introduced AgentsFeature for API endpoints, AgentManagementFeature for agent management, and AgentActivitiesFeature for activities. Updated service configurations and migrated files for cleaner and more maintainable code structure.

* Rename execute endpoints to invoke

Renamed the 'Execute' endpoints and related paths to 'Invoke' for better semantic clarity. Adjusted the namespace and updated the route configuration accordingly to reflect this change.

* Add permissions configuration to Agents Invoke endpoint

This commit adds a call to `ConfigurePermissions("agents:invoke")` in the Agents Invoke endpoint configuration. This change ensures that proper permissions are checked when this endpoint is accessed.

* Rename Management to Persistence with EF Core support

Renamed various files and classes from Management to Persistence. Introduced Entity Framework Core (EF Core) support for agents, including implementation of APIs for API key, service, and agent definitions. The changes also updated project configurations and added necessary database context and entity configurations.

* Add EntityFrameworkCore project for Agents persistence

Introduce a new Entity Framework Core provider to manage the persistence layer of the Agents module. The project includes references to the base persistence and common Entity Framework projects.

* Remove redundant exception handlers and add new SQLite migration.

Deleted unnecessary `DbExceptionTransformer` and `RethrowDbExceptionHandler` classes to streamline error handling. Introduced a new SQLite migration for the Agents module to create necessary database tables and indices. Also modified `Store` and `EntityStore` to utilize a service provider for exception handling.

* Add Sqlite configuration extension and sample updates

Introduce a new extension method for configuring EF Core to use Sqlite in the Elsa.Agents module. Updated the sample project to use a configurable Sqlite connection string and adjusted the appsettings.json accordingly. Also added necessary folder structure in the project file.

* Add agent, service, and API key CRUD operations

Introduced CRUD functionalities for agent, service, and API key entities. Added endpoint handlers, filtering capabilities, and memory/EF core stores to support these operations. Updated configuration mappings as well.

* Add Agent API endpoints and DTOs

Added CRUD endpoints for agents in Elsa.Agents.Api along with the required DTOs and Request models. These endpoints include create, read, update, delete, and list functionalities, facilitating the management of agent entities.

* Add support for agent service providers and discovery

Introduce interfaces and implementations for agent service providers and service discovery. Updated `KernelFactory` to integrate the new service discovery mechanism and restructured plugin discoverer naming for consistency.

* Add agent management features

Introduce agent management components, including pages for listing and handling agent workflows, dialogs for creating agents with Blazor components, and backend API endpoints for generating unique agent names. This enhances functionality for managing agent workflows within the application.

* Refactor exception handling and simplify event handling logic

Replaced DataProcessingDbExceptionHandler with DbExceptionTransformer for handling PostgreSQL exceptions. Streamlined event completion logic by renaming method and ensuring input retrieval before setting the result.

* Add bulk delete endpoint for agents

Introduced new API endpoint for bulk deleting agents. Implemented request and response models, and updated persistence layer to handle bulk operations. Also reorganized project structure for better modularity.

* Refactor API Key models and endpoint classes

Consolidate API Key request models into unified models and update endpoints to use these new models. Add new transformation methods to entity classes. Enhance configuration classes and streamline the dependency injection process.

* Refactor and streamline bulk delete and service APIs

Refactored the bulk delete endpoints for API keys, services, and agents, consolidating request and response models. Added new service input and model classes and updated the corresponding stores to support bulk deletions. Simplified the endpoint logic to use these new models and enhanced filtering capabilities.

* Set default values for AgentInputModel properties

Updated properties in `AgentInputModel` to use initialized default values instead of `default!`. This change ensures that string and object properties are instantiated properly, improving code stability and preventing potential null reference exceptions.

* Add plugin listing endpoint and related models

Introduced a new endpoint to list all registered plugins in the system. Added necessary models and methods for plugin description and retrieval, along with appropriate documentation comments.

* Refactor agent management to use `IAgentManager`.

Created new notifications for agent lifecycle events and introduced an `AgentManager` service for agent CRUD operations. Updated APIs and other components to use the new `IAgentManager` service instead of direct store access for a more cohesive architecture.

* Update agent activity descriptor names

Changed the activity descriptor names to use PascalCase for consistency and readability. Also added display names for input variables to improve user interfaces.

* Add overloaded RefreshDescriptorsAsync for single activity provider

Introduced a new overloaded method in IActivityRegistry and ActivityRegistry to refresh activity descriptors using a single IActivityProvider. Updated RefreshActivityRegistry to use the new method, ensuring more flexibility in descriptor refresh operations.
2024-09-01 09:03:14 +02:00
jdevillard 3083a400d8
Add Variable available in dropdown when configuring a activity input (#5918)
* Sample to provide a ExpressionDescriptorProvider

* fix variable accessor in VariableExpressionHandler

* remove Variable Expression to use default one

* Change the Variable Expression Provider to handle Variable Expression correctly

* need to check on UI Side and Server side is valid JSON is sent. For example, if a use change from a literal value to a Variable value, the value is sent to the API and result in an exception, we cannot handle invalid json  without using Try/Catch

---------

Co-authored-by: Jérémie DEVILLARD <jdevillard@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-08-30 12:01:44 +02:00
Sipke Schoorstra d0316687c1 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-14 19:44:39 +02:00
Sipke Schoorstra c8669d1bc1
Add optional refresh parameter (#5876)
* Add optional refresh parameter

Renamed several `RefreshDescriptors` methods to `RefreshDescriptorsAsync` to reflect their asynchronous nature consistently. Added an optional refresh parameter in the `ListActivityDescriptorsRequest` class and updated related endpoints to handle this parameter, triggering a registry refresh if necessary.

* Update version to 3.2.0-rc5 in GitHub Actions

Modified the `packages.yml` workflow to change the version from `3.2.0-rc4` to `3.2.0-rc5` for non-tagged releases. This ensures version consistency with the new release candidate.
2024-08-12 09:42:25 +02:00
Sipke Schoorstra 38998aa051
Add Orchard Core Integration + Agents Module (#5871)
* Add "integrations" project and update dependencies

A new project named "integrations" has been added to the Elsa solution. Related dependencies have been updated accordingly, with some connections reassigned to the new project. This includes both new connections established and existing connections removed in response to the project addition.

* Stash

* Refactor ActivityDescriber to clean up unused code

Removed commented out code lines related to input options in the `ActivityDescriber` class. This change enhances code readability and maintains the focus on active and necessary functionality. The main logic for getting UI properties remains intact.

* Remove deprecated webhook dispatchers and handlers

Eliminated classes and interfaces related to webhook dispatching and registration from the Elsa.Webhooks module as part of refactoring. Introduced new classes and features for handling webhook events within the Elsa.OrchardCore module. Updated several configuration files and references accordingly.

* Add OrchardCore integration sample and enhance webhooks

Added a new ASP.NET sample for integrating with OrchardCore and set up various related configuration files and project references. Improved the webhooks module by modifying the WebhookEventActivityProvider for better event handling and description management, and enhanced the WebhooksFeature to include the WebhookEventActivityProvider.

* Set activity descriptor description from eventType.DisplayName

The activity descriptor now sets its Description property using the eventType's DisplayName. This ensures that the description is consistent with the event type's display name across the system.

* Rename 'Description' to 'DisplayName' for activity descriptor

This change ensures that 'DisplayName' is assigned the correct value from 'eventType.DisplayName'. It helps avoid any confusion between 'Description' and 'DisplayName' properties in the activity descriptor object.

* Add PayloadType and Constructor logic for Webhooks

Added PayloadType to content item published events in `appsettings.json`. Updated `WebhookEventActivityProvider` to include constructor logic and manage payload types. Enhanced `OrchardCoreFeature` and `WebhooksFeature` with alias definitions for variable types.

* Add helper for webhook activity type names and refactor code to use it

Introduce `WebhookActivityTypeNameHelper` for consistent generation of webhook activity type names. Refactor relevant classes to utilize this helper, simplifying and standardizing the codebase. Adjust other minor settings, such as editor config line length and dependency configurations.

* Refactor and introduce Orchard core integration

Replace `ContentItemPublished` with a new handlers and payload structure. This change includes the creation of various classes and handlers necessary to support the OrchardCore integration, define the content item events, and configure them via app settings. This refactor aids in standardizing the handling of content item publication events through newly introduced classes and configurations.

* Refactor stimulus class name for content publication

Renamed `WebhookEventReceivedStimulus` to `ContentItemPublishedStimulus` to better reflect its purpose in handling content item publication events. Updated all references and imports accordingly to maintain consistency in the codebase.

* Refactor webhook payload handling

Switched from using only `ContentItemPublishedPayload` to incorporating both `WebhookEvent` and `ContentItemPublishedPayload` in the input dictionary. This change ensures more comprehensive data handling for webhook events.

* Add new dependency and correct project reference path

Included Elsa.Webhooks.ActivityProviders in WebhooksFeature to support new webhook activities. Corrected the project reference path in Elsa.Samples.AspNet.OrchardCoreIntegration.csproj to reflect the accurate module directory structure.

* Rename and enhance webhook handling functionality

Renamed `InvokeOrchardActivities` to `InvokeOrchardWebhookEventActivities` for clarity. Also added an `ActivityProvider` to improve webhook event handling.

* Refactor: replace Workflow with WorkflowGraph in context

Updated references from Workflow to WorkflowGraph across various files to reflect the new context structure. This change improves the accuracy of context information and aligns with recent architectural updates.

* Add Semantic Kernel modules and features

Introduced core classes and services to support Semantic Kernel agents, their configuration, and execution within workflows. Added API endpoints for agent execution, integrated kernel configuration settings, and ensured necessary extensions for skill and plugin integration.

* Update ActivityDescriber and AgentActivityProvider

Default `inputAttribute` to null in `GetUIHint` and use it in `AgentActivityProvider`. This change ensures the `GetUIHint` method is called directly, improving code reusability and simplifying input handling.

* Add GraphQL client with token authentication

Introduced a DefaultGraphQlClient to send GraphQL queries, incorporated token-based authentication using AuthenticatingDelegatingHandler, and included support for refresh tokens through DefaultSecurityTokenService. Updated projects and configuration accordingly.

* Rename OrchardWebhooks to OrchardCore

Updated method and related variables to use 'OrchardCore' instead of 'OrchardWebhooks' for consistency and improved clarity. Also added activity attributes for SendGraphQLQuery to better describe its purpose.

* Refactor webhook handling and add ProofreaderResult model

Refactor activities and event handling to support multiple webhook event types beyond content publishing. Introduce new models under OrchardCoreIntegration for ProofreaderResult and ProofreadError, and update existing methods for more generic payload handling.

* Add support for API key configuration and reference

Implemented a new `ApiKeyConfig` class to manage API keys. Updated various components to utilize API key references, enhancing configuration flexibility. Added new options and settings to facilitate these changes.

* Add translation functionality and improve agent activity displays

This commit introduces a new `TranslationResult` model and updates the project configuration to support translation features. Additionally, it adds two new project references and modifies the `appsettings.json` to include translation skills. The `AgentActivityProvider` class now has improved display names for agent activities for better readability.

* Add FactCheck and Tagging functionality

Introduced new models: FactCheckResult, GenerateTagsResult, and IncorrectFact. Integrated JetBrains.Annotations package and extended variable types in workflow management. Updated prompt templates and appsettings for proofreader, fact-checker, and tagger functions.

* Enhance 'Tagger' skill in appsettings.json

Updated the 'GenerateTags' function in the 'Tagger' skill to define the number of tags to generate and to ensure JSON response format. Added a 'Number' input variable and the 'ResponseFormat' property for clarity and precision.

* Add support for patching content items via OrchardCore API

Implement DefaultRestApiClient and IRestApiClient for GET and PATCH requests to the content API. Introduce Activity for patching content items and update app settings to handle TitleResult in JSON format.

* Add variable type aliasing and improve serialization

Introduced extension methods to add variable type and alias in WorkflowManagementFeature. Enhanced JSON serialization in AgentActivity and DefaultRestApiClient. Cleaned up solution by removing old project references and updated appsettings.json for revised prompt templates.

* Remove Proofreader models and update AgentActivity properties

Removed obsolete `ProofreaderResult` and `ProofreadError` classes to streamline the codebase. Updated `AgentActivity` properties to be internal and JSON-ignored to enhance encapsulation and reduce serialization overhead.

* Add localization support for content items in OrchardCore

Implemented a new activity `LocalizeContentItem` to create localized versions of content items based on a provided culture code. Added a corresponding API client method and request model to facilitate this feature, along with necessary format adjustments in the appsettings file.

* Enhance SkillExecutor functionality and update config details

Enabled 'AllowDangerouslySetContent' in SkillExecutor for improved flexibility with input variables. Streamlined the Proofread function prompt template and adjusted various execution settings in `appsettings.json`. Removed explicit MaxTokens limits to allow more dynamic content handling.

* Add attribute to BackgroundActivityStimulus

Added the `ExcludeFromHash` attribute to the `JobId` property in `BackgroundActivityStimulus` class, fixing background activity resumption.

* Rename SemanticKernel to Agents module for better clarity

This commit renames the SemanticKernel module to Agents across the entire project, including all relevant file paths, namespaces, and configuration settings. Major refactors encompass changing file names, class names, and namespaces to reflect the new module name, removing the SemanticKernel integration in favor of the more general Agents framework.

* Update configuration section for SemanticKernel

Changed the configuration section name from "Agents" to "SemanticKernel" in Program.cs and appsettings.json. This ensures the options are correctly bound to the relevant settings in appsettings.json.

* Add new entities and projects for Elsa Agents

Introduced new entities: `AgentDefinition`, `ApiKeyDefinition`, `ModelDefinition`, `PluginDefinition`, `SkillDefinition`. Added new projects: `Elsa.Agents.Management` and `Elsa.Agents.Persistence`. Renamed `ConfigureAgents.cs` to `ConfigureKernel.cs` and updated relevant references.

* Add ActivityHandle to workflow execution endpoint

Extended the endpoint to accept ActivityHandle as part of the workflow execution request. This change improves the ability to specify and manage activity-specific workflows. Additionally, updated `appsettings.json` to remove "Article" from the list of content types.

* Add nullable checks and new skills with functions

Improve null safety in AgentActivity and AgentActivityProvider classes by adding nullable checks. Additionally, update `appsettings.json` with corrected skill names and introduce new skills including ArticleWriter and Author.

* Register Random type in JavaScript engine

Added the Random type registration to both JintJavaScriptEvaluator and CommonTypesDefinitionProvider. This allows JavaScript code to utilize the Random class for generating random numbers.

* Add CreateContentItem activity and API client method

Introduce a new CreateContentItem activity to create and optionally publish content items. Added a new REST API client method to handle content creation, including the necessary request model.

* Update API endpoints and variable name for consistency

Changed API endpoints in DefaultRestApiClient.cs to use 'content-items' instead of 'content' for consistency with the project's URL structure. Also updated a variable name in CreateContentItem.cs for clarity and readability.

* Fix potential NullReferenceException on timer disposal.

Ensure that the timer is not null before calling Dispose in both ScheduledRecurringTask and ScheduledCronTask. This prevents potential NullReferenceExceptions when the timer event is fired.

* Add Image Generator and integrate OpenAI text-to-image feature

Introduced `ImageGenerator` for generating images from text using OpenAI's text-to-image functionality. Updated `KernelFactory` to support `OpenAITextToImage` service, and modified configurations and services to utilize the new image generation capabilities.

* Add UploadMedia activity and WorkflowInstanceStorageDriver

This commit introduces a new UploadMedia activity for uploading files to a media library and a WorkflowInstanceStorageDriver for managing workflow state storage. Additionally, it includes changes in IRestApiClient interface, DefaultRestApiClient implementation, and minor logging enhancements in ScheduledSpecificInstantTask. Unnecessary project references were removed and project files were updated accordingly.

* Mark `WorkflowStorageDriver` as obsolete and update condition.

Added [Obsolete] attribute to `WorkflowStorageDriver` indicating it will be removed in future releases. Modified `BackgroundActivityInvoker` to also recognize `WorkflowInstanceStorageDriver` for output capture.

* Handle JsonNode deserialization in ObjectConverter

Add support for deserializing JsonNode types in ObjectConverter.cs. This ensures proper handling of JsonNode along with JsonElement and JsonObject, enhancing the overall deserialization process.

* Add ResolveTags activity to OrchardCore module

Created a new ResolveTags activity that interacts with the Orchard Core API to resolve or create tags. Implemented the ResolveTagsRequest model and updated the IRestApiClient and DefaultRestApiClient to support this new functionality.

* Refactor project structure and update configuration

Renamed the Elsa.Agents project to Elsa.Agents.Activities and updated all relevant namespaces and file paths. Added a new setting to the appsettings.json to include the language parameter for article generation.

* Update project metadata in multiple csproj files

Revised descriptions and package tags across several projects to ensure consistent terminology related to Agents. Removed unused folder references in Elsa.Agents.Core.csproj.

* Simplify JSON node deserialization logic

Removed redundant JsonObject checks and unified the handling of JsonNode instances. This refactoring streamlines the code and ensures consistency in the deserialization process.
2024-08-08 19:35:43 +02:00
Sipke Schoorstra b2b29a7f4f Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-06 22:47:05 +02:00
Sipke Schoorstra d2fc7fb005
Rework Workflow Context Feature (#5861)
* Refactor workflow context extension structure

Removed unused WorkflowContextWorkflowDefinitionExtensions and renamed several extension and middleware classes for clarity and consistency. Simplified context provider type handling using JsonArray in custom properties.

* Refactor context provider and update email workflow

Refactored `CustomerWorkflowContextProvider` to use constructor parameter directly, removing redundant fields and constructors. Updated comments and list syntax in `CustomerCommunicationsWorkflow`. Improved README.md documentation for workflow contexts and context providers.

* Adjust logging levels and improve workflow logic

Modified `appsettings.json` to reduce the verbosity of logging by adjusting log levels. Updated `CustomerCommunicationsWorkflow` to include a timer activity and simplified the workflow context parameter setting.

* Remove PropertyBag and related functionalities.

Deleted the PropertyBag class and extension methods, and removed all instances where PropertyBag was utilized across various modules. This simplifies the workflow configuration and management by relying solely on CustomProperties for storing metadata.

* Add workflow context management extensions and optimizations

Introduces `WorkflowExecutionContextExtensions` for loading and saving workflow contexts more efficiently. Replaces `JsonArray` usage with `List<object>` for context provider types, simplifying the type management in the workflow execution context. Streamlines `WorkflowContextActivityExecutionMiddleware` by removing redundant service scope creation and enhancing readability.

* Remove redundant timer in CustomerCommunicationsWorkflow

The timer activity that was set to not start the workflow has been removed to streamline the workflow process. Updated the README to include instructions on how to start the CustomerCommunicationsWorkflow using the REST API or Elsa Studio.

* Refactor middleware to streamline context management

Removed dependency on `IServiceScopeFactory` and refactored middleware to directly load and save workflow contexts. Simplified workflow context provider invocation, enhancing code readability and maintainability.

* Remove redundant PropertyBag property

The PropertyBag property was commented out and is no longer needed in the WorkflowDefinition class. This change removes the commented-out code to clean up the class definition.

* Add JsonSerializerOptions for deserialization

Introduce JsonSerializerOptions with camelCase policy and case insensitivity for property names. This ensures consistent deserialization of custom properties in activity workflow context settings.
2024-08-06 22:36:24 +02:00
Sipke Schoorstra f96d594f6f Refactor folder paths and update method implementations
Updated Dockerfile paths to reflect 'apps' directory structure, removed redundant XML comments, and modified methods in WorkflowDefinitionExtensions to use FindWorkflowGraphAsync for consistent workflow retrieval.
2024-07-24 14:10:27 +02:00
Sipke Schoorstra 827680b3c4 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-18 18:25:49 +02:00
Sipke Schoorstra e5ba9a8a9d Refactor scheduling and system clock configurations
Moved specific services from Scoped to Singleton in SchedulingFeature and SystemClockFeature for better performance and consistency. Refactored DefaultWorkflowScheduler to use a constructor with an IScheduler parameter and removed redundant private field. Removed attribute RequiresUnreferencedCode in WorkflowManagementFeature.
2024-07-18 16:00:49 +02:00
Sipke Schoorstra f448e9520a
Fix reloading logic for workflow definitions (#5781)
* Add reloading logic for workflow definitions

Introduced a new mechanism to handle reloaded workflow definitions. This includes creating a "ReloadedWorkflowDefinition" model, updating the caching logic, and modifying notification handlers to work with the enhanced workflow reloading logic. This ensures workflow definitions are updated and managed correctly when published, retracted, or deleted.

* Improve RefreshActivityRegistry documentation

Updated the XML documentation to clarify that `RefreshActivityRegistry` refreshes the `IActivityRegistry` for `WorkflowDefinitionActivityProvider` whenever workflow definitions are reloaded, instead of when they are published, retracted, or deleted.

* Add 'materializer' to user dictionary

The term 'materializer' has been added to the user dictionary to improve code spelling and naming consistency. This change ensures that 'materializer' is recognized as a correct term in the codebase.

* Organize test files by adding a 'Workflows' directory

Renamed 'http-workflow.json' to indicate it belongs under 'Workflows'. This improves file organization and clarity within the 'WorkflowDefinitionReload' scenario.

* Add TestWorkflowMaterializer and TestWorkflowProvider

Introduce `TestWorkflowMaterializer` for deserializing workflows from `TestWorkflowProvider`. Added integration of these new components in the `ReloadWorkflowTests` and `WorkflowServer`. Also renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests`.

* Rename and expand workflow reload tests

Renamed `RemoveReloadWorkflowTests` to `ReloadWorkflowTests` to better reflect its purpose and expanded with additional test cases. Added tests to verify workflow and activity registry updates after source provider changes and workflow reloads.

* Update copy settings for workflow test scenarios

Reorganized and added 'CopyToOutputDirectory' settings for JSON files in workflow test scenarios. Ensured all necessary files are correctly included and copied during output directory builds to maintain test consistency.
2024-07-17 09:51:45 +02:00
Mohamed Ali a0846effae
bugfixing issues related to new PropertyBag in workflow definition (#5778) 2024-07-16 14:54:20 +02:00
Sipke Schoorstra 88f657e940 Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-11 10:28:24 +02:00
Sipke Schoorstra 558486920b
Add PropertyBag for non-polymorphic data transfer between server and client (#5735)
* Fix issues with workflow deletion, newline, and DB context.

Retract published workflows before deletion to avoid exceptions. Added missing newline at EOF for consistency. Removed redundant DisposeAsync call in migration service.

* Add PropertyBag for storing custom workflow properties

Replaced `CustomProperties` with `PropertyBag` class for more structured property management across workflows. Adjusted serializations, extensions, and middleware for seamless integration with the new `PropertyBag` structure.
2024-07-10 10:29:45 +02:00
Sipke Schoorstra 27d3a4e68e Merge remote-tracking branch 'origin/patch/3.2.x' 2024-07-07 11:45:26 +02:00