* Add initial implementation for Elsa Secrets modules
This commit introduces new projects for managing secrets within the Elsa framework: `Elsa.Secrets.Api`, `Elsa.Secrets.Core`, and `Elsa.Secrets.Management`. These projects include essential interfaces, models, entities, and endpoints to handle secret storage, retrieval, and management. Specific features include API endpoints for listing secrets, models for secret filtering, and interfaces for encryption key handling.
* Add weavers
* Refactor encryption handling in secrets management
Implemented a new architecture for handling encryption keys and algorithms within the secrets management system. Replaced old encryption key entities and related interfaces with a more modular and extensible approach. Added new services and models to improve encryption and decryption processes, enhancing maintainability and scalability.
* Add IEncryptor interface for encryption functionality
Introduced the IEncryptor interface to standardize encryption operations within the Elsa.Secrets.Management module. This interface includes the EncryptAsync method to handle encryption using a specified key ID and value.
* Add dependency on SecretsFeature and configure secrets provider
Integrate the SecretsFeature dependency and configure a secrets provider within the SecretsManagementFeature class. This adds the StoreSecretProvider to the service collection and ensures the secrets provider is correctly set up. Also, rename method from WithSecretsProvider to UseSecretsProvider for clarity.
* Add EF Core and SQLite support for Secrets module
Introduced Entity Framework Core and SQLite support for the Secrets module, including migration files, EF Core configurations, context factory, and store implementation. Added necessary extensions and configuration code to integrate with the existing API and features. Included updates to the main web application to utilize the new persistence providers.
* Update Microsoft.SemanticKernel to version 1.18.2
Upgrade Microsoft.SemanticKernel package to the latest version to ensure compatibility and new features. Remove unused Elsa.Agents.Persistence using directive from Program.cs for code cleanliness.
* Update workflows and add Agents module
Changed workflow branch targets from `main` to `feature/secrets`. Updated Docker image tags and added the `Agents` module to the Elsa Studio WebAssembly project.
* Enable agent activities in workflow configuration
This change introduces the `.UseAgentActivities()` method in the workflow configuration, enhancing the workflow capabilities. By doing so, it ensures that agent activities are appropriately integrated and available for use in the application.
* Add EF Core migrations for MySQL and SQL Server
Added Entity Framework Core migrations and related configurations to support MySQL and SQL Server for the Agents Persistence module. These changes include new migration files, context factories, and project configurations.
* Fix migration assembly reference and update method syntax
Changed the migration assembly reference in SqlServerProvidersExtensions. Updated method syntax in WorkflowManagementFeature to use array shorthand format.
* Add secret management functionalities
Introduced secret management services with CRUD operations, notifications, and bulk actions. Added unique name generation and validation for secrets, and implemented corresponding API endpoints.
* Enhance Secret Management Feature
Added Elsa.Extensions import and updated MemorySecretStore registration to use the AddMemoryStore method with Secret. This improves code modularity and adheres to the updated registration method conventions.
* Remove encryption services and update migration
Removed multiple files related to encryption services and their dependencies, including encryption algorithms and key providers. Also updated a migration script to reflect schema changes, removing specific columns and constraints.
* Implement versioning and retrieval for secrets management
Added "IsLatest" flag and cloning mechanism for secrets to support versioning. Introduced a new API endpoint for fetching decrypted secret input models. Refactored encryption and decryption logic to handle empty values gracefully.
* Add secret management functionalities
Introduced services and interfaces for secret name generation, validation, and updating. Updated secret handling to include expiration metadata. Refactored methods in ISecretManager to streamline secret creation and update processes.
* Remove DisableSyntaxSelection class and references
Deleted the DisableSyntaxSelection class and its references from various files. This includes removing its registration as a Scoped service and associated usage in the `RunJavaScript` activity.
* Add new migration for secrets and update DefaultSecretManager
Re-created migration files for V3_3 to include the ExpiresIn column. Updated DefaultSecretManager to utilize identityGenerator for generating Id and SecretId, and added additional fields like CreatedAt, UpdatedAt, and IsLatest.
Refactor property handling in ParallelForEachT to use JSON serialization for tag lists. Simplify the storage and retrieval of tags using helper methods GetTagList and SetTagList. Update storage driver types for variables to use WorkflowInstanceStorageDriver.
Enabled ProtoActor by setting `useProtoActor` to true. Integrated Elsa Agents API and persistence mechanisms. Refined tag handling in ParallelForEachT activity.
This commit introduces support for passing variables to iterations within the For activity. It includes a null check on the iterateNode and schedules the activity with currentValue as a variable, enhancing functionality without altering existing logic.
Updated Program.cs to bind HTTP options from the "Http" section instead of "Hosting". Added the new "Http" section in appsettings.json to separate concerns and better organize the configuration settings.
Changed the binding of HTTP options to 'Hosting' settings in Program.cs to ensure consistent configuration across the application. Removed redundant 'Http' configuration section from appsettings.json.
Replaced hardcoded base URL construction with a configuration-based approach for better management and flexibility. Updated `appsettings.json` to include the new `BaseUrl` setting under the `Hosting` section.
Switch workflows from tracking 'patch/3.2.x' branch to 'main', and update Docker image tags from 'v3-2-0-rc3' to 'v3-3-0-preview'. This ensures the latest changes are correctly integrated and deployed.
* Add variable support and engine configuration for JavaScript
Implemented handling of workflow variables in JavaScript expressions, including new handlers, notifications, and variable definitions. Enhanced type definition services and providers to include variable definitions, updated dependency injections, and applied modifications for improved backend API configuration.
* Add ObjectConverterHelper for JS object conversion
Implemented ObjectConverterHelper to convert .NET objects to JavaScript objects in EvaluateJavaScript context. Updated ConfigureEngineWithVariables handler to process and convert variables using the new helper utility.
* Add Customer and Order models and update Program.cs
Created new Customer and Order model classes in the Models namespace. Updated Program.cs to include and alias these models for use in the application.
* Add two new activities and integration test
Introduced `Activity1` and `Activity2` under `src/apps/Elsa.Server.Web/Activities`. Additionally, created a new integration test `VariablesInteropTests` to validate JavaScript variable modifications within workflows.
* Refactor to use IBookmarkQueue instead of IBookmarkResumer
Replaced IBookmarkResumer with IBookmarkQueue in various classes for enqueueing bookmark queue items. Added logging for better traceability and included additional helper imports for activity type name generation.
* Add correlationId tag to OpenTelemetry tracing
This change adds a correlationId tag to the tracing for workflow executions if the context contains a correlationId. This enhancement improves traceability and correlation across distributed systems.
* Set Correlation ID header in MassTransit messages
Added logic to set the "X-Correlation-ID" header in MassTransit messages if the CorrelationId is present. This ensures that the messages can be correlated properly across different parts of the system.
* Reduce logging verbosity in appsettings.json
Removed detailed debug logs for various Elsa workflows and middleware components from the appsettings.json. This change aims to streamline the log outputs, focusing on warnings and critical information to improve readability and debug efficiency.
* Add OpenTelemetry.Api package version 1.9.0
Include OpenTelemetry.Api to list of package versions in Directory.Packages.props. This addition aims to enhance application monitoring and observability.
* Add JavaScript variable handling integration test
Introduced integration tests for JavaScript activities to verify they can access and modify native variables. Added classes for data setup, test execution, and workflow definition with corresponding NUnit tests.
* Remove unused activities and models
Deleted several unused activity classes, models, and middleware to simplify the codebase. This cleanup helps reduce code complexity and improves maintainability. Updated Program.cs to reflect these deletions.
* Remove correlation ID header setting from dispatch
Simplified the workflow dispatching process by removing the redundant setting of the X-Correlation-ID header in two places. This change should improve code readability and maintainability.
* Format code block consistently
Corrected the indentation of the code block for better readability and consistency. This ensures all properties in the 'DispatchWorkflowInstance' initialization are properly aligned. No functional changes were made in this commit.
* Remove VariablesInteropTests.cs from integration tests
Deleted the VariablesInteropTests.cs file which contained a single test method testing JavaScript-to-JSON serialization. This cleanup removes unnecessary test code from the repository.
* fix - supressed warning IL2072
Changed DropInDescriptor from Record to Class to be able to add the suggested DynamicallyAccesedMembers decorator.
* Add Unconfigure method to DropIn class and interface
The `DropIn` class in `DropIn.cs` now includes a new `Unconfigure` method. This method handles the unconfiguration process when a drop-in is deleted. It retrieves the `ILogger<DropIn>` and `IActivityRegistry` services from the `IServiceProvider`, removes the `SampleActivity` from the activity registry, and logs the outcome.
The `IDropIn` interface in `IDropIn.cs` has been updated to include the new `Unconfigure` method, ensuring that it is called when the drop-in is deleted.
Additional using directives have been added to `DropIn.cs` for `Elsa.Workflows`, and `Microsoft.Extensions.Logging`.
* Refactor AssemblyLoader to use memory stream for loading
Modified the `LoadPath` method in the `AssemblyLoader` class to load assemblies from a memory stream instead of directly from the file. This change prevents file locking issues by copying the file contents to a memory stream and then loading the assembly from this stream.
* Dispose packageReader after loading assembly
Ensure the packageReader is disposed of after loading the assembly from the memory stream. This change adds a call to `packageReader.Dispose()` to close the package reader and the associated `.nupkg` file, which helps in releasing resources and preventing potential file locks or memory leaks.
* Add drop-in unloading on file deletion
Updated DropInDirectoryMonitorHostedService to handle unloading of drop-ins when files are deleted. Added `_debouncedUnloader` and `_installedDropIns` fields. Modified constructor to initialize these fields. Updated `ExecuteAsync` to call `LoadDropInAssemblyAsync` initially and handle file deletion events. Added `OnDeleted` method to manage file deletions and invoke the debounced unloader. Enhanced `LoadDropInAssemblyAsync` to track loaded drop-ins. Introduced `UnloadDropInAssemblyAsync` to remove drop-ins on file deletion.
* Load assembly from MemoryStream instead of FileStream
Modified the assembly loading process to use a MemoryStream
instead of a FileStream. This change ensures that the
FileStream is properly closed before the assembly is loaded,
improving resource management and preventing potential file
access issues.
* Improve logging and error handling in DropIn services
Removed logging from DropIn.Unconfigure method and added
ILogger dependency to DropInDirectoryMonitorHostedService.
Updated csproj to include Microsoft.Extensions.Logging.Abstractions.
Enhanced error handling in UnloadDropInAssemblyAsync method.
Minor formatting improvements for readability.
Added a missing period at the end of the FactChecker persona description. Ensures consistency in text descriptions across appsettings.json configuration.
Upgraded Elsa.Studio, Elsa.Studio.Core.BlazorWasm, and Elsa.Studio.Login.BlazorWasm packages from version 3.3.0-preview.508 to 3.3.0-preview.524. Ensures we are using the latest preview features and fixes.
Renamed methods and namespaces for clarity in `AgentsFeature` and extensions. Enhanced functionality of SEO Specialist and added new role for social media in sample app settings. Re-enabled persistence and API usage in Elsa server configuration.
Updated the `Kind` property of `activityDescriptor` to `Job` instead of `Task`. This change ensures the correct categorization and functionality within the agent activity provider.
Removed several unused `using` statements within multiple files to clean up the codebase. Additionally, enhanced the configuration for the `Agent` module in `appsettings.json` and enabled agent-related features in `Program.cs`.
Update the PromptTemplate to focus on text interpretation and simplify the example JSON format. Adjust the OutputVariable description to specify it as a URL to the created artwork.
* Remove `ServiceProfileConfig` and `SkillConfig`
Refactored the codebase to eliminate `ServiceProfileConfig` and `SkillConfig` in favor of simplifying configurations. Updated related entities and settings to consistently use `ServiceConfig` and `AgentConfig`. Refined function execution to enhance overall system coherence and maintainability.
* Refactor agent management and plugin system
This update eliminates AgentManager and related classes, introducing new management interfaces and memory-based stores. The plugin system is refactored with better separation of concerns and the addition of provider interfaces.
* Add new Agents features and services
Introduced AgentsFeature for API endpoints, AgentManagementFeature for agent management, and AgentActivitiesFeature for activities. Updated service configurations and migrated files for cleaner and more maintainable code structure.
* Rename execute endpoints to invoke
Renamed the 'Execute' endpoints and related paths to 'Invoke' for better semantic clarity. Adjusted the namespace and updated the route configuration accordingly to reflect this change.
* Add permissions configuration to Agents Invoke endpoint
This commit adds a call to `ConfigurePermissions("agents:invoke")` in the Agents Invoke endpoint configuration. This change ensures that proper permissions are checked when this endpoint is accessed.
* Rename Management to Persistence with EF Core support
Renamed various files and classes from Management to Persistence. Introduced Entity Framework Core (EF Core) support for agents, including implementation of APIs for API key, service, and agent definitions. The changes also updated project configurations and added necessary database context and entity configurations.
* Add EntityFrameworkCore project for Agents persistence
Introduce a new Entity Framework Core provider to manage the persistence layer of the Agents module. The project includes references to the base persistence and common Entity Framework projects.
* Remove redundant exception handlers and add new SQLite migration.
Deleted unnecessary `DbExceptionTransformer` and `RethrowDbExceptionHandler` classes to streamline error handling. Introduced a new SQLite migration for the Agents module to create necessary database tables and indices. Also modified `Store` and `EntityStore` to utilize a service provider for exception handling.
* Add Sqlite configuration extension and sample updates
Introduce a new extension method for configuring EF Core to use Sqlite in the Elsa.Agents module. Updated the sample project to use a configurable Sqlite connection string and adjusted the appsettings.json accordingly. Also added necessary folder structure in the project file.
* Add agent, service, and API key CRUD operations
Introduced CRUD functionalities for agent, service, and API key entities. Added endpoint handlers, filtering capabilities, and memory/EF core stores to support these operations. Updated configuration mappings as well.
* Add Agent API endpoints and DTOs
Added CRUD endpoints for agents in Elsa.Agents.Api along with the required DTOs and Request models. These endpoints include create, read, update, delete, and list functionalities, facilitating the management of agent entities.
* Add support for agent service providers and discovery
Introduce interfaces and implementations for agent service providers and service discovery. Updated `KernelFactory` to integrate the new service discovery mechanism and restructured plugin discoverer naming for consistency.
* Add agent management features
Introduce agent management components, including pages for listing and handling agent workflows, dialogs for creating agents with Blazor components, and backend API endpoints for generating unique agent names. This enhances functionality for managing agent workflows within the application.
* Refactor exception handling and simplify event handling logic
Replaced DataProcessingDbExceptionHandler with DbExceptionTransformer for handling PostgreSQL exceptions. Streamlined event completion logic by renaming method and ensuring input retrieval before setting the result.
* Add bulk delete endpoint for agents
Introduced new API endpoint for bulk deleting agents. Implemented request and response models, and updated persistence layer to handle bulk operations. Also reorganized project structure for better modularity.
* Refactor API Key models and endpoint classes
Consolidate API Key request models into unified models and update endpoints to use these new models. Add new transformation methods to entity classes. Enhance configuration classes and streamline the dependency injection process.
* Refactor and streamline bulk delete and service APIs
Refactored the bulk delete endpoints for API keys, services, and agents, consolidating request and response models. Added new service input and model classes and updated the corresponding stores to support bulk deletions. Simplified the endpoint logic to use these new models and enhanced filtering capabilities.
* Set default values for AgentInputModel properties
Updated properties in `AgentInputModel` to use initialized default values instead of `default!`. This change ensures that string and object properties are instantiated properly, improving code stability and preventing potential null reference exceptions.
* Add plugin listing endpoint and related models
Introduced a new endpoint to list all registered plugins in the system. Added necessary models and methods for plugin description and retrieval, along with appropriate documentation comments.
* Refactor agent management to use `IAgentManager`.
Created new notifications for agent lifecycle events and introduced an `AgentManager` service for agent CRUD operations. Updated APIs and other components to use the new `IAgentManager` service instead of direct store access for a more cohesive architecture.
* Update agent activity descriptor names
Changed the activity descriptor names to use PascalCase for consistency and readability. Also added display names for input variables to improve user interfaces.
* Add overloaded RefreshDescriptorsAsync for single activity provider
Introduced a new overloaded method in IActivityRegistry and ActivityRegistry to refresh activity descriptors using a single IActivityProvider. Updated RefreshActivityRegistry to use the new method, ensuring more flexibility in descriptor refresh operations.
* Sample to provide a ExpressionDescriptorProvider
* fix variable accessor in VariableExpressionHandler
* remove Variable Expression to use default one
* Change the Variable Expression Provider to handle Variable Expression correctly
* need to check on UI Side and Server side is valid JSON is sent. For example, if a use change from a literal value to a Variable value, the value is sent to the API and result in an exception, we cannot handle invalid json without using Try/Catch
---------
Co-authored-by: Jérémie DEVILLARD <jdevillard@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* Set event payload when event bookmark resumes
* Refactor to move common code into a function
* Typo in event input workflow key
---------
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Refactor DataProcessingException to remove unique key logic and create a dedicated UniqueKeyConstraintViolationException. Update DbExceptionTransformer to use the new exception for unique key violations, improving code clarity and separation of concerns.
Removed the `ConverterHelper` and `ExpandoObjectConverter` classes, along with their integration in `JintJavaScriptEvaluator`. Updated unit tests to skip unsupported functionality and added references to related issues for context.
Changed the fallback version in the GitHub Actions packages workflow from 3.2.0-rc5 to 3.2.0-rc6. This ensures that the workflows use the updated release candidate version for deployments.
Upgraded Elsa.Studio, Elsa.Studio.Core.BlazorWasm, and Elsa.Studio.Login.BlazorWasm to the latest preview version 3.3.0-preview.508. This ensures we are using the most recent features and fixes available in the preview release.
Reorder parameters in StartWorkflowAsync and ResumeWorkflowAsync for consistency. Update FindBookmarksAsync and ExecuteWorkflowAsync to include workflowInstanceId and correlationId, improving workflow lookup and execution logic.
* Modified POST execute API to show body in Swagger and resolved type-casting error in GET execute API
Modified the POST execute API to display the body in Swagger documentation and resolved the type-casting error in the GET execute API
* Added IExecutionRequest interface defines a contract for Request classes
* Moved the serialization logic into request classes
* Add missing import and reformat ActivityHandle property
Included the Elsa.Workflows.Models import in Models.cs to fix missing reference. Removed redundant ActivityHandle assignment in EndpointBase.cs to improve clarity and reduce redundancy.
* Added support to retrieve the workflow details(i.e., WorkflowDefinitionId, WorkflowDefinitionVersionId, WorkflowDefinitionVersion, WorkflowInstanceId) through Liquid script
---------
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
Introduce tests for resuming specific HTTP workflow instances. Enhanced `HttpWorkflowsMiddleware` to handle workflow instance IDs and correlation IDs for more precise control over workflow activation and resumption.
* Modified POST execute API to show body in Swagger and resolved type-casting error in GET execute API
Modified the POST execute API to display the body in Swagger documentation and resolved the type-casting error in the GET execute API
* Added IExecutionRequest interface defines a contract for Request classes
* Moved the serialization logic into request classes
* Add missing import and reformat ActivityHandle property
Included the Elsa.Workflows.Models import in Models.cs to fix missing reference. Removed redundant ActivityHandle assignment in EndpointBase.cs to improve clarity and reduce redundancy.
---------
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
Included Kubernetes YAML files for elsa-server, postgres, plant-uml, and trace-lens projects in the solution. This update ensures that essential configuration files for deployments, services, and roles are managed within the project structure.
Corrected paths in build.sh and build.ps1 to reference correct directories. Added "nuke" project to Elsa.sln for organizing build scripts under solution items.
Removed the 'nuke' project entry from the solution file and adjusted related NUKE tool directory paths within the build project file. These changes streamline the configuration and ensure the NUKE scripts are correctly located in the new directory structure.
Changed the `NukeRootDirectory` and `NukeScriptDirectory` paths to ensure correct directory structure. This update aims to resolve any discrepancies in the build process by pointing to the appropriate directories.
Changed the NukeScriptDirectory path to point one level up to streamline the directory structure. This minor adjustment helps in maintaining consistency across the project configuration.
Adjusted the indentation for the Release configuration of project {99F2B1DA-2F69-4D70-A2A3-AC985AD91EC4} in the solution file Elsa.sln. This change ensures proper alignment and readability of the project's configuration settings.
Refactor InputFunctionsDefinitionProvider to remove unused parameter. Simplify JintJavaScriptEvaluator object wrapping configuration and update Jint package to version 4.0.1. Add type Random to ConfigureEngineWithCommonTypes and enhance error handling in TaskReporter.
