Commit graph

6237 commits

Author SHA1 Message Date
Sipke Schoorstra 86826a89dc Add new Elsa logo image
Introduce a new Elsa logo image to the design assets. This image will be used in the custom 2x design section.
2024-10-03 11:16:14 +02:00
Sipke Schoorstra 108e928335 Update WebhooksCore to preview.17 and add HTTP client config
Upgraded WebhooksCore package version from 0.0.1-preview.16 to 0.0.1-preview.17. Introduced a new configurable HTTP client action in the WebhooksFeature class to support extended configurations.
2024-10-02 20:53:21 +02:00
Sipke Schoorstra bbedd61138
Implement Activity State Filtering and JavaScript Integration (#5993)
* Add secret scripting integration for JavaScript

Introduced a new `Elsa.Secrets.Scripting` module that provides secret management capabilities within JavaScript workflows. This includes configuring the Jint engine to use workflow variables, adding new type and variable definition providers, and integrating with existing secret management features.

* Refactor secret name extraction to a separate method

Moved the logic for extracting secret names from the main method to a dedicated private method `GetSecretNamesFromExpression`. This improves code readability and maintains the single responsibility principle by delegating secret name extraction to its own method.

* Add input evaluation, sensitive input handling, and middleware refactor

Introduced methods for evaluating activity input properties and handling inputs marked as sensitive. Refactored `ExecutionLogMiddleware` constructor for consistency. Enhanced `SendHttpRequestBase` to mark authorization inputs as potentially containing secrets. Removed obsolete entries and adjusted persistence logic for clarity.

* Refactor IActivityStateProtector interface

Remove unused using directives and unnecessary comments. Simplify the definition of the `ProtectedActivityStateContext` record.

* Add activity state filtering mechanism

Introduce an abstract filter base class, context, and result models to enable filtering of activity state. Implement a default filter manager to run these filters and apply a specific filter for obfuscating HTTP request headers. Update necessary dependencies and extension methods to integrate the new filtering functionality.

* Add expired secrets management

Implemented services to manage expired secrets by periodically checking and updating their status. Introduced a new hosted service to perform the sweep and configurable options for the sweep interval. Updated related classes and configurations accordingly.

* Update SweepInterval in appsettings.json

Changed the Secrets Management SweepInterval from 30 seconds to 4 hours. This adjustment aims to reduce the frequency of sweep operations and improve overall system performance.

* Update comment to reflect configuring engine with secrets

The comment was changed to better describe the handler's function, specifying that it configures the Jint engine with secrets instead of workflow variables. This clarifies the purpose and usage of the handler in the context of the code.

* Remove unused inputDescriptors variable

This commit removes the inputDescriptors variable, which was declared but never used in DefaultActivityExecutionMapper.cs. This helps in cleaning up the code and potentially reducing memory usage. Ensuring that all declared variables are utilized can improve code readability and maintainability.
2024-10-02 09:11:35 +02:00
Sipke Schoorstra fdb8061324 Update package versions to latest stable releases
Upgraded various packages to their latest stable versions to ensure compatibility and take advantage of new features and bug fixes. Notable upgrades include Microsoft.EntityFrameworkCore, Microsoft.Data.Sqlite, and Polly.
2024-09-30 14:31:51 +02:00
Sipke Schoorstra 3baacc83b3 Change WorkflowVersion property in WorkflowExecutionLogRecord
Modified the WorkflowVersion property from init-only to settable. This adjustment allows for updating the workflow version post-initialization, providing greater flexibility in handling workflow instances.
2024-09-27 17:55:47 +02:00
Sipke Schoorstra 750b4804fe Add GetOptions method to ISafeSerializer interface
This new method retrieves the JSON serializer options, enhancing flexibility in how JSON data is handled. It provides an additional way to customize serialization settings contextually.
2024-09-27 13:10:22 +02:00
Sipke Schoorstra 7d45db355d Refactor serializer options retrieval method name.
Renamed GetPayloadSerializerOptions to GetOptions for simplicity and updated method accessibility. This change ensures that the method name is more intuitive and aligns with common naming conventions. Additionally, it introduces a new public GetOptions method in the IPayloadSerializer interface.
2024-09-27 13:07:08 +02:00
Sipke Schoorstra 2e1ed995da Add synchronous serialization methods and mark async as obsolete
Implemented synchronous methods for serialization and deserialization while marking the asynchronous methods as obsolete across various storage and serialization services. This includes updates to handle serialization within the Save and Load methods, enhancing performance by avoiding unnecessary Task usage.
2024-09-27 12:59:59 +02:00
Sipke Schoorstra 7609b0e7ca Add factory method to PagedListResponse
This update introduces a static factory method `From` in the `PagedListResponse` class. It allows for the creation of `PagedListResponse` instances directly from `Page` objects, streamlining object instantiation.
2024-09-26 21:00:10 +02:00
Sipke Schoorstra 973489c961 Remove unused usings in HungryWorkflow.cs
This commit cleans up the HungryWorkflow.cs file by removing unused using directives. This improves code readability and reduces unnecessary dependencies.
2024-09-26 20:59:59 +02:00
Sipke Schoorstra 0492a7be43 Refactor to use primary constructor in Get class
Refactored the `Get` class to utilize a primary constructor for dependency injection of `IWorkflowExecutionLogStore`. This change simplifies the class definition and removes the need for a private field.
2024-09-26 14:09:28 +02:00
Sipke Schoorstra a62de77900 Update OpenTelemetry configurations and remove unused modules
Revised otel-collector and docker-compose configurations to streamline tracing and metrics collection, replacing Datadog connectors with OpenTelemetry. Removed unused scraping modules and unnecessary dependencies to improve setup efficiency and resource utilization.
2024-09-25 14:12:00 +02:00
Sipke Schoorstra 758b60767b Add OpenTelemetry auto-instrumentation support
Introduce OpenTelemetry configuration and dependencies for observability. Replace DataDog tracer with OpenTelemetry auto-instrumentation in the Dockerfile. Adjust project dependencies and log levels to incorporate OpenTelemetry without disrupting existing functionality.
2024-09-25 00:49:21 +02:00
Sipke Schoorstra df827b0fda Update Datadog config, add incident error serialization
Enhanced the Datadog tracing configuration to include priority sampling, rate limits, and sampling rules in the docker-compose-datadog.yml. Updated the agent image version and improved error message serialization in OpenTelemetry middleware to use custom JSON serializer options. Adjusted Dockerfile path to correct bundle location.
2024-09-24 00:27:51 +02:00
Sipke Schoorstra adf36927dc Update Docker and k8s configs, enhance OTEL tracing
Updated `docker-compose-datadog.yml` with new environment variables and updated image versions, enhancing trace sampling and service autodiscovery. Added Kubernetes deployment, service, and role files for `elsa-server`, `elsa-studio`, `plant-uml`, `postgres`, and `trace-lens`, improving the project's infrastructure. Enhanced OpenTelemetry middleware for better error handling and detailed tracing of activity and workflow executions.
2024-09-23 23:35:30 +02:00
Sipke Schoorstra 79b5bfc378 Add OpenTelemetry support for tracing workflow execution
This commit introduces the Elsa.OpenTelemetry module to provide OpenTelemetry sources for tracing workflow and activity execution. Additionally, it updates the program to include default workflow and activity execution pipelines with tracing middleware, and improves pipeline builder extensions to support middleware insertion.
2024-09-23 11:35:43 +02:00
Sipke Schoorstra 53172cded9 Refactor IdentityGraphService constructor and add logging.
Replaced private fields with constructor parameters in IdentityGraphService and introduced a logger. Added logging to handle cases where activity descriptors are not found, improving debugging and maintainability.
2024-09-20 15:59:35 +02:00
Sipke Schoorstra f9f8f05670 Improve exception messages for activity descriptor lookups
Enhanced exception messages to include activity type and version for better debugging. This provides clearer context when activity descriptors or inputs are not found.
2024-09-20 15:51:31 +02:00
Sipke Schoorstra a827ab241b
Fix Azure Service Bus module DI issues (#5975)
* Switch to Azure Service Bus for message brokering

Updated the message broker from in-memory to Azure Service Bus to enhance scalability. Added new configuration settings and dependencies to support Azure Service Bus integration, including scoped service initialization and queue/topic management.

* Refactor StartWorkers for dependency injection

Updated StartWorkers class to use constructor parameter injection and service scope factory for retrieving services. This refactor eliminates the need for field-level dependency storage, enhancing testability and adherence to the dependency injection principles. Additionally, annotated the class with [UsedImplicitly] to eliminate unused code warnings.

* Refactor Worker to use IServiceScopeFactory for dependency resolution

Refactored the Worker class to use IServiceScopeFactory for DI instead of directly injecting IWorkflowInbox. This enhances the flexibility and lifecycle management of the dependencies.

* Add Azure Service Bus integration toggle

Introduced a new configuration constant `useAzureServiceBus` to control the optional integration with Azure Service Bus. Implemented conditional logic to bind Azure Service Bus options from configuration if the flag is enabled.

* Add IComposite interface and implement Setup method

Introduced a new IComposite interface and added a default Setup method to the Composite class. Updated the ActivityFactory to call the Setup method when an activity is an instance of IComposite. The CompositeExample class demonstrates the use of the new interface and method.

* Remove AzureServiceBus configuration setup

The AzureServiceBus configuration setup was removed from the Program.cs file. This change aims to streamline configuration and remove unused or unnecessary setup, ensuring the code remains clean and maintainable.

* Switch MassTransit broker to in-memory

Changed the MassTransit broker from AzureServiceBus to Memory in the application configuration. This adjustment aims to simplify deployment and reduce dependencies in the current environment.
2024-09-19 20:48:09 +02:00
Sipke Schoorstra 7c31332529
Fix variable serialization (#5974)
* Improve dispatched workflow input handling

Addressed input handling in dispatch messages by adding `SerializedInput` property. Also removed initialization logic and moved input deserialization to a helper method, ensuring compatibility with both new and deprecated input property formats.

* Update workflow Docker images and version tags

Changed Docker image tags from v3-2-0-rc3 to v3-2-1-preview across multiple GitHub workflows. Updated the VERSION environment variable in packages.yml to reflect the new versioning scheme. These changes ensure consistency with the new preview release.

* Update versioning to include 'preview' in package workflow

Modified the workflow to append 'preview' to the version number for non-tagged builds. This ensures clearer differentiation between stable and non-stable versions in the CI pipeline.

* Add WorkflowInstanceStorageDriver for workflow variable storage

Introduced a new storage driver, WorkflowInstanceStorageDriver, to store workflow variables directly in the workflow state. Updated relevant classes and methods to incorporate this new storage driver, ensuring seamless read/write/delete operations and extending support for it throughout the codebase.

* Refactor object conversion and update variable retrieval.

Switched from JsonObject to JsonNode for object conversion and corrected a typo in the summary comment. Changed the return type of GetVariablesDictionary method and updated its implementation to use VariablesDictionary.

* Rename 'input' to 'serializedInput' in DispatchWorkflowDefinition.

This change clarifies that the input provided to the workflow should be serialized. It enhances the readability and accuracy of the code documentation, ensuring that developers understand the expected format of the input parameter.

* Add priority and deprecation attributes to storage drivers

Introduced a priority attribute to the `IStorageDriver` interface and implemented it in various storage drivers. Additionally, marked `WorkflowStorageDriver` as deprecated and reordered storage driver listing based on priority.

* Switch MassTransit broker to in-memory and refactor converter

Changed MassTransit broker from AzureServiceBus to in-memory for improved performance in development environment. Simplified PolymorphicObjectConverterFactory by removing redundant constructor and dependencies. Removed unused folder from the project file.
2024-09-19 16:18:56 +02:00
Marko Lahma 12f947cf9b
Upgrade Quartz and some warning fixes (#5949)
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-09-16 01:55:24 -07:00
Sipke Schoorstra 290eb9fe19 Update variable storage types and fix JavaScript handling
Changed storage driver types for 'Products' and 'Product' variables to `WorkflowInstanceStorageDriver`. Refactored JavaScript handling to properly process variables and exclude input names from overwriting. Re-enabled a previously skipped array sorting test.
2024-09-16 01:49:53 -07:00
Sipke Schoorstra 6b9079d61c Refactor Nuke script paths and update GitHub Actions
Simplify the Nuke directory structure by moving scripts to the root level. Adjust the GitHub Actions workflow to reflect the new script path. This improves script accessibility and project organization.
2024-09-15 17:17:27 -07:00
Sipke Schoorstra 28f171e472 Update workflows to main branch and preview images
Changed the workflow triggers from 'feature/secrets' branch to 'main' branch. Also updated Docker image tags from '-secrets' to '-preview' to better reflect their status.
2024-09-15 17:14:37 -07:00
Sipke Schoorstra 329a79be2e
Secrets API (#5967)
* Add initial implementation for Elsa Secrets modules

This commit introduces new projects for managing secrets within the Elsa framework: `Elsa.Secrets.Api`, `Elsa.Secrets.Core`, and `Elsa.Secrets.Management`. These projects include essential interfaces, models, entities, and endpoints to handle secret storage, retrieval, and management. Specific features include API endpoints for listing secrets, models for secret filtering, and interfaces for encryption key handling.

* Add weavers

* Refactor encryption handling in secrets management

Implemented a new architecture for handling encryption keys and algorithms within the secrets management system. Replaced old encryption key entities and related interfaces with a more modular and extensible approach. Added new services and models to improve encryption and decryption processes, enhancing maintainability and scalability.

* Add IEncryptor interface for encryption functionality

Introduced the IEncryptor interface to standardize encryption operations within the Elsa.Secrets.Management module. This interface includes the EncryptAsync method to handle encryption using a specified key ID and value.

* Add dependency on SecretsFeature and configure secrets provider

Integrate the SecretsFeature dependency and configure a secrets provider within the SecretsManagementFeature class. This adds the StoreSecretProvider to the service collection and ensures the secrets provider is correctly set up. Also, rename method from WithSecretsProvider to UseSecretsProvider for clarity.

* Add EF Core and SQLite support for Secrets module

Introduced Entity Framework Core and SQLite support for the Secrets module, including migration files, EF Core configurations, context factory, and store implementation. Added necessary extensions and configuration code to integrate with the existing API and features. Included updates to the main web application to utilize the new persistence providers.

* Update Microsoft.SemanticKernel to version 1.18.2

Upgrade Microsoft.SemanticKernel package to the latest version to ensure compatibility and new features. Remove unused Elsa.Agents.Persistence using directive from Program.cs for code cleanliness.

* Update workflows and add Agents module

Changed workflow branch targets from `main` to `feature/secrets`. Updated Docker image tags and added the `Agents` module to the Elsa Studio WebAssembly project.

* Enable agent activities in workflow configuration

This change introduces the `.UseAgentActivities()` method in the workflow configuration, enhancing the workflow capabilities. By doing so, it ensures that agent activities are appropriately integrated and available for use in the application.

* Add EF Core migrations for MySQL and SQL Server

Added Entity Framework Core migrations and related configurations to support MySQL and SQL Server for the Agents Persistence module. These changes include new migration files, context factories, and project configurations.

* Fix migration assembly reference and update method syntax

Changed the migration assembly reference in SqlServerProvidersExtensions. Updated method syntax in WorkflowManagementFeature to use array shorthand format.

* Add secret management functionalities

Introduced secret management services with CRUD operations, notifications, and bulk actions. Added unique name generation and validation for secrets, and implemented corresponding API endpoints.

* Enhance Secret Management Feature

Added Elsa.Extensions import and updated MemorySecretStore registration to use the AddMemoryStore method with Secret. This improves code modularity and adheres to the updated registration method conventions.

* Remove encryption services and update migration

Removed multiple files related to encryption services and their dependencies, including encryption algorithms and key providers. Also updated a migration script to reflect schema changes, removing specific columns and constraints.

* Implement versioning and retrieval for secrets management

Added "IsLatest" flag and cloning mechanism for secrets to support versioning. Introduced a new API endpoint for fetching decrypted secret input models. Refactored encryption and decryption logic to handle empty values gracefully.

* Add secret management functionalities

Introduced services and interfaces for secret name generation, validation, and updating. Updated secret handling to include expiration metadata. Refactored methods in ISecretManager to streamline secret creation and update processes.

* Remove DisableSyntaxSelection class and references

Deleted the DisableSyntaxSelection class and its references from various files. This includes removing its registration as a Scoped service and associated usage in the `RunJavaScript` activity.

* Add new migration for secrets and update DefaultSecretManager

Re-created migration files for V3_3 to include the ExpiresIn column. Updated DefaultSecretManager to utilize identityGenerator for generating Id and SecretId, and added additional fields like CreatedAt, UpdatedAt, and IsLatest.
2024-09-15 17:12:13 -07:00
Marko Lahma aa14e7f536
Upgrade to NUKE 8.1 (#5952)
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-09-12 09:56:06 -07:00
DnikitaA 51910810c1
Remove obsolete package MassTransit.Extensions.DependencyInjection (#5955)
Co-authored-by: Никита Данилов <ndanilov@loginet.ru>
2024-09-12 09:53:55 -07:00
Sipke Schoorstra 760a3e2cb5 Refactor context property handling with JSON serialization
Refactor property handling in ParallelForEachT to use JSON serialization for tag lists. Simplify the storage and retrieval of tags using helper methods GetTagList and SetTagList. Update storage driver types for variables to use WorkflowInstanceStorageDriver.
2024-09-06 22:45:38 +02:00
Sipke Schoorstra 48f0f72009 Enable ProtoActor and Add Agent Support
Enabled ProtoActor by setting `useProtoActor` to true. Integrated Elsa Agents API and persistence mechanisms. Refined tag handling in ParallelForEachT activity.
2024-09-06 22:45:31 +02:00
Sipke Schoorstra 2c0f5bb6b7 Update For activity to support variable passing to iterations
This commit introduces support for passing variables to iterations within the For activity. It includes a null check on the iterateNode and schedules the activity with currentValue as a variable, enhancing functionality without altering existing logic.
2024-09-06 20:18:54 +02:00
Sipke Schoorstra 21eb1034a5 Move HTTP configuration to separate section
Updated Program.cs to bind HTTP options from the "Http" section instead of "Hosting". Added the new "Http" section in appsettings.json to separate concerns and better organize the configuration settings.
2024-09-06 19:08:56 +02:00
Sipke Schoorstra 0827df7d33 Update HTTP configuration to use 'Hosting' settings
Changed the binding of HTTP options to 'Hosting' settings in Program.cs to ensure consistent configuration across the application. Removed redundant 'Http' configuration section from appsettings.json.
2024-09-06 19:06:59 +02:00
Sipke Schoorstra 83a8de6aae Change default BaseUrl port to 8080
Updated the appsettings.json to change the BaseUrl port from 5001 to 8080. This modification aligns with the new server configuration requirements.
2024-09-06 18:46:20 +02:00
Sipke Schoorstra 5b583f45d6 Update base URL retrieval to use configuration setting
Replaced hardcoded base URL construction with a configuration-based approach for better management and flexibility. Updated `appsettings.json` to include the new `BaseUrl` setting under the `Hosting` section.
2024-09-06 18:40:40 +02:00
Sipke Schoorstra b8083193bd Update GitHub workflows to track 'main' branch and new Docker tags
Switch workflows from tracking 'patch/3.2.x' branch to 'main', and update Docker image tags from 'v3-2-0-rc3' to 'v3-3-0-preview'. This ensures the latest changes are correctly integrated and deployed.
2024-09-06 18:06:16 +02:00
Sipke Schoorstra d6c14d9878
Simplify Workflow Variables with JS (#5946)
* Add variable support and engine configuration for JavaScript

Implemented handling of workflow variables in JavaScript expressions, including new handlers, notifications, and variable definitions. Enhanced type definition services and providers to include variable definitions, updated dependency injections, and applied modifications for improved backend API configuration.

* Add ObjectConverterHelper for JS object conversion

Implemented ObjectConverterHelper to convert .NET objects to JavaScript objects in EvaluateJavaScript context. Updated ConfigureEngineWithVariables handler to process and convert variables using the new helper utility.

* Add Customer and Order models and update Program.cs

Created new Customer and Order model classes in the Models namespace. Updated Program.cs to include and alias these models for use in the application.

* Add two new activities and integration test

Introduced `Activity1` and `Activity2` under `src/apps/Elsa.Server.Web/Activities`. Additionally, created a new integration test `VariablesInteropTests` to validate JavaScript variable modifications within workflows.

* Refactor to use IBookmarkQueue instead of IBookmarkResumer

Replaced IBookmarkResumer with IBookmarkQueue in various classes for enqueueing bookmark queue items. Added logging for better traceability and included additional helper imports for activity type name generation.

* Add correlationId tag to OpenTelemetry tracing

This change adds a correlationId tag to the tracing for workflow executions if the context contains a correlationId. This enhancement improves traceability and correlation across distributed systems.

* Set Correlation ID header in MassTransit messages

Added logic to set the "X-Correlation-ID" header in MassTransit messages if the CorrelationId is present. This ensures that the messages can be correlated properly across different parts of the system.

* Reduce logging verbosity in appsettings.json

Removed detailed debug logs for various Elsa workflows and middleware components from the appsettings.json. This change aims to streamline the log outputs, focusing on warnings and critical information to improve readability and debug efficiency.

* Add OpenTelemetry.Api package version 1.9.0

Include OpenTelemetry.Api to list of package versions in Directory.Packages.props. This addition aims to enhance application monitoring and observability.

* Add JavaScript variable handling integration test

Introduced integration tests for JavaScript activities to verify they can access and modify native variables. Added classes for data setup, test execution, and workflow definition with corresponding NUnit tests.

* Remove unused activities and models

Deleted several unused activity classes, models, and middleware to simplify the codebase. This cleanup helps reduce code complexity and improves maintainability. Updated Program.cs to reflect these deletions.

* Remove correlation ID header setting from dispatch

Simplified the workflow dispatching process by removing the redundant setting of the X-Correlation-ID header in two places. This change should improve code readability and maintainability.

* Format code block consistently

Corrected the indentation of the code block for better readability and consistency. This ensures all properties in the 'DispatchWorkflowInstance' initialization are properly aligned. No functional changes were made in this commit.

* Remove VariablesInteropTests.cs from integration tests

Deleted the VariablesInteropTests.cs file which contained a single test method testing JavaScript-to-JSON serialization. This cleanup removes unnecessary test code from the repository.
2024-09-06 18:03:47 +02:00
jdevillard 2f584bb730
Merge pull request #5947 from jdevillard/chore/update_nuget_artefact_action
[chore] Update upload-artifact major version for CI
2024-09-06 10:03:31 +02:00
Jérémie DEVILLARD d113e786aa chore: update upload-artifact major version after change by dependabot (#5941) to be compatible 2024-09-06 09:23:54 +02:00
RenatoCapelo c24dca1ba6
Enh/5943 DropIns - Prevention of File Locks and Unloading (#5944)
* fix - supressed warning IL2072
Changed DropInDescriptor from Record to Class to be able to add the suggested DynamicallyAccesedMembers decorator.

* Add Unconfigure method to DropIn class and interface

The `DropIn` class in `DropIn.cs` now includes a new `Unconfigure` method. This method handles the unconfiguration process when a drop-in is deleted. It retrieves the `ILogger<DropIn>` and `IActivityRegistry` services from the `IServiceProvider`, removes the `SampleActivity` from the activity registry, and logs the outcome.

The `IDropIn` interface in `IDropIn.cs` has been updated to include the new `Unconfigure` method, ensuring that it is called when the drop-in is deleted.

Additional using directives have been added to `DropIn.cs` for `Elsa.Workflows`, and `Microsoft.Extensions.Logging`.

* Refactor AssemblyLoader to use memory stream for loading

Modified the `LoadPath` method in the `AssemblyLoader` class to load assemblies from a memory stream instead of directly from the file. This change prevents file locking issues by copying the file contents to a memory stream and then loading the assembly from this stream.

* Dispose packageReader after loading assembly

Ensure the packageReader is disposed of after loading the assembly from the memory stream. This change adds a call to `packageReader.Dispose()` to close the package reader and the associated `.nupkg` file, which helps in releasing resources and preventing potential file locks or memory leaks.

* Add drop-in unloading on file deletion

Updated DropInDirectoryMonitorHostedService to handle unloading of drop-ins when files are deleted. Added `_debouncedUnloader` and `_installedDropIns` fields. Modified constructor to initialize these fields. Updated `ExecuteAsync` to call `LoadDropInAssemblyAsync` initially and handle file deletion events. Added `OnDeleted` method to manage file deletions and invoke the debounced unloader. Enhanced `LoadDropInAssemblyAsync` to track loaded drop-ins. Introduced `UnloadDropInAssemblyAsync` to remove drop-ins on file deletion.

* Load assembly from MemoryStream instead of FileStream

Modified the assembly loading process to use a MemoryStream
instead of a FileStream. This change ensures that the
FileStream is properly closed before the assembly is loaded,
improving resource management and preventing potential file
access issues.

* Improve logging and error handling in DropIn services

Removed logging from DropIn.Unconfigure method and added
ILogger dependency to DropInDirectoryMonitorHostedService.
Updated csproj to include Microsoft.Extensions.Logging.Abstractions.
Enhanced error handling in UnloadDropInAssemblyAsync method.
Minor formatting improvements for readability.
2024-09-05 18:09:31 +02:00
jdevillard 0c6cc6b062
Change to backendConfig after update of AddRemoteBackend properties (#5945)
Co-authored-by: Jérémie DEVILLARD <jdevillard@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
2024-09-05 17:33:16 +02:00
dependabot[bot] 537899e2f9
Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows (#5941)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 3 to 4.1.7.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v3...v4.1.7)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-05 13:41:26 +02:00
Sipke Schoorstra 05c432267b Correct punctuation in appsettings.json
Added a missing period at the end of the FactChecker persona description. Ensures consistency in text descriptions across appsettings.json configuration.
2024-09-01 21:44:25 +02:00
Sipke Schoorstra 2b64ec7d90 Update Elsa.Studio packages to version 3.3.0-preview.524
Upgraded Elsa.Studio, Elsa.Studio.Core.BlazorWasm, and Elsa.Studio.Login.BlazorWasm packages from version 3.3.0-preview.508 to 3.3.0-preview.524. Ensures we are using the latest preview features and fixes.
2024-09-01 20:32:33 +02:00
Sipke Schoorstra 3acb1fd055 Update agent management features and refine configuration
Renamed methods and namespaces for clarity in `AgentsFeature` and extensions. Enhanced functionality of SEO Specialist and added new role for social media in sample app settings. Re-enabled persistence and API usage in Elsa server configuration.
2024-09-01 18:50:28 +02:00
Sipke Schoorstra 1940938021 Change ActivityKind from Task to Job
Updated the `Kind` property of `activityDescriptor` to `Job` instead of `Task`. This change ensures the correct categorization and functionality within the agent activity provider.
2024-09-01 14:40:39 +02:00
Sipke Schoorstra 223536c90a Remove unused using statements and configure agents.
Removed several unused `using` statements within multiple files to clean up the codebase. Additionally, enhanced the configuration for the `Agent` module in `appsettings.json` and enabled agent-related features in `Program.cs`.
2024-09-01 14:24:43 +02:00
Sipke Schoorstra 92c1e9a582 Refine prompt and description in appsettings.json
Update the PromptTemplate to focus on text interpretation and simplify the example JSON format. Adjust the OutputVariable description to specify it as a URL to the created artwork.
2024-09-01 09:36:15 +02:00
Sipke Schoorstra 2d62a373e0
Add Agents Module with Semantic Kernel Integration (#5937)
* Remove `ServiceProfileConfig` and `SkillConfig`

Refactored the codebase to eliminate `ServiceProfileConfig` and `SkillConfig` in favor of simplifying configurations. Updated related entities and settings to consistently use `ServiceConfig` and `AgentConfig`. Refined function execution to enhance overall system coherence and maintainability.

* Refactor agent management and plugin system

This update eliminates AgentManager and related classes, introducing new management interfaces and memory-based stores. The plugin system is refactored with better separation of concerns and the addition of provider interfaces.

* Add new Agents features and services

Introduced AgentsFeature for API endpoints, AgentManagementFeature for agent management, and AgentActivitiesFeature for activities. Updated service configurations and migrated files for cleaner and more maintainable code structure.

* Rename execute endpoints to invoke

Renamed the 'Execute' endpoints and related paths to 'Invoke' for better semantic clarity. Adjusted the namespace and updated the route configuration accordingly to reflect this change.

* Add permissions configuration to Agents Invoke endpoint

This commit adds a call to `ConfigurePermissions("agents:invoke")` in the Agents Invoke endpoint configuration. This change ensures that proper permissions are checked when this endpoint is accessed.

* Rename Management to Persistence with EF Core support

Renamed various files and classes from Management to Persistence. Introduced Entity Framework Core (EF Core) support for agents, including implementation of APIs for API key, service, and agent definitions. The changes also updated project configurations and added necessary database context and entity configurations.

* Add EntityFrameworkCore project for Agents persistence

Introduce a new Entity Framework Core provider to manage the persistence layer of the Agents module. The project includes references to the base persistence and common Entity Framework projects.

* Remove redundant exception handlers and add new SQLite migration.

Deleted unnecessary `DbExceptionTransformer` and `RethrowDbExceptionHandler` classes to streamline error handling. Introduced a new SQLite migration for the Agents module to create necessary database tables and indices. Also modified `Store` and `EntityStore` to utilize a service provider for exception handling.

* Add Sqlite configuration extension and sample updates

Introduce a new extension method for configuring EF Core to use Sqlite in the Elsa.Agents module. Updated the sample project to use a configurable Sqlite connection string and adjusted the appsettings.json accordingly. Also added necessary folder structure in the project file.

* Add agent, service, and API key CRUD operations

Introduced CRUD functionalities for agent, service, and API key entities. Added endpoint handlers, filtering capabilities, and memory/EF core stores to support these operations. Updated configuration mappings as well.

* Add Agent API endpoints and DTOs

Added CRUD endpoints for agents in Elsa.Agents.Api along with the required DTOs and Request models. These endpoints include create, read, update, delete, and list functionalities, facilitating the management of agent entities.

* Add support for agent service providers and discovery

Introduce interfaces and implementations for agent service providers and service discovery. Updated `KernelFactory` to integrate the new service discovery mechanism and restructured plugin discoverer naming for consistency.

* Add agent management features

Introduce agent management components, including pages for listing and handling agent workflows, dialogs for creating agents with Blazor components, and backend API endpoints for generating unique agent names. This enhances functionality for managing agent workflows within the application.

* Refactor exception handling and simplify event handling logic

Replaced DataProcessingDbExceptionHandler with DbExceptionTransformer for handling PostgreSQL exceptions. Streamlined event completion logic by renaming method and ensuring input retrieval before setting the result.

* Add bulk delete endpoint for agents

Introduced new API endpoint for bulk deleting agents. Implemented request and response models, and updated persistence layer to handle bulk operations. Also reorganized project structure for better modularity.

* Refactor API Key models and endpoint classes

Consolidate API Key request models into unified models and update endpoints to use these new models. Add new transformation methods to entity classes. Enhance configuration classes and streamline the dependency injection process.

* Refactor and streamline bulk delete and service APIs

Refactored the bulk delete endpoints for API keys, services, and agents, consolidating request and response models. Added new service input and model classes and updated the corresponding stores to support bulk deletions. Simplified the endpoint logic to use these new models and enhanced filtering capabilities.

* Set default values for AgentInputModel properties

Updated properties in `AgentInputModel` to use initialized default values instead of `default!`. This change ensures that string and object properties are instantiated properly, improving code stability and preventing potential null reference exceptions.

* Add plugin listing endpoint and related models

Introduced a new endpoint to list all registered plugins in the system. Added necessary models and methods for plugin description and retrieval, along with appropriate documentation comments.

* Refactor agent management to use `IAgentManager`.

Created new notifications for agent lifecycle events and introduced an `AgentManager` service for agent CRUD operations. Updated APIs and other components to use the new `IAgentManager` service instead of direct store access for a more cohesive architecture.

* Update agent activity descriptor names

Changed the activity descriptor names to use PascalCase for consistency and readability. Also added display names for input variables to improve user interfaces.

* Add overloaded RefreshDescriptorsAsync for single activity provider

Introduced a new overloaded method in IActivityRegistry and ActivityRegistry to refresh activity descriptors using a single IActivityProvider. Updated RefreshActivityRegistry to use the new method, ensuring more flexibility in descriptor refresh operations.
2024-09-01 09:03:14 +02:00
Sipke Schoorstra 9e421a166d Merge remote-tracking branch 'origin/patch/3.2.x' 2024-08-30 12:23:38 +02:00
rosca-sabina 18b0d2e70a
Add alteration logs to workflow execution logs (#5897) (#5911) 2024-08-30 12:02:41 +02:00