Commit graph

76 commits

Author SHA1 Message Date
Sipke Schoorstra b854c0272a
Remove unused Docker services, volumes, and solution references.
Removed unnecessary Docker services, configuration files, and volumes from `docker-compose.yml`. Cleaned up unused project references in `Elsa.sln`. Streamlined local development setup by eliminating redundant resources.
2025-12-19 19:31:01 +01:00
Sipke Schoorstra 068b2079cc
Update Dockerfiles to remove "bookworm-slim" tag from .NET SDK and ASP.NET runtime images 2025-12-10 21:44:48 +01:00
Sipke Schoorstra cd0cd10fb1
Update Dockerfiles to use .NET SDK and ASP.NET runtime 10.0-bookworm-slim 2025-12-10 21:42:07 +01:00
Sipke Schoorstra f24b4394cf
Add WorkflowStateCommitted notification support and update state handling logic
- Introduced `WorkflowStateCommitted` notification to encapsulate workflow execution context, state, and instance details.
- Updated `DefaultCommitStateHandler` to publish `WorkflowStateCommitted` via `IMediator`.
- Adjusted `DispatchWorkflowExtensions` to use `WorkflowStateCommitted` for workflow completion.

Updates KubernetesClient and Microsoft packages (#6917)

* Remove Proto.Cluster.Kubernetes dependency due to vulnerability

- Temporarily removed `Proto.Cluster.Kubernetes` package and provider integration because of a vulnerability in its dependency (https://avd.aquasec.com/nvd/2025/cve-2025-9708).
- Adjusted related cluster provider and remote configuration logic.
- Updated `PortAttribute` default parameter for clarity.

* Revert "Remove Proto.Cluster.Kubernetes dependency due to vulnerability"

This reverts commit 0720d970968e4f7338825407258b34ddffb1d2a4.

* Add KubernetesClient package and update MicrosoftVersion to 9.0.9

- Added `KubernetesClient` package to the project dependencies.
- Updated `MicrosoftVersion` to `9.0.9` in `Directory.Packages.props`.
Update Polly packages

- Bump Polly and Polly.Extensions package versions to 8.6.3.

Update `Microsoft.AspNetCore.Authorization` to use `MicrosoftVersion` property

Ensure Docker images ship CA trust and add TLS smoke tests (#6918)

Remove TlsSmoke project and related solution references

- Deleted `TlsSmoke` project files (`Program.cs` and `TlsSmoke.csproj`).
- Removed `TlsSmoke` project reference from the solution file (`Elsa.sln`).

Add comprehensive Copilot coding agent instructions for repository onboarding (#6920)

* Initial plan

* Add comprehensive .github/copilot-instructions.md with validated build instructions

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Add ForEach tests, introduce asynchronous workflow runner and enhance workflow events. (#6926)

* Introduce asynchronous workflow runner and enhance workflow events.

- Added `AsyncWorkflowRunner` to enable asynchronous workflow execution and result tracking.
- Introduced new event arguments, such as `ActivityExecutedEventArgs` and `WorkflowStateCommittedEventArgs`.
- Expanded `WorkflowEvents` class to include `ActivityExecuted`, `ActivityExecutedLogUpdated`, and `WorkflowStateCommitted` events.
- Refactored event arguments into the `Elsa.Testing.Shared.EventArgs` namespace.
- Enhanced tests with `AsyncWorkflowRunner` and new event-driven workflow scenarios.

* Refactor event argument classes to unify namespace and simplify inheritance

* Add shared component DotSettings file to support namespace exclusions
Refactor `WaitAsync` call in `DispatchWorkflowsTests` to remove unnecessary generic type.
2025-09-25 20:58:21 +02:00
Sipke Schoorstra 72ad243ed4
Merge 3.5.1 into 3.6.0 (#6907)
* Introduce `IWorkflowResumer` and `ActivityInputEvaluatorContext`, refactor endpoint handling, extend logging, and improve bookmark queue processing.

* Remove deprecated WorkflowContexts module and optimize project.

Deleted the Elsa.Studio.WorkflowContexts module and references from solution files. Corrected minor errors in remaining code and updated project configurations to align with the new structure.```

* Update GitHub workflows to track `develop/3.6.0` branch instead of `patch/3.5.1`.

* Fix inconsistent formatting in `InputDescriptor` constructor and properties.

* Add XML documentation for `DictionaryValueEvaluator` in `UIHints/Dictionary` module

* Refactor `DictionaryValueEvaluator` to improve readability and simplify dictionary evaluation logic.
2025-09-13 13:05:52 +02:00
Sipke Schoorstra f0b84f8090
Add Azurite service to docker-compose.yml
Integrated Azurite for local Azure Storage emulation, including Blob, Queue, and Table services. Configured ports, volumes, and debug options for development purposes. This addition aims to enhance local testing and reduce dependencies on external Azure services.
2025-04-28 19:07:24 +02:00
Sipke Schoorstra af63b1b59a
Ignore docker compose for local testing 2025-03-17 13:42:57 +01:00
Sipke Schoorstra 32da99e3ec
Add docker-compose file for Datadog and OpenTelemetry setup
Introduce a new docker-compose configuration to integrate Datadog and OpenTelemetry with services like PostgreSQL, RabbitMQ, Redis, and Elsa. Updated the Datadog API key in an existing compose file for security purposes. This setup enables detailed monitoring, tracing, and metrics collection for development environments.
2025-03-16 19:49:35 +01:00
Sipke Schoorstra fd31b5b605
Update OTEL fields and tags
Added and updated multiple package references, including OpenTelemetry, Datadog.Trace.Bundle, and various Microsoft.Extensions libraries. This ensures compatibility with the latest dependencies and introduces enhanced features for resilience and tracing.
2025-03-14 23:17:54 +01:00
Sipke Schoorstra e288ecb954
Add SQL Server service to Docker Compose configuration
Introduced an SQL Server service using the official 2022 image. Configured environment variables, ports, and a volume for persistent data storage. This addition supports development and testing scenarios requiring SQL Server.
2025-03-13 11:54:54 +01:00
Sipke Schoorstra 73ab3c9859 Simplify Oracle setup directory structure
Moved Oracle setup scripts to a more intuitive directory (`docker/setup`) and updated corresponding volume mappings in `docker-compose.yml`. Adjusted `.gitignore` to exclude the new `docker/data/` directory. This change improves organization and clarity for Oracle-related files.
2025-02-05 15:08:46 +01:00
Sipke Schoorstra 6c0d1ea66c Refactor database setup and EFCore provider configurations.
Revised database initialization scripts to better handle Postgres and Oracle environments, introduced schema-specific configurations for Oracle EFCore, and cleaned up obsolete or redundant entity model setup. Streamlined project structure by relocating and renaming files for SQLite and Oracle EFCore setups, improving maintainability and readability.
2025-01-31 19:51:24 +01:00
Sipke Schoorstra 26e2b1fb56 Add Oracle database support to the project
This commit introduces Oracle database integration by adding necessary configurations, entity mappings, and Docker Compose setup. Oracle-specific mappings ensure compatibility with NCLOB for large data handling. The changes also include updates to appsettings, enum for SqlDatabaseProvider, and project references to support Oracle.
2025-01-31 00:31:09 +01:00
Sipke Schoorstra 658630078b Add SMTP server configuration and refactor email activity for improved error handling and attachment processing 2025-01-24 23:06:29 +01:00
Sipke Schoorstra e07ece5875 Regenerate EF Core migrations for Runtime 2024-12-31 13:54:19 +01:00
Sipke Schoorstra 021795f53a Enable MySQL primitive collections support and update configs
Add support for EF Core primitive collections in MySQL by specifying `EnablePrimitiveCollectionsSupport()` in the DbContext configuration. Updated the Docker Compose file to rename the MySQL volume and adjusted appsettings.json to include a MySQL connection string and set MySQL as the default database provider.
2024-12-28 18:56:29 +01:00
Sipke Schoorstra cb6a7dfc11 Update RabbitMQ image to version 4-management
Upgraded RabbitMQ from version 3-management to 4-management across tests and the Docker setup. This ensures compatibility with the latest features and improvements in RabbitMQ while maintaining consistency across environments.
2024-12-14 11:44:52 +01:00
Robin Sue 4d6b7a17fb Add .NET 9.0 target 2024-12-10 21:06:42 +01:00
Sipke Schoorstra f53e024d25
Add Elsa.Kafka Module for Kafka Integration with Message Sending and Receiving Activities (#6108)
* Add Kafka module with integration and example setup

Introduced the Kafka module providing consumer integration and activities into the project. This includes new classes for consumer handling, configuration, and activities. An example setup using Docker Compose is also added to facilitate development and testing.

* Refactor KafkaTransportMessage to inline Timestamp namespace

Simplify the namespace usage for the Timestamp type within the KafkaTransportMessage record. This change eliminates the need for a separate using directive for Timestamp, enhancing code readability and maintainability.

* Add support for handling Kafka transport messages

This commit introduces the capability to handle and trigger workflows based on Kafka transport messages. It adds a new handler, notifications, and updates the message stimulus to include correlating fields. Additionally, the Kafka consumers are now managed more modularly with updated startup tasks and mediator integration.

* Enable Kafka integration and fix Kafka options naming

Added support for Kafka integration in Elsa.Server.Web by setting up Kafka configurations in appsettings.json and updating Program.cs. Also, renamed `ConsumerConfigs` to `ConsumerDefinitions` in Kafka options for clarity.

* Add consumer definition enumeration and dropdown support

Introduced `IConsumerDefinitionEnumerator` for managing consumer definitions across providers and implemented in `ConsumerDefinitionEnumerator` class. Enhanced `KafkaFeature` to register these services and updated the `MessageReceived` activity to use a dropdown UI hint for consuming definitions. Improved `StartConsumersTask` by refactoring consumer definition retrieval logic.

* Add SendMessage activity and refine Kafka messaging

Introduce a new SendMessage activity for Kafka, enabling message publishing to specific topics. Refine KafkaTransportMessage model by removing headers and timestamp fields. Adjust the StimulusSender logic to streamline the bookmark queuing process and fix key-value pairing in dropdown options. Update appsettings for corrected Kafka bootstrap server and topic configurations.

* Add producer and topic management support

Introduced interfaces and implementations for managing producer and topic definitions along with their respective enumerators and list providers. Updated `SendMessage` activity to include producer selection and refactored consumer definition providers for better consistency.

* Refactor Kafka configuration property names

Renamed Kafka configuration properties for better consistency and readability across the codebase. Updated property names from `ProducerDefinitions` to `Producers`, `ConsumerDefinitions` to `Consumers`, and `TopicDefinitions` to `Topics`. Added missing input attribute in `SendMessage.cs` and registered additional handlers in `KafkaFeature.cs`.

* Add custom serializers for Kafka message handling

Introduced `DefaultSerializers` class for custom serialization and deserialization of Kafka messages. Updated `MessageReceived` and `SendMessage` activities to use these custom serializers, and modified `KafkaOptions` to include them.

* Fix ExpandoObject serialization method parameter

Changed the serialization type from `ExpandoObject` to the actual type of the object to ensure proper serialization. This ensures that derived types are correctly handled during the serialization process.

* Add Producer and Consumer workflows for Kafka

Introduced two new workflows: `ProducerWorkflow` and `ConsumerWorkflow` for handling Kafka messages. Updated `DefaultSerializers` to use camelCase property naming and modified `appsettings.json` to include `topic-2` and format entries.

* Add JSON serialization to log output in ConsumerWorkflow

This change enhances the log output by serializing messages to JSON format before writing them. The addition of System.Text.Json ensures that the message content is presented in a structured and standardized format in logs.

* Add correlation strategies and update Kafka features

Implemented HeaderCorrelationStrategy and NullCorrelationStrategy, and updated KafkaFeature to support customizable correlation strategies. Added correlation ID handling to Kafka transport messages and updated config and handlers accordingly.

* Add tenant accessor to ConsumerDefinitionWorkflowContextProvider

Integrated ITenantAccessor to the provider to support tenant-specific context loading. Updated the constructor and LoadAsync method to retrieve the tenant information and use it for context-specific operations.

* Switch to MySQL and disable Kafka

This commit changes the SQL database provider from SQLite to MySQL and disables Kafka use. It also includes necessary adjustments such as adding MySQL handling in configuration and connection setups, updating `docker-compose` to include MySQL services, and referencing MySQL projects in the `.csproj` file.

* Add UI property handlers to multiple features

This commit introduces various UI property handlers across several features such as Python, JavaScript, CSharp, and Workflow features to enhance user interface property handling. It also updates the property UI handler resolution logic to better manage cases where providers are not available. Furthermore, adjustments were made in the server configuration to switch database providers and enable Kafka.

* Refactor property UI handler retrieval logic

Modified the logic to fetch property UI handlers by preloading them into a list and then filtering. This change improves readability and potentially performance by reducing repetitive service provider calls.

* Incremental work on Kafka workers and predicate evaluation

* Merge BookmarkInvoker with BookmarkResumer

* Register IWorkerManager

* Change lifetime scope of WorkerManager to Singleton

* **Introduce topic subscription handling for Kafka workers**

Added `IWorkerTopicSubscriber` interface and its implementation for managing topic subscriptions. Enhanced workers to bind triggers and bookmarks dynamically based on existing data. Updated several classes and methods to support topic-based subscriptions and headers.

* Refactor trigger matching logic.

Extract trigger matching conditions into `IsMatchAsync` method for reuse. This enhances code maintainability and readability by reducing redundancy. The new `GetTopic` helper method isolates the topic retrieval logic.

* Add Name property to MassTransitActivityTypeProvider

This commit inserts the Name property in the returned object within the MassTransitActivityTypeProvider class. It ensures that the typeName is included, providing a clearer definition of the activity type.

* Add handling for deleted bookmarks and refactor bookmark removal

Added a new event handler for `BookmarksDeleted` to ensure removed bookmarks are processed correctly. Refactored the bookmark removal logic into a helper method to reduce code duplication and streamline the workflow.

* Switch to asynchronous bookmark queue processing

Refactored the `TriggerWorkflows` handler to use `IBookmarkQueue` instead of directly invoking the `IBookmarkResumer`. This change aims to improve scalability by queueing bookmark resumption requests, enabling better load distribution and async processing. Added necessary helpers and configuration options to support this functionality.

* Remove unused IBookmarkResumer dependency

Simplify the constructor by removing the unused IBookmarkResumer dependency. This cleanup reduces potential confusion and improves code maintainability without impacting functionality.

* Add support for local message processing

Introduced an `IsLocal` property to `MessageReceivedStimulus` for determining if the message event is local to a specific workflow instance. Updated `BookmarkBinding` and related handler methods to utilize `CorrelationId` for local event matching. Removed unused `CorrelatingFields` from `MessageReceived` activity.

* Add nullability checks to IWorker retrieval methods

Updated `GetWorker` methods to return nullable `IWorker` to handle cases where a worker might not exist. Modified code to include null checks and conditional operations to prevent potential null reference exceptions when accessing worker methods.

* Add filtering based on activity type name for triggers and bookmarks

This commit introduces filtering for triggers and bookmarks based on the `MessageReceived` activity type name. It also adds an option to mark messages as local in the `SendMessage` activity, where local messages are delivered to the current workflow instance only. These changes help enhance the management and targeted delivery of messages within the workflow framework.

* Add new Kafka topics and clean up producers config

New topics "topic-3" and "topic-4" were added to the Kafka settings. Unused topic references were removed from the producers configuration to simplify and improve clarity.

* Add predicate to KafkaConsumerActivity in ConsumerWorkflow

Introduced a predicate to the KafkaConsumerActivity using JavaScript expressions to filter messages based on OrderId. This ensures only relevant messages are processed in the workflow.
2024-11-18 13:42:54 +01:00
Sipke Schoorstra ff487fad80 Merge remote-tracking branch 'origin/blueberry' into enh/multi-tenancy-refactorings 2024-10-11 19:45:40 +02:00
Sipke Schoorstra 078f34fd79 Remove obsolete tag settings and correct tag schemas
Eliminated redundant `correlationId` tag setting. Adjusted `workflowInstance` and `activityInstance` tag schemas for better clarity. Simplified the processors list in `otel-collector-config.yaml`.
2024-10-05 14:12:00 +02:00
Sipke Schoorstra a62de77900 Update OpenTelemetry configurations and remove unused modules
Revised otel-collector and docker-compose configurations to streamline tracing and metrics collection, replacing Datadog connectors with OpenTelemetry. Removed unused scraping modules and unnecessary dependencies to improve setup efficiency and resource utilization.
2024-09-25 14:12:00 +02:00
Sipke Schoorstra 758b60767b Add OpenTelemetry auto-instrumentation support
Introduce OpenTelemetry configuration and dependencies for observability. Replace DataDog tracer with OpenTelemetry auto-instrumentation in the Dockerfile. Adjust project dependencies and log levels to incorporate OpenTelemetry without disrupting existing functionality.
2024-09-25 00:49:21 +02:00
Sipke Schoorstra df827b0fda Update Datadog config, add incident error serialization
Enhanced the Datadog tracing configuration to include priority sampling, rate limits, and sampling rules in the docker-compose-datadog.yml. Updated the agent image version and improved error message serialization in OpenTelemetry middleware to use custom JSON serializer options. Adjusted Dockerfile path to correct bundle location.
2024-09-24 00:27:51 +02:00
Sipke Schoorstra adf36927dc Update Docker and k8s configs, enhance OTEL tracing
Updated `docker-compose-datadog.yml` with new environment variables and updated image versions, enhancing trace sampling and service autodiscovery. Added Kubernetes deployment, service, and role files for `elsa-server`, `elsa-studio`, `plant-uml`, `postgres`, and `trace-lens`, improving the project's infrastructure. Enhanced OpenTelemetry middleware for better error handling and detailed tracing of activity and workflow executions.
2024-09-23 23:35:30 +02:00
Sipke Schoorstra 631036f404
Proto.Actor implementation for ChangeTokenSignalPublisher (#5817)
* **Refactor ProtoActor modules and integrate new core module**

Removed obsolete proto actor-related files and introduced a new core module under `Elsa.ProtoActor.Core` to centralize ProtoActor functionalities. Updated services and extensions to align with the new core structure, focusing on efficient persistence and actor system configurations.

* Add Proto.Actor-based distributed caching module

Introduces a new module `Elsa.Caching.Distributed.ProtoActor` for Proto.Actor-based distributed caching, including configuration extensions, proto files, and required services. Refactors some existing Proto.Actor-related features and updates Dockerfile and example projects to use the new module.

* Refactor ProtoActor cache handling and virtual actor setup.

Reorganize the distributed caching by introducing LocalCacheVirtualActorProvider and StartLocalCacheActor. Update WorkflowInstanceVirtualActorProvider for better cluster kind handling. Adjust namespaces in Protobuf definitions for consistency.

* Refactor LocalCacheImpl to use IChangeTokenSignalInvoker

Replace IChangeTokenSignaler with IChangeTokenSignalInvoker to align with updated dependency contract. Adjust method call to use InvokeAsync for triggering token signals with cancellation support.

* Refactor ConfigureClusterConfig and config mutation

Change ConfigureClusterConfig from Action to Func for better flexibility. Update clusterConfig and remoteConfig to support reassignment from configuration methods.

* Add ProtoActor support for distributed caching

Introduced ProtoActor as a new distributed caching transport option. Updated the configuration and workflow runtime settings to utilize ProtoActor. Added necessary project reference for Elsa.Caching.Distributed.ProtoActor in the .csproj file.

* Add LocalNodeStrategy and integrate it in actor provider

Introduced `LocalNodeStrategy` to handle member placement on the current node. Integrated the new strategy in `LocalCacheVirtualActorProvider`, ensuring it uses `LocalNodeStrategy` for member management.

* Prevent duplicate member additions based on ID.

Updated the member checking logic to include member IDs. In addition, this change improves the robustness of the member management in `LocalNodeStrategy.cs`.

* Refactor virtual actor configuration into a separate method

Moved virtual actor setup logic from `ProtoActorFeature` to a new `AddVirtualActors` method to improve code readability and reusability. Updated related files to maintain consistency and enhance documentation clarity.

* Refactor LocalCache to use PubSub for change token signals

Replaced direct event stream usage with PubSub in LocalCache implementation. Updated service and hosted service to support PubSub subscription and publishing. Removed obsolete Start and Stop RPC methods from LocalCache service definition.

* Add UsedImplicitly attribute to notification handler

This change introduces the [UsedImplicitly] attribute to the DistributedWorkflowDefinitionNotificationsHandler class. The attribute is intended to prevent any accidental removal by static analysis tools, ensuring the class remains available for dynamic usage scenarios.

* Refactor caching and signal handling mechanisms

Replaced `TriggerChangeTokenSignalConsumer` with `ChangeTokenSignalInvoker` and added new decorators for change token handling. Renamed namespaces and file paths for better consistency and clarity. Updated test files to align with these changes.

* Remove unnecessary interface dependencies from services

Eliminated the ISignalManager and related interfaces to streamline dependency management. Updated services and test components to use concrete implementations directly, reducing complexity and improving maintainability.

* Remove Shared.proto and associated imports

Deleted the Shared.proto file and removed related import statements across multiple files. This cleanup also involved modifying the proto actor provider and project file to exclude references to Shared.proto.

* Simplify namespaces in component test helpers

Consolidated several namespaces into 'Elsa.Workflows.ComponentTests.Helpers' to reduce redundancy and improve maintainability. Removed unnecessary using directives in multiple test files for cleaner and more readable code.

* Refactor imports in component tests

Consolidated various helper imports in component tests by removing redundant specific references and utilizing general 'Elsa.Workflows.ComponentTests.Helpers'. This change simplifies the dependency management and ensures cleaner and more maintainable code.

* Remove redundant state persistence calls

Eliminated multiple calls to PersistStateAsync in WorkflowInstanceImpl.cs as they were unnecessary given that the WorkflowRunner already invokes the commit handler. This change simplifies the workflow execution and cancellation logic by avoiding redundant state persistence operations.

* Add workflowInstanceId to response mapping

Updated methods to include workflowInstanceId in response mapping functions for consistency and clarity. Additionally, fixed project reference paths and added error handling for missing workflow variables in tests.

* Remove unnecessary variable existence check

Removed a redundant check for the existence of the "Workflow1:variable-1" key in the variables dictionary. This streamlines the test and relies on the assumption that the key exists as expected without explicit validation.

* Add Kubernetes deployment and service configurations

Introduced a Deployment and Service configuration for the Kubernetes cluster. Updated Dockerfiles and build script to align with port 8080 configuration and renamed images for consistency. Updated solution file to include new deployment files.

* Add Kubernetes cluster integration

Introduced Kubernetes cluster provider for Proto.Actor and configured the application to use it if running in a Kubernetes environment. Added necessary RBAC roles, role bindings, and service accounts to support Kubernetes integration. Updated deployment configuration and package references to include Proto.Cluster.Kubernetes.

* Refactor deployment configurations and add service support.

Reorganized deployment YAML files into designated subdirectories for elsa-server, postgres, plant-uml, and trace-lens. Introduced new configuration maps, service accounts, roles, and service bindings. Updated .NET environment variables and solution structure to reflect these changes.

* Update service configurations and environment variables

Renamed and split services in trace-lens to isolate the OTEL collector. Updated environment variables in elsa-server to enhance instrumentation, connection strings, and profiling settings. Adjusted OTEL exporter endpoint to match the new service naming.

* Increase deployment replicas to 3

Updated the 'replicas' field in the deployment configuration to enhance the system's availability and load balancing. This change ensures that three instances of 'elsa-server' will be running simultaneously.

* Rename LocalCacheImpl to LocalCache and add logging

Renamed `LocalCacheImpl` class to `LocalCache` to better reflect its purpose. Added a logging statement in `OnReceive` method to log incoming `ProtoTriggerChangeTokenSignal` messages. These changes improve code readability and debugging.

* Disable OTEL console exporters and set session affinity

Disabled console exporters for logs, metrics, and traces in the OTEL configuration to reduce unnecessary console output. Additionally, set session affinity to 'None' in the elsa-server service configuration for load balancing.

* Rename WorkflowInstanceImpl to WorkflowInstance

Updated the class name from WorkflowInstanceImpl to WorkflowInstance for clarity and simplicity. Adjusted all relevant references and instances in the codebase to match the new class name.

* Remove ActivityIncidentStateMapper and Update ProtoBuf Mappings

Removed the unused ActivityIncidentStateMapper class to streamline the codebase. Updated all related ProtoBuf mappings and imports to ensure consistency and remove redundancy across the project.

* Remove duplicate actor spawn verification timeout setting

The code had a redundant setting for actor spawn verification timeout, which was specified twice. This commit removes the duplicate line to ensure cleaner and more maintainable configuration.

* Remove unused imports from Program.cs

Eliminated unnecessary imports for ActivityExecution, WorkflowExecution, and k8s libraries. This cleanup helps reduce the code footprint and may improve compile time.

* Remove debug logging from LocalCache actor

The `Console.WriteLine` statement was removed from the `OnReceive` method in `LocalCache.cs`. This change eliminates unnecessary console output during the token signal handling, improving performance and reducing log clutter.
2024-07-24 22:23:33 +02:00
Sipke Schoorstra f96d594f6f Refactor folder paths and update method implementations
Updated Dockerfile paths to reflect 'apps' directory structure, removed redundant XML comments, and modified methods in WorkflowDefinitionExtensions to use FindWorkflowGraphAsync for consistent workflow retrieval.
2024-07-24 14:10:27 +02:00
Sipke Schoorstra d3f9d5ef8f Merge branch 'main' of github.com:elsa-workflows/elsa-core 2024-07-22 14:36:43 +02:00
Sipke Schoorstra 70b8582b23
Add OpenTelemetry module (#5810)
* Add OpenTelemetry integration for workflow tracing

Introduced a new module, Elsa.OpenTelemetry, to provide OpenTelemetry sources for tracing workflow and activity execution. Updated various components and pipeline extensions to support OpenTelemetry tracing throughout the workflow execution process.

* Refactor workflow execution pipelines

Deleted `WorkflowsFeatureExtensions` and migrated methods to `PipelineWorkflowsFeatureExtensions` with added configurability. Enhanced `ActivityExecutionMiddlewareExtensions` and `ActivityExecutionPipelinePipelineBuilder` to support middleware insertion. Added comprehensive tracing to `OpenTelemetryTracingWorkflowExecutionMiddleware`.

* Add OpenTelemetry tracing to activity execution

Introduced `OpenTelemetryTracingActivityExecutionMiddleware` to capture tracing information for activity execution within workflows. This middleware logs activity execution start and end events, attaching pertinent activity tags. Added extension method to register this middleware in the workflow execution pipeline.

* Set `PYTHONNET_PYDLL` consistently and update workflow pipelines

Update Dockerfiles to set the `PYTHONNET_PYDLL` environment variable consistently without spaces. Additionally, refactor `Program.cs` to streamline workflow and activity execution pipeline configurations by using the `WithDefaultWorkflowExecutionPipeline` and `WithDefaultActivityExecutionPipeline` methods.

* Enhance OpenTelemetry Tracing Middleware Implementation

Add missing activity tags and events to improve telemetry data. Simplify middleware installation syntax for both workflow and activity execution tracing pipelines, ensuring consistent and clear tracing across modules.
2024-07-22 14:36:33 +02:00
Sipke Schoorstra 4f49252fbf Add dependencies to docker-compose and update README
Updated `docker-compose.yml` to include `depends_on` for `trace-lens`, ensuring it starts after `plantuml-server` and `postgres`. Modified the README to include `Elsa.Workflows` in `OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES` and corrected formatting for `OTEL_RESOURCE_ATTRIBUTES`.
2024-07-22 14:36:02 +02:00
Sipke Schoorstra b4cd4dd465
Replace custom webhooks implementation with WebhooksCore (#5811)
* Replace custom webhooks implementation with WebhooksCore

Replaced the existing custom webhooks registration and dispatching system with the WebhooksCore library. This involved removing all previous custom webhook classes and interfaces, updating project dependencies, and modifying appsettings and program files to use the new configuration. The update simplifies the webhook handling architecture and leverages the features provided by WebhooksCore.

* Hide API key in Datadog Docker compose file

This change replaces the exposed Datadog API key with a placeholder "<HIDDEN>". This enhances security by preventing the API key from being visible in the version control system.
2024-07-21 16:48:38 +02:00
Sipke Schoorstra 7c34f22d56 Remove commented OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES
Deleted an unused commented configuration line in docker-compose-datadog.yml to improve readability and maintainability. This change does not affect the existing tracing configuration or application behavior.
2024-07-20 00:05:49 +02:00
Sipke Schoorstra 8acf9b7212 Add pull_policy and environment variables for Datadog
Added `pull_policy: always` to `elsa-server` and `elsa-studio` services in the Docker Compose file to ensure the latest images are always pulled. Updated `elsa-server` environment variables to improve Datadog tracing and monitoring. Adjusted the Dockerfile path for the `elsa-studio` service.
2024-07-19 23:51:46 +02:00
Sipke Schoorstra 6924bd0770 Update build paths, add Datadog tracing, and adjust versions
Corrected Dockerfile paths for Elsa.Server and Elsa.Studio, and added Datadog tracing and related environment variables. Included Datadog.Trace package in Elsa.Server.Web project and updated Elsa.Studio package versions to 3.3.0-preview.466.
2024-07-19 23:36:31 +02:00
Sipke Schoorstra 107d660210 Update Dockerfile paths for consistency
Changed paths from "bundles" to "apps" in multiple Dockerfiles to standardize the directory structure. This ensures all Dockerfiles use a consistent directory naming convention across the project.
2024-07-19 21:01:46 +02:00
Sipke Schoorstra 283824fcfd
Enable Proto Actor Tracing for TraceLens (#5800)
* Add database initialization script and update dependencies

Added a script to initialize the 'tracelens' database and modified the Docker setup to include this script. Refactored and improved the ProtoActorFeature class, added OpenTelemetry dependencies, and updated project settings.

* Enable OpenTelemetry integration for Proto.Actor

Added OpenTelemetry environment configuration details to the README and included the Proto.OpenTelemetry package in the project file. Updated the ProtoActorFeature to apply tracing with OpenTelemetry to WorkflowInstanceActor.

* Refactor VariablePersistenceManager to use primary constructor

This refactor simplifies the VariablePersistenceManager by moving the storageDriverManager initialization into the primary constructor. It removes the redundant field and constructor, aligning with the concise nature of modern C# syntax, and ensures consistency in accessing the storageDriverManager throughout the class.

* Remove unused Open Telemetry code from Program.cs

The code for configuring Open Telemetry was commented out but not removed, cluttering the file. This commit cleans up Program.cs by deleting these unused lines, maintaining a cleaner and more readable codebase.

* Add metrics and tracing configurations for ProtoActorFeature

Introduced methods to enable metrics and tracing in ProtoActorFeature. Removed redundant properties and updated the workflow runtime to utilize the new configurations.

* Add Directory.Build.props for shared project settings

Introduce Directory.Build.props to centralize common project settings and dependencies. Consolidate target framework, language version, and package references to reduce duplication. Remove redundant property definitions from Elsa.Server.Web.csproj.

* Move apps from bundles to apps folder and Elsa module to modules folder
2024-07-19 20:23:32 +02:00
Sipke Schoorstra 9b9454403b
Minor improvements and bug fixes following the 3.1 release (#5168)
* Move DynamicActivity.cs to Activities directory

The DynamicActivity.cs file has been moved from the Models directory to the Activities directory. This reorganization aims to ensure that the file's location correctly reflects its namespace.

* Add GetOutput method in ActivityExtensions

A new GetOutput method has been added to the ActivityExtensions.cs file. This method allows the retrieval of output with a specific name from an activity. Useful for handling complex types in workflow activities.

* Add feature check and refactor dependencies in Elsa

The commit introduces a new feature check in the `Module` class and refactors the dependencies in MassTransit features. Specifically, it enables querying for a specific feature before configuring the dispatcher endpoints, increasing flexibility and control. In addition, the responsibility for creating `IEndpointChannelFormatter` has been shifted from `MassTransitWorkflowDispatcherFeature` to `MassTransitFeature`, aligning with responsibility distribution.

Fixes #5165

* Add HasFeature method to IModule interface

The IModule interface has been updated to include two methods, HasFeature<T>() and HasFeature(Type featureType). These methods are designed to check if a specific type of feature has been configured, enhancing the functionality provided by the interface.

* Add WorkflowRuntimeFeature dependency

Removed unused namespaces from WorkflowsApiFeature class and added a new dependency on WorkflowRuntimeFeature. This change enhances the code cleanliness and ensures all required dependencies are correctly linked.

* Add activity completion functionality to multiple contexts

This commit introduces multiple methods to handle activity completion across various contexts, including ActivityExecutionContext and ActivityCompletedContext. It also includes updates to bookmark serialization and the WorkflowRuntime. The resulting changes should improve handling of activity outcomes and status updates in the application flow.

* Handle null options in DefaultWorkflowRuntime

Added null-conditional operators to prevent potential NullReferenceExceptions in DefaultWorkflowRuntime. This change ensures that even if the 'options' object is null, the code will not throw an exception and will instead use default values where applicable.

* Add ElsaDbContextOptions to DbContextOptionsBuilder

A line of code is added to enable applying ElsaDbContextOptions as default in DbContextOptionsBuilder within PersistenceFeatureBase. This change specifies the use of ElsaDbContextOptions when configuring the context options, enhancing the database context setup in the EntityFrameworkCore.Common module.

* Remove whitespace in Elsa.Server.Web.csproj

This commit removes unnecessary whitespaces at the end of the ProjectReference and PackageReference elements, in the Elsa.Server.Web.csproj file. This improves the readability and alignment of the code and follows the best practice for XML file format.

* Add MongoDB to docker-compose.yml

A MongoDB service has been added to the docker-compose file. The configuration includes port mapping and volume mapping for MongoDB data storage. This allows more flexibility in our environment setup with MongoDB now being spun up automatically.

* Add collection check in MongoDbStore before bulk save

Adjusted code structure, and divided longer lines of code into smaller, multi-line chunks for better readability. This refactoring makes the underlying operations and structuring of the code more apparent, aiding in future code maintenance and understanding.

* Change target branch in packages.yml workflow

This commit modifies the Github actions workflow for packaging. The branch from which to fetch changes is now specified explicitly as 'origin/patch/3.1.1' instead of the default 'origin/main'. This adjustment is specific for package creation under certain conditions.
2024-04-02 07:41:46 +02:00
Sipke Schoorstra 83dd776233 Add new docker-compose files, new activities and services
Create new docker-compose files for non-data dog and data dog services. New services include Postgres, cockroachdb, rabbitmq, redis, and elsa-server. Along with the services, a new bash script for migration and various new activities and endpoints are also added for the Elsa server. This set of changes broadens the operational environment of Elsa.
2024-03-24 20:28:19 +01:00
Sipke Schoorstra b553ffe0d9 Add Docker build script for Elsa Studio
This commit introduces a Dockerfile for the Elsa Studio application and a corresponding GitHub workflow for building and deploying the Docker image. These make it possible to automatically build and deploy Elsa Studio as a Docker image.
2023-12-27 22:15:57 +01:00
Sipke Schoorstra e0b9356c47 Update Dockerfiles for Python 3.11 installation
Added lines in Dockerfiles to install Python 3.11 and set the PYTHONNET_PYDLL environment variable. This enhancement helps to ensure compatibility with recent Python updates and improves general performance.
2023-12-27 21:45:01 +01:00
Sipke Schoorstra 062158b4d2 Update Dockerfiles and add Python 3.11
Working directories in both ElsaServer.Dockerfile and ElsaServerAndStudio.Dockerfile have been updated for better consistency. Additionally, Python 3.11 has been installed in the ElsaServer.Dockerfile, accompanied by relevant environment variables and symbolic links to improve interoperability.
2023-12-27 20:44:28 +01:00
Sipke Schoorstra f17966c592 Refactor project structure and Docker configurations
The changes included in this commit are made to refactor the project structure and Docker configurations. Previously, the application was structured around the AllInOneWeb solution, but it has been split into Server.Web and ServerAndStudio.Web solutions for better separation of concerns. Additionally, the Docker configurations have been updated to reflect the new structure, which should streamline the build and deployment process.
2023-12-27 20:30:33 +01:00
Sipke Schoorstra 0146a8aeb0 Update AllInOne dockerfile 2023-12-04 14:25:34 +01:00
Sipke Schoorstra f5962501bc Update targets 2023-11-27 20:20:22 +01:00
Sipke Schoorstra 86ffcfab5d Update Dockerfile 2023-11-27 20:03:50 +01:00
Sipke Schoorstra bec8cbc322
Target .NET 8 (#4660) 2023-11-27 11:26:16 +01:00
Sipke Schoorstra cb08cad32b
Replace Stencil dashboard with Blazor in AllInOne.Web project (#4554) 2023-10-19 01:03:55 +02:00
Sipke Schoorstra ab53ba4d8b Update All In One dockerfile to build client assets 2023-04-01 10:50:52 +02:00
Sipke Schoorstra f31a950d68 Try --platform=$BUILDPLATFORM 2022-12-30 19:42:52 +01:00
Sipke Schoorstra 3017804980 Try Node 19.0.0 2022-12-30 19:36:56 +01:00