diff --git a/doc/wiki/README.md b/doc/wiki/README.md index e49c35cd7..e5bb48117 100644 --- a/doc/wiki/README.md +++ b/doc/wiki/README.md @@ -47,6 +47,7 @@ flowchart LR | [HTTP, Scheduling, And Resilience](http-scheduling-resilience.md) | Inbound HTTP workflows, outbound HTTP, scheduled triggers, and resilience strategies. | | [Persistence](persistence.md) | In-memory stores, EF Core stores, provider packages, migrations, and multi-provider rules. | | [Diagnostics Structured Logs](diagnostics-structured-logs.md) | `ILogger` capture, live feed, REST/SignalR surface, redaction, and SQLite persistence. | +| [Diagnostics Console Logs](diagnostics-console-logs.md) | Raw stdout/stderr capture, live feed, REST/SignalR surface, and redaction. | | [Identity, Tenancy, And Security](identity-tenancy-security.md) | Users, applications, roles, API keys, tenant resolution, and authorization touch points. | | [Testing Guide](testing-guide.md) | Test project layout, fixture choices, and targeted commands. | | [Extension Guide](extension-guide.md) | How to add features, activities, expression providers, stores, endpoints, and ingress sources. | @@ -63,7 +64,7 @@ flowchart LR - Runtime feature: [src/modules/Elsa.Workflows.Runtime/Features/WorkflowRuntimeFeature.cs](../../src/modules/Elsa.Workflows.Runtime/Features/WorkflowRuntimeFeature.cs) - API feature: [src/modules/Elsa.Workflows.Api/Features/WorkflowsApiFeature.cs](../../src/modules/Elsa.Workflows.Api/Features/WorkflowsApiFeature.cs) - Reference server: [src/apps/Elsa.Server.Web/Program.cs](../../src/apps/Elsa.Server.Web/Program.cs) -- Active structured-log persistence plan: [specs/005-structured-log-persistence/plan.md](../../specs/005-structured-log-persistence/plan.md) +- Structured-log persistence design: [specs/005-structured-log-persistence/plan.md](../../specs/005-structured-log-persistence/plan.md) ## Contributor Workflow diff --git a/doc/wiki/activities-and-authoring.md b/doc/wiki/activities-and-authoring.md index ed3090a38..872327f58 100644 --- a/doc/wiki/activities-and-authoring.md +++ b/doc/wiki/activities-and-authoring.md @@ -95,6 +95,28 @@ Tests: - [CachingAndWorkflowDefinitionActivity](../../test/component/Elsa.Workflows.ComponentTests/Scenarios/CachingAndWorkflowDefinitionActivity) - [WorkflowReferenceGraph](../../test/component/Elsa.Workflows.ComponentTests/Scenarios/WorkflowReferenceGraph) +## StateMachine Activity + +The [StateMachine](../../src/modules/Elsa.Workflows.Core/Activities/StateMachine/Activities/StateMachine.cs) activity models named states with trigger-driven transitions. It lives in `Elsa.Workflows.Core` and is registered as a built-in activity. + +Key concepts: + +- **States** ([StateMachineState](../../src/modules/Elsa.Workflows.Core/Activities/StateMachine/Models/StateMachineState.cs)): Named states, each with an optional `Entry` and `Exit` activity. +- **Transitions** ([Transition](../../src/modules/Elsa.Workflows.Core/Activities/StateMachine/Models/Transition.cs)): Directed From→To pairs with an optional `Trigger` activity, optional `Condition` expression, and optional `Action` activity. +- **InitialState**: The first state to enter when execution starts. + +Execution cycle: + +1. On start, the machine enters `InitialState` and runs its `Entry` activity if present. +2. All outbound transition triggers for the current state are scheduled concurrently. +3. When a trigger completes and its `Condition` evaluates to true, competing triggers are canceled, the optional `Action` runs, then the current state's `Exit` activity runs. +4. The machine moves to the `To` state and repeats from step 1. +5. If the current state has no valid outbound transitions, the machine completes. + +When a condition evaluates to false the failed trigger is re-armed and remains waiting alongside other triggers for the state. + +Spec: [specs/006-state-machine-activity/spec.md](../../specs/006-state-machine-activity/spec.md). + ## Adding A New Activity Typical steps: diff --git a/doc/wiki/architecture.md b/doc/wiki/architecture.md index 9b894a867..50f5751ad 100644 --- a/doc/wiki/architecture.md +++ b/doc/wiki/architecture.md @@ -67,7 +67,7 @@ sequenceDiagram ## Activities And Control Flow -Activities are the unit of work. Core activity types live under [Elsa.Workflows.Core/Activities](../../src/modules/Elsa.Workflows.Core/Activities). Control flow includes `Sequence`, `If`, `Switch`, `Fork`, `For`, `ForEach`, `While`, `Parallel`, `Flowchart`, and flowchart node activities. +Activities are the unit of work. Core activity types live under [Elsa.Workflows.Core/Activities](../../src/modules/Elsa.Workflows.Core/Activities). Control flow includes `Sequence`, `If`, `Switch`, `Fork`, `For`, `ForEach`, `While`, `Parallel`, `Flowchart`, `StateMachine`, and flowchart node activities. Flowchart execution has a token-centric model documented in [ADR 0005](../adr/0005-token-centric-flowchart-execution-model.md), with explicit join behavior documented in [ADR 0007](../adr/0007-adoption-of-explicit-merge-modes-for-flowchart-joins.md). @@ -118,6 +118,8 @@ Provider-specific packages such as SQLite, SQL Server, PostgreSQL, MySQL, and Or ## Diagnostics Layer -[Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Diagnostics.StructuredLogs) captures semantic `ILogger` events, redacts them, keeps recent events, exposes REST endpoints, and streams live events through SignalR. The default store is in-memory; the active `005-structured-log-persistence` work adds SQLite persistence through a shared relational package. +[Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Diagnostics.StructuredLogs) captures semantic `ILogger` events, redacts them, keeps recent events, exposes REST endpoints, and streams live events through SignalR. The default store is in-memory; SQLite persistence is available through the relational and SQLite persistence packages. -See [Diagnostics Structured Logs](diagnostics-structured-logs.md). +[Elsa.Diagnostics.ConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs) captures raw `stdout` and `stderr` from the host process, buffers recent lines, and streams live console output to authorized callers through SignalR. It is separate from structured logs and does not write to durable storage. + +See [Diagnostics Structured Logs](diagnostics-structured-logs.md) and [Diagnostics Console Logs](diagnostics-console-logs.md). diff --git a/doc/wiki/build-run-operate.md b/doc/wiki/build-run-operate.md index 6d0206f4e..079ac17e7 100644 --- a/doc/wiki/build-run-operate.md +++ b/doc/wiki/build-run-operate.md @@ -130,6 +130,11 @@ Structured log diagnostics endpoints include: - `GET /elsa/api/diagnostics/structured-logs/sources` - `GET /elsa/api/diagnostics/structured-logs/storage` +Console log diagnostics endpoints include (when `Elsa.Diagnostics.ConsoleLogs` is enabled): + +- `POST /elsa/api/diagnostics/console-logs/recent` +- `GET /elsa/api/diagnostics/console-logs/sources` + Health checks are mapped to `/` in the reference server. ## Runtime Knobs diff --git a/doc/wiki/diagnostics-console-logs.md b/doc/wiki/diagnostics-console-logs.md new file mode 100644 index 000000000..1d8d76ac8 --- /dev/null +++ b/doc/wiki/diagnostics-console-logs.md @@ -0,0 +1,120 @@ +# Diagnostics Console Logs + +`Elsa.Diagnostics.ConsoleLogs` is an opt-in module that captures raw `stdout` and `stderr` from the Elsa host process, buffers recent lines, and streams live console output to authorized callers over SignalR. It is intentionally separate from `Elsa.Diagnostics.StructuredLogs`; the two modules cover different diagnostic surfaces. + +Start in [src/modules/Elsa.Diagnostics.ConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs). + +## Scope + +This module captures raw process console output only. It does not parse `ILogger` records, write to durable audit storage, call orchestrator log APIs, or implement trace or metric exploration. For semantic `ILogger` capture see [Diagnostics Structured Logs](diagnostics-structured-logs.md). + +## Feature Wiring + +[ConsoleLogsFeature](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Features/ConsoleLogsFeature.cs): + +- registers FastEndpoints assembly +- calls `AddConsoleLogsServices` +- adds FastEndpoints from the module + +[AddConsoleLogsServices](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Extensions/ServiceCollectionExtensions.cs) registers: + +- SignalR +- `ConsoleLogsOptions` +- source registry +- redactor +- line formatter +- in-memory console log provider +- subscription manager +- `ConsoleCaptureTee` as `IConsoleLogCapture` +- `ConsoleLogCaptureHostedService` hosted service + +The hosted service installs a `TextWriter` tee that writes to both the original console destination and the capture pipeline. Original stdout and stderr destinations are preserved. + +## Core Contracts + +| Contract | Purpose | +| --- | --- | +| [IConsoleLogCapture](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Contracts/IConsoleLogCapture.cs) | Capture pipeline entry point. | +| [IConsoleLogProvider](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Contracts/IConsoleLogProvider.cs) | REST/SignalR facade used by endpoints and subscription manager. | +| [IConsoleLogSourceRegistry](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Contracts/IConsoleLogSourceRegistry.cs) | Tracks source metadata and health. | +| [IConsoleLogRedactor](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Contracts/IConsoleLogRedactor.cs) | Redacts text before buffering and streaming. | +| [IConsoleLogDroppedLineReporter](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Contracts/IConsoleLogDroppedLineReporter.cs) | Reports dropped-line counts when buffers overflow. | + +## Event Flow + +Captured console writes pass through redaction before reaching any consumer: + +```mermaid +sequenceDiagram + participant Console as Process stdout/stderr + participant Tee as ConsoleCaptureTee + participant Original as Original TextWriter + participant Redactor as IConsoleLogRedactor + participant Buffer as ConsoleLineBuffer + participant Manager as ConsoleLogSubscriptionManager + participant Hub as ConsoleLogsHub + participant Client as Authorized caller + + Console->>Tee: write bytes/chars + Tee->>Original: pass through (preserved) + Tee->>Redactor: redact line text + Redactor->>Buffer: append redacted line + Redactor->>Manager: publish redacted line + Manager->>Hub: live event + Hub->>Client: SignalR stream + Client->>Buffer: recent query through REST +``` + +## REST And SignalR Surface + +REST endpoints: + +- `POST /elsa/api/diagnostics/console-logs/recent` +- `GET /elsa/api/diagnostics/console-logs/sources` + +Endpoint code is under [Endpoints/ConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Endpoints/ConsoleLogs). + +SignalR: + +- Hub: [ConsoleLogsHub](../../src/modules/Elsa.Diagnostics.ConsoleLogs/RealTime/ConsoleLogsHub.cs) +- Route: `/elsa/hubs/diagnostics/console-logs` +- Mapping: [MapConsoleLogsHub](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Extensions/EndpointRouteBuilderExtensions.cs) +- App extension: [UseConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Extensions/ApplicationBuilderExtensions.cs) + +## Authorization + +All endpoints and the SignalR hub require `read:diagnostics:console-logs`, defined in [ConsoleLogsPermissions](../../src/modules/Elsa.Diagnostics.ConsoleLogs/Permissions/ConsoleLogsPermissions.cs). + +## Safety Boundaries + +- Redaction runs before recent buffering, live streaming, and endpoint responses. +- ANSI escape sequences are stripped by default (`StripAnsiEscapeSequences = true`). +- Partial writes (no trailing newline) are buffered until the line completes, reaches the maximum line length, or an idle flush occurs. +- Lines longer than `MaxLineLength` are truncated to one event and marked as truncated. +- Dropped-line counts are reported through `IConsoleLogDroppedLineReporter` when buffers or subscriber queues overflow. + +## Configuration + +```csharp +services.AddElsa(elsa => +{ + elsa.UseConsoleLogs(options => + { + options.RecentLogCapacity = 5_000; + options.SubscriberChannelCapacity = 1_000; + options.MaxRecentQuerySize = 1_000; + options.MaxLineLength = 16_384; + options.StripAnsiEscapeSequences = true; + }); +}); +``` + +Map the live hub after routing is configured: + +```csharp +app.UseConsoleLogs(); +``` + +## Design Spec + +[specs/006-diagnostics-console-logs/spec.md](../../specs/006-diagnostics-console-logs/spec.md) defines requirements for capture, buffering, endpoints, SignalR, permissions, source identity, and redaction. diff --git a/doc/wiki/diagnostics-structured-logs.md b/doc/wiki/diagnostics-structured-logs.md index aeba6c284..8e9b9ffd2 100644 --- a/doc/wiki/diagnostics-structured-logs.md +++ b/doc/wiki/diagnostics-structured-logs.md @@ -6,7 +6,7 @@ Start in [src/modules/Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Di ## Scope -This module captures structured `ILogger` records only. It does not capture raw stdout/stderr console streams, traces, metrics, or OpenTelemetry spans. Those are intentionally separate future diagnostics concerns. +This module captures structured `ILogger` records only. It does not capture raw stdout/stderr console streams, traces, metrics, or OpenTelemetry spans. Raw console output is covered by the sibling [Elsa.Diagnostics.ConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs) module — see [Diagnostics Console Logs](diagnostics-console-logs.md). ## Feature Wiring @@ -100,7 +100,7 @@ Events pass through `IStructuredLogRedactor` before buffering or streaming. Conf ## SQLite Persistence -The active structured-log persistence work adds durable SQLite storage: +Durable SQLite storage is available through the relational and SQLite packages: - design plan: [specs/005-structured-log-persistence/plan.md](../../specs/005-structured-log-persistence/plan.md) - quickstart: [specs/005-structured-log-persistence/quickstart.md](../../specs/005-structured-log-persistence/quickstart.md) diff --git a/doc/wiki/repository-map.md b/doc/wiki/repository-map.md index b53904862..2f1877dd9 100644 --- a/doc/wiki/repository-map.md +++ b/doc/wiki/repository-map.md @@ -33,7 +33,7 @@ Elsa Core is organized as a large multi-project .NET solution. The repo favors s | Transport/activity packages | [Elsa.Http](../../src/modules/Elsa.Http), [Elsa.Scheduling](../../src/modules/Elsa.Scheduling), [Elsa.Resilience](../../src/modules/Elsa.Resilience) | HTTP triggers and calls, scheduled triggers, resilience strategies. | | Persistence | [Elsa.Persistence.EFCore](../../src/modules/Elsa.Persistence.EFCore), provider packages under `Elsa.Persistence.EFCore.*`, and structured-log persistence packages | EF Core stores and provider-specific configuration/migrations. | | Security and tenancy | [Elsa.Identity](../../src/modules/Elsa.Identity), [Elsa.Tenants](../../src/modules/Elsa.Tenants), [Elsa.Tenants.AspNetCore](../../src/modules/Elsa.Tenants.AspNetCore), [Elsa.SasTokens](../../src/modules/Elsa.SasTokens) | Users, applications, roles, API keys, tenants, tenant-aware routing, SAS tokens. | -| Diagnostics | [Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Diagnostics.StructuredLogs), [Relational](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Relational), [Sqlite](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Sqlite) | Structured `ILogger` capture, live feed, REST/SignalR endpoints, in-memory and SQLite storage. | +| Diagnostics | [Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Diagnostics.StructuredLogs), [Relational](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Relational), [Sqlite](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Sqlite), [Elsa.Diagnostics.ConsoleLogs](../../src/modules/Elsa.Diagnostics.ConsoleLogs) | Structured `ILogger` capture, raw console capture, live feed, REST/SignalR endpoints, in-memory and SQLite storage. | | Shells and modular hosting | [Elsa.Shells.Api](../../src/modules/Elsa.Shells.Api), CShells-facing shell feature classes throughout modules | Runtime-configurable feature loading for modular hosts. | ## Reference Hosts diff --git a/doc/wiki/specs-and-adrs.md b/doc/wiki/specs-and-adrs.md index b4ee88a61..a278e64f7 100644 --- a/doc/wiki/specs-and-adrs.md +++ b/doc/wiki/specs-and-adrs.md @@ -35,6 +35,8 @@ Current ADRs: | [003 live server logs](../../specs/003-live-server-logs/spec.md) | Diagnostics precursor | Earlier live server logs work that led to structured diagnostics. | | [004 diagnostics structured logs](../../specs/004-diagnostics-structured-logs/spec.md) | Diagnostics | Refactors server logs into structured log diagnostics with semantic `ILogger` capture. | | [005 structured log persistence](../../specs/005-structured-log-persistence/spec.md) | Diagnostics persistence | Adds storage abstraction, relational persistence, SQLite durability, migrations, write queue, and retention. | +| [006 diagnostics console logs](../../specs/006-diagnostics-console-logs/spec.md) | Diagnostics | Defines capture, buffering, endpoints, SignalR hub, permissions, source identity, and redaction for raw console output. | +| [006 state machine activity](../../specs/006-state-machine-activity/spec.md) | Workflow core | Adds a state machine activity with named states and trigger-driven transitions to the workflow engine. | Each spec folder usually contains: