# Feature Specification: Diagnostics Structured Logs **Feature Branch**: `004-diagnostics-structured-logs` **Created**: 2026-05-10 **Status**: Draft **Input**: User description: "Refactor the current server log streaming module into Elsa.Diagnostics.StructuredLogs, keep it as the structured logging module, and prepare separate future specs for console streaming and OpenTelemetry exploration." ## Clarifications ### Session 2026-05-10 - Q: Should the current module become raw console streaming? -> A: No. Keep it as structured logging and specify raw stdout/stderr console streaming separately later. - Q: What should the diagnostic namespace umbrella be? -> A: Use `Elsa.Diagnostics.*`. - Q: What should this module be called? -> A: `Elsa.Diagnostics.StructuredLogs`. - Q: Should OpenTelemetry visualization be part of this module? -> A: No. This module should expose trace/span correlation fields and links, while a future `Elsa.Diagnostics.OpenTelemetry` module owns trace and metric exploration. - Manual clarification pass: No remaining high-impact ambiguities were found; implementation should proceed with a consistent breaking rename from the unpublished `Elsa.ServerLogs` feature to `Elsa.Diagnostics.StructuredLogs`. ## User Scenarios & Testing *(mandatory)* ### User Story 1 - Install a clearly named structured logs module (Priority: P1) An Elsa host developer enables structured log streaming through a diagnostics-specific module name and API that no longer suggests raw server console capture. **Why this priority**: The current name conflates structured `ILogger` events with stdout/stderr console logs. A precise module identity prevents the wrong user expectations before adding console streaming as a separate module. **Independent Test**: Build a sample host using the renamed package, namespace, feature class, fluent extension, shell feature, endpoint mappings, and installed feature name without references to the previous `Elsa.ServerLogs` module identity. **Acceptance Scenarios**: 1. **Given** an Elsa host references the diagnostics structured logs package, **When** the host enables the old-style module API, **Then** it uses diagnostics-specific names such as `UseStructuredLogs` from `Elsa.Diagnostics.StructuredLogs`. 2. **Given** a shell-based host configures features through appsettings, **When** it enables the structured logs shell feature, **Then** the feature appears under a diagnostics structured logs remote feature name. 3. **Given** Studio checks installed backend features, **When** structured logs are enabled, **Then** the advertised remote feature name uniquely identifies structured logs and does not mention console streaming. --- ### User Story 2 - Preserve semantic log data for inspection (Priority: P2) An operator inspects logs as structured records with levels, categories, message templates, rendered messages, scopes, properties, exceptions, and workflow/correlation context. **Why this priority**: The module's value is semantic logging. If it only renders formatted text, it overlaps with the future console logs module and loses the Aspire-style structured log experience. **Independent Test**: Emit `ILogger` records with message templates, named properties, scopes, exceptions, workflow context, and active `Activity` trace/span IDs; verify the captured event contains the semantic fields after redaction. **Acceptance Scenarios**: 1. **Given** an `ILogger` call uses a message template and named arguments, **When** the event is captured, **Then** the module stores both the rendered message and the original message template plus named properties. 2. **Given** an `ILogger` scope is active, **When** a log event is captured inside the scope, **Then** the event includes scope values unless redaction removes them. 3. **Given** a log event occurs inside an active trace/span, **When** the event is captured, **Then** trace ID and span ID are exposed for Studio and future OpenTelemetry cross-links. --- ### User Story 3 - Keep the structured logs contract separate from console logs and telemetry exploration (Priority: P3) An operator understands which diagnostics surface they are using and can rely on stable boundaries between structured logs, raw console logs, and future OpenTelemetry views. **Why this priority**: The diagnostics area is expected to grow. Strong boundaries now avoid naming churn and duplicated responsibilities later. **Independent Test**: Review API routes, hub routes, models, permissions, documentation, and feature metadata to confirm this module only promises structured logs and explicitly excludes stdout/stderr capture and trace visualization. **Acceptance Scenarios**: 1. **Given** the structured logs module is enabled, **When** an activity writes directly to stdout without `ILogger`, **Then** this module is not expected to capture it. 2. **Given** a user needs raw stdout/stderr, **When** they read this module's README or quickstart, **Then** it points to the future Console Logs/Console Streaming module rather than claiming console capture. 3. **Given** a user needs trace waterfall or metrics charts, **When** they read this module's README or quickstart, **Then** it identifies those capabilities as future OpenTelemetry module responsibilities. ### Edge Cases - Existing unpublished code, docs, and specs still use `Elsa.ServerLogs` or `ServerLogStreaming`. - Shell feature names are persisted in appsettings and must be intentionally changed before release. - API clients generated from the old route names may still exist in Studio. - Scope values can be nested objects, anonymous objects, dictionaries, or non-string values. - Message templates can be missing when log events originate from custom providers. - Redaction can remove fields that would otherwise be used for filtering or trace links. - Structured log capture can recursively capture its own internal diagnostics logs. ## Requirements *(mandatory)* ### Functional Requirements **Naming and module identity** - **FR-001**: The Core module project MUST be renamed from `Elsa.ServerLogs` to `Elsa.Diagnostics.StructuredLogs`. - **FR-002**: The root namespace MUST be `Elsa.Diagnostics.StructuredLogs`. - **FR-003**: Public types that currently use `ServerLog` or `ServerLogs` MUST be renamed to `StructuredLog` or `StructuredLogs` unless the old name is retained only as an explicitly obsolete compatibility shim. - **FR-004**: Public types that currently use `ServerLogStreaming` MUST be renamed to `StructuredLogs` or `StructuredLogStreaming` only where "streaming" describes a transport detail. - **FR-005**: The old-style feature class MUST be renamed to a structured logs feature, with a fluent extension such as `UseStructuredLogs`. - **FR-006**: The shell feature MUST be renamed to a structured logs shell feature and MUST expose configuration through bindable public properties only. - **FR-007**: The remote installed-feature name MUST become diagnostics-specific, for example `Elsa.Diagnostics.StructuredLogs.ShellFeatures.StructuredLogsFeature`, and Studio MUST be able to gate against that value. - **FR-008**: Package metadata, README files, sample host wiring, solution entries, project references, test project names, and docs MUST use the diagnostics structured logs name. **Structured log capture** - **FR-009**: The module MUST continue to capture structured `ILogger` events through an `ILoggerProvider`. - **FR-010**: Captured events MUST include timestamp, received timestamp, sequence, level, category, event ID, event name, rendered message, message template, exception summary/detail, scopes, structured properties, trace ID, span ID, correlation ID, tenant ID, workflow definition ID, workflow instance ID, and source ID when available. - **FR-011**: The logger provider MUST populate message template values from `{OriginalFormat}` when present. - **FR-012**: The logger provider MUST capture active logging scopes, including dictionary-like and key-value scope values. - **FR-013**: The module MUST preserve named structured properties separately from the rendered message. - **FR-014**: The module MUST keep a bounded recent-history buffer and bounded subscriber behavior with visible dropped-event counts. - **FR-015**: The module MUST prevent recursive capture of its own diagnostics logs by default. **API, hub, and permissions** - **FR-016**: REST and SignalR route names SHOULD move from server-log naming to diagnostics structured-log naming, for example `/elsa/api/diagnostics/structured-logs` and `/elsa/hubs/diagnostics/structured-logs`. - **FR-017**: The permission name MUST move from `read:server-logs` to a diagnostics structured logs permission such as `read:diagnostics:structured-logs`. - **FR-018**: The API contract MUST continue to support recent queries, source listing, live subscriptions, filter updates, and dropped-event summaries. - **FR-019**: Filters MUST continue to support minimum level, exact levels, category prefix, free-text query, tenant ID, workflow definition ID, workflow instance ID, trace ID, span ID, correlation ID, source ID, and time range. - **FR-020**: Trace ID and span ID fields MUST be stable enough for future `Elsa.Diagnostics.OpenTelemetry` deep links. **Configuration and safety** - **FR-021**: Configuration options MUST be renamed from server-log wording to structured-log wording. - **FR-022**: Shell feature properties MUST mirror bindable structured log options and copy those values into the service options during service registration. - **FR-023**: Redaction MUST continue to run before buffering or streaming. - **FR-024**: Default redaction MUST continue to mask common secret names and values in messages, exceptions, scopes, and structured properties. - **FR-025**: The module MUST explicitly document that direct stdout/stderr capture is out of scope and belongs to a future diagnostics console logs module. ### Key Entities *(include if feature involves data)* - **Structured Log Event**: A semantic log record captured from `ILogger`. - **Structured Log Source**: The process, pod, container, machine, or backend source that produced a structured log event. - **Structured Log Filter**: Query/subscription criteria for recent and live structured logs. - **Structured Log Provider**: Pluggable source of recent and live structured log events. - **Structured Log Redactor**: Redaction service applied before events leave the backend. - **Structured Log Subscription**: Live SignalR subscription plus mutable filters and backpressure state. ## Success Criteria *(mandatory)* ### Measurable Outcomes - **SC-001**: No production source file, test file, project file, or README in the structured logs module retains `Elsa.ServerLogs` as the active namespace, assembly, package, or module name. - **SC-002**: A host can enable the renamed module with old-style code configuration and shell-based appsettings configuration. - **SC-003**: Studio can detect the renamed remote feature and load structured logs without relying on the old remote feature name. - **SC-004**: Tests verify message template capture from `{OriginalFormat}`. - **SC-005**: Tests verify logging scope capture and redaction. - **SC-006**: Existing recent query, source list, filtering, live stream, and dropped-event behavior continue to pass after the rename. - **SC-007**: Documentation clearly separates structured logs from future console logs and OpenTelemetry explorer modules. ## Assumptions - The current branch has not shipped as a stable public package, so breaking renames are acceptable if performed consistently. - Studio will implement a paired `Elsa.Studio.Diagnostics.StructuredLogs` spec under the same feature ID. - Console streaming will be specified separately as `Elsa.Diagnostics.ConsoleLogs` or an equivalent final name. - OpenTelemetry visualization will be specified separately as `Elsa.Diagnostics.OpenTelemetry`. - Long-term durable log storage remains out of scope for this module unless provided later by a pluggable provider.