Commit graph

6 commits

Author SHA1 Message Date
Sipke Schoorstra 895d494c3f
fix(external-auth)!: wildcard-aware permission grant boundary, and startup smoke tests for both hosts (#7985)
* fix(external-auth)!: match permission grant boundaries as patterns

The deployment allow/deny boundary and the delegation authorizer compared
permission strings with ordinal equality, so under the {resource}:{verb}
vocabulary they could not see wildcards. A deny list naming
'workflows/*:delete' did not deny 'workflows/definitions:delete', and a grant
of 'workflows/*:delete' outflanked a deny naming that leaf.

The bypass was reachable. ElsaRolePermissionGrantSource passes a role's
permissions to the boundary verbatim, survivors land in the issued token as
permission claims, and PermissionEvaluator does expand wildcards there. So an
ordinary role plus a deny list was enough, on every external sign-in, with no
privileged actor involved. Restoring the ordinal boundary under the new tests
fails seven of them.

Deny is now matched in both directions, allow one-directionally, both through
PermissionMatcher. A grant that is not a well-formed permission is dropped
with a warning rather than carried into a token it cannot authorize anything
in.

Five non-endpoint checks -- delegation, role-reference removal, unsafe
settings confirmation, the recovery override and the boundary itself -- also
still compared against the legacy ExternalAuthenticationPermissions
constants. Those carry two colons, so Permission.TryParse rejects them and no
principal can hold one, while the migration guide tells operators to replace
exactly those strings. All five now route through IPermissionEvaluator, and
the module registers AddElsaAuthorization itself instead of depending on host
ordering.

Non-core verbs move to ExternalAuthenticationVerbs, declared beside the
resources they apply to so a delegation check cannot spell one differently
from the endpoint it guards.

Refs #7982

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: apply IDE code cleanup to the diagnostics and identity modules

Redundant namespace qualifiers and usings removed, and primary-constructor
and record syntax applied, across Elsa.Diagnostics.ConsoleLogs,
Elsa.Diagnostics.StructuredLogs, Elsa.Expressions.JavaScript and
Elsa.Identity. Produced by a solution-wide IDE cleanup that ran alongside the
authorization work; separated from it so the permission changes can be
reviewed on their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(hosts): boot both hosts and assert their gated routes challenge

This repo runs two parallel feature systems, the classic Features/ path and
the CShells ShellFeatures/ path, and every module has to register in both.
Nothing exercised either. The unit and integration suites construct services
directly, so a module registered in one path and not the other, or a service
missing from one container, passes every test and fails only when a host
starts. Three bugs in #7980 were found by running these two hosts by hand,
two of them shell-versus-classic divergences.

Each host is booted through WebApplicationFactory, running its real Program
with full feature registration, and asked for a handful of routes it is
expected to serve behind a permission. A 404 means the module was never
registered, a 5xx means the endpoint was found but its dependencies could not
be constructed, and a 200 means no gate ran; only 401 passes. All routes are
reported together, so a feature system that stops registering a group of
modules reads as one failure rather than a queue of identical ones.

Removing AddExternalAuthenticationServices from the shell feature -- the
divergence this is built to catch -- fails the shell host on all five of its
routes while the classic host stays green.

The assertions go through HTTP rather than the container on purpose. The
hosts have different topologies: the classic host's root provider holds
everything and registers 125 routes, while CShells gives each shell its own
provider and mounts routes per shell, leaving 6 in the root. A container or
route-table assertion would have to encode that difference and would break
whenever CShells changed internally. Behaviour at the edge is host-agnostic,
and it is what actually has to match.

Each host gains a namespaced entry-point marker because both already declare
a Program in the global namespace, which a test project referencing both
cannot tell apart.

Coverage is off for this project: it references both hosts, so every module
either pulls in would enter its denominator without adding real coverage, and
coverlet cannot instrument a graph that size. TreatAsLocalProperty keeps CI's
/p:CollectCoverage=true from overriding that.

Refs #7982

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(external-auth)!: fail closed on an unparseable grant boundary

Two findings from review, both real.

The grant boundary parsed its allow and deny lists and silently dropped what
would not parse. An allow list of nothing but malformed entries therefore
reduced to an empty set, and an empty allow list means unrestricted -- so a
typo turned the boundary off entirely and let external grant sources put
permissions straight into issued tokens. The deny side had the mirror of it:
a malformed entry quietly stopped denying what it named.

A boundary that does not parse now admits nothing, and
ExternalAuthenticationOptionsValidator rejects the configuration at startup,
so the mistake reaches an operator rather than a token. Failing startup is
what makes the runtime behaviour safe to be strict about: it cannot be hit by
someone mid-edit, only by validation having been bypassed.

ConnectionEndpointSupport.HasPermission was a sixth ad-hoc permission check,
missed when the other five were converted. It compared claim values against
the legacy ExternalAuthenticationPermissions constants at four call sites --
policy management on create and update, session revocation, and unsafe
settings confirmation -- and those constants carry two colons, so nothing can
hold one once a deployment follows the migration guide. It now routes through
IPermissionEvaluator like the rest, resolved from the request with a fallback
to the shared evaluator, the same way EndpointSecurity does it.

Refs #7982

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style(external-auth): filter permission patterns with Where

Addresses a review nit on ValidatePermissionPatterns. Behaviour is unchanged:
a null list still iterates nothing, only malformed entries are reported, and
the message text is identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(external-auth)!: apply the grant boundary to role permissions too

Token issuance concatenated the user's Elsa role permissions raw alongside
the boundary-filtered external grants. A permission the boundary had just
excluded during grant resolution therefore reappeared in the issued token
from the same roles, which made the deny list unenforceable for anything a
role carried and left ElsaRolePermissionGrantSource filtering nothing that
was not added back a moment later. The bypass did not even need that grant
source configured: role permissions reached the token regardless of which
sources a connection selected.

Both origins now pass the same boundary. Re-applying it at issuance also
picks up a boundary that changed since sign-in, since refreshing reissues.

This is a behaviour change for deployments that configured a boundary
expecting it to bound only claim-mapped permissions: an external login may
now carry fewer permissions than before. Deployments with no boundary
configured, the default, are unaffected -- every well-formed permission
passes. The migration guide describes both directions.

Refs #7982

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 05:25:29 +02:00
Sipke Schoorstra c2fb027c41
Refactor: Overhauls workflow JSON type serialization (#7549)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Remove `ConsoleCaptureTee` and related services and tests

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Filter live console logs by workflow instance

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment

* Add diagnostics OpenTelemetry backend

* Introduce dedicated workflow JSON type registry and hardening

This change addresses GitHub issue #7541 by establishing a separate type registry (`IWorkflowJsonTypeRegistry`) for workflow JSON serialization. This decouples workflow type resolution from expression type aliases, enforcing a strict trust boundary.

Key aspects:
- New workflow JSON emits preferred aliases for registered types.
- Existing persisted workflows can be loaded via registered legacy names.
- Unknown, abstract, interface, open generic, or inappropriate collection types are rejected during deserialization, enhancing security.
- Public APIs (e.g., incident strategies) now expose consistent workflow JSON type identifiers.

This ensures secure, predictable, and backward-compatible handling of types within workflow definitions and payloads.

* Remove unused project references and streamline console log endpoint

* Move serialization type aliases to Elsa.Common

* Update serialization integration fixtures for aliases

* Stabilize missing rate limiter policy test
2026-05-31 11:09:39 +02:00
Sipke Schoorstra 842cf7c162
[codex] Fix console log metadata and type resolution (#7542)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Add OpenTelemetry diagnostics backend foundation

* Add OTLP HTTP ingestion parsing

* Document OpenTelemetry diagnostics setup

* Enforce OpenTelemetry hub permissions

* Remove `ConsoleCaptureTee` and related services and tests

* Add OpenTelemetry HTTP ingestion integration test

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Add OpenTelemetry ingestion security tests

* Add OpenTelemetry API authorization tests

* Filter live console logs by workflow instance

* Add OpenTelemetry hub tests

* Add OpenTelemetry gRPC metadata hook

* Assert OpenTelemetry workflow tags survive ingestion

* Mark OpenTelemetry core build verified

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment

* Wire OpenTelemetry diagnostics into core sample

* Address Core diagnostics review feedback

* Address Core Copilot follow-up feedback

* Add OpenTelemetry metric instrument names

* Address Core Copilot provider feedback

* Address Core Copilot diagnostics follow-up

* Address Core Copilot live feed feedback

* Address Core Copilot store feedback

* Integrate OpenTelemetry for logging, tracing, and metrics in ModularServer and update launch settings and docker-compose configuration.

* Refactor to replace `ConsoleLogStream.Core` with `ConsoleLogStreaming.Core` across codebase and update `ConsoleStreamHook` installation.

* Add diagnostics OpenTelemetry backend

* Fix OpenTelemetry live hub subscription

* Fix modular OpenTelemetry exporter endpoints

* Add CShells logging configuration in appsettings.json

* Remove obsolete unit tests and helper classes

* Restore default activity exception handling

* Simplify type serialization and alias management

This commit refactors the internal type serialization and alias management system to reduce boilerplate, improve robustness, and simplify the developer experience:

-   Removed numerous explicit `ExpressionOptions` type alias registrations across various modules.
-   Updated `TypeJsonConverter` and polymorphic serialization to reliably handle types using assembly-qualified names when a short alias is not explicitly registered.
-   Streamlined `ExcludeFromHashConverter` to strictly adhere to `ExcludeFromHashAttribute` for hash calculations, removing complex `JsonIgnoreCondition` logic.
-   Eliminated several helper classes (`WorkflowJsonTypeResolver`, `WorkflowTypeValidator`, `IWorkflowTypeRegistry`, `WorkflowFactoryDictionary`, `JavaScriptExceptionTypeAliasRegistrar`, `WorkflowRuntimeTypeAliasRegistrar`) and their associated unit tests, simplifying the codebase.

Additionally, this commit introduces a comprehensive markdown document (`product-website-feature-source.md`) outlining Elsa's core features, Studio capabilities, extension ecosystem, and architectural selling points, intended as source material for the product website.

* Refine type serialization for improved robustness and alias handling

This commit further enhances the type serialization and deserialization mechanisms:

*   Centralizes type resolution and alias management through `IWellKnownTypeRegistry` and `WorkflowJsonTypeResolver`.
*   Prioritizes registered type aliases when serializing type metadata in `PolymorphicObjectConverter`, resulting in more concise JSON output.
*   Enhances deserialization in `PolymorphicObjectConverter` and `VariableMapper` to gracefully handle unknown or non-instantiable types, providing fallbacks and logging warnings.
*   Simplifies `TypeJsonConverter` by delegating complex type resolution logic to the `WorkflowJsonTypeResolver`.
*   Adds `JsonArray` to the well-known type aliases for direct recognition.

* Fix console logs packaging and workflow type resolution

* Fix console log metadata and type resolution

* Address Copilot review feedback

* Enhance type resolution, improve console log handling, and update tests

- Streamlined `WorkflowDictionaryExtensions` for better workflow registration validation.
- Refined `ConsoleLogsAuthorizationTests` with the new `SetJsonRequest` helper to improve test requests handling.
- Updated `OrderDefinition` to ignore JSON serialization for `KeySelector`.
- Enhanced `WorkflowRuntimeFeature` for improved workflow registration and type alias configuration.
- Added tests to ensure `ConsoleLogProvider` metadata filtration in various scenarios.
- Improved type serialization logic in `WorkflowJsonTypeResolver`.
- Updated README to fix references related to diagnostics.
- Optimized `ExcludeFromHashConverter` for property serialization conditions.
- Modified `TriggerIndexer` for streamlined trigger management.
- Tested payload checks in `PublishEventTests`.
- Adjusted `Endpoint` in `ConsoleLogs` for automatic JSON request handling.
- Ensured registration of workflow type aliases in `WorkflowsFeature`.

* Restore CLR workflow registration compatibility

* Align JSON island serialization fixtures

* Add Console Logs Services and Enhance Endpoint Handling

- Introduced `ActivityExecutionsEndpointTests` to validate route exposure.
- Added `ConsoleLogCaptureHostedService` for console log streaming.
- Implemented `ConsoleStreamJsonConverter` for JSON conversion of console streams.
- Developed `ElsaConsoleLogRecentBuffer` to handle recent log buffering.
- Updated `ConsoleLogsAuthorizationTests` with new test cases for stream filter mapping.
- Consolidated console log provider dependencies and registration, including recent buffering.
- Enhanced `ElsaConsoleLogProvider` to use recent buffer for filtering.
- Adjusted `Program.cs` for streamlined logging service setup.

* Enhance type resolution and test coverage; streamline console log integration

- Added `ConsoleStreamHook` for streamlined log streaming.
- Updated `WorkflowJsonTypeResolverTests` to improve type resolution and test new scenarios.
- Simplified type resolution by removing trusted assembly checks.

* Fix CI smoke and package restore failures

* Fix Docker smoke image project paths

* Fix Docker Python runtime packages

* Fix Docker CA smoke teardown

* Refresh Elsa roadmap

* Implement background processors and mediation coordination

- Added `BackgroundCommandProcessor`, `BackgroundJobProcessor`, and `BackgroundNotificationProcessor` classes for handling commands, jobs, and notifications, respectively.
- Introduced `MediatorBackgroundProcessingCoordinator` to coordinate the execution of all background processors.
- Implemented `MediatorBackgroundTask` for wrapping `MediatorBackgroundProcessingCoordinator` in `BackgroundTask`.
- Added unit tests for `MediatorBackgroundTask` to ensure proper start and stop behavior.
- Refactored `BackgroundCommandSenderHostedService` to utilize `BackgroundCommandProcessor`.
- Introduced 'elsa-roadmap-refresh' skill configuration for roadmap updates.

* Address workflow type resolution review feedback

* Address follow-up review feedback

* Restore recent console logs execute path

* Address Copilot follow-up review

* Decouple workflow JSON aliases from expressions

* Fix workflow management unit test setup

* Fix console logs recent endpoint handler shape

* Respect workflow JSON strict type aliases

* Remove unused console log contracts reference

* Address Copilot review feedback

* Address Copilot follow-up comments

* Synchronize ring buffer dropped count

* Address background processor strategy replay
2026-05-30 22:52:01 +02:00
Sipke Schoorstra 5245599131
Enhance console logging with improved context and lifecycle (#7536)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Remove `ConsoleCaptureTee` and related services and tests

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Filter live console logs by workflow instance

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment
2026-05-27 00:14:28 +02:00
Sipke Schoorstra 8e301d4e1e
[codex] Scope console logs to workflow instances (#7535)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines
2026-05-25 11:49:51 +02:00
Sipke Schoorstra 43108c2e48
Add diagnostics console logs (#7462)
* feat: add diagnostics console logs

* test: avoid secret-like redaction fixtures

* fix: address console logs review feedback

* fix: harden console log capture lifecycle

* fix: report console log drop summaries

* fix: address diagnostics review cleanups
2026-05-18 02:16:46 +02:00