* fix(core): a throwing FaultSignal handler must not escape the middleware (#7911)
The signal is sent from inside the catch whose whole job is to stop exceptions
escaping the activity pipeline. A handler that threw went straight through it:
no incident, no strategy, and the original fault lost along with it.
The send is now guarded. A handler that throws is treated as not having handled
the fault, so the incident strategy runs exactly as it would with no handler
present. That is the conservative direction: a handler that failed part way
through may have left the faulted activity in any state, and an incident is a
better answer than silence. Its exception is logged at error level, because a
broken fault handler is a defect in its own right rather than a workflow
outcome.
Covered both ways, since a handler that already claimed the fault before
throwing is the case that could plausibly have been mistaken for success.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(core): let cancellation from a fault handler propagate
The handler guard caught OperationCanceledException along with everything else,
so a cancellation raised while an ancestor was being offered a fault was logged
as a broken handler and handed to the incident strategy. A deliberately
cancelled run reported itself faulted.
Cancellation is excluded now, matching how this repository already keeps the two
apart: the workflow-level exception middleware cancels and rethrows before its
general catch, and WorkflowRunner declines to record cancellation as the
workflow's exception.
Caught by review on #7924.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(core): pin that a handled fault stays in the execution log
The justification for dropping the incident is that the journal keeps the
evidence. That was asserted in several places and guarded nowhere.
It holds because ExecutionLogMiddleware writes the Faulted entry from a catch
that rethrows, and it is registered inside ExceptionHandlingMiddleware, so the
entry lands before the fault is ever offered to an ancestor. Swapping those two
registrations would make a handled failure disappear from the record with
nothing failing, which is what this test now prevents.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(core): a fault a container claimed is not an incident (#7911)
RecoverFromFault reset the counts and the status but left behind the two other
things Fault recorded: the ActivityIncident and the exception. So a container
that successfully handled a child's fault still left the workflow carrying an
incident.
That is not cosmetic. Code reads a non-empty WorkflowExecutionContext.Incidents
as "this workflow failed" without looking further; HttpWorkflowsMiddleware is
one, and it hands the caller a fault response. A workflow whose container caught
the error and finished normally was reported to its caller as failed.
RecoverFromFault is now the inverse of Fault: it removes the incident Fault
appended, matched on this activity's node id and most recent first so an
activity that faults, recovers and faults again keeps the incident that was
never recovered, and it clears the recorded exception so the activity does not
sit in Running carrying one.
The execution log still records the failure, so nothing is hidden from anyone
reading the journal. Two integration assertions that encoded the old behaviour
are updated; they were written from the reasoning this change corrects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(core): tie an incident to the execution that raised it, not its node
Recovery matched the incident to remove on ActivityNodeId, which identifies the
static workflow node rather than an execution of it. A node inside a loop,
retried, or run concurrently raises one incident per execution, all under the
same node id, so recovering one execution could remove another's incident and
leave its own behind.
ActivityIncident now carries the ActivityInstanceId of the execution that raised
it, and recovery matches on that. Within a single execution the most recent is
still taken, so fault, recover, fault again keeps the incident that was never
recovered. The property is optional: an incident recorded against the workflow
itself has no execution, and so do incidents persisted before this existed.
Caught by review on #7923.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(api-client): mirror ActivityInstanceId on the client incident model
The server model gained the property in the previous commit and the API client
carries a hand-maintained copy of it. Left alone, a client deserializing an
incident would silently drop the only field that says which execution raised it.
Also records two consequences of recovery that were implicit: it relies on the
incident collection preserving insertion order to pick an execution's newest
incident, which holds only because the collection is list-backed; and clearing
the exception also clears it from the activity's execution record, which is
intended for the same reason the incident goes, with the journal keeping the
evidence either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Review caught that the contract advertised something that cannot work. It
offered handlers three ways to terminalize the faulted activity - cancel,
complete, or reschedule - but CompleteActivityAsync returns immediately unless
the activity is Running, and throughout the handler it is still Faulted, since
recovery runs only after the handler returns. Completing inline did nothing at
all, silently, leaving the child Running.
Measured, same container, handler completing the faulted child:
inline complete -> child Running, no output, Running/Suspended
TransitionTo(Running), complete -> child Completed, "after", Finished/Finished
So a supported path exists; it just needed writing down. Document it on
FaultSignal, note that it is not licence to call RecoverFromFault (which also
rewrites the fault counts), and note that cancelling and rescheduling need no
equivalent step. Cover it with an integration test asserting that completing the
child with a substitute result fires the container's completion callback and
resumes its sequencing.
Refs #7911
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review raised that TrySendSignalAsync delivers to the faulting activity before
walking ancestors, so an activity that throws and also handles FaultSignal can
claim its own fault and suppress the incident strategy.
That is real, but it is the channel's existing dispatch, which #7911 chose
deliberately over a variant of it, and SignalContext.IsSelf exists so handlers
can discriminate. It also grants no capability: an activity that catches its own
exception never faults at all, ending Finished/Finished with zero incidents,
which is a cleaner suppression than self-handling (incident still recorded,
activity left Running, workflow suspended).
So dispatch is unchanged. What was missing is that none of this was written
down: the contract describes the handler as an enclosing container and never
mentioned self-receipt. Document it on FaultSignal, including how a handler
that wants ancestors-only semantics opts out, and add a test so the behavior is
pinned rather than incidental.
Refs #7911
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A container activity had no way to learn that one of its children faulted.
ExceptionHandlingMiddleware caught the exception, called context.Fault(e) and
handed off to the workflow-global IIncidentStrategy; the container's completion
callback never fired, because the child never completed.
Add a seam on the ancestor-bubbling signal channel that already exists:
- FaultSignal(Exception, ActivityExecutionContext), beside CancelSignal. Its XML
doc carries the contract, including why a handler must not call
RecoverFromFault and why the CompleteActivityAsync sweep is a backstop rather
than the mechanism.
- An internal bool-returning TrySendSignalAsync, since SignalContext
.StopPropagationRequested is internal and SendSignalAsync reported nothing.
SendSignalAsync keeps its public signature and delegates to it.
- ExceptionHandlingMiddleware sends the signal after faulting and, when an
ancestor stops propagation, calls RecoverFromFault once and returns instead of
raising an incident.
RecoverFromFault now transitions to Running only when the activity is still
Faulted. It is called after the handler runs, so the unconditional transition
would otherwise undo a handler that cancelled or completed the faulted child.
The counts are still reset unconditionally, and the one pre-existing caller is
unaffected.
Behavior is unchanged when nobody handles the signal: verified by running the
new unhandled-fault theory against the pre-change middleware, and by
IncidentStrategyTests and Primitives/FaultTests passing unmodified.
Refs #7911
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
