* fix: persist Interrupted after drain force-cancel commits Cancelled
Deadline-breach force-cancel makes the runner persist Finished/Cancelled.
#8059 then skipped every Finished row, so Interrupted never landed and
Packages CI failed DeadlineBreachPersistsInterrupted. Treat Cancelled as
interruptible and promote it to Running+Interrupted so recovery can
requeue it, while still refusing naturally completed rows.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: do not promote user cancellations to Interrupted on drain
Gate Cancelled→Interrupted on instances that were not already Cancelled
when drain snapshotted live cycles. Deadline-breach force-cancel still
promotes the runner's Finished/Cancelled commit; ordinary client
cancellations stay Cancelled and are not requeued.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: confine Cancelled→Interrupted promote to Drain
Restore TryMarkInterruptedAsync to refuse every Finished row by default
(#8052). Drain PersistInterrupted alone may pass allowFinishedCancelled
when the instance is Finished/Cancelled and in this drain's
force-cancelled set. User cancellations stay cancelled.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* docs: document TryMarkInterruptedAsync parameters
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: bound pre-cancel snapshot so stalled Find cannot block Cancel
WaitAsync the instance-store snapshot under a short shutdown budget so
a hang or ignored cancellation token cannot delay handle.Cancel().
Unknown pre-state is not treated as already Cancelled; observed
user cancellations are still preserved.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* test: restore Fact on terminal-race drain persist skip
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: exclude unknown snapshot rows from drain-induced promote
A timed-out or failed pre-cancel Find no longer joins drainInduced.
Only a successful read that is clearly not already Cancelled may be
promoted. Cancel still proceeds without waiting on store latency.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: give each pre-cancel snapshot its own timeout
A shared 250ms overallSnapshotCts let a stalled first Find cancel later
Finds before they started, so those instances were excluded from
drainInduced and never persisted as Interrupted after Phase A Cancel.
Each Find now uses an independent CTS linked only to the host token.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: snapshot live instances concurrently before Phase A Cancel
Independent per-find 250ms budgets kept recovery, but a serial foreach
still delayed every handle.Cancel by up to N×250ms. Run those bounded
Finds with Task.WhenAll so Cancel waits one timeout window, not N.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: give each parallel snapshot Find its own DI scope
Task.WhenAll was sharing one scoped IWorkflowInstanceStore. EF DbContext
is not thread-safe; Phase C already persists sequentially for that reason.
Each snapshot task now CreateScope()s its own store and disposes it.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix: stop requeuing Finished+Interrupted workflow instances
InterruptedRecoveryScanner now requires Status=Running, matching the
sibling crash-recovery task. DrainOrchestrator skips already-terminal
instances so a runner-clobber race cannot stamp Interrupted onto a
Finished row. Fixes#8052.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: conditionally mark Interrupted so drain cannot clobber Finished
PersistInterruptedAsync no longer SaveAsync-es the Find snapshot. The
store now applies Interrupted only when Status is still non-terminal
(EF: ExecuteUpdate WHERE Status != Finished; memory: mutate the live
row). A runner that commits Finished between read and write keeps its
terminal state, so startup recovery cannot requeue completed work.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* test: disambiguate NSubstitute Returns for TryMarkInterruptedAsync
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: make in-memory Interrupted mark atomic against completion
Lock the memory-store check and mutations together, then abort if Status
became Finished in-place so drain cannot record Interrupted on a
completed instance.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
* fix: share memory-store lock between Interrupted mark and Save
TryMarkInterruptedAsync now serializes with Save/Update/SaveMany so a
runner's terminal persist cannot land between the non-terminal check
and the Interrupted mutations. A finishing Save therefore cannot leave
Finished+Interrupted.
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Implement Weaver AI Copilot core
* Address Greptile review feedback
* Address Greptile persistence feedback
* Address Greptile orchestration feedback
* Address Greptile tool isolation feedback
* Wire chat audit events
* Stream chat events over SSE
* Use server identity for AI endpoints
* Validate AI proposal persistence
* Isolate AI audit failures
* Enforce AI tool lookup scope
* Support AI tool result continuations
* Handle AI chat reconnects safely
* Tighten AI context and reconnect behavior
* Guard AI conversation and persistence setup
* Persist AI tool-loop progress
* Tighten AI tool registry and reconnect cleanup
* Handle AI preparation failures cleanly
* Order AI tool messages after assistant turns
* Initialize AI provider sessions
* Align AI context capabilities
* Prevent completed AI reconnect replay
* Enforce AI conversation ownership
* Default AI proposal creation time
* Persist AI session and retention defaults
* Allow AI context provider overrides
* Scope AI tool results per turn
* Apply AI provider configuration
* Scope AI proposal reads
* Avoid duplicate AI tool continuations
* Resolve AI tool registry scopes
* Tighten AI reconnect cleanup
* Honor default AI proposal tools
* Pass AI provider session to turns
* Close AI observability gaps
* Fix AI capabilities options alias
* Harden AI orchestration lifetimes
* Track actual AI reconnect conversation
* Address AI audit and context review findings
* Fix AI reconnect and persistence capabilities
* Handle AI session startup failures
* Tighten AI orchestration review gaps
* Warn on placeholder AI context
* Filter disabled AI provider tools
* Add durable AI conversation persistence
* Fix AI orchestrator persistence lifetime
* Handle failed AI reconnect edge cases
* Harden AI reconnect failure handling
* Address AI reconnect and cleanup review gaps
* Tighten AI audit and cleanup persistence
* Keep expired AI cleanup best effort
* Tighten AI tool lookup and cleanup fallback
* Handle AI provider and tenant edge cases
* Tighten AI proposal and agent authorization
* Address AI tool scope cleanup review
* Close remaining AI greptile findings
* Harden AI stores and tool defaults
* Harden AI conversation persistence edge cases
* Cover AI proposal and tool visibility guards
* Fix AI capabilities and audit batch resilience
* Fix AI conversation truncation for unicode
* Resolve remaining AI persistence review nits
* Wire AI conversation persistence option
* Address AI audit and proposal style review
* Fix AI stream truncation surrogate handling
* Address AI context and cleanup review
* Preserve AI titles and tenant tool defaults
* Guard AI conversation user ownership
* Align in-memory AI conversation ownership
* Fix expired AI conversation cleanup tracking
* Harden AI proposal persistence retry
* Tighten AI proposal reads and cleanup SQL
* Harden AI reconnect and provider defaults
* Optimize AI tool listing and message trimming
* Preserve AI conversation timestamps
* Address final AI persistence review nits
* Normalize AI acronym casing
* Address Copilot AI review comments
* Normalize default tenant handling for AI stores
* Harden AI registry and message truncation
* Make AI tool filtering explicit
* Align AI contracts with implementation
* Align remaining AI review contracts
* address greptile ai persistence feedback
* Address Copilot AI persistence feedback
* Address Copilot AI host feedback
* Order persisted AI conversation messages
* Address Copilot chat and cleanup feedback
* Release unused AI reconnect reservations
* Address Copilot AI review feedback
* Address Copilot tool and conversation feedback
* Address Copilot governance feedback
* Address Copilot tool test feedback
* Address AI review follow-ups
* Address Copilot AI follow-ups
* Clean up AI persistence tests
* Address IAITool disposal review
* Address AI integration review follow-ups
* Address AI chat persistence review
* Address AI registry and truncation review
* Enable read-only AI tools by default
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
