elsa-core/specs/005-structured-log-persistence/contracts/persistence-contract.md
Sipke Schoorstra 827ad6bc6b
[codex] Add structured log SQLite persistence (#7445)
* Add structured log persistence spec

* Clarify structured log persistence spec

* Plan structured log persistence implementation

* Regenerate structured log persistence tasks

* Address structured log persistence analysis findings

* Add structured log SQLite persistence

* Address structured log persistence review

* Harden structured log write buffer shutdown

* Start structured log SQLite migrations before buffer
2026-05-13 15:57:06 +02:00

3.4 KiB

Persistence Contract: Structured Log Persistence

IStructuredLogSink

Append-only destination for redacted structured log events.

public interface IStructuredLogSink
{
    ValueTask WriteAsync(StructuredLogEvent logEvent, CancellationToken cancellationToken = default);

    ValueTask WriteManyAsync(IReadOnlyCollection<StructuredLogEvent> logEvents, CancellationToken cancellationToken = default);
}

IStructuredLogStore

Queryable storage for redacted structured log events.

public interface IStructuredLogStore : IStructuredLogSink
{
    ValueTask<RecentStructuredLogsResult> QueryAsync(StructuredLogFilter filter, CancellationToken cancellationToken = default);

    ValueTask<IReadOnlyCollection<StructuredLogSource>> ListSourcesAsync(CancellationToken cancellationToken = default);
}

IStructuredLogLiveFeed

Live subscription source for SignalR.

public interface IStructuredLogLiveFeed
{
    ValueTask PublishAsync(StructuredLogEvent logEvent, CancellationToken cancellationToken = default);

    IAsyncEnumerable<StructuredLogStreamItem> SubscribeAsync(StructuredLogFilter filter, CancellationToken cancellationToken = default);
}

IRelationalStructuredLogConnectionFactory

Provider-owned connection creation.

public interface IRelationalStructuredLogConnectionFactory
{
    ValueTask<DbConnection> OpenConnectionAsync(CancellationToken cancellationToken = default);
}

IRelationalStructuredLogDialect

Provider-owned SQL differences.

public interface IRelationalStructuredLogDialect
{
    string ProviderName { get; }

    string QuoteIdentifier(string identifier);

    string ApplyLimit(string sql, int take);
}

Additional methods can be added when implementation reveals provider-specific differences for timestamps, free-text filtering, or JSON handling.

IStructuredLogSchemaMigrator

FluentMigrator-backed schema setup.

public interface IStructuredLogSchemaMigrator
{
    ValueTask MigrateAsync(CancellationToken cancellationToken = default);
}

IStructuredLogRetentionService

Durable cleanup boundary.

public interface IStructuredLogRetentionService
{
    ValueTask CleanupAsync(CancellationToken cancellationToken = default);
}

IStructuredLogWriteBuffer

Bounded async write queue used by durable stores.

public interface IStructuredLogWriteBuffer : IAsyncDisposable
{
    long DroppedWriteCount { get; }

    ValueTask EnqueueAsync(StructuredLogEvent logEvent, CancellationToken cancellationToken = default);

    ValueTask FlushAsync(CancellationToken cancellationToken = default);
}

The SQLite implementation drops newly received events when the queue is full, increments DroppedWriteCount, and logs warning summaries. It does not block logging calls and does not allocate unbounded memory. Existing REST and SignalR code continues to use IStructuredLogProvider as the facade.

Registration Sketch

Default in-memory:

services.AddElsa(elsa => elsa.UseStructuredLogs());

SQLite:

services.AddElsa(elsa =>
{
    elsa.UseStructuredLogs(structuredLogs =>
    {
        structuredLogs.UseSqliteStorage("Data Source=elsa-structured-logs.db");
    });
});

SQLite migrations run on startup by default. Retention deletes no records unless MaxAge, MaxRows, or both are configured. The exact fluent API may change during implementation to match existing Elsa feature conventions.