162 lines
8.6 KiB
Markdown
162 lines
8.6 KiB
Markdown
# Persistence
|
|
|
|
Elsa uses replaceable stores. Most features default to memory stores, then provider packages replace those stores with EF Core or other persistence implementations. Diagnostics structured logs also have a separate relational/SQLite persistence path that deliberately does not use EF Core.
|
|
|
|
## Store Replacement Pattern
|
|
|
|
Feature classes expose store factories. Persistence features replace those factories during `Configure()`.
|
|
|
|
Example from [EFCoreWorkflowRuntimePersistenceFeature](../../src/modules/Elsa.Persistence.EFCore/Modules/Runtime/WorkflowRuntimePersistenceFeature.cs):
|
|
|
|
- replace `WorkflowRuntimeFeature.TriggerStore`
|
|
- replace `BookmarkStore`
|
|
- replace `BookmarkQueueStore`
|
|
- replace `WorkflowExecutionLogStore`
|
|
- replace `ActivityExecutionLogStore`
|
|
- replace key-value store
|
|
|
|
Management persistence follows the same idea for definition and instance stores.
|
|
|
|
## EF Core Shared Infrastructure
|
|
|
|
Shared EF Core infrastructure lives in [Elsa.Persistence.EFCore.Common](../../src/modules/Elsa.Persistence.EFCore.Common) and [Elsa.Persistence.EFCore](../../src/modules/Elsa.Persistence.EFCore).
|
|
|
|
Important base types:
|
|
|
|
- [PersistenceFeatureBase](../../src/modules/Elsa.Persistence.EFCore.Common/PersistenceFeatureBase.cs)
|
|
- [PersistenceShellFeatureBase](../../src/modules/Elsa.Persistence.EFCore.Common/PersistenceShellFeatureBase.cs)
|
|
- [CombinedPersistenceShellFeatureBase](../../src/modules/Elsa.Persistence.EFCore.Common/CombinedPersistenceShellFeatureBase.cs)
|
|
- [ElsaDbContextBase](../../src/modules/Elsa.Persistence.EFCore.Common/ElsaDbContextBase.cs)
|
|
|
|
`PersistenceFeatureBase` registers `IDbContextFactory<TDbContext>`, migration options, tenant-aware context factory decoration, and tenant model handlers.
|
|
|
|
## EF Core Module Slices
|
|
|
|
The shared EF Core module contains slices for:
|
|
|
|
- [Management](../../src/modules/Elsa.Persistence.EFCore/Modules/Management)
|
|
- [Runtime](../../src/modules/Elsa.Persistence.EFCore/Modules/Runtime)
|
|
- [Identity](../../src/modules/Elsa.Persistence.EFCore/Modules/Identity)
|
|
- [Tenants](../../src/modules/Elsa.Persistence.EFCore/Modules/Tenants)
|
|
- [Labels](../../src/modules/Elsa.Persistence.EFCore/Modules/Labels)
|
|
- [Alterations](../../src/modules/Elsa.Persistence.EFCore/Modules/Alterations)
|
|
|
|
Each slice has a DbContext, configurations, store implementations, feature classes, and shell feature classes.
|
|
|
|
## Provider Packages
|
|
|
|
Provider packages configure database-specific EF Core options and migrations:
|
|
|
|
- [Elsa.Persistence.EFCore.Sqlite](../../src/modules/Elsa.Persistence.EFCore.Sqlite)
|
|
- [Elsa.Persistence.EFCore.SqlServer](../../src/modules/Elsa.Persistence.EFCore.SqlServer)
|
|
- [Elsa.Persistence.EFCore.PostgreSql](../../src/modules/Elsa.Persistence.EFCore.PostgreSql)
|
|
- [Elsa.Persistence.EFCore.MySql](../../src/modules/Elsa.Persistence.EFCore.MySql)
|
|
- [Elsa.Persistence.EFCore.Oracle](../../src/modules/Elsa.Persistence.EFCore.Oracle)
|
|
|
|
Combined provider shell features such as `SqliteWorkflowPersistenceShellFeature` let modular hosts configure workflow persistence once and share settings with dependent definition, instance, and runtime persistence features.
|
|
|
|
## Typical Host Configuration
|
|
|
|
The reference server configures SQLite persistence for management and runtime separately:
|
|
|
|
```csharp
|
|
elsa.UseWorkflowManagement(management =>
|
|
{
|
|
management.UseEntityFrameworkCore(ef => ef.UseSqlite());
|
|
});
|
|
|
|
elsa.UseWorkflowRuntime(runtime =>
|
|
{
|
|
runtime.UseEntityFrameworkCore(ef => ef.UseSqlite());
|
|
});
|
|
```
|
|
|
|
See [src/apps/Elsa.Server.Web/Program.cs](../../src/apps/Elsa.Server.Web/Program.cs).
|
|
|
|
## Migrations
|
|
|
|
EF Core migrations are controlled by feature options such as `RunMigrations`. The base persistence feature registers startup tasks that run migrations when enabled.
|
|
|
|
Migration-related files:
|
|
|
|
- [MigrationOptions](../../src/modules/Elsa.Persistence.EFCore.Common/MigrationOptions.cs)
|
|
- [RunMigrationsStartupTask](../../src/modules/Elsa.Persistence.EFCore.Common/RunMigrationsStartupTask.cs)
|
|
- [scripts/migrations/README.md](../../scripts/migrations/README.md)
|
|
|
|
When adding an entity or changing persisted shape, check every provider package and test provider-specific migration behavior where practical.
|
|
|
|
## Tenant Awareness
|
|
|
|
EF Core persistence decorates `IDbContextFactory<TDbContext>` with `TenantAwareDbContextFactory<TDbContext>` and registers model/saving handlers:
|
|
|
|
- `ApplyTenantId`
|
|
- `SetTenantIdFilter`
|
|
|
|
Tenant conventions are documented in ADRs:
|
|
|
|
- [ADR 0008: Empty String As Default Tenant ID](../adr/0008-empty-string-as-default-tenant-id.md)
|
|
- [ADR 0009: Asterisk Sentinel Value For Tenant-Agnostic Entities](../adr/0009-asterisk-sentinel-value-for-tenant-agnostic-entities.md)
|
|
|
|
## Structured Log Persistence
|
|
|
|
Structured log persistence is intentionally separate from EF Core. The active feature plan is [005 structured log persistence](../../specs/005-structured-log-persistence/plan.md).
|
|
|
|
Packages:
|
|
|
|
- [Elsa.Diagnostics.StructuredLogs](../../src/modules/Elsa.Diagnostics.StructuredLogs): core capture, API, hub, provider/store contracts, default in-memory store.
|
|
- [Elsa.Diagnostics.StructuredLogs.Persistence.Relational](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Relational): provider-neutral relational store, SQL builder, mapper, retention service, write buffer.
|
|
- [Elsa.Diagnostics.StructuredLogs.Persistence.Sqlite](../../src/modules/Elsa.Diagnostics.StructuredLogs.Persistence.Sqlite): SQLite connection factory, dialect, FluentMigrator runner, startup migration/cleanup service.
|
|
|
|
This path uses explicit SQL and FluentMigrator. It stores timestamps as UTC ISO-8601 text and JSON payloads as text in SQLite.
|
|
|
|
## Adding A Store
|
|
|
|
When adding a new store implementation:
|
|
|
|
1. Identify the feature contract that owns the store.
|
|
2. Keep the core module provider-neutral.
|
|
3. Add the concrete store in the persistence/provider module.
|
|
4. Replace the feature's store factory in the persistence feature.
|
|
5. Add unit tests for store-specific query behavior if logic is nontrivial.
|
|
6. Add integration tests for provider behavior, migrations, and multi-target concerns where practical.
|
|
|
|
## Persistence vNext
|
|
|
|
Persistence vNext is a next-generation, provider-neutral persistence system being developed in parallel with the existing EF Core path. It is currently in early design and not yet used by production Elsa modules.
|
|
|
|
### Design Goals
|
|
|
|
- Module authors declare **storage manifests** (storage units, fields, keys, and indexes) once, without referencing any database-specific package.
|
|
- Separate **planners** (relational and document) turn manifests into provider-specific plans.
|
|
- A **portable document/index store** wraps provider-specific backends so simple modules do not need custom EF Core, SQL, or MongoDB code.
|
|
- **Schema versioning** replaces per-provider EF Core migration packages.
|
|
|
|
### Package Layout
|
|
|
|
| Package | Role |
|
|
| --- | --- |
|
|
| [Elsa.Persistence.VNext](../../src/modules/Elsa.Persistence.VNext) | Core abstractions: storage manifests, document/index contracts, planner interfaces. |
|
|
| [Elsa.Persistence.VNext.Extensions](../../src/modules/Elsa.Persistence.VNext.Extensions) | Feature registration and options. |
|
|
| [Elsa.Persistence.VNext.Runtime](../../src/modules/Elsa.Persistence.VNext.Runtime) | Runtime-specific entity definitions and evaluation helpers. |
|
|
| [Elsa.Persistence.VNext.Relational](../../src/modules/Elsa.Persistence.VNext.Relational) | Shared relational store behavior and SQL rendering. |
|
|
| [Elsa.Persistence.VNext.Sqlite](../../src/modules/Elsa.Persistence.VNext.Sqlite) | SQLite physicalization: DDL renderer, type mapper, schema version runner, document store. |
|
|
| [Elsa.Persistence.VNext.PostgreSql](../../src/modules/Elsa.Persistence.VNext.PostgreSql) | PostgreSQL physicalization. |
|
|
| [Elsa.Persistence.VNext.SqlServer](../../src/modules/Elsa.Persistence.VNext.SqlServer) | SQL Server physicalization. |
|
|
| [Elsa.Persistence.VNext.MongoDb](../../src/modules/Elsa.Persistence.VNext.MongoDb) | Native MongoDB physicalization using collections and indexes. |
|
|
|
|
Spec: [specs/011-persistence-vnext/spec.md](../../specs/011-persistence-vnext/spec.md).
|
|
|
|
### When To Use Persistence vNext
|
|
|
|
Today, use the existing EF Core path for all production Elsa modules. Persistence vNext work belongs in the `011-persistence-vnext` feature branch. Consult the spec and plan before contributing to avoid duplicating EF Core infrastructure for new module entities.
|
|
|
|
## Persistence Risk Checklist
|
|
|
|
- Does the change affect multiple target frameworks?
|
|
- Does it need provider-specific migrations?
|
|
- Does it preserve tenant filtering?
|
|
- Does it update both definition and instance stores if both shapes changed?
|
|
- Does it require API/client DTO updates?
|
|
- Does it alter runtime recovery, bookmark, or trigger semantics?
|
|
- Does it need retention, cleanup, or migration documentation?
|