elsa-core/doc/qa/test-guidelines-integrators.md

49 lines
3.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

- **Bulk provisioning & isolation** — tests should support bulk import of workflow definitions for large-suite runs and ensure clean, isolated state per test.
- **Endtoend tests**: Deploy Elsa Server (or host app) in Docker/K8s with a real DB, then run workflows via REST and assert via durable traces (journal/DB/events). Keep E2E suite small and targeted.
### Importing and publishing workflow definitions for testing workflows
Two main patterns:
**A. Codefirst registration (recommended for component tests)**
- Register workflows as code inside test setup. This is fast and avoids serialization roundtrips.
- Good for tests that validate runtime behavior without involving persistence or designer serialization.
**B. Serialized definitions (recommended for integration & E2E tests)**
- Store JSON workflow definition artifacts in the `tests/definitions/` folder in the repo, commit them with semantic versions, and let tests import them into the engine via the same publish/import APIs used in production.
- Tests are responsible for *publishing* the definitions into the test host if the scenario requires persistence (e.g. testing versioned definitions or import behavior).
**Which to use?**
- Component tests: code-first definitions or workflows created via designer and exported to JSON.
- Integration/E2E tests: use serialized artifacts to validate persistence, designer output and versioning behavior.
### Bulk import
- Implement a **test importer utility** that accepts a directory of workflow definition artifacts (JSON/YAML) and publishes them via the engine's public API or directly seeds the persistence store. The utility should:
- Validate schema and version.
- Report conflicts or duplicate IDs.
- Run in parallel but enforce deterministic ordering when versions matter.
- For very large import workloads, support an optimized DB seed path used only in tests (direct DB insert) to avoid the overhead of the full publish pipeline. Mark this as *test-only*.
### Working with Docker, env, K8s cluster deployments
- **Test strategy split**:
- Local developer tests: use inprocess hosts and inmemory/ephemeral DBs (SQLite in-memory or Testcontainers-based DB). Fast and deterministic.
- CI Docker Compose: spin up a lightweight containerized environment with the host app, a real DB (Postgres, SQL Server or Mongo) and optional message broker; use Testcontainers (or Docker Compose) to orchestrate in CI.
- K8s E2E: run a small suite that deploys a test namespace with Helm or apply manifests. Use ephemeral resources and ensure cleanup. Keep these tests in a separate CI stage.
- **Use Testcontainers** (or equivalent) to provision ephemeral DBs/brokers in CI; this keeps environments close to production while still being isolated and reproducible.
- **Configuration**: Keep environment variables and k8s manifests in `tests/ci/` and parametrize connection strings so tests can switch between inprocess and containerized runs.
### CI recommendations
- **Local unit tests**: run in `dotnet test` step (fast). Use inmemory stores and determinism.
- **Integration tests**: run with Testcontainers (or similar) to provide real DB/broker. Run these in a separate CI job because they are slower.
- **K8s smoke tests**: optional separate stage. Deploy to ephemeral namespace via Helm and run a small suite of smoke E2E tests; tear down after.
- **Parallelization**: run multiple test matrices (DB providers) but avoid running heavy E2E jobs in parallel unless you have isolated resources.
- **Flaky test detection**: enable a flaky test retry policy for known nondeterministic tests, but treat retries as signals to fix the underlying determinism problems.