elsa-core/.github/workflows/packages.yml
Sipke Schoorstra ca268c16ad
Enhances distributed lock handling with resilience (#7161)
* Add retry mechanism for distributed locks with transient error handling and logging

- Introduced Polly-based retry pipeline for distributed lock acquisition in `DistributedWorkflowClient` to handle transient errors such as network issues or database connection failures.
- Added detailed logging for retry attempts and lock release errors.
- Updated project dependencies to include Polly.

* Refactor transient exception handling to shared resilience module.

Migrated transient exception detection logic from scheduling module to a new shared resilience module. Updated services, jobs, and features to utilize the centralized `ITransientExceptionDetectionService`. This change improves maintainability and promotes reusability across modules.

* Add unit tests for transient exception detection and resilience strategy evaluation.

- Introduced comprehensive unit tests for `DefaultTransientExceptionDetector`, `ResilienceStrategyCatalog`, `ResilienceStrategyConfigEvaluator`, and `TransientExceptionDetectionService`.
- Added helper classes and test data factories to facilitate reusable test patterns for resilience modules.
- Updated solution to include `Elsa.Resilience.Core.UnitTests` project.

* Add component tests for distributed lock resilience

- Introduced new tests to verify retry behavior during transient lock acquisition and release failures.
- Added `TestDistributedLockProvider` and related mocks for simulating transient failures.
- Updated `WorkflowServer` test services to support the new distributed lock test scenarios.

* Refactor distributed lock resilience tests

- Consolidated test logic: streamlined test providers, injected services, and reusable test patterns.
- Simplified `TestDistributedLockProvider` implementation with enhanced initialization and failure simulation.
- Reorganized tests for transient acquisition/release failures to use parameterized `Theory` for improved maintainability.

* Refactor transient exception handling: rename interfaces and classes for consistency, update references across codebase, and improve code readability.

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Simplify WorkflowServer setup and DistributedLockResilienceTests by replacing IDistributedLockProvider with TestDistributedLockProvider.

* Remove unused `using` directives in unit tests to improve code cleanliness.

* Remove `TransientExceptionTypes` helper and inline its usage in tests for improved maintainability.

* Add descriptive `DisplayName` attributes to unit tests for improved test clarity.

* Fix redundant exception checking in TransientExceptionDetector (#7162)

* Initial plan

* Fix redundant exception checking in TransientExceptionDetector

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Extract MaxRetryAttempts constant in DistributedLockResilienceTests (#7164)

* Initial plan

* Extract MaxRetryAttempts constant to eliminate hardcoded magic number

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Make TestDistributedLockProvider thread-safe with Interlocked operations (#7163)

* Initial plan

* Make TestDistributedLockProvider thread-safe using Interlocked operations

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/component/Elsa.Workflows.ComponentTests/Scenarios/DistributedLockResilience/DistributedLockResilienceTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/modules/Elsa.Resilience.Core/Services/DefaultTransientExceptionStrategy.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Include `CancellationToken` in distributed lock handling methods for improved cancellation support.

* Initial plan (#7168)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

* Cache detector list in TransientExceptionDetector to avoid repeated allocations (#7167)

* Initial plan

* Cache detector list in field to avoid repeated allocations

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Use IReadOnlyList instead of List for better intent expression

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Fix TestDistributedLockProvider registration to properly decorate IDistributedLockProvider (#7166)

* Initial plan

* Fix TestDistributedLockProvider registration to use Decorate pattern and fix variable reference bug

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add runtime check for TestDistributedLockProvider registration

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Refactor `DistributedWorkflowClient` to simplify `Lazy<ResiliencePipeline>` initialization.

* Add integration tests for DistributedWorkflowClient lock resilience (#7165)

* Initial plan

* Fix compilation error: use correct parameter name transientExceptionDetector

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Add integration tests for DistributedWorkflowClient lock resilience

- Add SimpleWorkflow for testing distributed lock scenarios
- Add tests exercising RunInstanceAsync with transient lock failures
- Verify retry logic works correctly with actual workflow execution
- Test both acquisition and release failure scenarios
- Decorate IDistributedLockProvider to use TestDistributedLockProvider

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

* Address code review feedback

- Add explanatory comment for TestDistributedLockProvider cast
- Remove unnecessary blank line for consistent formatting

Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>

* Simplify transient exception strategy by refactoring message pattern matching logic.

* Enhance distributed lock mock to support per-lock failure configuration and improve resilience tests.

* Refactor `TestDistributedLockProvider` to streamline failure handling logic and improve code clarity.

* Remove unused methods and redundant test case from `DistributedLockResilienceTests`.

* Refactor `DistributedLockResilienceTests` to simplify workflow client creation, consolidate assertion logic, and remove redundant test cases.

* Format `ResilienceStrategyCatalogTests` by removing redundant line breaks in test setup.

* Refactor `TransientExceptionDetectorTests` to simplify test setup, consolidate test cases, and remove redundant logic.

* Handle `InvalidOperationException` in `XunitLogger` to suppress logging errors during inactive tests.

* Update workflows to use .NET 10 and adjust resilience tests project configuration.

* Refactor distributed runtime feature and integrations to improve resilience handling, configure services fluently, and add cancellation safeguards in background services.

* Update `base_version` to `3.7.0` in GitHub workflow configuration.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
2025-12-29 19:53:48 +01:00

215 lines
7.1 KiB
YAML

name: Packages
on:
workflow_dispatch:
push:
branches:
- 'main'
- 'bug/*'
- 'perf/*'
- 'patch/*'
- 'feat/*'
- 'enh/*'
- 'rc/*'
- 'develop/*'
- 'codex/*'
release:
types: [prereleased, published]
env:
base_version: '3.7.0'
feedz_feed_source: 'https://f.feedz.io/elsa-workflows/elsa-3/nuget/index.json'
nuget_feed_source: 'https://api.nuget.org/v3/index.json'
jobs:
test:
name: Test with coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up .NET SDKs
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x
9.0.x
10.0.x
- name: Cache NuGet packages
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/Directory.Packages.props', '**/*.csproj') }}
restore-keys: |
${{ runner.os }}-nuget-
- name: Restore solution
run: dotnet restore Elsa.sln
- name: Build solution
run: dotnet build Elsa.sln --no-restore --configuration Release
- name: Prepare coverage directory
run: |
rm -rf artifacts/coverage
mkdir -p artifacts/coverage
- name: Run tests with coverage
run: |
set -euo pipefail
mapfile -t projects < <(find test -type f -name '*.csproj' | sort)
if [ ${#projects[@]} -eq 0 ]; then
echo "No test projects were found in the test directory." >&2
exit 1
fi
for project in "${projects[@]}"; do
echo "Running tests with coverage for ${project}"
dotnet test "$project" --configuration Release --no-build --logger "GitHubActions;report-warnings=false" /p:CollectCoverage=true
done
- name: Dump docker logs on failure
if: failure()
run: |
echo "=== Docker containers ==="
docker ps -a || true
echo "=== Docker logs ==="
for container in $(docker ps -aq); do
echo "--- Logs for container $container ---"
docker logs "$container" 2>&1 | tail -100 || true
done
- name: Install ReportGenerator
run: dotnet tool install -g dotnet-reportgenerator-globaltool
- name: Generate HTML coverage report
run: |
reportgenerator \
"-reports:./artifacts/coverage/**/coverage.cobertura.xml" \
"-targetdir:./artifacts/coverage-report" \
"-reporttypes:Html;Cobertura;TextSummary" \
"-verbosity:Info"
- name: Upload coverage reports
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-reports
path: artifacts/coverage
- name: Upload Pages artifact
if: always() && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop/3.6.0')
uses: actions/upload-pages-artifact@v3
with:
path: './artifacts/coverage-report'
build:
name: Build packages
needs: test
runs-on: ubuntu-latest
timeout-minutes: 30
if: ${{ github.event_name != 'pull_request' }}
steps:
- name: Extract branch name
run: |
BRANCH_NAME=${{ github.ref }} # e.g., refs/heads/main
BRANCH_NAME=${BRANCH_NAME#refs/heads/} # remove the refs/heads/ prefix
# Extract the last part after the last slash of the branch name, if any, e.g., feature/issue-123 -> issue-123 and use it as the version prefix.
PACKAGE_PREFIX=$(echo $BRANCH_NAME | rev | cut -d/ -f1 | rev | tr '_' '-')
# If the branch name is main, use the preview version. Otherwise, use the branch name as the version prefix.
if [[ "${BRANCH_NAME}" == "main" || "${BRANCH_NAME}" =~ ^develop/ ]]; then
PACKAGE_PREFIX="preview"
fi
echo "Ref: ${{ github.ref }}"
echo "Branch name: ${BRANCH_NAME}"
echo "Package prefix: ${PACKAGE_PREFIX}"
echo "BRANCH_NAME=${BRANCH_NAME}" >> $GITHUB_ENV
echo "PACKAGE_PREFIX=${PACKAGE_PREFIX}" >> $GITHUB_ENV
- name: Checkout
uses: actions/checkout@v4
- name: Verify commit exists in branch
run: |
if [[ "${{ github.ref }}" == refs/tags/* && "${{ github.event_name }}" == "release" && ("${{ github.event.action }}" == "published" || "${{ github.event.action }}" == "prereleased") ]]; then
git fetch --no-tags --prune --depth=1 origin +refs/heads/*:refs/remotes/origin/*
git branch --remote --contains | grep origin/main
else
git fetch --no-tags --prune --depth=1 origin +refs/heads/*:refs/remotes/origin/*
git branch --remote --contains | grep origin/${BRANCH_NAME}
fi
- name: Set VERSION variable
run: |
if [[ "${{ github.ref }}" == refs/tags/* && "${{ github.event_name }}" == "release" && ("${{ github.event.action }}" == "published" || "${{ github.event.action }}" == "prereleased") ]]; then
TAG_NAME=${{ github.ref }} # e.g., refs/tags/3.0.0
TAG_NAME=${TAG_NAME#refs/tags/} # remove the refs/tags/ prefix
echo "VERSION=${TAG_NAME}" >> $GITHUB_ENV
else
echo "VERSION=${{env.base_version}}-preview.${{github.run_number}}" >> $GITHUB_ENV
fi
- uses: actions/setup-dotnet@v4
with:
dotnet-version: 10.x
- name: Compile+Pack
run: ./build.sh Compile+Pack --version ${VERSION} --analyseCode true
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: elsa-nuget-packages
path: packages/*nupkg
if: ${{ github.event_name == 'release' || github.event_name == 'push'}}
publish_preview_feedz:
name: Publish to feedz.io
needs: build
runs-on: ubuntu-latest
timeout-minutes: 10
if: ${{ github.event_name == 'release' || github.event_name == 'push'}}
steps:
- name: Download Packages
uses: actions/download-artifact@v4.1.7
with:
name: elsa-nuget-packages
- name: Publish to feedz.io
run: dotnet nuget push *.nupkg -k ${{ secrets.FEEDZ_API_KEY }} -s ${{ env.feedz_feed_source }} --skip-duplicate
publish_nuget:
name: Publish release to nuget.org
needs: build
runs-on: ubuntu-latest
timeout-minutes: 10
if: ${{ github.event.action == 'published' }}
steps:
- name: Download Packages
uses: actions/download-artifact@v4.1.7
with:
name: elsa-nuget-packages
- name: Publish to nuget.org
run: dotnet nuget push *.nupkg -k ${{ secrets.NUGET_API_KEY }} -s ${{ env.nuget_feed_source }} --skip-duplicate
deploy_coverage:
name: Deploy coverage to GitHub Pages
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop/3.6.0'
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4