elsa-core/test/unit/Elsa.Resilience.Core.UnitTests/TransientExceptionDetectorTests.cs

152 lines
5.9 KiB
C#
Raw Normal View History

Enhances distributed lock handling with resilience (#7161) * Add retry mechanism for distributed locks with transient error handling and logging - Introduced Polly-based retry pipeline for distributed lock acquisition in `DistributedWorkflowClient` to handle transient errors such as network issues or database connection failures. - Added detailed logging for retry attempts and lock release errors. - Updated project dependencies to include Polly. * Refactor transient exception handling to shared resilience module. Migrated transient exception detection logic from scheduling module to a new shared resilience module. Updated services, jobs, and features to utilize the centralized `ITransientExceptionDetectionService`. This change improves maintainability and promotes reusability across modules. * Add unit tests for transient exception detection and resilience strategy evaluation. - Introduced comprehensive unit tests for `DefaultTransientExceptionDetector`, `ResilienceStrategyCatalog`, `ResilienceStrategyConfigEvaluator`, and `TransientExceptionDetectionService`. - Added helper classes and test data factories to facilitate reusable test patterns for resilience modules. - Updated solution to include `Elsa.Resilience.Core.UnitTests` project. * Add component tests for distributed lock resilience - Introduced new tests to verify retry behavior during transient lock acquisition and release failures. - Added `TestDistributedLockProvider` and related mocks for simulating transient failures. - Updated `WorkflowServer` test services to support the new distributed lock test scenarios. * Refactor distributed lock resilience tests - Consolidated test logic: streamlined test providers, injected services, and reusable test patterns. - Simplified `TestDistributedLockProvider` implementation with enhanced initialization and failure simulation. - Reorganized tests for transient acquisition/release failures to use parameterized `Theory` for improved maintainability. * Refactor transient exception handling: rename interfaces and classes for consistency, update references across codebase, and improve code readability. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Simplify WorkflowServer setup and DistributedLockResilienceTests by replacing IDistributedLockProvider with TestDistributedLockProvider. * Remove unused `using` directives in unit tests to improve code cleanliness. * Remove `TransientExceptionTypes` helper and inline its usage in tests for improved maintainability. * Add descriptive `DisplayName` attributes to unit tests for improved test clarity. * Fix redundant exception checking in TransientExceptionDetector (#7162) * Initial plan * Fix redundant exception checking in TransientExceptionDetector Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Extract MaxRetryAttempts constant in DistributedLockResilienceTests (#7164) * Initial plan * Extract MaxRetryAttempts constant to eliminate hardcoded magic number Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Make TestDistributedLockProvider thread-safe with Interlocked operations (#7163) * Initial plan * Make TestDistributedLockProvider thread-safe using Interlocked operations Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/component/Elsa.Workflows.ComponentTests/Scenarios/DistributedLockResilience/DistributedLockResilienceTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/Elsa.Workflows.Runtime.Distributed/Services/DistributedWorkflowClient.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/Elsa.Resilience.Core/Services/DefaultTransientExceptionStrategy.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Include `CancellationToken` in distributed lock handling methods for improved cancellation support. * Initial plan (#7168) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Cache detector list in TransientExceptionDetector to avoid repeated allocations (#7167) * Initial plan * Cache detector list in field to avoid repeated allocations Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Use IReadOnlyList instead of List for better intent expression Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Fix TestDistributedLockProvider registration to properly decorate IDistributedLockProvider (#7166) * Initial plan * Fix TestDistributedLockProvider registration to use Decorate pattern and fix variable reference bug Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Add runtime check for TestDistributedLockProvider registration Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Refactor `DistributedWorkflowClient` to simplify `Lazy<ResiliencePipeline>` initialization. * Add integration tests for DistributedWorkflowClient lock resilience (#7165) * Initial plan * Fix compilation error: use correct parameter name transientExceptionDetector Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Add integration tests for DistributedWorkflowClient lock resilience - Add SimpleWorkflow for testing distributed lock scenarios - Add tests exercising RunInstanceAsync with transient lock failures - Verify retry logic works correctly with actual workflow execution - Test both acquisition and release failure scenarios - Decorate IDistributedLockProvider to use TestDistributedLockProvider Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> * Address code review feedback - Add explanatory comment for TestDistributedLockProvider cast - Remove unnecessary blank line for consistent formatting Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com> Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com> * Simplify transient exception strategy by refactoring message pattern matching logic. * Enhance distributed lock mock to support per-lock failure configuration and improve resilience tests. * Refactor `TestDistributedLockProvider` to streamline failure handling logic and improve code clarity. * Remove unused methods and redundant test case from `DistributedLockResilienceTests`. * Refactor `DistributedLockResilienceTests` to simplify workflow client creation, consolidate assertion logic, and remove redundant test cases. * Format `ResilienceStrategyCatalogTests` by removing redundant line breaks in test setup. * Refactor `TransientExceptionDetectorTests` to simplify test setup, consolidate test cases, and remove redundant logic. * Handle `InvalidOperationException` in `XunitLogger` to suppress logging errors during inactive tests. * Update workflows to use .NET 10 and adjust resilience tests project configuration. * Refactor distributed runtime feature and integrations to improve resilience handling, configure services fluently, and add cancellation safeguards in background services. * Update `base_version` to `3.7.0` in GitHub workflow configuration. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfmskywalker <938393+sfmskywalker@users.noreply.github.com>
2025-12-29 18:53:48 +00:00
using NSubstitute;
namespace Elsa.Resilience.Core.UnitTests;
public class TransientExceptionDetectorTests
{
[Fact(DisplayName = "Service with no registered strategies should return false for any exception")]
public void IsTransient_NoDetectors_ReturnsFalse()
{
var detector = CreateDetector();
var exception = new Exception("test");
Assert.False(detector.IsTransient(exception));
}
[Fact(DisplayName = "Service should return true when any strategy detects the exception as transient")]
public void IsTransient_DetectorReturnsTrue_ReturnsTrue()
{
var exception = new Exception("test");
var strategy = CreateStrategy((exception, true));
var detector = CreateDetector(strategy);
Assert.True(detector.IsTransient(exception));
}
[Fact(DisplayName = "Service with multiple strategies should return true if any one detects as transient")]
public void IsTransient_MultipleDetectorsOneReturnsTrue_ReturnsTrue()
{
var exception = new Exception("test");
var strategy1 = CreateStrategy((exception, false));
var strategy2 = CreateStrategy((exception, true));
var detector = CreateDetector(strategy1, strategy2);
Assert.True(detector.IsTransient(exception));
}
[Fact(DisplayName = "Service should return false when all strategies detect the exception as non-transient")]
public void IsTransient_AllDetectorsReturnFalse_ReturnsFalse()
{
var exception = new Exception("test");
var strategy1 = Substitute.For<ITransientExceptionStrategy>();
var strategy2 = Substitute.For<ITransientExceptionStrategy>();
strategy1.IsTransient(Arg.Any<Exception>()).Returns(false);
strategy2.IsTransient(Arg.Any<Exception>()).Returns(false);
var detector = CreateDetector(strategy1, strategy2);
Assert.False(detector.IsTransient(exception));
}
[Theory(DisplayName = "Service should walk the inner exception chain to find transient exceptions")]
[MemberData(nameof(InnerExceptionChainTestCases))]
public void IsTransient_InnerExceptionChainHasTransient_ReturnsTrue(Exception exception, Exception transientException)
{
var strategy = Substitute.For<ITransientExceptionStrategy>();
strategy.IsTransient(transientException).Returns(true);
strategy.IsTransient(Arg.Is<Exception>(e => e != transientException)).Returns(false);
var detector = CreateDetector(strategy);
Assert.True(detector.IsTransient(exception));
}
[Theory(DisplayName = "Service should inspect AggregateException inner exceptions")]
[MemberData(nameof(AggregateExceptionTestCases))]
public void IsTransient_AggregateException_ChecksInnerExceptions(
AggregateException aggregateException,
Action<ITransientExceptionStrategy> configureDetector,
bool expectedResult)
{
var strategy = Substitute.For<ITransientExceptionStrategy>();
configureDetector(strategy);
var detector = CreateDetector(strategy);
Assert.Equal(expectedResult, detector.IsTransient(aggregateException));
}
[Fact(DisplayName = "AggregateException with mixed inner exceptions should be transient if any inner is transient")]
public void IsTransient_AggregateExceptionWithMultipleInnerOneTransient_ReturnsTrue()
{
var transientException = new TimeoutException("timeout");
var nonTransientException = new InvalidOperationException("invalid");
var aggregateException = new AggregateException("aggregate", nonTransientException, transientException);
var strategy = CreateStrategy(
(aggregateException, false),
(nonTransientException, false),
(transientException, true));
var detector = CreateDetector(strategy);
Assert.True(detector.IsTransient(aggregateException));
}
public static IEnumerable<object[]> InnerExceptionChainTestCases
{
get
{
var transientException = new TimeoutException("timeout");
yield return
[
new Exception("outer", transientException),
transientException
];
yield return
[
new Exception("outer", new("middle", transientException)),
transientException
];
}
}
public static IEnumerable<object[]> AggregateExceptionTestCases
{
get
{
// Scenario 1: One transient inner exception
var timeoutException = new TimeoutException("timeout");
yield return
[
new AggregateException("aggregate", timeoutException),
(Action<ITransientExceptionStrategy>)(detector =>
{
detector.IsTransient(Arg.Is<TimeoutException>(e => e.Message == "timeout")).Returns(true);
detector.IsTransient(Arg.Is<Exception>(e => e.GetType() != typeof(TimeoutException))).Returns(false);
}),
true
];
// Scenario 2: No transient inner exceptions
yield return
[
new AggregateException("aggregate", new InvalidOperationException("error1"), new ArgumentException("error2")),
(Action<ITransientExceptionStrategy>)(detector =>
{
detector.IsTransient(Arg.Any<Exception>()).Returns(false);
}),
false
];
}
}
private static ITransientExceptionStrategy CreateStrategy(params (Exception exception, bool isTransient)[] behaviors)
{
var detector = Substitute.For<ITransientExceptionStrategy>();
foreach (var (exception, isTransient) in behaviors)
detector.IsTransient(exception).Returns(isTransient);
return detector;
}
private static TransientExceptionDetector CreateDetector(params ITransientExceptionStrategy[] detectors) => new(detectors);
}