Testing Payment Retries Without Charging Customers Twice

Payment processing is one of the areas where retry logic needs to be designed carefully. A temporary network failure can make a payment request appear unsuccessful even when the payment provider has already processed it.

For example, a customer clicks the Pay button, and your application sends a request to a payment gateway. The gateway charges the card successfully, but the response never reaches your application because the connection times out. If your application immediately sends another payment request, the customer could be charged twice.

The difficult part is that a timeout does not tell you whether the payment failed. It only tells you that your application did not receive a response within the expected time.

Safe payment retries require idempotency, reliable payment-state tracking, and tests that simulate ambiguous outcomes. In this article, we will explore how to implement these safeguards in C#, how to test failure scenarios without making real charges, and how to verify that retry logic does not create duplicate payments.

Why Payment Retries Can Cause Duplicate Charges

Consider a checkout service that sends a payment request to a provider.

The provider processes the payment and charges the customer. Before the response reaches the application, the network connection breaks. The application receives a timeout exception and assumes that the request failed.

A retry sends the same payment request again. If the provider treats it as a new operation, it may create another charge.

The problem is that the application cannot reliably determine the outcome from the network exception alone.

Scenario	What the application knows	Correct response
The request fails before reaching the provider	The payment may not have been submitted	Retry according to the provider's rules
The provider processes the payment, but the response is lost	The payment outcome is unknown	Retry using the same idempotency key or query the provider
The provider returns a confirmed decline	The payment was declined	Do not retry blindly; follow the payment method and decline policy
The provider returns a temporary server error	The outcome depends on the provider's documented semantics	Follow its retry guidance and reconcile uncertain outcomes
The application receives a duplicate webhook	The event may already have been processed	Deduplicate the event and make processing idempotent

A reliable payment system distinguishes between a confirmed failure and an unknown outcome. Treating both as the same condition is a common cause of duplicate charges.

Use an Idempotency Key for Every Logical Payment

An idempotency key identifies one logical payment operation. When the application retries that operation, it sends the same key instead of creating a new one.

For example, a checkout request might contain:

POST /payments
Idempotency-Key: checkout-order-7842-attempt-1
Content-Type: application/json

{
  "orderId": "ORD-7842",
  "amount": 1499.00,
  "currency": "INR"
}

The key should be generated and persisted before the first provider request. If the request times out, the application reuses that key when retrying the same payment.

Do not generate a new key for every HTTP attempt. Doing so makes each retry appear to be a separate payment.

Also, do not blindly derive an idempotency key from an amount or customer ID. A customer may legitimately make two purchases for the same amount. The key should identify the logical payment operation, not merely its attributes.

Most importantly, the payment provider must support idempotency for the key to prevent duplicate provider-side operations. An idempotency key generated only in your own application cannot stop a gateway from processing two independent requests as two charges.

The provider's rules determine how long keys are retained, which request fields must remain identical, and what happens when a request with the same key is repeated. Your retry policy must respect those rules.

Model Payment States Explicitly

Payment operations should not be represented by a single Boolean such as IsPaid. A payment can be pending, confirmed, declined, or in an uncertain state while the application investigates the result.

A simple state model might look like this:

public enum PaymentStatus
{
    Pending,
    Processing,
    Succeeded,
    Failed,
    OutcomeUnknown
}

public sealed class PaymentAttempt
{
    public Guid Id { get; set; }

    public required string OrderId { get; set; }

    public required string IdempotencyKey { get; set; }

    public required string Currency { get; set; }

    public decimal Amount { get; set; }

    public PaymentStatus Status { get; set; }

    public string? ProviderPaymentId { get; set; }

    public DateTimeOffset CreatedAtUtc { get; set; }
}

The OutcomeUnknown state is particularly important. It represents a situation where the application cannot yet establish whether the provider processed the payment.

For example, a timeout after submitting the request should not automatically change the payment to Failed. The application should retain the idempotency key and reconcile the result through the provider's supported mechanisms.

Persist payment attempts in a database. A process restart should not cause the application to forget which key belongs to an order or whether a previous attempt might already have succeeded.

Separate Payment Creation from Retry Attempts

A payment retry should be a continuation of the same logical payment operation, not the creation of a new one.

The following example illustrates the structure of a payment service. The provider client and repository are application-defined abstractions; the actual implementation must follow the payment gateway's API contract.

public sealed record PaymentRequest(
    string OrderId,
    decimal Amount,
    string Currency,
    string IdempotencyKey);

public sealed record PaymentResult(
    bool IsSuccess,
    bool IsOutcomeUnknown,
    string? ProviderPaymentId);

public interface IPaymentProvider
{
    Task<PaymentResult> ChargeAsync(
        PaymentRequest request,
        CancellationToken cancellationToken);
}

public interface IPaymentRepository
{
    Task<PaymentAttempt?> FindByOrderAsync(
        string orderId,
        CancellationToken cancellationToken);

    Task SaveAsync(
        PaymentAttempt payment,
        CancellationToken cancellationToken);
}

The service should load the existing payment attempt or create one with a stable key before contacting the provider.

public sealed class PaymentService
{
    private readonly IPaymentProvider _provider;
    private readonly IPaymentRepository _repository;

    public PaymentService(
        IPaymentProvider provider,
        IPaymentRepository repository)
    {
        _provider = provider;
        _repository = repository;
    }

    public async Task<PaymentResult> ProcessAsync(
        PaymentAttempt payment,
        CancellationToken cancellationToken)
    {
        if (payment.Status == PaymentStatus.Succeeded)
        {
            return new PaymentResult(
                true,
                false,
                payment.ProviderPaymentId);
        }

        payment.Status = PaymentStatus.Processing;

        await _repository.SaveAsync(
            payment, cancellationToken);

        var request = new PaymentRequest(
            payment.OrderId,
            payment.Amount,
            payment.Currency,
            payment.IdempotencyKey);

        try
        {
            PaymentResult result = await _provider.ChargeAsync(
                request, cancellationToken);

            if (result.IsOutcomeUnknown)
            {
                payment.Status = PaymentStatus.OutcomeUnknown;
            }
            else if (result.IsSuccess)
            {
                payment.Status = PaymentStatus.Succeeded;
                payment.ProviderPaymentId =
                    result.ProviderPaymentId;
            }
            else
            {
                payment.Status = PaymentStatus.Failed;
            }

            await _repository.SaveAsync(
                payment, cancellationToken);

            return result;
        }
        catch (OperationCanceledException)
            when (cancellationToken.IsCancellationRequested)
        {
            // Cancellation does not prove that the provider
            // failed to process the request.
            payment.Status = PaymentStatus.OutcomeUnknown;

            await _repository.SaveAsync(
                payment, CancellationToken.None);

            throw;
        }
        catch (TimeoutException)
        {
            payment.Status = PaymentStatus.OutcomeUnknown;

            await _repository.SaveAsync(
                payment, cancellationToken);

            return new PaymentResult(
                false,
                true,
                payment.ProviderPaymentId);
        }
    }
}

This code illustrates the state transitions, but it is not a complete payment implementation. In particular, the repository must enforce concurrency rules, and the provider adapter must classify errors according to the gateway's documented behavior. A real HTTP client may report timeouts through exceptions other than TimeoutException, so the adapter should normalize those outcomes deliberately.

The example also highlights a difficult failure window: the provider can complete a charge while the application fails before saving the result. A database status alone cannot eliminate that uncertainty. A recovery process must query the provider or use its supported webhook and reconciliation mechanisms.

Do not automatically treat every exception as a confirmed decline. When the outcome is uncertain, retain the original key and reconcile before creating a new logical payment.

Prevent Two Requests from Starting the Same Payment

Idempotency at the payment provider is important, but your own application also needs to prevent duplicate payment attempts caused by concurrent requests.

Imagine that a customer double-clicks Pay or that a mobile client retries an HTTP request while the first request is still running. Two application instances may try to create a payment attempt for the same order.

A database uniqueness constraint can help enforce the business rule. For example, if an order may have only one active logical payment, the application can maintain a unique operation record for that order and reuse its idempotency key.

The exact constraint depends on the payment model. An order may legitimately have multiple payment attempts after a confirmed decline, but it should not accidentally create two concurrent charges for the same checkout operation.

Use a database transaction to create the payment attempt and its idempotency key before calling the provider. Do not hold a database transaction open while waiting for a remote payment request. Instead, persist the operation, call the provider, and update the state afterward.

For complex workflows, a durable background worker and an outbox pattern can help ensure that payment work is not lost when a process restarts.

Test Payment Retries Without Real Charges

The safest way to test retry behavior is to separate the payment logic from the actual gateway implementation. A test can inject a fake provider that simulates timeouts, successful charges, declines, and repeated requests.

The following example uses xUnit and a fake provider to verify that a payment retry reuses the same idempotency key.

using Xunit;

public sealed class FakePaymentProvider : IPaymentProvider
{
    public List<string> ReceivedKeys { get; } = [];

    private int _calls;

    public Task<PaymentResult> ChargeAsync(
        PaymentRequest request,
        CancellationToken cancellationToken)
    {
        ReceivedKeys.Add(request.IdempotencyKey);
        _calls++;

        if (_calls == 1)
        {
            throw new TimeoutException(
                "Simulated response timeout.");
        }

        return Task.FromResult(
            new PaymentResult(
                true,
                false,
                "provider-payment-123"));
    }
}

public sealed class PaymentRetryTests
{
    [Fact]
    public async Task Retry_UsesSameIdempotencyKey()
    {
        var provider = new FakePaymentProvider();

        var request = new PaymentRequest(
            "ORD-7842",
            1499.00m,
            "INR",
            "payment-operation-7842");

        try
        {
            await provider.ChargeAsync(
                request, CancellationToken.None);
        }
        catch (TimeoutException)
        {
            // Simulate retrying the same logical operation.
        }

        PaymentResult result = await provider.ChargeAsync(
            request, CancellationToken.None);

        Assert.True(result.IsSuccess);
        Assert.Equal(2, provider.ReceivedKeys.Count);
        Assert.Equal(
            provider.ReceivedKeys[0],
            provider.ReceivedKeys[1]);
    }
}

This test verifies that the caller sends the same key twice. It does not prove that a real gateway will deduplicate the requests. The fake provider does not model provider-side persistence or charge creation, so it cannot establish that duplicate charges are impossible.

A stronger test double can store results by idempotency key and return the original result for repeated requests. Then the test can assert that two requests with the same key create only one simulated charge.

Use the gateway's sandbox environment for integration tests. Confirm that repeated requests with the same key behave as documented, that key retention matches the retry window, and that the provider's status lookup can resolve uncertain outcomes. Sandbox testing should not use live payment credentials.

Test the Failure Scenarios That Matter

A reliable payment test suite should cover more than a single timeout.

Test scenario	Expected behavior
Provider returns success	The payment is marked as succeeded
Provider returns a confirmed decline	The payment is marked as failed according to the provider's response
Request times out after submission	The outcome becomes unknown until reconciled
Retry follows a timeout	The same idempotency key is reused
Two requests arrive concurrently	The application prevents duplicate logical payment creation
Provider succeeds but database update fails	Recovery discovers the provider result without creating a new charge
Duplicate success webhook arrives	The event is processed idempotently
Provider status lookup is temporarily unavailable	The payment remains unresolved and is retried according to policy
Client cancels its HTTP request	The application does not assume the provider cancelled the payment

For the most important test, simulate the provider completing the payment and then dropping the response. This is the failure scenario that exposes the difference between a failed request and an unknown payment outcome.

Also test the database recovery path. A payment may succeed at the provider while the application crashes before recording the provider's payment ID. Your recovery process should be able to find the existing payment using the idempotency key or another stable provider reference.

Use Webhooks and Reconciliation to Resolve Unknown Outcomes

When the payment provider supports webhooks, a signed payment event can notify your application that a payment succeeded or failed. Webhook handlers should verify signatures, deduplicate event IDs, and update payment state safely.

Webhooks are not a replacement for idempotency. Providers may retry webhook delivery, and events may arrive out of order. The handler should check the current payment state and apply only valid transitions.

A reconciliation worker can periodically inspect payments that remain in OutcomeUnknown. It can query the provider using the stored payment reference or idempotency key, depending on the gateway's API.

If the provider confirms success, the application updates the payment record and order state. If it confirms failure, the application records that outcome. If the provider still cannot establish the result, the payment remains unresolved until the next check or manual investigation.

Do not create a new payment operation simply because the reconciliation request timed out. The original operation may still exist at the provider.

Common Mistakes to Avoid

Generating a new key on every retry. This turns one logical payment into multiple independent operations.
Treating a timeout as a decline. The provider may have completed the charge.
Retrying every error automatically. Confirmed declines and invalid requests generally require different handling from temporary failures.
Relying only on client-side button disabling. Requests can be duplicated by network retries, multiple devices, or concurrent server calls.
Ignoring duplicate webhooks. Repeated event delivery can trigger the same state transition more than once.
Testing only successful responses. The most dangerous failure occurs when the provider succeeds but the application never receives the response.
Assuming a database transaction includes the gateway. A local database transaction cannot roll back an external payment.
Logging sensitive payment data. Logs should contain operation IDs and safe diagnostic details, not secrets or full payment credentials.

Best Practices for Production Payment Systems

Create and persist the idempotency key before sending the first payment request.
Reuse that key for every retry of the same logical payment.
Follow the provider's documented key format, retention period, and request-matching rules.
Distinguish confirmed failures from unknown outcomes.
Enforce uniqueness and concurrency rules in the database.
Use signed, deduplicated webhooks and a reconciliation worker where supported.
Keep payment state transitions explicit and auditable.
Test provider success followed by response loss and database failure.
Use sandbox credentials for integration tests and verify behavior against the real provider contract.
Alert on payments that remain unresolved beyond an agreed period.

Advantages and Limitations

Advantages

Idempotency keys help prevent duplicate provider-side operations.
Explicit payment states make uncertain outcomes visible.
Automated tests can simulate failures without charging real customers.
Webhooks and reconciliation provide recovery paths for lost responses.
Database constraints help prevent concurrent requests from creating duplicate logical payments.

Limitations

Idempotency depends on the payment provider's support and retention rules.
Network failures can leave payment outcomes uncertain.
A local database transaction cannot make a remote gateway call atomic.
Webhooks may be delayed, duplicated, or delivered out of order.
Integration tests cannot reproduce every production network failure.

Summary

Payment retries must account for the possibility that a provider completed a charge even though the application received a timeout. Treating every timeout as a failed payment and sending a new request can charge customers twice.

Use a stable idempotency key for each logical payment, persist payment state, enforce concurrency rules, and distinguish confirmed failures from unknown outcomes. Test the same-key retry behavior with a fake provider, then verify the gateway's actual semantics in its sandbox.

For production systems, combine these safeguards with signed webhook handling and reconciliation. The goal is not simply to retry failed HTTP requests. It is to ensure that one logical payment remains one logical payment, even when responses are lost, workers restart, or requests arrive concurrently.