Python asyncio Cancellation: How to Stop Tasks Without Leaving Resources Open

Python's asyncio makes it possible to run many I/O-bound operations without blocking the entire application. It is commonly used for web requests, database access, background workers, and other tasks that spend time waiting for external operations.

But asynchronous applications also need to stop work when it is no longer required. A user may cancel a request, a timeout may expire, or an application may shut down while a task is still running.

This is where cancellation becomes important.

In Python, cancelling an asyncio task does not instantly terminate everything it is doing. Instead, cancellation is delivered to the coroutine, giving it an opportunity to stop safely and release resources such as files, database connections, locks, and network sessions.

If cancellation is handled incorrectly, an application may leave resources open, keep locks held, or continue performing unnecessary work.

In this article, you will learn how asyncio cancellation works, how to clean up resources safely, how to handle timeouts, and how to avoid common mistakes in production applications.

What Is Cancellation in Python asyncio?

Cancellation is the process of requesting that an asynchronous task stop running.

Consider a task that waits for a remote service:

import asyncio

async def fetch_data():
    print("Starting request")

    await asyncio.sleep(10)

    print("Request completed")

async def main():
    task = asyncio.create_task(fetch_data())

    await asyncio.sleep(1)

    task.cancel()

    try:
        await task
    except asyncio.CancelledError:
        print("Task was cancelled")

asyncio.run(main())

The task begins execution and waits for ten seconds. After one second, the main coroutine requests cancellation by calling task.cancel().

When the task reaches an appropriate cancellation point, asyncio raises CancelledError inside the coroutine. Because the task is awaiting asyncio.sleep(), cancellation can interrupt that wait.

The main coroutine catches CancelledError when it awaits the cancelled task.

The important distinction is that task.cancel() requests cancellation. It does not forcibly terminate arbitrary Python code that is currently executing without yielding control.

How asyncio Cancellation Works

Understanding the cancellation process helps you write reliable cleanup logic.

When you call cancel() on an asyncio task, the runtime arranges for CancelledError to be raised in the task, normally at its next opportunity to process cancellation.

A typical sequence looks like this:

A coroutine starts running.
The application requests cancellation.
The coroutine receives CancelledError at a cancellation opportunity.
The coroutine's cleanup code runs.
The coroutine either propagates the cancellation or handles it according to its design.
The task finishes in a cancelled state if cancellation was not suppressed.

This process allows a coroutine to release resources before it stops.

However, cancellation may be delayed if the coroutine performs a long-running synchronous operation without yielding to the event loop.

For example, a CPU-intensive loop can prevent other coroutines from running and delay cancellation. Such work may need to be divided into smaller operations or moved to a worker thread or process.

Step 1: Use try/finally to Release Resources

The safest general pattern is to use try/finally for cleanup.

Suppose an asynchronous task acquires a resource before performing work. If cancellation occurs during the operation, the cleanup block still runs.

import asyncio

async def process_data():
    resource = await acquire_resource()

    try:
        await perform_operation(resource)
    finally:
        await resource.close()

Here, acquire_resource(), perform_operation(), and resource.close() represent application-specific asynchronous operations.

The finally block runs when the coroutine completes normally or when an exception, including cancellation, unwinds the relevant block.

This pattern is useful for releasing connections, closing sessions, removing temporary files, and performing other required cleanup.

There is an important detail: cleanup code should be designed to finish reliably. If cleanup itself fails, that failure can affect the exception that ultimately propagates. Production code should log and handle cleanup failures where appropriate without accidentally hiding the original error.

Step 2: Use async with for Managed Resources

Many asynchronous libraries provide asynchronous context managers that handle setup and cleanup automatically.

For example, an asynchronous resource may support the async with statement:

async def process_request(client):
    async with client.session() as session:
        response = await session.fetch(
            "https://api.example.com/data"
        )

        return response

This example assumes that client.session() returns an asynchronous context manager and that the session provides the shown fetch() method.

When the block exits, the context manager's cleanup logic runs, including when cancellation interrupts the operation.

This is generally preferable to manually managing resource lifetimes when the library provides a suitable context manager.

Common examples include asynchronous database sessions, network clients, and locks that support the asynchronous context manager protocol.

However, async with does not guarantee that every cleanup operation can never fail. The resource library still needs correct cancellation and shutdown behavior.

Step 3: Handle CancelledError Correctly

One common mistake is catching every exception and treating cancellation as an ordinary failure.

Consider this example:

import asyncio

async def worker():
    try:
        await do_work()
    except asyncio.CancelledError:
        print("Worker received cancellation")
        raise

The coroutine catches CancelledError to perform any necessary cancellation-specific work, then re-raises the exception.

Re-raising preserves the expected cancellation behavior and allows callers to recognize that the task was cancelled.

In most application code, you should not suppress CancelledError unless you have a specific reason and understand the consequences.

For example, this pattern is dangerous:

async def worker():
    try:
        await do_work()
    except asyncio.CancelledError:
        return

The coroutine consumes the cancellation request and returns normally. Code awaiting the task may no longer observe it as cancelled.

That can make shutdown behavior confusing and may cause higher-level code to believe the operation completed successfully.

If cancellation needs special handling, perform the necessary cleanup and normally allow the exception to propagate.

Step 4: Use Timeouts to Cancel Slow Operations

Timeouts are a common reason for cancellation.

Python provides asyncio.timeout() in modern Python versions to establish a time limit for a block of asynchronous work.

import asyncio

async def fetch_with_timeout():
    try:
        async with asyncio.timeout(5):
            await fetch_remote_data()
    except TimeoutError:
        print("The operation exceeded five seconds")

If the operation does not finish within the specified time, the timeout context requests cancellation of the current task and converts the resulting cancellation into TimeoutError outside the context.

The exception handler therefore catches TimeoutError, not the internal cancellation used to implement the timeout.

The operation should still clean up its resources correctly when cancellation occurs.

For compatibility with older Python versions, asyncio.wait_for() is another option:

import asyncio

async def fetch_with_timeout():
    try:
        result = await asyncio.wait_for(
            fetch_remote_data(),
            timeout=5
        )

        return result
    except TimeoutError:
        print("The operation timed out")
        raise

asyncio.wait_for() cancels the awaited operation when the timeout expires and waits for cancellation handling to complete. As a result, the total elapsed time can exceed the configured timeout if the task takes time to finish its cleanup.

A timeout is not a hard guarantee that every underlying operation stops immediately. For example, a library that wraps a thread or native operation may have different cancellation limitations.

Step 5: Cancel and Await Background Tasks

Applications often create background tasks for polling, message processing, or periodic maintenance.

When shutting down, it is not enough to call cancel() and immediately forget about the task. The application should normally await the task so its cleanup can finish.

import asyncio

async def background_worker():
    try:
        while True:
            await process_next_item()
    except asyncio.CancelledError:
        print("Worker is stopping")
        raise
    finally:
        await close_worker_resources()

async def main():
    task = asyncio.create_task(background_worker())

    try:
        await run_application()
    finally:
        task.cancel()

        try:
            await task
        except asyncio.CancelledError:
            pass

asyncio.run(main())

The example illustrates a basic shutdown pattern.

When the application is ready to stop, it requests cancellation and awaits the worker. The worker can then release its resources before the shutdown sequence continues.

In a production application, consider how cleanup failures should be reported and how long shutdown is allowed to take.

If the application manages multiple independent background tasks, use a structured concurrency approach where possible. Python's asyncio.TaskGroup, available in modern Python versions, helps coordinate related tasks and their lifetimes.

Step 6: Understand Cancellation in TaskGroup

asyncio.TaskGroup provides a structured way to manage groups of related tasks.

import asyncio

async def process_item(item_id):
    await handle_item(item_id)

async def main():
    async with asyncio.TaskGroup() as group:
        for item_id in range(10):
            group.create_task(
                process_item(item_id)
            )

asyncio.run(main())

The task group waits for its child tasks when the context exits.

If a child task fails with an exception other than cancellation, the group generally cancels its remaining tasks and reports the failure after coordinating their completion.

This helps prevent background tasks from being created and forgotten.

However, normal exit from a TaskGroup does not mean that every child task is automatically cancelled. The group waits for the tasks to finish. If a child task runs forever, the group can remain open indefinitely.

For long-running workers, design an explicit shutdown signal or cancellation strategy rather than assuming that leaving the context will stop them.

Step 7: Avoid Leaving Locks and Shared State Inconsistent

Cancellation can happen while a task is waiting for a lock or performing a multi-step operation.

Use asynchronous context managers for locks when available:

import asyncio

lock = asyncio.Lock()
shared_data = {}

async def update_data(key, value):
    async with lock:
        shared_data[key] = value

        await persist_data(shared_data)

The lock is released when execution leaves the async with block, including when cancellation propagates.

However, releasing a lock does not automatically undo changes made to shared state before cancellation.

In this example, shared_data is modified before persist_data() finishes. If persistence fails or the task is cancelled, the in-memory state may differ from the stored state.

To prevent inconsistent state, design the operation around a clear commit point. Depending on the application, you might validate data first, write changes transactionally, or update shared state only after persistence succeeds.

Cancellation-safe code must consider both resource cleanup and the consistency of partially completed operations.

Common Cancellation Mistakes to Avoid

Assuming cancel() stops a task immediately: Cancellation is cooperative and may not be processed until the task reaches a suitable opportunity.
Swallowing CancelledError: Suppressing cancellation can interfere with shutdown and higher-level task management.
Forgetting cleanup logic: Resources can remain open if the code does not use finally or a suitable context manager.
Cancelling tasks without awaiting them: Cleanup may not finish before the application exits.
Assuming a timeout stops all underlying work: Threads, native operations, and external services may continue independently.
Using unbounded cleanup operations: Shutdown can hang if cleanup never completes.
Assuming locks solve every consistency problem: Releasing a lock does not roll back partial changes to application state.

Best Practices for Cancellation-Safe asyncio Applications

Use try/finally to release resources.
Prefer async with when a library provides a suitable asynchronous context manager.
Catch CancelledError only when necessary, and normally re-raise it.
Apply timeouts to operations that must not wait indefinitely.
Cancel and await background tasks during shutdown.
Use TaskGroup to manage related tasks where appropriate.
Keep synchronous CPU-intensive operations from blocking the event loop.
Make multi-step operations safe against partial completion.
Test cancellation during network calls, database operations, and cleanup.
Log cancellation and cleanup failures without exposing sensitive data.

Conclusion

Python asyncio cancellation is a request to stop a task, not an immediate forceful termination. The coroutine needs a safe way to release resources, handle partially completed operations, and propagate cancellation correctly.

Use try/finally and asynchronous context managers to manage resource lifetimes. Apply timeouts to operations that can hang, and cancel and await background tasks during shutdown.

Most importantly, test cancellation at different points in an operation. A task that works correctly during normal execution may still leave connections open or shared state inconsistent when cancellation occurs.

When cancellation is part of the application's design rather than an afterthought, asynchronous services become easier to shut down, recover, and operate reliably.