Most rate-limiting examples stop at "refuse the request." A refusal without a Retry-After value forces every client to guess, and the guesses come back as retry storms. This article presents a small dependency-free Python token bucket whose refusals always carry the wait time, plus the unit tests that prove its behavior and a demo that shows a well-behaved client recovering on the first retry. The full source and tests are attached as a download.
Background
A token bucket holds up to capacity tokens and refills at a fixed rate. A request consumes tokens; when the bucket is empty, the request must wait deficit / refill_rate seconds. That wait value is exactly what an HTTP 429 response should put in its Retry-After header (whole seconds, rounded up, per RFC 9110).
Using the code
from token_bucket import TokenBucket, RateLimited, retry_after_header
bucket = TokenBucket(capacity=100, refill_per_second=100/60) # 100 req/min
wait = bucket.try_acquire()
if wait > 0:
# in a real handler: return 429 with headers
print("Retry-After:", retry_after_header(wait))
acquire_or_raise raises RateLimited with a retry_after attribute for frameworks that prefer exceptions. The clock is injectable, so tests run instantly with a fake clock; the default is time.monotonic, which is safe across system clock changes.
The tests are the specification
The download includes seven unit tests covering burst-then-refuse, exact Retry-After math (0.25 s for a 1-token deficit at 4 tokens/s), refill over time, the capacity ceiling, and header rounding:
Ran 7 tests in 0.001s
OK
And the demo output, six requests against a 3-token bucket refilling at 1/s:
request 1: 200 OK
request 2: 200 OK
request 3: 200 OK
request 4: 429 Too Many Requests | Retry-After: 1s
request 4 (retry): 200 OK
request 5: 429 Too Many Requests | Retry-After: 1s
request 5 (retry): 200 OK
Every 429 is answered with the exact wait, the client sleeps exactly that long, and the retry succeeds. No guessing, no exponential backoff storm.
Points of interest
Two details matter more than the algorithm. First, monotonic time: using wall time makes the limiter behave erratically when NTP steps the clock. Second, rounding Retry-After up: a value of 0 tells some clients to retry immediately, which recreates the storm.
The header conventions around limits (X-RateLimit-*, documentation links in error bodies, and the difference between rate limits and spending caps) are covered in this companion piece: API rate limits and spending caps: protecting your backend and your bill.
Summary
A rate limiter that only says "no" pushes work onto every client. One that says "no, come back in exactly N seconds" keeps clients polite and servers calm. The attached source is dependency-free and covered by tests you can run in seconds.
This article was created with the help of AI; the code and test output above were produced by actually running the included source (python -m unittest, python demo.py).
Join the conversation! Your thoughts help the community grow.