Introduction
Python is easy to write, but it is often slow when handling heavy tasks. This is because Python runs most of its code using the Global Interpreter Lock (GIL), which allows only one thread to execute Python code at a time. However, Python gives us tools like multithreading and multiprocessing to run tasks faster—if used correctly. Many developers misuse them, resulting in little or no performance improvement. In this article, we will explain how to use threads and processes correctly to make your Python programs run faster, with simple explanations and clean examples.
Understanding the Difference Between Multithreading and Multiprocessing
Before optimizing performance, you must understand when to use each one.
Multithreading
Runs multiple threads inside the same Python process.
Best for:
I/O-bound tasks (waiting on files, API calls, network operations)
Programs that spend the most time waiting
Multiprocessing
Runs multiple processes, each with its own Python interpreter and memory.
Best for:
CPU-bound tasks (math operations, loops, image processing)
Heavy computations
Simple Rule
Use multithreading for I/O-bound work
Use multiprocessing for CPU-bound work
Understanding the GIL (Global Interpreter Lock)
The GIL allows only one thread to execute Python bytecode at a time.
Because of the GIL:
Threads do NOT run Python code in parallel
Threads CAN run I/O operations in parallel
Processes CAN run CPU code in parallel
This is why choosing between threads and processes is important.
Multithreading Example (I/O-Bound Tasks)
Example: Downloading multiple URLs
import threading
import time
import requests
def download(url):
print(f"Downloading {url}")
response = requests.get(url)
print(f"Completed {url}")
urls = [
"https://example.com",
"https://www.google.com",
"https://www.github.com"
]
threads = []
for url in urls:
t = threading.Thread(target=download, args=(url,))
threads.append(t)
t.start()
for t in threads:
t.join()Why This Works
While one thread waits for the network, others continue running
Significant improvement over sequential requests
Multiprocessing Example (CPU-Bound Tasks)
Example: Computing squares of large numbers
from multiprocessing import Process
import time
def compute(n):
for _ in range(10_000_000):
n * n
processes = []
for i in range(4):
p = Process(target=compute, args=(i,))
processes.append(p)
p.start()
for p in processes:
p.join()Why This Works
Each process runs in parallel using separate CPU cores
No GIL blocking
Using multiprocessing.Pool for Easy Parallel Mapping
Example: Parallel execution with a pool
from multiprocessing import Pool
def square(n):
return n * n
with Pool(processes=4) as pool:
results = pool.map(square, [1, 2, 3, 4, 5])
print(results)Benefits
Simple API
Automatically manages worker processes
Using concurrent.futures for Cleaner Code
concurrent.futures provides modern wrappers for both threads and processes.
ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor
def fetch(url):
return requests.get(url).status_code
with ThreadPoolExecutor(max_workers=5) as executor:
results = executor.map(fetch, urls)ProcessPoolExecutor
from concurrent.futures import ProcessPoolExecutor
def square(n):
return n * n
with ProcessPoolExecutor() as executor:
results = executor.map(square, range(10))
Join the conversation! Your thoughts help the community grow.