Concurrency vs Parallelism in Python: Asyncio, Threads, Processes, and Interpreters

Someone adds concurrency to a slow service and the service gets slower. Throughput drops, latency climbs, and the incident channel fills with timeouts. The code now has more workers, more tasks, and more moving parts, and every one of them is fighting for the same core.

The mistake is rarely the library. It is the model. Concurrency and parallelism get used as synonyms, and "more workers" gets treated as a synonym for "more speed." They are not the same thing, and in Python the difference is load-bearing.

Here is the invariant I want you to leave with: the bottleneck decides the execution model, and every model charges a coordination tax. Pick the model whose switching and sharing semantics match what your program is actually waiting on or burning CPU on. Then pay the tax deliberately instead of discovering it in production.

Concurrency and Parallelism Are Different Questions

Concurrency is about overlapping waiting. Multiple tasks make progress by interleaving, and most of that interleaving happens while one task is blocked on something external: a socket, a disk read, a subprocess. Parallelism is about overlapping computing. Multiple tasks execute at the same instant on separate cores.

These are orthogonal. You can have concurrency without parallelism: a single-threaded event loop juggling ten thousand open sockets is concurrent and runs on one core. You can have parallelism without meaningful concurrency: a batch of independent CPU jobs that never talk to each other. You can have both, and you usually want both in a real service.

The reason this distinction bites harder in Python than in most languages is the global interpreter lock. In CPython, the GIL serializes bytecode execution within a single interpreter. Threads can overlap waiting, because the GIL is released around blocking I/O and inside many C extensions, but they cannot overlap pure-Python computation. Two threads running a tight Python loop take turns; they do not run side by side.

The GIL is a CPython implementation choice, not a language law. It has changed across versions, and free-threaded builds remove it. Treat "Python is slow at threads" as a statement about a specific interpreter build, not about the language.

So the real question is not "threads or async or processes." The real question is: what is my program waiting on, and what is it burning CPU on? Answer that first, and the model choice mostly falls out.

Classify the Workload Before Choosing a Model

I/O-bound work spends its time blocked. The process is idle while the clock runs: waiting on a network response, a database round trip, a file read, a subprocess. CPU-bound work spends its time executing bytecode: parsing, numeric loops, serialization, regex over large inputs.

The decision boundary is simple to state and easy to get wrong:

• If removing the wait would not make the program faster, the bottleneck is compute.
• If the CPU is idle while wall-clock time accumulates, the bottleneck is waiting.

Measure before you decide. Wall-clock time versus CPU time tells you most of it. If a stage takes 4 seconds of wall clock and 0.2 seconds of CPU, it is waiting. If it takes 4 seconds of wall clock and 3.9 seconds of CPU, it is computing. A profiler or a quick time.perf_counter() around the stage will settle the argument faster than any design discussion.

The trap is classifying per program instead of per stage. Real workloads are mixed. A request handler fetches from an API, transforms the payload, writes to a database, and returns. The fetch is waiting. The transform may be computing. The write is waiting again. Each stage has its own bottleneck, and each stage may deserve a different model.

I have watched engineers wrap a CPU-bound loop in a thread pool, observe no speedup, and conclude that "Python concurrency is slow." The model was wrong for the workload. Threads overlap waiting; they do not overlap pure-Python computation. The tool did exactly what it promised.

Asyncio: Cooperative Concurrency on One Thread

Asyncio runs one thread and one event loop. Coroutines yield control at await points, and the event loop does not preempt them. Scheduling is cooperative, which means the tasks decide when to give up control.

That single fact produces both the strength and the most common production failure. A blocking call inside a coroutine stalls the entire loop. Not one task. The whole loop. Every other coroutine waiting on that loop stops making progress until the blocking call returns. A synchronous database driver, a requests.get, a CPU-heavy regex, a time.sleep — any of these inside an async function turns your concurrent service into a serial one, and the symptom looks like mysterious latency spikes rather than an obvious bug.

The payoff is scale in the count of simultaneous waits. A task is far cheaper than a thread, so an event loop can hold tens of thousands of in-flight operations that would exhaust a thread pool. The win is not raw speed per operation. The win is the number of waits you can keep open at once.

Cancellation and timeouts are first-class in asyncio, but they require cancellation-aware code. A coroutine that catches CancelledError and swallows it breaks shutdown: the loop asks the task to stop, the task refuses, and your graceful shutdown hangs. If you catch broad exceptions, re-raise CancelledError or let it propagate.

When not to use it: CPU-bound stages, code paths dominated by synchronous libraries with no async equivalent, and small scripts where the event-loop ceremony buys nothing. If you have three sequential HTTP calls and no concurrency requirement, asyncio is overhead with extra syntax.

Threads: Preemptive Concurrency With Shared Memory

Threads are scheduled preemptively by the operating system. The runtime does not need cooperation, so a thread that blocks in a C-level call releases the GIL and lets other threads run. This is why threads work well for I/O and for libraries that drop the GIL internally — many numerical and compression libraries do exactly that.

Threads share memory. That is the feature and the hazard in the same sentence. Shared mutable state requires locks, and lock discipline is where thread bugs live. The failure mode is not a clean crash; it is a race that reproduces once a week under load and never on your laptop.

Thread pools bound resource use, which is good, but they also bound throughput. Pool size and queue depth become tuning parameters with real failure modes: too small and you leave the machine idle, too large and you thrash on context switches and memory. Under load, an unbounded queue in front of a bounded pool converts a latency problem into a memory problem.

When not to use it: pure-Python CPU-bound loops, or when the coordination cost of shared state exceeds the work being parallelized. If two threads spend more time contending for a lock than doing work, you have built a slower single-threaded program with extra steps.

Processes: True Parallelism and Its Serialization Tax

Multiprocessing gives you separate interpreters, separate memory, and separate GILs. That is what enables real multi-core execution of Python bytecode. If your bottleneck is compute, this is the model that actually removes it.

The cost is serialization. Arguments and results cross a process boundary, which means they get pickled on one side and unpickled on the other. For short tasks, that overhead can dominate the compute you were trying to parallelize. A function that takes 2 milliseconds to run and 5 milliseconds to serialize is a net loss no matter how many cores you throw at it.

Startup semantics matter too. fork and spawn behave differently across platforms, and spawn re-imports the module, which breaks naive top-level code that runs work at import time. If your worker module does anything at module scope, guard it.

Failure semantics change as well. A crashed worker does not take down the parent, which is good isolation, but it also does not share exceptions naturally. Error propagation must be designed: you decide what a worker failure means, how it is transported back, and whether the parent retries, fails, or degrades.

When not to use it: I/O-bound work, where threads or async are cheaper; tiny tasks where serialization exceeds compute; and stateful objects that cannot be pickled. If your data does not cross the boundary cleanly, you will spend your time building a serialization layer instead of solving the actual problem.

Interpreters and Free-Threading: Two Different Bets

Sub-interpreters and free-threaded builds get grouped together because both are newer than the GIL-bound threading model, but they answer different questions. Keep them separate.

Sub-interpreters give you isolated execution contexts inside one process. Each interpreter has its own state and its own GIL, so combining threads with multiple interpreters enables full multi-core parallelism without paying process startup and full serialization costs. Communication is message-passing oriented, which makes the mental model closer to the actor model or CSP than to threads: isolated contexts exchanging messages. The decision question is narrow — do you have CPU-bound work where process startup and pickling are the limiting factor, and can your data cross a message boundary cleanly?

Free-threaded builds remove the GIL constraint entirely. The decision question is different: do you have a workload where thread-level parallelism is the natural fit, and can you verify that your C extensions and libraries are actually thread-safe without the GIL serializing them? Code that was safe because the GIL serialized it may no longer be safe, and the failure modes are the classic data races Python developers have not had to think about.

Treat both as options to evaluate for new CPU-bound work, not as drop-in replacements for existing thread or process designs. Behavior depends on the Python version, the build, and library support. Verify against your target runtime before committing.

Coordination, Failure, and Observability Tradeoffs

Speed is the easy axis to compare. The axes that actually break systems are coordination, backpressure, failure isolation, and observability.

Backpressure deserves its own line, and it needs to be stated as an invariant rather than a suggestion. Every model needs a bound on in-flight work, and the bound must govern a specific resource: accepted requests, queued tasks, active workers, or outstanding bytes. The invariant is: accepted work has a bounded owner, each permit is released on success, failure, or cancellation, and downstream capacity is never exceeded by upstream fan-out. An event loop that spawns a task per request without a semaphore will happily accept more work than it can complete, and the queue becomes the outage. Bound concurrency in every model, and decide what happens when the bound is hit.

Observability is where mixed systems get painful. Correlating work across threads, processes, and event-loop tasks requires context propagation and structured logging as part of the design, not an afterthought. A request ID that survives a process boundary and a task boundary is worth more than any micro-optimization.

A Mixed Workload, Traced End to End

The composition step is where most real systems live, so let me trace one request through it. The handler accepts an HTTP request, fetches a document from an external API, runs a CPU-heavy transform on the payload, writes the result to a database, and returns.

Stage 1 — Fetch (async I/O). The handler awaits an HTTP client. The event loop holds the wait open and serves other requests. No thread is consumed. This is the model's strength: thousands of concurrent fetches on one thread.

Stage 2 — Transform (CPU-bound). The transform is pure Python and takes 200 ms of CPU. If it runs inline in the coroutine, it blocks the event loop for 200 ms and every other request stalls. The fix is to submit it to a bounded process pool: await loop.run_in_executor(pool, transform, payload). The payload is pickled to the worker, the result is pickled back, and the event loop stays free. The serialization cost is real — if the payload is large, measure it, because it may exceed the compute you are parallelizing.

Stage 3 — Write (async I/O). The result returns to the coroutine and awaits a database write. Same pattern as stage 1.

Now the failure and cancellation paths, which are where the design actually lives:

• Client cancels mid-request. The coroutine receives CancelledError. If it is awaiting the executor future, the future is cancelled, but the process-pool worker may already be running and cannot be interrupted. The worker finishes, its result is discarded, and the permit is released. If your pool is small and transforms are long, cancellation does not free capacity immediately — plan for that.
• Worker crashes. The executor future raises. The coroutine must decide: retry, fail the request, or degrade. The exception does not propagate automatically across the process boundary; you transport it explicitly.
• Backpressure. The process pool has a fixed worker count and a bounded queue. When the queue is full, run_in_executor submissions must be rejected or awaited, not silently accumulated. The permit is released when the future resolves, whether by success, exception, or cancellation.

The coordination tax here is paid twice: once at the async-to-process boundary (pickling, queue wait) and once at the process-to-async return (result transport, error mapping). That is fine. It is a deliberate cost, not an accident.

A Decision Procedure for Mixed Workloads

Here is the procedure I actually use.

1. Profile and classify each stage as waiting or computing. Per stage, not per program.
2. For waiting stages, prefer async when the wait count is high and the libraries are async-native. Prefer threads when the code is synchronous or the wait count is modest. Both are legitimate; the choice is about scale and library fit.
3. For computing stages, move to processes. Consider interpreters or free-threaded builds when serialization or startup cost is the limiting factor, and verify runtime support first.
4. Compose. An async front end that dispatches CPU-bound stages to a bounded process pool is a legitimate and common architecture. Name the boundary explicitly, because that boundary is where serialization and failure semantics live.
5. Bound concurrency, define cancellation and timeout behavior, and decide how failures cross each boundary before shipping.

What to Do Next

Do not rewrite your service. Profile the current hot path and label each stage as waiting or computing. Then find the single worst-mismatched stage — the CPU-bound loop wrapped in threads, or the blocking call inside a coroutine — and rewrite just that stage using the model the classification implies.

Measure more than throughput, because throughput alone hides the failure modes you just paid to avoid. Capture stage wall time and CPU time, queue wait, active in-flight count, serialization or boundary time, peak memory, and p95 latency under a bounded load. If the rewrite helped, you should see the bottleneck stage's wall time drop without queue wait or tail latency rising. If queue wait or p95 latency climbed while throughput stayed flat, you moved the bottleneck instead of removing it — and the numbers just told you where it went.