Threading in Python

Python's threading module lets you run multiple tasks concurrently within a single process. Understanding threads — and the Global Interpreter Lock that governs them — is essential for writing responsive I/O-bound programs.

Creating and Starting Threads

A thread is created by instantiating threading.Thread with a target function. Call .start() to launch it and .join() to wait for it to finish:

The key insight: both threads run simultaneously. While thread A sleeps for 2 seconds, thread B completes its 1-second sleep and finishes first.

Passing Arguments

Use args= for positional arguments and kwargs= for keyword arguments:

Daemon Threads

A daemon thread runs in the background and is automatically killed when all non-daemon threads (including the main thread) exit. Use them for background tasks like monitoring or cleanup that shouldn't prevent program shutdown:

Set daemon=True before calling .start(), or use thread.daemon = True.

Race Conditions

A race condition happens when two threads read and write shared data simultaneously, producing unpredictable results:

The problem: counter += 1 compiles to three bytecode operations. A thread switch can happen between any two of them, causing one thread to overwrite the other's update.

threading.Lock — Mutual Exclusion

A Lock (mutex) ensures only one thread executes a critical section at a time:

Always use with lock: rather than manually calling lock.acquire() / lock.release() — the context manager guarantees release even if an exception occurs.

threading.RLock — Reentrant Locks

A regular Lock deadlocks if the same thread tries to acquire it twice. RLock (reentrant lock) allows the same thread to acquire it multiple times:

Use RLock when you have recursive functions or methods that all need the same lock.

The GIL — Global Interpreter Lock

CPython (the standard Python interpreter) has a Global Interpreter Lock — a mutex that allows only one thread to execute Python bytecode at a time, even on multi-core machines.

What this means in practice:

• I/O-bound tasks: Threads work great. When a thread waits for network/disk I/O, it releases the GIL, letting other threads run. Web scraping, file processing, database queries — threading speeds these up.
• CPU-bound tasks: Threads don't help. All threads compete for the single GIL, so you get no parallelism. For CPU-heavy work, use multiprocessing instead.

threading.Event — Signaling Between Threads

Event is a simple communication primitive: one thread signals an event, others wait for it:

threading.Semaphore — Limiting Concurrency

A Semaphore limits how many threads can access a resource simultaneously. It's like a lock with a counter:

Thread-Local Storage

Each thread can have its own copy of a variable using threading.local():

Quick Reference

The GIL means Python threads are best suited for I/O-bound tasks. For CPU parallelism, reach for multiprocessing or concurrent.futures.ProcessPoolExecutor.