# Python GIL **Python GIL** (Global Interpreter Lock) is a mutex that prevents multiple threads from executing Python bytecode simultaneously in CPython. Only one thread can hold the GIL at a time, making CPU-bound multithreading ineffective. The GIL exists because CPython's memory management isn't thread-safe; removing it hurts single-threaded performance. Use multiprocessing for CPU-bound parallelism, threading for I/O-bound concurrency, or async for high-concurrency I/O. ## Example This example demonstrates GIL limitations and workarounds. ```python # run: python3 gil.py # description: GIL effects on threading and multiprocessing import threading import time import multiprocessing def cpu_bound_task(n): """Pure CPU work (no I/O).""" total = 0 for i in range(n): total += i return total # Threading: GIL prevents parallelism print("Threading (GIL limited):") start = time.time() def thread_worker(): cpu_bound_task(100_000_000) threads = [] for _ in range(2): t = threading.Thread(target=thread_worker) threads.append(t) t.start() for t in threads: t.join() thread_elapsed = time.time() - start print(f" 2 threads: {thread_elapsed:.3f}s") # Multiprocessing: separate processes, no GIL print("\nMultiprocessing (no GIL):") start = time.time() processes = [] for _ in range(2): p = multiprocessing.Process(target=cpu_bound_task, args=(100_000_000,)) processes.append(p) p.start() for p in processes: p.join() process_elapsed = time.time() - start print(f" 2 processes: {process_elapsed:.3f}s") print(f"\nSpeedup: {thread_elapsed / process_elapsed:.1f}x") # I/O-bound: threading is fine print("\n" + "="*60) print("I/O-bound threading (GIL released during I/O):") def io_bound_task(): """I/O work (releases GIL).""" time.sleep(1) # GIL released during sleep start = time.time() threads = [] for _ in range(3): t = threading.Thread(target=io_bound_task) threads.append(t) t.start() for t in threads: t.join() io_elapsed = time.time() - start print(f" 3 I/O tasks in threads: {io_elapsed:.3f}s") print(f" (should be ~1s, not ~3s)") # GIL release points print("\n" + "="*60) print("GIL is released during:") print(" - I/O operations (read, write, network)") print(" - time.sleep()") print(" - C extensions (NumPy, etc.) often release GIL") print(" - threading.Lock/RLock acquisition") # sys.getswitchinterval() and GIL import sys print(f"\nGIL switch interval: {sys.getswitchinterval()}s") print(" (how often to switch between threads)") ``` ## Common patterns **GIL behavior**: - CPython has GIL; PyPy, Jython, IronPython don't - Released during I/O operations - Acquired before executing Python bytecode - Prevents true parallelism in CPU-bound code **For CPU-bound work**: - Use `multiprocessing`: separate process, separate GIL - Each process has its own interpreter and GIL - Trade-off: more overhead, but true parallelism **For I/O-bound work**: - Use `threading`: efficient, GIL released during I/O - Great for network requests, file I/O, databases - Each thread can wait independently **For high-concurrency I/O**: - Use `asyncio` or `trio`: even lighter than threads - Thousands of concurrent tasks with minimal overhead - No thread switching; cooperative multitasking **Alternatives to threading**: - `multiprocessing.Pool`: pool of worker processes - `concurrent.futures.ThreadPoolExecutor`: thread pool for I/O - `concurrent.futures.ProcessPoolExecutor`: process pool for CPU - `asyncio`: event loop for I/O **Thread safety without GIL**: - GIL doesn't prevent all race conditions - Still need locks for shared mutable state - Even atomic operations (e.g., `x += 1`) aren't thread-safe without locking **NumPy and GIL**: - NumPy releases GIL for large operations - Can use threading with NumPy for some parallelism - True NumPy parallelism better with multiprocessing **GIL internals**: - `sys.getswitchinterval()`: how often to switch threads (default 5ms) - `sys.setswitchinterval(seconds)`: adjust (rarely needed) - GIL contention visible with many threads competing **Common misconceptions**: - "Threading is slow in Python": not for I/O - "Multiprocessing always better": overhead not worth it for I/O - "Locks solve all concurrency": still need careful design - "PyPy has no GIL": true, but PyPy is not CPython **When to use each**: - CPU-bound: multiprocessing - I/O-bound (<100 concurrent): threading - I/O-bound (100+): asyncio - Mixed: multiprocessing + asyncio inside processes