How can concurrency primitives like Locks, Semaphores, and Events be used in Python's threading or multiprocessing?

In Python, Locks, Semaphores, and Events are used to coordinate threads or processes' access to shared resources. 1. Locks ensures that only one thread or process executes specific code blocks at a time to prevent race conditions; 2. Semaphores limits the number of resources accessed simultaneously, and is suitable for connection pools and other scenarios; 3. Events are used for thread or inter-process communication, waiting for signals through .wait() and sending signals. In addition, pay attention to the reentry of locks, the complexity of state sharing between processes, and avoiding excessive locking and performance degradation and deadlock problems.

In Python, concurrency primitives like Locks , Semaphores , and Events are essential for coordinating access to shared resources when working with threads or processes. They help prevent race conditions, manage resource availability, and signal between concurrent units of execution.

Here's how these tools work in practice with both threading and multiprocessing .

Locks: Preventing Simultaneous Access

A Lock is the most basic synchronization primitive. It ensures that only one thread or process can execute a particular block of code at a time.

• In threading , you use threading.Lock() .
• In multiprocessing , you use multiprocessing.Lock() (or get it from a Manager if needed across processes).

Why it matters: Without locks, multiple threads or processes might try to update a shared variable at the same time, leading to inconsistent data.

Example usage:

 import threading

counter = 0
lock = threading.Lock()

def increment():
    Global counter
    with lock:
        counter = 1

This guaranteees that only one thread will be inside the with lock: block at any given moment.

Note: For multiprocessing, you'll usually create the lock inside a Manager context or pass it through shared memory constructs.

Semaphores: Limiting Access to Resources

A Semaphore is like a counter that limits how many threads or processes can access a resource concurrently.

• Use threading.Semaphore(n) to allow up to n concurrent accesses.
• Use multiprocessing.Semaphore(n) similarly for processes.

When to use it: Great for controlling access to limited resources — think connection pools, limited file handles, or bandwidth caps.

Example scenario: You're downloading files and want to limit the number of simulateneous downloads to 3.

 import threading
import time

sem = threading.Semaphore(3)

def download_file(i):
    with sem:
        print(f"Downloading file {i}")
        time.sleep(2)
        print(f"Finished file {i}")

for i in range(5):
    threading.Thread(target=download_file, args=(i,)).start()

This ensures no more than 3 threads run the critical section at once.

Events: Signaling Between Threads or Processes

An Event is a simple communication mechanism used to signal that something has happened.

• Use threading.Event() or multiprocessing.Event() .
• One thread/process waits on .wait() , and another signals via .set() .

Use case: You might have a background thread waiting for a trigger before proceeding.

Example:

 import threading
import time

event = threading.Event()

def wait_for_signal():
    print("Waiting for signal...")
    event.wait()
    print("Signal received!")

threading.Thread(target=wait_for_signal).start()

time.sleep(2)
print("Sending signal")
event.set()

This pattern is useful for pausing and recovering operations, or triggering actions after setup completes elsewhere.

In multiprocessing, events behave similarly but must be shared explicitly — often using multiprocessing.Manager() or passed through Process arguments.

A Few Gotchas

• Locks are re-entrant? Not all are. If you need a thread to acquire the same lock multiple times without deadlocking, use threading.RLock() instead of a regular Lock .
• Shared state is tricky in multiprocessing. Variables aren't automatically shared between processes, so use Value , Array , or Manager objects when sharing state.
• Avoid over-locking. Only protect what needs protection. Excessive locking can hurt performance and introduce deadlocks.

These concurrency primitives are powerful, but they're also easy to misuse. Understanding which one to use — and when — makes writing safe, efficient concurrent code in Python much easier.

Basically that's it.

The above is the detailed content of How can concurrency primitives like Locks, Semaphores, and Events be used in Python's threading or multiprocessing?. For more information, please follow other related articles on the PHP Chinese website!