This content originally appeared on Level Up Coding - Medium and was authored by Faaruk
Unlock the Deadlocks
Transform your multithreaded applications for better performance and safety
What is a Mutex?
A mutex (mutual exclusion) is a synchronization primitive that allows only one thread to access a shared resource at a time. It has two states
either 0 / 1 or unlocked / locked.
They’re often used to ensure only one thread enters a critical section of code at a time. The thread that wants to run the critical section code, must acquire the mutex lock first and the lock must be released later by that same thread.
Someone in stackoverflow gave a very good analogy:
Lets check some useful usecases of Mutex…
Preventing Deadlocks in Database Resource Allocation
If we envision any system with good traffic, we can assume it with multiple transactions happening at the same time that are hitting the same shared database resource. Its likely to occur deadlocks when two different threads acquire lock on one resource (A) and wait for the other to release their locks on resource (B).
Here locking the resources in proper order matters, say what if both the threads locked resource A first then B or resource B first then A.
Proper locking order along with mutexes will help us in resolving deadlocks.
# Here is an example of a purchase transaction and refund transaction
# happening simultaneously for the same user where
# both threads want to modify the user data and order data
# and gets deadlock when thread1 got lock for user data and thread2 got lock
# for order data but now they can't release their locks because they also want
# to modify the data that the other threads are holding on to.
# Below code use mutex and locking order to solve the deadlock.
import threading
import time
# Data storage, the shared resource
user_account = {'balance': 1000}
order_record = {'status': 'pending', 'total': 200}
# Mutex locks
user_account_lock = threading.Lock()
order_record_lock = threading.Lock()
# Purchase
def process_purchase(thread_id):
print(f"Thread {thread_id} attempting to process purchase...")
# Define lock acquisition order: first user account, then order
user_account_lock.acquire()
print(f"Thread {thread_id} acquired user account lock")
time.sleep(1) # Simulate some purchase processing
order_record_lock.acquire()
print(f"Thread {thread_id} acquired order record lock")
# Simulate purchase: deduct from balance and update order status
user_account['balance'] -= order_record['total']
order_record['status'] = 'completed'
print(f"Thread {thread_id}: Purchase completed, new balance: {user_account['balance']}")
# Release locks in reverse order
order_record_lock.release()
user_account_lock.release()
print(f"Thread {thread_id} released locks and finished purchase")
# Refund
def process_refund(thread_id):
print(f"Thread {thread_id} attempting to process refund...")
# Follow the same lock order: first user account, then order
user_account_lock.acquire()
print(f"Thread {thread_id} acquired user account lock")
time.sleep(1) # Simulate some processing time
order_record_lock.acquire()
print(f"Thread {thread_id} acquired order record lock")
# Simulate refund: return amount to balance and update order status
user_account['balance'] += order_record['total']
order_record['status'] = 'refunded'
print(f"Thread {thread_id}: Refund processed, new balance: {user_account['balance']}")
# Release locks in reverse order
# This is important so we release the lock first which we acquired latest
# otherwise deadlock can still occur
order_record_lock.release()
user_account_lock.release()
print(f"Thread {thread_id} released locks and finished refund")
# Simulate concurrent operations
# purchase and refund happening at the same time
purchase_thread = threading.Thread(target=process_purchase, args=(1,))
refund_thread = threading.Thread(target=process_refund, args=(2,))
purchase_thread.start()
refund_thread.start()
purchase_thread.join()
refund_thread.join()
# Output
Thread 1 attempting to process purchase...
Thread 1 acquired user account lock
Thread 2 attempting to process refund...
Thread 1 acquired order record lock
Thread 1: Purchase completed, new balance: 800
Thread 1 released locks and finished purchase
Thread 2 acquired user account lock
Thread 2 acquired order record lock
Thread 2: Refund processed, new balance: 1000
Thread 2 released locks and finished refund
Note: the code between lock.acquire() and lock.release() is called critical section
Preserve Consistency in Database Write Operation
Envision a banking system, where user accounts are present and transactions occur where balance is added or deducted. Also we need to make sure when we check balance, it should show up correctly according to the order of operations.
So its very likely you are a rich person and lot of transactions occur to your bank account at the same time. If we pick two of those add transactions being carried out by two different threads, now without mutex we could end up with lesser money if race condition happens. Now who’d want that!
Here a mutex ensures that no two threads write to the same account at the same time.
# In below example you would see, we have both the threads who has fetched
# the same bank balance for the user, without mutex locking both the transactions
# would be successful but would eventually lead to wrong bank balance.
# Below code use mutex to preserve consistency in bank balance.
import threading
# Data storage, the shared resource
account = {'user':'Cristiano', 'balance': 777}
bank_transaction_lock = threading.Lock()
def withdraw(amount):
bank_transaction_lock.acquire()
try:
if account['balance'] >= amount:
account['balance'] -= amount
print(f"Withdrawn {amount}, remaining balance: {account['balance']}")
else:
print("Insufficient balance")
finally:
bank_transaction_lock.release()
# Simulate concurrent transactions for withdrawal
thread1 = threading.Thread(target=withdraw, args=(400,))
thread2 = threading.Thread(target=withdraw, args=(300,))
thread3 = threading.Thread(target=withdraw, args=(700,))
thread1.start()
thread2.start()
thread3.start()
thread1.join()
thread2.join()
thread3.join()
# Output:
Withdrawn 400, remaining balance: 377
Withdrawn 300, remaining balance: 77
Insufficient balance
Log File Management: Handling High Number of Concurrent File Writes
We can assume that our log file(shared) is being written by multiple threads simultaneously and since they are append only, all of the threads are writing at the same location which is the last part of the content in the log file.
Thus it has high chances of simply appending log messages but at random order which is later found undecipherable.
A mutex ensures that only one thread writes to the log file at any given moment, while other threads wait for their turn.
# Below code simulates multiple threads writing to one shared log file
import threading
logging_write_lock = threading.Lock()
def write_log(log_message):
with logging_write_lock:
with open('app.log', 'a') as log_file:
log_file.write(log_message + '\n')
# writes some fake logs
def worker(thread_id):
import datetime
from faker import Faker
faker = Faker()
for i in range(3):
write_log(f"Thread {thread_id}: Log entry ip:{faker.ipv4()},"
f"referer: {faker.uri()}, time: {datetime.datetime.now().strftime('%d/%b/%Y:%H:%M:%S')}")
# Simulate concurrent log writing
threads = []
for i in range(2):
thread = threading.Thread(target=worker, args=(i,))
threads.append(thread)
thread.start()
for thread in threads:
thread.join()
with open('app.log', 'r') as log_file:
print(log_file.read())
# Output in below image
Summary:
We have seen how Mutexes can be used to manage concurrency in Python, helping prevent race conditions, controlling access to limited resources and preserving consistency.
A fun fact, the requirement of mutual exclusion(mutex) was first identified and solved by Dijkstra in his 1965 paper, Solution of a problem in concurrent programming control which is credited as the first topic in the study of concurrent algorithms.
Let me know how you have used mutexes in your work or projects.
Thank you for reading!
Follow me for more insights and updates!
If you found this article useful, you can motivate me with some claps.
It helps a lot!
Also, did you know you could clap upto 50 times? 😄
Cheers!
Faaruk
Power of Mutex: The Secret to Smarter Python Code was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Faaruk
Faaruk | Sciencx (2024-10-21T23:22:11+00:00) Power of Mutex: The Secret to Smarter Python Code. Retrieved from https://www.scien.cx/2024/10/21/power-of-mutex-the-secret-to-smarter-python-code/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.