Redis distributed lock for Python multithreading

preface


In many Internet product applications, some scenarios need to be locked, such as double 11 second kill, global incremental ID, floor generation, etc.

Most of the solutions are implemented based on DB. Redis is a single process and single thread mode. The queue mode is used to change concurrent access into serial access, and there is no competition between multiple clients for redis connection.

Secondly, Redis provides some commands SETNX and GETSET, which can facilitate the implementation of distributed locking mechanism.

 

Python code implementation

import time
import redis
import threading
 
#Connect using connection pool mode redis
redis_pool=redis.ConnectionPool(host="127.0.0.1",port=6379)
redis_conn=redis.Redis(connection_pool=redis_pool)
 
#definition redis class
class RedisLock():
    def __init__(self):
        self.redis_conn = redis_conn
        print("init the redis connection")
 
    #Acquire lock
    def get_lock(self,name,value):
        while True:
            # set(name, value, ex=None, px=None, nx=False, xx=False)
            # nx - If set to True,Only name When it does not exist, the current set Operation only
            # ex - Expiration time (seconds)
            result=self.redis_conn.set(name,value,nx=True,ex=3)
            # print(result)
            if(result):
                # Get result Terminate after while loop
                break
            # time.sleep(0.5)
 
    #Release lock
    def release_lock(self,name,value):
        #Get original name key Corresponding value
        old_value = redis_conn.get(name)
        print("--------------------------------the key =%s ;the name=%s"%(name,old_value))
        #Judge original value Is it the same as the value to be released
        if(old_value == value):
            #Same from redis Release inside
            self.redis_conn.delete(name)
            print("release the lock is success")
 
def redis_lock_test(lock,name,value):
    try:
        print("% --start to work"%name)
        print("% --ready get the lock and execute lock operation"%name)
        lock.get_lock(name,value)#Here is the lock acquisition operation
        print("% --get the lock and continue to operation"%name)
    except Exception as e:
        print("the exception is: %s"%str(e))
    finally:
        print("% --ready release the lock"%name)
        lock.release_lock(name,value)#Finally, the lock operation must be released
        print("% --release the lock is over"%name)
 
if __name__ == '__main__':
    start_time=time.time()
    rs=RedisLock()
    tasks=[]
    for i in range(1,3):
        # Create thread
        t = threading.Thread(target=redis_lock_test(rs,"task-name%"%i,"lock%d"%i))
        # Put the created thread into the list
        tasks.append(t)
        # Start thread
        t.start()
    #The daemon thread is not set here and is not set join Functional timeout Parameter, the main thread will wait until all the sub threads end, the main process ends and the program exits
    [t.join() for t in tasks]
    print("total waster time is:",time.time()-start_time)

 

be careful:
1. thread.setDaemon(True) when the parameter timeout=2 of the daemon thread join function is set, the main thread will wait for the accumulation of the timeout of multiple sub threads and such a period of time. When the time comes, the main thread ends, kills the unfinished sub threads, and the program exits.

 

2. When the daemon thread is not set and the timeout parameter of the join function is 2, the main thread will wait for the accumulation of the timeout of multiple sub threads and such a period of time. Once the main thread ends, but the sub thread is not killed, the sub thread can still continue to execute until all the sub threads end and the program exits.

 

Welcome to the "test of quantity" official account, reply [resource]

Python+Unittest framework API automation

Python+Unittest framework API automation

Python+Pytest framework API automation

Python + pandas + pyechards big data analysis

Python+Selenium framework Web UI automation

UI automation of Python+Appium framework APP

Python Programming learning resources dry goods

Resources and code are free~
Below the official account is a two-dimensional code, which can be swept directly by WeChat.

Remarks: my official account has been officially opened and dedicated to sharing IT Internet technology.

Including: data analysis, big data testing, machine learning, test development, API interface automation, test operation and maintenance, UI automation, performance testing, code testing, programming technology, etc.

The official account of WeChat search: "the way of testing without quantity", or the following two-dimensional code is scanned:

Add attention, let's grow together!

Tags: Python

Posted on Mon, 29 Nov 2021 04:57:44 -0500 by geetakhurana