What is distributed lock

Speaking of Redis, The first function we thought of was to cache data , besides ,Redis Because single process 、 High performance features , It's also often used for distributed locking .

We all know , The function in the program is the synchronization tool , Ensure that shared resources can only be accessed by one thread at the same time ,Java We're all familiar with the locks in , image synchronized 、Lock We use them all the time , however Java The lock is only valid when it is single machine , There's nothing that can be done in a distributed cluster environment , At this time, we need to use distributed lock .

Distributed lock , seeing the name of a thing one thinks of its function , It's the lock used in distributed project development , It can be used to control synchronous access to shared resources between distributed systems , Generally speaking , There are several features that distributed locks need to satisfy :

1、 Mutual exclusivity : At any moment , For the same data , Only one application can acquire the distributed lock ;

2、 High availability : In a distributed scenario , A small number of server downtime does not affect normal use , In this case, the services that provide distributed locks need to be deployed in a cluster ;

3、 Prevent lock timeout : If the client does not actively release the lock , The server will release the lock automatically after a period of time , Prevent client downtime or network unreachable When the birth and death lock ;

4、 Exclusivity : Locking and unlocking must be done by the same server , That is, the lock holder can release the lock , You can't show the lock you put on , You've been unlocked ;

There are many tools in the industry that can implement the effect of distributed locking , But the operation is nothing more than these : Lock 、 Unlock 、 Prevent lock timeout .

Since this article is about Redis Distributed lock , Then, of course, we should take Redis To extend the knowledge of .

The command to implement the lock

First introduce Redis Several orders of ,

1、SETNX, Usage is SETNX key value

SETNX yes 『 SET if Not eXists』( If it doesn't exist , be SET) Abbreviation , If the setting is successful, return to 1, Otherwise return to 0.

It can be seen that , When put key by lock Is set to "Java" after , Set it to another value and it will fail , It looks simple , It's also like monopolizing the lock , But there's a fatal problem , Namely key No expiration time , thus , Unless you delete key Or get the lock and set the expiration time , Otherwise, other threads will never get the lock .

In this case , We give key Add an expiration date , Let the thread acquire the lock directly and perform two steps :

SETNX Key 1
EXPIRE Key Seconds

There are also problems with this plan , Because getting lock and setting expiration time are divided into two steps , It's not atomic , There may be Get lock succeeded but set time failed , It's a waste of money .

But don't worry , Such thing Redis The government has already considered for us , So here's the command

2、SETEX, usage SETEX key seconds value

Will value value Related to key , And will key The lifetime of is set to seconds ( In seconds ). If key Already exist ,SETEX The command will override the old value .

This command is similar to the following two commands :

SET key value
EXPIRE key seconds  #  Set the lifetime 

These two steps are atomic , It will be done at the same time .

3、PSETEX , usage PSETEX key milliseconds value

This command and SETEX Command similar , But it's set in milliseconds key Survival time , Not like it SETEX Order that , In seconds .

however , from Redis 2.6.12 Version start ,SET Commands can be implemented by parameters and SETNX、SETEX、PSETEX   Effect of three commands .

Like this command

SET key value NX EX seconds

add NX、EX After the parameter , The effect is equivalent to SETEX, This is also Redis Get the most common lock writing .

How to release the lock

The command to release the lock is simple , Delete directly key Just go , But as we said earlier , Because distributed locks must be released by the lock holder himself , So we have to make sure that the thread releasing the lock is the holder , No problem, delete it , thus , It's two steps , It seems to be against atomicity again , What shall I do? ?

Don't panic , We can use lua The script assembles the two steps , It's like this :

if redis.call("get",KEYS[1]) == ARGV[1]
then
    return redis.call("del",KEYS[1])
else
    return 0
end

KEYS[1] It is the present. key The name of ,ARGV[1] It can be of the current thread ID( Or other unfixed values , Can identify the thread ), This prevents threads holding expired locks , Or other threads delete existing locks by mistake .

Code implementation

After knowing the principle , We can do it by hand Redis The function of distributed lock has been improved , Because the purpose of this article is to explain the principle , Not to teach you how to write distributed locks , So I implemented it in pseudo code .

First of all redis Lock tool class , Contains the basic methods of locking and unlocking :

public class RedisLockUtil {
    private String LOCK_KEY = "redis_lock";
    // key The holding time of ,5ms
    private long EXPIRE_TIME = 5;
    //  Waiting timeout ,1s
    private long TIME_OUT = 1000;
    // redis Command parameter , amount to nx and px The command set of
    private SetParams params = SetParams.setParams().nx().px(EXPIRE_TIME);
    // redis Connection pool , It's local redis client
    JedisPool jedisPool = new JedisPool("127.0.0.1", 6379);
    /**
     *  Lock
     *
     * @param id
     *             Thread id, Or other fields that recognize the current thread and do not repeat
     * @return
     */
    public boolean lock(String id) {
        Long start = System.currentTimeMillis();
        Jedis jedis = jedisPool.getResource();
        try {
            for (;;) {
                // SET Command return OK , It is proved that the lock acquisition is successful
                String lock = jedis.set(LOCK_KEY, id, params);
                if ("OK".equals(lock)) {
                    return true;
                }
                //  Otherwise, the loop waits , stay TIME_OUT The lock has not been obtained in time , Get failed
                long l = System.currentTimeMillis() - start;
                if (l >= TIME_OUT) {
                    return false;
                }
                try {
                    //  Sleep for a while , Otherwise, repeated execution of the loop will always fail
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        } finally {
            jedis.close();
        }
    }
    /**
     *  Unlock
     *
     * @param id
     *             Thread id, Or other fields that recognize the current thread and do not repeat
     * @return
     */
    public boolean unlock(String id) {
        Jedis jedis = jedisPool.getResource();
        //  Delete key Of lua Script
        String script = "if redis.call('get',KEYS[1]) == ARGV[1] then" + "   return redis.call('del',KEYS[1]) " + "else"
            + "   return 0 " + "end";
        try {
            String result =
                jedis.eval(script, Collections.singletonList(LOCK_KEY), Collections.singletonList(id)).toString();
            return "1".equals(result);
        } finally {
            jedis.close();
        }
    }
}

The specific code function comments have been written very clearly , Then we can write a demo Class to test the effect :

public class RedisLockTest {
    private static RedisLockUtil demo = new RedisLockUtil();
    private static Integer NUM = 101;
    public static void main(String[] args) {
        for (int i = 0; i < 100; i++) {
            new Thread(() -> {
                String id = Thread.currentThread().getId() + "";
                boolean isLock = demo.lock(id);
                try {
                 //  If you get the lock , Just subtract one from the shared parameter
                    if (isLock) {
                        NUM--;
                        System.out.println(NUM);
                    }
                } finally {
                 //  Release the lock must pay attention to finally
                    demo.unlock(id);
                }
            }).start();
        }
    }
}

We created 100 Two threads to simulate concurrency , The result of implementation is like this :

It can be seen that , The effect of the lock is , Thread safety is guaranteed .

Of course , The above code is just a simple implementation of the effect , The function is definitely incomplete , There are many aspects to consider in a sound distributed lock , Actually, it's not that easy to design .

Our purpose is just to learn and understand the principle , It's unrealistic to write an industrial distributed lock tool , It's not necessary. , There are a lot of similar open source tools (Redisson), The principle is almost the same , And it has already been tested by the industry , Just use it .

Although the function is realized , But actually in terms of design , This kind of distributed lock has a big flaw , This is what this article wants to focus on , What are the defects ?

The flaws of distributed locks

One 、 Lock invalidation caused by long-term blocking of client

client 1 Got the lock. , Because of network problems or GC And so on , And then the lock expired before the business program finished executing , At this point, the client 2 You can also get the lock normally , May cause thread safety problems .

So how to prevent such an exception ? Let's not talk about solutions , Let's talk about it after introducing other defects .

Two 、redis Server clock drift problem

If redis The server's machine clock jumps forward , Will lead to this key Premature timeout failure , For example, the client 1 When I get the lock ,key The expiration time of is 12:02 branch , but redis The server's own clock is faster than the client's 2 minute , Lead to key stay 12:00 It doesn't work when it's too late , Now , If the client 1 If the lock hasn't been released , It may lead to the problem that multiple clients hold the same lock at the same time .

3、 ... and 、 Single point instance security issues

If redis It's single master Mode , When this machine goes down , Then all the clients can't get the lock , To improve usability , Maybe you'll give this master Add one more slave, But because redis The master-slave synchronization of is asynchronous , There may be clients 1 After setting the lock ,master Hang up ,slave Upgrade to master, Because of the nature of asynchronous replication , client 1 The set lock is missing , At this point, the client 2 Setting locks also works , Causes the client to 1 And the client 2 Have locks at the same time .

In order to solve Redis A single point of the problem ,redis The author of put forward RedLock Algorithm .

RedLock Algorithm

The premise of this algorithm is Redis It has to be multi node , Can effectively prevent a single point of failure , The specific implementation idea is as follows :

1、 Get the current timestamp (ms);

2、 First set key Effective time (TTL), Beyond this time, it will be released automatically , then client( client ) Try using the same key and value For all redis Instance , Every link redis Instance, set a ratio of TTL A lot of short time outs , This is to avoid waiting too long for the closed redis service . And try to get the next redis example .

such as :TTL( Which is the expiration time ) by 5s, Then the time-out time for obtaining the lock can be set to 50ms, So if 50ms Can't get lock inside , Just give up getting the lock , To try to get the next lock ;

3、client Subtract the time of the first step from the time after acquiring all available locks , also redis The clock drift error of the server , And then the time difference is less than TTL Time and the number of successful lock setting instances >= N/2 + 1(N by Redis Number of instances ), So the lock is successful

such as TTL yes 5s, Connect redis It's time to get all the locks 2s, And then subtract the clock drift ( Suppose the error is 1s about ), Then the real effective time of the lock is 2s 了 ;

4、 If the client fails to acquire the lock for some reason , Will start unlocking all redis example .

According to this algorithm , We assume that there is 5 individual Redis For example , that client Just get one of them 3 The lock above the platform is a success , It's like this with a flow chart :

Okay , The algorithm is also introduced , In terms of design , without doubt ,RedLock The main idea of the algorithm is to effectively prevent Redis A single point of failure , And it's designing TTL The error of server clock drift is also taken into account , It improves the security of distributed lock .

But is that really the case ? Anyway, I personally feel that the effect is general ,

First and foremost , We can see , stay RedLock In the algorithm, , The effective time of the lock subtracts the connection Redis The duration of the instance , If this process takes too long due to network problems , In the end, the effective time left for the lock will be greatly reduced , Client access to shared resources is very short , It's very likely that the lock will expire during the processing of the program . and , The effective time of the lock also needs to subtract the clock drift of the server , But how much should be reduced , If this value is not set properly , It's easy to have problems .

And then the second point , Although this algorithm takes into account the use of multiple nodes to prevent Redis A single point of failure , But if a node crashes and restarts , It is still possible for multiple clients to acquire locks at the same time .

Let's say there are 5 individual Redis node :A、B、C、D、E, client 1 and 2 Lock them separately

  1. client 1 It's locked A,B,C, Lock acquired successfully ( but D and E There is no lock ).
  2. node C Of master Hang up , And then the lock hasn't synced to slave,slave Upgrade to master Lost the client after 1 Lock added .
  3. client 2 This is the time to get the lock , lock C,D,E, Lock acquired successfully .

such , client 1 And the client 2 I got the lock at the same time , The hidden danger of program security still exists . besides , If one of these nodes has time drift , It can also lead to lock security problems .

So , Although the availability and reliability are improved through multi instance deployment , but RedLock It's not completely solved Redis The hidden danger of single point of failure , It doesn't solve clock drift 、 Lock timeout failure caused by long-term blocking of client .

From this point of view ,RedLock The algorithm does not guarantee the security of the lock .

Conclusion

Someone may want to ask further , So what can we do to ensure the absolute safety of the lock ?

I can only say , You can't have both fish and bear paws , The reason why we use Redis As a tool for distributed locking , It's largely because Redis It has the characteristics of high efficiency and single process , Even in the case of high concurrency, it can guarantee the performance very well , But a lot of times , Performance and safety cannot be taken into account , If you have to keep the lock safe , You can use other middleware such as db、zookeeper To control , These tools can ensure the safety of the lock , But the performance can only be said to be unsatisfactory , Otherwise, people would have used it for a long time .

Generally speaking , use Redis If the control of shared resources and data security requirements are high , The ultimate solution is to make idempotent control over business data , thus , Even if multiple clients get locks, the consistency of data will not be affected . Of course , Not all scenes are suitable for this , You need to deal with the specific choice , After all , There's no perfect technology , Only the right one is the best .


If you find the article useful , Welcome to like or forward to support , It would be the best encouragement for me to create !

author : I'm Xue , An Internet person who is not limited to technology , Like to use easy to understand language to deconstruct the knowledge of back-end technology , I want to read more interesting articles to pay attention to my official account , WeChat search 【 I'm Xue 】 Can focus on