1. Redis Fragmentation mechanism

=============

1.1 Why fragmentation is needed

If you need to store huge amounts of memory data , If you only use one redis, No guarantee redis The efficiency of work . A lot of time is wasted in addressing . So we need a mechanism to meet this requirement .
Using the fragmentation mechanism to achieve :

1.2 Redis Build in pieces

1.2.1 Construction precautions

Redis The startup of the service depends on redis.conf Configuration file for . If you need to prepare 3 platform redis. You need to prepare 3 individual redis.conf Configuration of .

Prepare port number :
1.6379
2.6380
3.6381

1.2.2 Sharding implementation

Modify port number : Modify the respective port number .

start-up 3 platform redis The server

Verify that the server is running properly

1.2.3 Notes on slicing

1. Problem description ：
When you start multiple redis After the server , More than one redis There is no necessary connection for the time being , Each is a separate entity . It can store data . As shown in the figure .
2. If the partition is operated in the way of program , To put 3 platform redis As a whole , So it's totally different from the above operation . There won't be a key Save to more than one at the same time redis The phenomenon of .

1.3 Introduction case of segmentation

 `/**
* test Redis Fragmentation mechanism
* reflection : shards How to determine which one should be stored in redis What about China? ???
*/
@Test
public void testShards(){
List<JedisShardInfo> shards = new ArrayList<>();
shards.add(new JedisShardInfo("192.168.126.129",6379));
shards.add(new JedisShardInfo("192.168.126.129",6380));
shards.add(new JedisShardInfo("192.168.126.129",6381));
// Ready to slice objects
ShardedJedis shardedJedis = new ShardedJedis(shards);
shardedJedis.set("shards","redis Slice test ");
System.out.println(shardedJedis.get("shards"));
}`

1.4 Uniformity hash Algorithm

1.4.0 Common sense says

common sense 1: General hash yes 8 position 16 Hexadecimal number . 0-9 A-F (24)8 = 2^32
common sense 2: If you do the same data hash operation The result must be the same .
common sense 3: A data 1M And data 1G Of hash The speed of operation is the same .

1.4.1 Uniformity hash Algorithm is introduced

The consistency hash algorithm 1997 Proposed by MIT in , It's a special hash algorithm , The purpose is to solve the problem of distributed cache . [1] When removing or adding a server , Be able to change the mapping relationship between existing service requests and processing request servers as little as possible . Consistent hashing solves the problem of simple hashing algorithms in distributed hashes ( Distributed Hash Table,DHT) Dynamic scaling and other problems in [2] .

1.4.2 characteristic 1- Balance

Concept : Balance means hash The results should be distributed equally among the nodes , In this way, the load balancing problem is solved from the algorithm [4] .( Roughly average )
Problem description : It's all due to the nodes hash How to calculate . So it may appear as shown in the figure ., It leads to serious load imbalance

resolvent : Introduce virtual nodes

1.4.3 characteristic 2- monotonicity

characteristic : Monotonicity is defined as in newly added perhaps Abridge Node time , It does not affect the normal operation of the system [4] .

1.4.4 characteristic 3- Dispersion

The proverb, : Don't put eggs in a basket .
③ Decentralization refers to the fact that data should be stored in distributed cluster nodes ( The node itself can have backup ), It's not necessary for every node to store all the data [4]

1.5 SpringBoot Integrate Redis Fragmentation

1.5.1 Edit profile

`# To configure redis Single server
redis.host=192.168.126.129
redis.port=6379
# To configure redis Fragmentation mechanism
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381`

1.5.2 Edit configuration class

`@Configuration
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {
@Value("${redis.nodes}")
private String nodes; //node,node,node.....
// To configure redis Fragmentation mechanism
@Bean
public ShardedJedis shardedJedis(){
nodes = nodes.trim(); // Remove extra space on both sides
List<JedisShardInfo> shards = new ArrayList<>();
String[] nodeArray = nodes.split(",");
for (String strNode : nodeArray){ //strNode = host:port
String host = strNode.split(":")[0];
int port = Integer.parseInt(strNode.split(":")[1]);
JedisShardInfo info = new JedisShardInfo(host, port);
shards.add(info);
}
return new ShardedJedis(shards);
}
}`

1.5.3 modify AOP The injection term

2 Redis Sentinel mechanism

2.1 About Redis It's divided into sections

advantage : Realize the expansion of memory data .
shortcoming : If redis There is a problem with a node in the shard ., Then the whole redis There must be some problems with the fragmentation mechanism Directly affect the use of users .
Solution : Realization redis High availability .

2.2 To configure redis The structure of master-slave

Strategy Division : 1 Lord 2 from 6379 Lord 6380/6381 from

1. Copy the partitioned directory Change your name sentinel

1. Restart three redis The server
   
   3. Check redis The master-slave state of the node
   
   4. Realize the master-slave mount
   
   5. Check the status of the host
   

2.3 How the sentry works

Principle that :
1. To configure redis The structure of master-slave .
2. When sentinel service starts , Will monitor the current host . At the same time, get the details of the host ( The structure of master-slave )
3. When the sentinel uses the heartbeat detection mechanism (PING-PONG) continuity 3 If no feedback is received from the host, the host will be judged to be down .
4. When the sentry finds out that the host is down , The election mechanism will be opened , Choose one of the current slaves Redis As a mainframe .
5. Take the rest redis The node is set as the slave of the new host .

2.4 Edit sentinel profile

1). Copy profile
cp sentinel.conf sentinel/

2). Change protection mode

3). Turn on background operation

4). Set up sentinel surveillance
Among them 1 The number of votes in effect There is only one sentry at present, so write 1

5). Modify the downtime

6). When the election failed
explain : If the election does not end beyond the specified time , Re election .

7). Start sentinel service

2.5 Redis Sentinel high availability implementation

testing procedure :
1. Check the status of the host
2. take redis Main server down wait for 10 second Then check whether the slave is selected as the new host

3. restart 6379 The server ., Check whether it becomes the slave of the new host .

2.6 Sentinel entry case

`/**
* test Redis sentry
*/
@Test
public void testSentinel(){
Set<String> set = new HashSet<>();
//1. Pass the sentinel configuration information
set.add("192.168.126.129:26379");
JedisSentinelPool sentinelPool =
new JedisSentinelPool("mymaster",set);
Jedis jedis = sentinelPool.getResource();
jedis.set("aa"," sentinel test ");
System.out.println(jedis.get("aa"));
}`

2.7 SpringBoot Integrate Redis sentry (10 minute )

2.7.1 edit pro The configuration file

`# To configure redis Single server
redis.host=192.168.126.129
redis.port=6379
# To configure redis Fragmentation mechanism
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381
# Configure sentinel nodes
redis.sentinel=192.168.126.129:26379`

2.7.2 edit redis Configuration class

`@Configuration
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {
@Value("${redis.sentinel}")
private String sentinel; // For the time being, there's only one
@Bean
public JedisSentinelPool jedisSentinelPool(){
Set<String> sentinels = new HashSet<>();
sentinels.add(sentinel);
return new JedisSentinelPool("mymaster",sentinels);
}
}`

2.7.3 modify CacheAOP The injection term in

`package com.jt.aop;
import com.jt.anno.CacheFind;
import com.jt.config.JedisConfig;
import com.jt.util.ObjectMapperUtil;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.*;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.stereotype.Service;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisSentinelPool;
import redis.clients.jedis.ShardedJedis;
import java.lang.reflect.Method;
import java.util.Arrays;
@Aspect // I am a AOP Section class
@Component // Give the class to spring Container management
public class CacheAOP {
@Autowired
//private Jedis jedis; // A single redis
//private ShardedJedis jedis; // Fragmentation mechanism
private JedisSentinelPool jedisSentinelPool;
/**
* section = The breakthrough point + Notification method
* Annotations associated + Surrounding the notification Control whether the target method is implemented
*
* difficulty :
* 1. How to get annotation objects
* 2. Dynamic generation key prekey + User parameter array
* 3. How to get the return value type of a method
*/
@Around("@annotation(cacheFind)") // Parameter transfer variable transfer
//@Around("@annotation(com.jt.anno.CacheFind)")
public Object around(ProceedingJoinPoint joinPoint,CacheFind cacheFind){
// Get... From the pool jedis object
Jedis jedis = jedisSentinelPool.getResource();
Object result = null;
try {
//1. Splicing redis Storing data key
Object[] args = joinPoint.getArgs();
String key = cacheFind.preKey() +"::" + Arrays.toString(args);
//2. Inquire about redis Then judge whether there is data
if(jedis.exists(key)){
//redis There's a record in , There is no need to execute the target method
String json = jedis.get(key);
// Get the return value type of the method dynamically Look up Shape down
MethodSignature methodSignature = (MethodSignature) joinPoint.getSignature();
Class returnType = methodSignature.getReturnType();
result = ObjectMapperUtil.toObj(json,returnType);
System.out.println("AOP Inquire about redis cache ");
}else{
// Indicates that the data does not exist , Need to query database
result = joinPoint.proceed(); // Implementation target method and notification
// Save the query results to redis In the middle
String json = ObjectMapperUtil.toJSON(result);
// Determine whether the data needs a timeout
if(cacheFind.seconds()>0){
jedis.setex(key,cacheFind.seconds(),json);
}else {
jedis.set(key, json);
}
System.out.println("aop Execute target method query database ");
}
} catch (Throwable throwable) {
throwable.printStackTrace();
}
jedis.close(); // Remember to close the completed link .
return result;
}
}`

Homework

1. preview Redis Cluster building steps
2. understand redis How clusters work