Stage 3: day14 redis fragmentation mechanism redis sentinel mechanism

Half bump 2020-11-09 12:09:59
stage day14 day redis fragmentation


  1. Redis Fragmentation mechanism

=============

1.1 Why fragmentation is needed

If you need to store huge amounts of memory data , If you only use one redis, No guarantee redis The efficiency of work . A lot of time is wasted in addressing . So we need a mechanism to meet this requirement .
Using the fragmentation mechanism to achieve :
 Insert picture description here

1.2 Redis Build in pieces

1.2.1 Construction precautions

Redis The startup of the service depends on redis.conf Configuration file for . If you need to prepare 3 platform redis. You need to prepare 3 individual redis.conf Configuration of .

Prepare port number :
1.6379
2.6380
3.6381

1.2.2 Sharding implementation

 Insert picture description here
Modify port number : Modify the respective port number .
 Insert picture description here

start-up 3 platform redis The server
 Insert picture description here
Verify that the server is running properly
 Insert picture description here

1.2.3 Notes on slicing

1. Problem description :
When you start multiple redis After the server , More than one redis There is no necessary connection for the time being , Each is a separate entity . It can store data . As shown in the figure .
2. If the partition is operated in the way of program , To put 3 platform redis As a whole , So it's totally different from the above operation . There won't be a key Save to more than one at the same time redis The phenomenon of .
 Insert picture description here

1.3 Introduction case of segmentation

 `/**
* test Redis Fragmentation mechanism
* reflection : shards How to determine which one should be stored in redis What about China? ???
*/
@Test
public void testShards(){
List<JedisShardInfo> shards = new ArrayList<>();
shards.add(new JedisShardInfo("192.168.126.129",6379));
shards.add(new JedisShardInfo("192.168.126.129",6380));
shards.add(new JedisShardInfo("192.168.126.129",6381));
// Ready to slice objects
ShardedJedis shardedJedis = new ShardedJedis(shards);
shardedJedis.set("shards","redis Slice test ");
System.out.println(shardedJedis.get("shards"));
}`

1.4 Uniformity hash Algorithm

1.4.0 Common sense says

common sense 1: General hash yes 8 position 16 Hexadecimal number . 0-9 A-F (24)8 = 2^32
common sense 2: If you do the same data hash operation The result must be the same .
common sense 3: A data 1M And data 1G Of hash The speed of operation is the same .

1.4.1 Uniformity hash Algorithm is introduced

The consistency hash algorithm 1997 Proposed by MIT in , It's a special hash algorithm , The purpose is to solve the problem of distributed cache . [1] When removing or adding a server , Be able to change the mapping relationship between existing service requests and processing request servers as little as possible . Consistent hashing solves the problem of simple hashing algorithms in distributed hashes ( Distributed Hash Table,DHT) Dynamic scaling and other problems in [2] .
 Insert picture description here

1.4.2 characteristic 1- Balance

Concept : Balance means hash The results should be distributed equally among the nodes , In this way, the load balancing problem is solved from the algorithm [4] .( Roughly average )
Problem description : It's all due to the nodes hash How to calculate . So it may appear as shown in the figure ., It leads to serious load imbalance
 Insert picture description here
resolvent : Introduce virtual nodes
 Insert picture description here

1.4.3 characteristic 2- monotonicity

characteristic : Monotonicity is defined as in newly added perhaps Abridge Node time , It does not affect the normal operation of the system [4] .
 Insert picture description here

1.4.4 characteristic 3- Dispersion

The proverb, : Don't put eggs in a basket .
③ Decentralization refers to the fact that data should be stored in distributed cluster nodes ( The node itself can have backup ), It's not necessary for every node to store all the data [4]

1.5 SpringBoot Integrate Redis Fragmentation

1.5.1 Edit profile

`# To configure redis Single server
redis.host=192.168.126.129
redis.port=6379
# To configure redis Fragmentation mechanism
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381`

1.5.2 Edit configuration class

`@Configuration
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {
@Value("${redis.nodes}")
private String nodes; //node,node,node.....
// To configure redis Fragmentation mechanism
@Bean
public ShardedJedis shardedJedis(){
nodes = nodes.trim(); // Remove extra space on both sides
List<JedisShardInfo> shards = new ArrayList<>();
String[] nodeArray = nodes.split(",");
for (String strNode : nodeArray){ //strNode = host:port
String host = strNode.split(":")[0];
int port = Integer.parseInt(strNode.split(":")[1]);
JedisShardInfo info = new JedisShardInfo(host, port);
shards.add(info);
}
return new ShardedJedis(shards);
}
}`

1.5.3 modify AOP The injection term

 Insert picture description here

2 Redis Sentinel mechanism

2.1 About Redis It's divided into sections

advantage : Realize the expansion of memory data .
shortcoming : If redis There is a problem with a node in the shard ., Then the whole redis There must be some problems with the fragmentation mechanism Directly affect the use of users .
Solution : Realization redis High availability .

2.2 To configure redis The structure of master-slave

Strategy Division : 1 Lord 2 from 6379 Lord 6380/6381 from

1. Copy the partitioned directory Change your name sentinel
 Insert picture description here

  1. Restart three redis The server
     Insert picture description here
    3. Check redis The master-slave state of the node
     Insert picture description here
    4. Realize the master-slave mount
     Insert picture description here
    5. Check the status of the host
     Insert picture description here

2.3 How the sentry works

 Insert picture description here
Principle that :
1. To configure redis The structure of master-slave .
2. When sentinel service starts , Will monitor the current host . At the same time, get the details of the host ( The structure of master-slave )
3. When the sentinel uses the heartbeat detection mechanism (PING-PONG) continuity 3 If no feedback is received from the host, the host will be judged to be down .
4. When the sentry finds out that the host is down , The election mechanism will be opened , Choose one of the current slaves Redis As a mainframe .
5. Take the rest redis The node is set as the slave of the new host .

2.4 Edit sentinel profile

1). Copy profile
cp sentinel.conf sentinel/

2). Change protection mode
 Insert picture description here
3). Turn on background operation
 Insert picture description here
4). Set up sentinel surveillance
Among them 1 The number of votes in effect There is only one sentry at present, so write 1
 Insert picture description here
5). Modify the downtime
 Insert picture description here
6). When the election failed
explain : If the election does not end beyond the specified time , Re election .
 Insert picture description here
7). Start sentinel service
 Insert picture description here

2.5 Redis Sentinel high availability implementation

testing procedure :
1. Check the status of the host
2. take redis Main server down wait for 10 second Then check whether the slave is selected as the new host
 Insert picture description here
3. restart 6379 The server ., Check whether it becomes the slave of the new host .
 Insert picture description here

2.6 Sentinel entry case

`/**
* test Redis sentry
*/
@Test
public void testSentinel(){
Set<String> set = new HashSet<>();
//1. Pass the sentinel configuration information
set.add("192.168.126.129:26379");
JedisSentinelPool sentinelPool =
new JedisSentinelPool("mymaster",set);
Jedis jedis = sentinelPool.getResource();
jedis.set("aa"," sentinel test ");
System.out.println(jedis.get("aa"));
}`

2.7 SpringBoot Integrate Redis sentry (10 minute )

2.7.1 edit pro The configuration file

`# To configure redis Single server
redis.host=192.168.126.129
redis.port=6379
# To configure redis Fragmentation mechanism
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381
# Configure sentinel nodes
redis.sentinel=192.168.126.129:26379`

2.7.2 edit redis Configuration class

`@Configuration
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {
@Value("${redis.sentinel}")
private String sentinel; // For the time being, there's only one
@Bean
public JedisSentinelPool jedisSentinelPool(){
Set<String> sentinels = new HashSet<>();
sentinels.add(sentinel);
return new JedisSentinelPool("mymaster",sentinels);
}
}`

2.7.3 modify CacheAOP The injection term in

`package com.jt.aop;
import com.jt.anno.CacheFind;
import com.jt.config.JedisConfig;
import com.jt.util.ObjectMapperUtil;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.*;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.stereotype.Service;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisSentinelPool;
import redis.clients.jedis.ShardedJedis;
import java.lang.reflect.Method;
import java.util.Arrays;
@Aspect // I am a AOP Section class
@Component // Give the class to spring Container management
public class CacheAOP {
@Autowired
//private Jedis jedis; // A single redis
//private ShardedJedis jedis; // Fragmentation mechanism
private JedisSentinelPool jedisSentinelPool;
/**
* section = The breakthrough point + Notification method
* Annotations associated + Surrounding the notification Control whether the target method is implemented
*
* difficulty :
* 1. How to get annotation objects
* 2. Dynamic generation key prekey + User parameter array
* 3. How to get the return value type of a method
*/
@Around("@annotation(cacheFind)") // Parameter transfer variable transfer
//@Around("@annotation(com.jt.anno.CacheFind)")
public Object around(ProceedingJoinPoint joinPoint,CacheFind cacheFind){
// Get... From the pool jedis object
Jedis jedis = jedisSentinelPool.getResource();
Object result = null;
try {
//1. Splicing redis Storing data key
Object[] args = joinPoint.getArgs();
String key = cacheFind.preKey() +"::" + Arrays.toString(args);
//2. Inquire about redis Then judge whether there is data
if(jedis.exists(key)){
//redis There's a record in , There is no need to execute the target method
String json = jedis.get(key);
// Get the return value type of the method dynamically Look up Shape down
MethodSignature methodSignature = (MethodSignature) joinPoint.getSignature();
Class returnType = methodSignature.getReturnType();
result = ObjectMapperUtil.toObj(json,returnType);
System.out.println("AOP Inquire about redis cache ");
}else{
// Indicates that the data does not exist , Need to query database
result = joinPoint.proceed(); // Implementation target method and notification
// Save the query results to redis In the middle
String json = ObjectMapperUtil.toJSON(result);
// Determine whether the data needs a timeout
if(cacheFind.seconds()>0){
jedis.setex(key,cacheFind.seconds(),json);
}else {
jedis.set(key, json);
}
System.out.println("aop Execute target method query database ");
}
} catch (Throwable throwable) {
throwable.printStackTrace();
}
jedis.close(); // Remember to close the completed link .
return result;
}
}`

Homework

1. preview Redis Cluster building steps
2. understand redis How clusters work

版权声明
本文为[Half bump]所创,转载请带上原文链接,感谢

  1. 【计算机网络 12(1),尚学堂马士兵Java视频教程
  2. 【程序猿历程,史上最全的Java面试题集锦在这里
  3. 【程序猿历程(1),Javaweb视频教程百度云
  4. Notes on MySQL 45 lectures (1-7)
  5. [computer network 12 (1), Shang Xuetang Ma soldier java video tutorial
  6. The most complete collection of Java interview questions in history is here
  7. [process of program ape (1), JavaWeb video tutorial, baidu cloud
  8. Notes on MySQL 45 lectures (1-7)
  9. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  10. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  11. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  12. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  13. 【递归,Java传智播客笔记
  14. [recursion, Java intelligence podcast notes
  15. [adhere to painting for 386 days] the beginning of spring of 24 solar terms
  16. K8S系列第八篇(Service、EndPoints以及高可用kubeadm部署)
  17. K8s Series Part 8 (service, endpoints and high availability kubeadm deployment)
  18. 【重识 HTML (3),350道Java面试真题分享
  19. 【重识 HTML (2),Java并发编程必会的多线程你竟然还不会
  20. 【重识 HTML (1),二本Java小菜鸟4面字节跳动被秒成渣渣
  21. [re recognize HTML (3) and share 350 real Java interview questions
  22. [re recognize HTML (2). Multithreading is a must for Java Concurrent Programming. How dare you not
  23. [re recognize HTML (1), two Java rookies' 4-sided bytes beat and become slag in seconds
  24. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  25. RPC 1: how to develop RPC framework from scratch
  26. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  27. RPC 1: how to develop RPC framework from scratch
  28. 一次性捋清楚吧,对乱糟糟的,Spring事务扩展机制
  29. 一文彻底弄懂如何选择抽象类还是接口,连续四年百度Java岗必问面试题
  30. Redis常用命令
  31. 一双拖鞋引发的血案,狂神说Java系列笔记
  32. 一、mysql基础安装
  33. 一位程序员的独白:尽管我一生坎坷,Java框架面试基础
  34. Clear it all at once. For the messy, spring transaction extension mechanism
  35. A thorough understanding of how to choose abstract classes or interfaces, baidu Java post must ask interview questions for four consecutive years
  36. Redis common commands
  37. A pair of slippers triggered the murder, crazy God said java series notes
  38. 1、 MySQL basic installation
  39. Monologue of a programmer: despite my ups and downs in my life, Java framework is the foundation of interview
  40. 【大厂面试】三面三问Spring循环依赖,请一定要把这篇看完(建议收藏)
  41. 一线互联网企业中,springboot入门项目
  42. 一篇文带你入门SSM框架Spring开发,帮你快速拿Offer
  43. 【面试资料】Java全集、微服务、大数据、数据结构与算法、机器学习知识最全总结,283页pdf
  44. 【leetcode刷题】24.数组中重复的数字——Java版
  45. 【leetcode刷题】23.对称二叉树——Java版
  46. 【leetcode刷题】22.二叉树的中序遍历——Java版
  47. 【leetcode刷题】21.三数之和——Java版
  48. 【leetcode刷题】20.最长回文子串——Java版
  49. 【leetcode刷题】19.回文链表——Java版
  50. 【leetcode刷题】18.反转链表——Java版
  51. 【leetcode刷题】17.相交链表——Java&python版
  52. 【leetcode刷题】16.环形链表——Java版
  53. 【leetcode刷题】15.汉明距离——Java版
  54. 【leetcode刷题】14.找到所有数组中消失的数字——Java版
  55. 【leetcode刷题】13.比特位计数——Java版
  56. oracle控制用户权限命令
  57. 三年Java开发,继阿里,鲁班二期Java架构师
  58. Oracle必须要启动的服务
  59. 万字长文!深入剖析HashMap,Java基础笔试题大全带答案
  60. 一问Kafka就心慌?我却凭着这份,图灵学院vip课程百度云