What exactly does zookeeper solve?

Java technology stack 2020-11-06 20:58:51
exactly zookeeper solve

The goal is

ZooKeeper Is very popular , There's a basic question :

  • ZooKeeper What is it used for ?
  • Not before ZK, Why was it born ZK?

OK, Answer the question above :( Here's what I said intuitively )

  • ZooKeeper Distributed application development is used to simplify , Shield developers from the underlying details of the distributed application development process
  • ZooKeeper The exposure is simple API, For supporting distributed application development
  • ZooKeeper While providing the above functions , It's still a High performance 、 High availability 、 Highly reliable distributed cluster

That's all it says , To sum up ,ZK Distributed application development can solve the problem ,ZK Can solve the problem very well . To this step , There are more questions :

  1. Distributed application development , What are the common problems ?ZK How to shield these underlying details ?
  2. ZooKeeper Exposed to the outside world API? these API How to support distributed application development ? these API Can it be simplified ?API How about the semantic nature of ?
  3. ZooKeeper Itself is a high performance 、 High availability 、 Highly reliable distributed cluster , There's a simple question :
    • What does high performance mean ?ZooKeeper In order to achieve high performance , What has been done ?
    • High availability is the same as
    • High reliability is the same as

Note: This article wiki It is to solve the first question .( Other questions will be in other blog Step by step )

Why ZooKeeper

An application , When multiple processes are involved in collaboration , There are a lot of complex process collaboration logic in business logic code .

The above multi process collaboration logic , Yes 2 Characteristics :

  • Deal with complexity
  • Processing logic is reusable

therefore , Consider the common problems of multi process collaboration , As infrastructure , Give Way RD More focus on business logic development , namely :

ZooKeeper It is one of the basic services of multi process cooperation .

ZooKeeper Characteristics

ZooKeeper There are several simple features :

  • ZooKeeper Of API: from file system API The inspiration , Provide simple API
  • ZooKeeper Running on a dedicated server , Separate from business logic , To ensure the High fault tolerance and Extensibility

ZooKeeper It's a storage facility , But pay special attention to

  • ZK The data stored on is focused on : Collaborative data Metadata ), Instead of applying data , Application data has its own storage scheme , for example HDFS etc.
  • ZK Essentially , It can be seen as a kind of special FS

In particular :

Application data and metadata , Due to different scenarios , There are differences in requirements for consistency and persistence , therefore , Architecture design 、 In the process of data governance , Should be 2 Class data is viewed independently 、 Independent storage .

ZooKeeper The mission of

ZK The core problem to be solved :

ZK The goal is : Simplify distributed application development , Multi process collaboration problem . For distributed applications , Provide Efficient reliable The distributed coordination service ( Basic services ), for example :

  • Unified naming service
  • Distributed lock
  • Process crash detection
  • Leader The election
  • Configuration Management : When the configuration changes , Send it to all in time Client.

A simple question : What is multi process collaboration ? Nima , Over and over , You have everything , Facing this crazy skull , Let's answer .

Multi process collaboration , The whole is divided into 2 class :

  1. Collaboration : Many processes need to be dealt with together , Some processes take action, and others work properly , for example : Master-slave structure ,M towards S Assigned tasks ,S Will execute , otherwise S Just stay idle
  2. competition : Two processes can't work at the same time , One process must wait for another process to finish executing , for example : Master-slave structure ,M After node failure , quite a lot S All want to be M, At this time , You need mutexes , Only the first to get the lock S Become M

In particular :

  1. No collaboration across networks : Multi process , It can be on the same physical host , Synchronization primitives are very convenient ( such as ? The Conduit 、 Shared memory 、 Message queue 、 Semaphore )
  2. Collaboration across networks : Multi process , Distributed on different physical hosts ,ZK Focus on this category

Multi process collaboration across networks , Process of communication , The basic idea is 2 individual :

  1. Message mechanism : Through the network , Direct information exchange , Multi message passing algorithm , Implement synchronization primitives
  2. Shared memory : Using external shared storage , Achieve multi process collaboration , requirement Shared memory Provide orderly access to ,ZK In this way

In real systems , Cross network communication , There are several common problems :

  1. Message delay : Because of the Internet , Send later and arrive first
  2. Processor performance : Due to system scheduling , When the message arrives , Delays in processing
  3. Clock offset : Different physical hosts , Clock offset

ZK Carefully designed for Block the above 3 A common problem , Make these problems completely transparent at the application service level .

ZooKeeper characteristic

ZooKeeper The essence of the solution

Consistency of distributed systems :

  1. The messaging : Delay , Message sent first , You don't have to arrive first ;
  2. The messaging : Loss of sex , Message sent , May be lost ;
  3. Node crash : Within a distributed system , Any node can crash ;

under these circumstances , How to ensure data consistency ?

  1. Vote on the proposal : Based on voting strategy ,2PC
  2. The election vote : Based on voting strategy , Cast The highest priority node ( The node that contains the latest data )

Paxos The goal is : solve Distributed consistency problem , Improve distributed systems Fault tolerance The consistency algorithm .

Paxos The essence : be based on The messaging Of Highly fault-tolerant Of Consistency algorithm

ZooKeeper location

ZooKeeper yes :

  1. Distributed coordination services
  2. Efficient 、 reliable
  3. Convenient applications , focusing Business logic development , And you don't have to pay too much attention to Details of distributed inter process collaboration

ZooKeeper No direct exposure The original language , It is , Expose a part of Calling method Composed of API, File system like API, Support applications to implement their own The original language .

ZooKeeper characteristic

ZooKeeper The following distributed consistency features can be guaranteed :

  • Sequential consistency : The same Client Initiated transaction request , Strictly follow the order of initiation
  • Atomicity : Transaction request , Or apply to all nodes , Either one node has no application
  • A single view :Client No matter which node you connect to , The server-side data you see is consistent (Note: inaccurate , It's actually Final consistency
  • reliability : Once the transaction is successfully executed , The state remains permanently
  • The real time : Once the transaction is successfully executed ,Client You can't see the latest data immediately , but ZooKeeper Guarantee Final consistency

ZooKeeper Design objectives

ZooKeeper Committed to providing High performance High availability Sequential consistency The distributed coordination service , Guaranteed data Final consistency .

Goal one : High performance ( A simple data model )

  1. use A tree structure Organize data nodes ;
  2. Full data nodes , All stored in memory ;
  3. Follower and Observer Handle non transactional requests directly ;

Goal two : High availability ( Build clusters )

  1. More than half of the machines survive , The service will run normally
  2. Automatically Leader The election

Goal three : Sequential consistency ( The order of transaction operations )

  1. Each transaction request , Will be forwarded to Leader Handle
  2. Every business , The global allocation will be incremental id(zxid,64 position :epoch + Self increasing id)

Goal four : Final consistency

  1. By proposing a vote , Ensure the reliability of transaction commit
  2. Propose a way to vote , Only guarantee Client After receiving the transaction commit successfully , More than half of the nodes can see the latest data

ZooKeeper Before appearance

ZK Before appearance , There are two ways of distributed system , Achieve multi process collaboration :

  1. Distributed lock manager
  2. Distributed database

ZK More focused on process collaboration , It does not provide any lock interface and general storage data interface .( doubt :ZK It can also be provided , We don't need to use it )

application server , common 2 Needs :

  1. Master-Slave Leader The election : Asked to provide Master Node election function
  2. Process response tracking Crash detection : Request to provide a trace of the process's survival state
  3. Distributed lock : Exclusive lock

ZK For the above 2 These strategies provide the basis API.

ZooKeeper Scenarios that are not applicable :

  1. Mass data storage :ZK The essence is special FS, but ZK Used to store Metadata , It needs to be stored separately Application data

Introduction to terms

The term explain
Distributed systems Across multiple physical hosts , A system consisting of multiple independently running nodes
The original language Business indivisible elements / The process , give an example : Distributed lock primitives , Can expose establish Inquire about Release Several ways

Reference source

  1. ZooKeeper-Distributed Process Coordination Chapter 1 brief introduction
  2. from Paxos To Zookeeper Principle and practice of distributed consistency Chapter 4 First time to know ZooKeeper

Recent hot article recommends :

1.Java 15 Official release , 14 A new feature , Refresh your mind !!

2. Finally, I got it through open source projects IntelliJ IDEA Activation code , It's delicious !

3. I use Java 8 Wrote a piece of logic , I can't understand it , You try ..

4. To hang up Tomcat ,Undertow It's very powerful !!

5.《Java Development Manual ( Song Mountain version )》 The latest release , Download it quickly !

I think it's good , Don't forget to like it + Forward !

本文为[Java technology stack]所创,转载请带上原文链接,感谢

  1. 【计算机网络 12(1),尚学堂马士兵Java视频教程
  2. 【程序猿历程,史上最全的Java面试题集锦在这里
  3. 【程序猿历程(1),Javaweb视频教程百度云
  4. Notes on MySQL 45 lectures (1-7)
  5. [computer network 12 (1), Shang Xuetang Ma soldier java video tutorial
  6. The most complete collection of Java interview questions in history is here
  7. [process of program ape (1), JavaWeb video tutorial, baidu cloud
  8. Notes on MySQL 45 lectures (1-7)
  9. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  10. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  11. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  12. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  13. 【递归,Java传智播客笔记
  14. [recursion, Java intelligence podcast notes
  15. [adhere to painting for 386 days] the beginning of spring of 24 solar terms
  16. K8S系列第八篇(Service、EndPoints以及高可用kubeadm部署)
  17. K8s Series Part 8 (service, endpoints and high availability kubeadm deployment)
  18. 【重识 HTML (3),350道Java面试真题分享
  19. 【重识 HTML (2),Java并发编程必会的多线程你竟然还不会
  20. 【重识 HTML (1),二本Java小菜鸟4面字节跳动被秒成渣渣
  21. [re recognize HTML (3) and share 350 real Java interview questions
  22. [re recognize HTML (2). Multithreading is a must for Java Concurrent Programming. How dare you not
  23. [re recognize HTML (1), two Java rookies' 4-sided bytes beat and become slag in seconds
  24. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  25. RPC 1: how to develop RPC framework from scratch
  26. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  27. RPC 1: how to develop RPC framework from scratch
  28. 一次性捋清楚吧,对乱糟糟的,Spring事务扩展机制
  29. 一文彻底弄懂如何选择抽象类还是接口,连续四年百度Java岗必问面试题
  30. Redis常用命令
  31. 一双拖鞋引发的血案,狂神说Java系列笔记
  32. 一、mysql基础安装
  33. 一位程序员的独白:尽管我一生坎坷,Java框架面试基础
  34. Clear it all at once. For the messy, spring transaction extension mechanism
  35. A thorough understanding of how to choose abstract classes or interfaces, baidu Java post must ask interview questions for four consecutive years
  36. Redis common commands
  37. A pair of slippers triggered the murder, crazy God said java series notes
  38. 1、 MySQL basic installation
  39. Monologue of a programmer: despite my ups and downs in my life, Java framework is the foundation of interview
  40. 【大厂面试】三面三问Spring循环依赖,请一定要把这篇看完(建议收藏)
  41. 一线互联网企业中,springboot入门项目
  42. 一篇文带你入门SSM框架Spring开发,帮你快速拿Offer
  43. 【面试资料】Java全集、微服务、大数据、数据结构与算法、机器学习知识最全总结,283页pdf
  44. 【leetcode刷题】24.数组中重复的数字——Java版
  45. 【leetcode刷题】23.对称二叉树——Java版
  46. 【leetcode刷题】22.二叉树的中序遍历——Java版
  47. 【leetcode刷题】21.三数之和——Java版
  48. 【leetcode刷题】20.最长回文子串——Java版
  49. 【leetcode刷题】19.回文链表——Java版
  50. 【leetcode刷题】18.反转链表——Java版
  51. 【leetcode刷题】17.相交链表——Java&python版
  52. 【leetcode刷题】16.环形链表——Java版
  53. 【leetcode刷题】15.汉明距离——Java版
  54. 【leetcode刷题】14.找到所有数组中消失的数字——Java版
  55. 【leetcode刷题】13.比特位计数——Java版
  56. oracle控制用户权限命令
  57. 三年Java开发,继阿里,鲁班二期Java架构师
  58. Oracle必须要启动的服务
  59. 万字长文!深入剖析HashMap,Java基础笔试题大全带答案
  60. 一问Kafka就心慌?我却凭着这份,图灵学院vip课程百度云