Hand in hand teaching you to read and debug large open source project zookeeper

To cut the chill 2021-04-08 09:48:37
hand hand teaching read debug

The author of this article :HelloGitHub- Lao Xun

Hi, Here is HelloGitHub To launch the HelloZooKeeper series , Free and open source 、 Interesting 、 Entry level ZooKeeper course , For beginners with programming foundation .

Project address :https://github.com/HelloGitHub-Team/HelloZooKeeper

I'm going to take you all in today ZooKeeper Source code internal !

One 、 The source code to debug

Teaching a man to fish is better than giving him a fish

I always believed “ It's on paper ”, In the end, readers want to really understand ZK internals , Reading the source code is essential , If you have naked eyes like me Debug The ability of , In fact, you don't have to spend a lot of time building a source code debugging environment , The direct front is hard .

But if not , hold ZK Download the source code , Using a hand scale IDE Just run , And then interrupt where you need to learn , Isn't it beautiful

1.1 Download the source code

ZooKeeper 3.6.2 Source code download page

Choose one of the links above that has a high download speed , Click to download the compressed package , After the download is completed, unzip and you will get the following directory structure

├── zookeeper-server
├── zookeeper-recipes
├── zookeeper-metrics-providers
├── zookeeper-jute
├── zookeeper-it
├── zookeeper-docs
├── zookeeper-contrib
├── zookeeper-compatibility-tests
├── zookeeper-client
├── zookeeper-assembly
├── zk-merge-pr.py
├── pom.xml
├── owaspSuppressions.xml
├── excludeFindBugsFilter.xml
├── dev
├── conf
├── checkstyleSuppressions.xml
├── checkstyle-strict.xml
├── checkstyle-simple.xml
├── bin
├── README_packaging.md
├── README.md
├── NOTICE.txt
├── LICENSE.txt
├── Jenkinsfile-PreCommit
└── Jenkinsfile

There are... In the catalog pom.xml therefore ZK Need to pass through maven Compile the entire project , Make sure you have maven It's installed

$ mvn --version
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T02:33:14+08:00)
Maven home: /your/maven/home/apache-maven-3.5.4
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac"

If there is such an output description maven The installation was successful , I'll skip the specific installation process here , If you have difficulties , You can leave us a message

1.2 Compile the project

Access and pom.xml And enter

$ mvn install -DskipTests=true

You'll see the project compiling , Wait until the final output BUILD SUCCESS, That means the project is compiled

[INFO] Reactor Summary:
[INFO] Apache ZooKeeper 3.6.2 ............................. SUCCESS [ 3.621 s]
[INFO] Apache ZooKeeper - Documentation ................... SUCCESS [ 2.086 s]
[INFO] Apache ZooKeeper - Jute ............................ SUCCESS [ 10.633 s]
[INFO] Apache ZooKeeper - Server .......................... SUCCESS [ 19.246 s]
[INFO] Apache ZooKeeper - Metrics Providers ............... SUCCESS [ 0.108 s]
[INFO] Apache ZooKeeper - Prometheus.io Metrics Provider .. SUCCESS [ 1.286 s]
[INFO] Apache ZooKeeper - Client .......................... SUCCESS [ 0.083 s]
[INFO] Apache ZooKeeper - Recipes ......................... SUCCESS [ 0.092 s]
[INFO] Apache ZooKeeper - Recipes - Election .............. SUCCESS [ 0.244 s]
[INFO] Apache ZooKeeper - Recipes - Lock .................. SUCCESS [ 0.259 s]
[INFO] Apache ZooKeeper - Recipes - Queue ................. SUCCESS [ 0.295 s]
[INFO] Apache ZooKeeper - Assembly ........................ SUCCESS [ 5.425 s]
[INFO] Apache ZooKeeper - Compatibility Tests ............. SUCCESS [ 0.072 s]
[INFO] Apache ZooKeeper - Compatibility Tests - Curator 3.6.2 SUCCESS [ 0.432 s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.263 s
[INFO] Finished at: 2021-01-22T13:49:30+08:00
[INFO] ------------------------------------------------------------------------

1.3 Open and configure the project

Then you can go through your IDE Open this directory , What I use here is idea

Then start configuration Run/Debug Configurations

Click on + Add a new configuration

choice Application

1.3.1 Stand alone boot configuration

Then fill in or select according to the figure below

  1. Let's give this configuration a strong name first
  2. choice Modify options Open the submenu
  3. Make sure that all three sub options in the menu are selected ( There is √)

Then let's look at the specific configuration

The unzipped project path on my computer is /Users/junjiexun/Desktop/apache-zookeeper-3.6.2 Please revise it according to your own situation

  1. Choose your local jdk ( I'm a local 1.8 I don't know if other versions are OK , The lower version certainly can't , Because the source code uses 1.8 Some ways to write )
  2. choice zookeeper
  3. To configure VM options, The content is -Dlog4j.configuration=file:/Users/junjiexun/Desktop/apache-zookeeper-3.6.2/conf/log4j.properties, If not configured , Unable to output log
  4. Specify the startup class org.apache.zookeeper.server.ZooKeeperServerMain
  5. Stand alone startup requires command line parameters , The content is 2181 /Users/junjiexun/Desktop/apache-zookeeper-3.6.2/data
  6. This should not be modified , It will be filled in automatically , Anyway, the content is /Users/junjiexun/Desktop/apache-zookeeper-3.6.2
  7. Click... In the middle + Add package path , The content is org.apache.zookeeper.server.*

And then click Apply as well as OK Finish saving .

Then click on the bug to start

2021-01-22 15:12:16,319 [myid:] - INFO [main:NIOServerCnxnFactory@674] - binding to port
2021-01-22 15:12:16,413 [myid:] - INFO [main:WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2021-01-22 15:12:16,413 [myid:] - INFO [main:WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2021-01-22 15:12:16,413 [myid:] - INFO [main:ZKDatabase@132] - zookeeper.snapshotSizeFactor = 0.33
2021-01-22 15:12:16,413 [myid:] - INFO [main:ZKDatabase@152] - zookeeper.commitLogCount=500
2021-01-22 15:12:16,429 [myid:] - INFO [main:SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
2021-01-22 15:12:16,432 [myid:] - INFO [main:FileSnap@85] - Reading snapshot /Users/junjiexun/Desktop/apache-zookeeper-3.6.2/data/version-2/snapshot.2
2021-01-22 15:12:16,444 [myid:] - INFO [main:DataTree@1737] - The digest value is empty in snapshot
2021-01-22 15:12:16,480 [myid:] - INFO [main:ZKDatabase@289] - Snapshot loaded in 67 ms, highest zxid is 0x2, digest is 1371985504
2021-01-22 15:12:16,481 [myid:] - INFO [main:FileTxnSnapLog@470] - Snapshotting: 0x2 to /Users/junjiexun/Desktop/apache-zookeeper-3.6.2/data/version-2/snapshot.2
2021-01-22 15:12:16,488 [myid:] - INFO [main:ZooKeeperServer@529] - Snapshot taken in 6 ms
2021-01-22 15:12:16,544 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@136] - PrepRequestProcessor (sid:0) started, reconfigEnabled=false
2021-01-22 15:12:16,546 [myid:] - INFO [main:RequestThrottler@74] - zookeeper.request_throttler.shutdownTimeout = 10000
2021-01-22 15:12:16,623 [myid:] - INFO [main:ContainerManager@83] - Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0
2021-01-22 15:12:16,628 [myid:] - INFO [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.

See log output , If there is no error, it is a success !

Then we can test it with the client

ZooKeeper client = new ZooKeeper("", 3000, null);
List<String> children = client.getChildren("/", false);

Output is


The stand-alone version is done ! Let's try the cluster version next

1.3.2 Cluster version boot configuration

We sometimes need to debug the cluster version ZK There is only logic , The previous stand-alone version is not enough , And I recommend compressing the previous source code here , Unzip to two different directories , And then through IDE Open these two directories separately , To completely simulate two different nodes . The configuration of cluster version is similar to that of stand-alone version , Let's see what's different ? I'll start two nodes here myid Namely 1 and 2.

  1. First of all, the default zoo_sample.cfg Copy and rename to zoo.cfg, You can also rename it directly
  2. newly build data Catalog ( If not ), And create a new text file under it myid The text is 1

And then edit zoo.cfg

# modify
# Add the following two lines

The specific configuration is as follows :

  1. The startup class is different , The purpose of cluster is org.apache.zookeeper.server.quorum.QuorumPeerMain
  2. The command line parameters are different , What's coming in is zoo.cfg route , My path is /Users/junjiexun/Desktop/apache-zookeeper-3.6.2/conf/zoo.cfg

Then there is the configuration of the second node , Let me assume that the project directory of the second node is /Users/junjiexun/Desktop/apache-zookeeper-3.6.2-bak

The second node puts myid The content of the document should be changed to 2

zoo.cfg The content of is

# modify
# modify , Because my two nodes are in one machine , So ports can't be duplicated
# Also add the following two lines

The command line arguments are /Users/junjiexun/Desktop/apache-zookeeper-3.6.2-bak/conf/zoo.cfg

Other nodes I didn't mention 1 It's the same .

Let's start two nodes and try

2021-01-22 15:44:08,461 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2021-01-22 15:44:08,461 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2021-01-22 15:44:08,471 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):Learner@677] - Learner received NEWLEADER message
2021-01-22 15:44:08,471 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@1811] - Dynamic reconfig is disabled, we don't store the last seen config.
2021-01-22 15:44:08,471 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):FileTxnSnapLog@470] - Snapshotting: 0x28100000001 to /Users/junjiexun/Desktop/apache-zookeeper-3.6.2/data/version-2/snapshot.28100000001
2021-01-22 15:44:08,472 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):ZooKeeperServer@529] - Snapshot taken in 1 ms
2021-01-22 15:44:08,525 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):Learner@661] - Learner received UPTODATE message
2021-01-22 15:44:08,525 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@868] - Peer state changed: following - synchronization
2021-01-22 15:44:08,537 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):CommitProcessor@476] - Configuring CommitProcessor with readBatchSize -1 commitBatchSize 1
2021-01-22 15:44:08,537 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):CommitProcessor@438] - Configuring CommitProcessor with 4 worker threads.
2021-01-22 15:44:08,544 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):RequestThrottler@74] - zookeeper.request_throttler.shutdownTimeout = 10000
2021-01-22 15:44:08,567 [myid:1] - INFO [QuorumPeer[myid=1](plain=[0:0:0:0:0:0:0:0]:2181)(secure=disabled):QuorumPeer@863] - Peer state changed: following - broadcast

final Peer state changed The representative election is complete , The posted node 1 yes Follower, Be accomplished !

Then when you want to learn the flow of the source code , Start the server directly , Is it beautiful ~

1.4 Source code reading North

  • Server startup , colony QuorumPeerMain#main, stand-alone ZooKeeperServerMain#main
  • client ZooKeeper
  • Analysis of configuration related problems ,QuorumPeerConfig#parse
  • Memory model ( Little red book )DataTree
  • Callback notice ( Little Huang Ben )IWatchManager Look at the interface implementation
    • Default implementation WatchManager
    • Optimization plan WatchManagerOptimized
  • The election FastLeaderElection#lookForLeader
  • Server instance , Set up the pipeline setupRequestProcessors Method
    • Leader node LeaderZooKeeperServer
    • Follower node FollowerZooKeeperServer
    • Observer node ObserverZooKeeperServer
  • Every assembly line employee RequestProcessor Look at the implementation of the interface
  • Persistence log FileTxnLog,snapshot FileSnap
  • session management SessionTrackerImpl#run
  • agreement Record Look at the implementation of the interface

1.5 Source code reading experience

Reading the source code of a large project must be a time-consuming and laborious work , I'll also talk about my reading here ZK Source experience :

  • Don't pick details ! Large projects usually have more source code , If you look at every detail of logic , Will be lost in the sea of source code .
  • Usually read the source code with a purpose . for example :ZK How to do protocol conversion ,ZK How the election was held, and so on . With a purpose , Look at the relevant source code is to selectively ignore some other irrelevant details , You can use method names or comments , Let's have a perceptual understanding of the specific code block first .
  • Where I can't read , You can go to the Internet to see if anyone has written a similar blog , Standing on the shoulders of giants , It's very likely that you'll get through with someone else .
  • stay ZK In general, indirect or direct inheritance ZooKeeperThread All thread objects , The main logic can be seen run Method .
  • The important attribute of any class must be in the member field , You can roughly infer the data structure behind this class by looking at the member fields .
  • If there is a field blocking the queue in the member property , Probably the producer - The embodiment of the consumer model , You can focus on the use of the blocking queue , When to put in and take out elements .

1.6 Summary

I used some graphic space to introduce how to debug locally ZK Source code , And how to read the source code scientifically . My local environment is Mac, With IDE yes idea, If your environment or tools are different from mine , In case of difficulties , You can also leave us a message ~

Two 、ZK The design patterns applied to the design of

ZK It's a distributed application in itself , It's also an excellent open source project , I'd like to talk about the application I saw in reading the source code ZK The design pattern in this article

2.1 Producer consumer

This is ZK The most representative design pattern in is applied ,ZK Itself is C/S Architecture design , The request is the data sent by the client to the server , The response is the data sent by the server to the client , and ZK Some functions are not implemented by calling different methods in linear order , It's usually done by the producer thread , Blocking queues and consumer threads are made up of , The producer thread puts some request objects received upstream into the blocking queue , The current method returns , After that, the consumer thread gets the information from the blocking queue through the loop , And then complete the business logic . give an example :

  • PrepRequestProcessor, The blocking queue is submittedRequests
  • SyncRequestProcessor, The blocking queue is queuedRequests

2.2 Factory mode

There are some interface implementations ,ZK Itself provides the default choice , But if the user configures other implementations in the configuration ,ZK The factory will automatically create those other implementations . give an example :

  • Creating ClientCnxnSocket when , Will be based on zookeeper.clientCnxnSocket To select the client's IO Realization
  • Creating IWatchManager when , Will be based on zookeeper.watchManagerName To select the server's watch Management implementation
  • Creating ServerCnxnFactory when , Will be based on zookeeper.serverCnxnFactory To select the server's IO Factory realization

2.3 The chain of responsibility model

I've learned before ,ZK The business logic processing of the server side is through one by one XxxProcessor String together to achieve ,Processor They don't care about the call order , Just passed nextProcessor relation , Different server roles can also greatly reuse code in this way

  • In stand-alone mode :PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
  • Cluster mode Leader :LeaderRequestProcessor -> PrepRequestProcessor -> ProposalRequestProcessor -> CommitProcessor -> Leader.ToBeAppliedRequestProcessor -> FinalRequestProcessor
  • Cluster mode Follower :FollowerRequestProcessor -> CommitProcessor -> FinalRequestProcessor
  • Cluster mode Observer :ObserverRequestProcessor -> CommitProcessor -> FinalRequestProcessor

2.4 The strategy pattern

zookeeper.snapshot.compression.method It can be configured into different snapshot Compression algorithm , When you need to generate snapshot When you file , It will be executed according to different compression algorithms :

  • gzGZIPInputStream
  • snappySnappyInputStream
  • Default :BufferedInputStream

2.5 Decorator mode

Or just the compression algorithm , What is offered to the outside world is CheckedInputStream The unified processing object of , Use CheckedInputStream Package the above three compression implementations , These objects are all InputStream Subclasses of

switch ( Depending on the configuration ) {
// The embodiment of the strategic model
case GZIP:
is = new GZIPInputStream(fis);
case SNAPPY:
is = new SnappyInputStream(fis);
is = new BufferedInputStream(fis);
// It's all packaged in CheckedInputStream
// The embodiment of Decorator Pattern
return new CheckedInputStream(is, new Adler32());

3、 ... and 、 summary

Today I'm going to talk about how to go directly from ZK Source code DEBUG, Introduced some ZK The design pattern used in , If you have any questions about reading the source code , Welcome to leave me a message . This article was first published in 「HelloGitHub」 official account

The next issue introduces ZK The advanced usage of is purely practical , Look forward to it ~

Old rules , If you have any questions about the article, it can also be suggestions or right ZK Questions about the principle part , Welcome to the warehouse issue Give us , Or talk about the topic .

Address :https://www.yuque.com/kaixin1002/yla8hz

本文为[To cut the chill]所创,转载请带上原文链接,感谢

  1. How to realize encryption and decryption of spring boot interface parameters gracefully?
  2. A new way to play in spring 5! This kind of URL request makes me see better!
  3. Can parameters in spring MVC be passed like this? It's up!
  4. Hand in hand to teach you how to develop mybatis plug-ins
  5. Fine spring boot + thymeleaf, there are so many fun details!
  6. Spring boot logs all kinds of posture, it's time to clear!
  7. Web 3.0踏浪而来,分布式存储举足轻重|时空云邀请您参加Web3.0中国峰会暨分布式存储行业大会
  8. spring-aop 进不了切面方法的解决办法
  9. Web 3.0 is coming, distributed storage is very important | spatiotemporal cloud invites you to attend Web3.0 China Summit and distributed storage industry conference
  10. The solution of spring AOP can't enter the section method
  11. Linux中如何启用root用户
  12. How to enable root in Linux
  13. 踩坑 MySQL 索引,看看你真的会用吗?
  14. Hive优化之配置参数的优化(一)
  15. Step on the MySQL index to see if you really know how to use it?
  16. Optimization of configuration parameters for hive optimization (1)
  17. Linux入门教程资料分享
  18. Introduction to Linux
  19. 外部连接mysql docker容器异常
  20. Exception of external connection MySQL docker container
  21. Zookeeper分布式锁?
  22. Zookeeper distributed lock?
  23. 嵌入式Linux_Framebuffer_03点阵显示ASCII字符串
  24. 嵌入式Linux_Framebuffer_02字符编码
  25. Embedded Linux_ Framebuffer_ 03 dot matrix display ascii string
  26. Embedded Linux_ Framebuffer_ 02 character encoding
  27. Looking forward to new product launch of Xiaomi in spring CNMO takes you to see 11 new products in advance
  28. An inventory of the commonly used garbage collectors in Java
  29. Why is it so easy to get started with HBase?
  30. Implementation of PRC framework based on netty
  31. 2021 Java back end engineer must know knowledge - (Dubbo, distributed RPC framework)
  32. 关于spring advisor和元数据 同时来管理事务的问题
  33. How to manage transactions with spring advisor and metadata at the same time
  34. 使用Playwright对Java API实现自动视觉测试 - applitools
  35. Using playwright to implement automatic visual testing for Java API - applitools
  36. Dubbo和Spring cloud、Istio对比图
  37. Comparison of Dubbo with spring cloud and istio
  38. HttpServletRequest、通过request获得请求头、请求体等、解决中文乱码等问题
  39. Mybatis学习笔记-一对一,一对多,多对多关联查询
  40. Mybatis学习笔记-基本概念与操作
  41. HttpServletRequest, obtaining request header and request body through request, solving Chinese garbled code and other problems
  42. Mybatis learning notes - one to one, one to many, many to many association query
  43. Mybatis learning notes - basic concepts and operation
  44. Spring Cloud 升级之路 - 2020.0.x - 3. Undertow 的 accesslog 配置
  45. Spring cloud upgrade road - 2020.0. X - 3. Accesslog configuration of undertow
  46. 被Java培训机构坑骗后,我在这里找回了自信
  47. After being cheated by java training institutions, I found my confidence here
  48. Linux下安装Mysql出现的常见问题以及解决办法
  49. Common problems and solutions of installing MySQL under Linux
  50. java并发编程JUC第十二篇:AtomicInteger原子整型
  51. Java Concurrent Programming JUC Chapter 12: atomicinteger atomic integer
  52. 面经手册 · 第29篇《Spring IOC 特性有哪些,不会读不懂源码!》
  53. Chapter 29 "what are the features of spring IOC? I can't understand the source code! 》
  54. 浅析linux容器--Docker
  55. Analysis of Linux container -- docker
  56. 换种方法学操作系统,轻松入门Linux内核
  57. 浅析linux容器--Docker
  58. Another way to learn operating system, easy access to Linux kernel
  59. Analysis of Linux container -- docker
  60. 手摸手教你阅读和调试大型开源项目 ZooKeeper