2021 summary of the latest Java common open source libraries, the top java development tools necessary for developers

Programmer Xiao Gang 2021-10-14 08:06:37
summary latest java common open


ZooKeeper Is very popular , There's a basic question :

  • ZooKeeper What is it used for ?
  • Not before ZK, Why was it born ZK?

OK, Answer the question above :( Here's what I said intuitively )

  • ZooKeeper Distributed application development is used to simplify , Shield developers from the underlying details of the distributed application development process
  • ZooKeeper The exposure is simple API, For supporting distributed application development
  • ZooKeeper While providing the above functions , It's still a High performance 、 High availability 、 Highly reliable distributed cluster

That's all it says , To sum up ,ZK Distributed application development can solve the problem ,ZK Can solve the problem very well . To this step , There are more questions :

  1. Distributed application development , What are the common problems ?ZK How to shield these underlying details ?
  2. ZooKeeper Exposed to the outside world API? these API How to support distributed application development ? these API Can it be simplified ?API How about the semantic nature of ?
  3. ZooKeeper Itself is a high performance 、 High availability 、 Highly reliable distributed cluster , There's a simple question :
  • What does high performance mean ?ZooKeeper In order to achieve high performance , What has been done ?
  • High availability is the same as
  • High reliability is the same as

Note: This article wiki It is to solve the first question .( Other questions will be in other blog Step by step )

Why ZooKeeper

An application , When multiple processes are involved in collaboration , There are a lot of complex process collaboration logic in business logic code .

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java

The above multi process collaboration logic , Yes 2 Characteristics :

  • Deal with complexity
  • Processing logic is reusable

therefore , Consider the common problems of multi process collaboration , As infrastructure , Give Way RD More focus on business logic development , namely :

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java_02

ZooKeeper It is one of the basic services of multi process cooperation .

ZooKeeper Characteristics

ZooKeeper There are several simple features :

  • ZooKeeper Of API: from file system API The inspiration , Provide simple API
  • ZooKeeper Running on a dedicated server , Separate from business logic , To ensure the High fault tolerance and Extensibility

ZooKeeper It's a storage facility , But pay special attention to

  • ZK The data stored on is focused on : Collaborative data Metadata ), Instead of applying data , Application data has its own storage scheme , for example HDFS etc.
  • ZK Essentially , It can be seen as a kind of special FS

In particular :

Application data and metadata , Due to different scenarios , There are differences in requirements for consistency and persistence , therefore , Architecture design 、 In the process of data governance , Should be 2 Class data is viewed independently 、 Independent storage .

ZooKeeper The mission of

ZK The core problem to be solved :

ZK The goal is : Simplify distributed application development , Multi process collaboration problem . For distributed applications , Provide Efficient reliable The distributed coordination service ( Basic services ), for example :

  • Unified naming service
  • Distributed lock
  • Process crash detection
  • Leader The election
  • Configuration Management : When the configuration changes , Send it to all in time Client.

A simple question : What is multi process collaboration ? Nima , Over and over , You have everything , Facing this crazy skull , Let's answer .

Multi process collaboration , The whole is divided into 2 class :

  1. Collaboration : Many processes need to be dealt with together , Some processes take action, and others work properly , for example : Master-slave structure ,M towards S Assigned tasks ,S Will execute , otherwise S Just stay idle
  2. competition : Two processes can't work at the same time , One process must wait for another process to finish executing , for example : Master-slave structure ,M After node failure , quite a lot S All want to be M, At this time , You need mutexes , Only the first to get the lock S Become M

In particular :

  1. No collaboration across networks : Multi process , It can be on the same physical host , Synchronization primitives are very convenient ( such as ? The Conduit 、 Shared memory 、 Message queue 、 Semaphore )
  2. Collaboration across networks : Multi process , Distributed on different physical hosts ,ZK Focus on this category

Multi process collaboration across networks , Process of communication , The basic idea is 2 individual :

  1. Message mechanism : Through the network , Direct information exchange , Multi message passing algorithm , Implement synchronization primitives
  2. Shared memory : Using external shared storage , Achieve multi process collaboration , requirement Shared memory Provide orderly access to ,ZK In this way

In real systems , Cross network communication , There are several common problems :

  1. Message delay : Because of the Internet , Send later and arrive first
  2. Processor performance : Due to system scheduling , When the message arrives , Delays in processing
  3. Clock offset : Different physical hosts , Clock offset

ZK Carefully designed for Block the above 3 A common problem , Make these problems completely transparent at the application service level .

ZooKeeper characteristic

ZooKeeper The essence of the solution

Consistency of distributed systems :

  1. The messaging : Delay , Message sent first , You don't have to arrive first ;
  2. The messaging : Loss of sex , Message sent , May be lost ;
  3. Node crash : Within a distributed system , Any node can crash ;

under these circumstances , How to ensure data consistency ?

  1. Vote on the proposal : Based on voting strategy ,2PC
  2. The election vote : Based on voting strategy , Cast The highest priority node ( The node that contains the latest data )

Paxos The goal is : solve Distributed consistency problem , Improve distributed systems Fault tolerance The consistency algorithm .

Paxos The essence : be based on The messaging Of Highly fault-tolerant Of Consistency algorithm

ZooKeeper location

ZooKeeper yes :

  1. Distributed coordination services
  2. Efficient 、 reliable
  3. Convenient applications , focusing Business logic development , And you don't have to pay too much attention to Details of distributed inter process collaboration

ZooKeeper No direct exposure The original language , It is , Expose a part of Calling method Composed of API, File system like API, Support applications to implement their own The original language .

ZooKeeper characteristic

ZooKeeper The following distributed consistency features can be guaranteed :

  • Sequential consistency : The same Client Initiated transaction request , Strictly follow the order of initiation
  • Atomicity : Transaction request , Or apply to all nodes , Either one node has no application
  • A single view :Client No matter which node you connect to , The server-side data you see is consistent (Note: inaccurate , It's actually Final consistency
  • reliability : Once the transaction is successfully executed , The state remains permanently
  • The real time : Once the transaction is successfully executed ,Client You can't see the latest data immediately , but ZooKeeper Guarantee Final consistency

ZooKeeper Design objectives

ZooKeeper Committed to providing High performance High availability Sequential consistency The distributed coordination service , Guaranteed data Final consistency .

Goal one : High performance ( A simple data model )

  1. use A tree structure Organize data nodes ;
  2. Full data nodes , All stored in memory ;
  3. Follower and Observer Handle non transactional requests directly ;

Goal two : High availability ( Build clusters )

  1. More than half of the machines survive , The service will run normally
  2. Automatically Leader The election

Goal three : Sequential consistency ( The order of transaction operations )

  1. Each transaction request , Will be forwarded to Leader Handle
  2. Every business , The global allocation will be incremental id(zxid,64 position :epoch + Self increasing id)

Goal four : Final consistency

  1. By proposing a vote , Ensure the reliability of transaction commit
  2. Propose a way to vote , Only guarantee Client After receiving the transaction commit successfully , More than half of the nodes can see the latest data

ZooKeeper Before appearance

ZK Before appearance , There are two ways of distributed system , Achieve multi process collaboration :

  1. Distributed lock manager
  2. Distributed database

ZK More focused on process collaboration , It does not provide any lock interface and general storage data interface .( doubt :ZK It can also be provided , We don't need to use it )

application server , common 2 Needs :

  1. Master-Slave?Leader The election : Asked to provide Master Node election function
  2. Process response tracking ? Crash detection : Request to provide a trace of the process's survival state
  3. Distributed lock : Exclusive lock

ZK For the above 2 These strategies provide the basis API.

ZooKeeper Scenarios that are not applicable :

  1. Mass data storage :ZK The essence is special FS, but ZK Used to store Metadata , It needs to be stored separately Application data


Why don't I fully advocate self-study ?
Daniel on the platform basically has many years of working experience , Have you ever thought about the threshold of the industry before , What is the industry threshold now ? In the past, enterprises did not have such high requirements for programmers' ability , Even more than a decade ago, if you could write “Hello World”, You can get started in this industry , So you can get started before .
② Now there are some excellent young cattle , They may also be self-taught , But they must have excellent learning ability , Excellent self-management skills ( time management , Meditation, persistence, etc ) And be good at finding and summarizing problems .
If you think your goal is very clear , Can you do the first ② The points mentioned in point , In the current market , You are really suitable for self-study .

besides , For most people , Registering for a class must be the best way to grow quickly . But there's a problem , At present, the quality of training institutions in the market is uneven , If you don't find a good training class , It's a total waste of energy , Time and money , This needs to be selected by yourself .

I personally suggest that online is more cost-effective than offline , The price of offline training is basically unchanged 2W You can't get down , Online education is now more mature , During this outbreak , Students have basically experienced the online learning mode . Compared with offline , The advantages of online are mainly in the following aspects according to my understanding :
① Price : The online price is basically half of the offline price ;
② teacher : Relatively speaking, the teachers of online education are stronger and richer than offline education , Better coordination of resources ;
③ Time : Learning time is relatively free , Don't learn naked words , Suitable for learning while working , Reduce the stress of life ;
④ Course : In terms of course content , It really goes deeper than offline .

What technologies should be learned to meet the requirements of the enterprise ?( The following figure summarizes )

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java_03

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _ The backend development _04

This article has been  CODING Open source project :【 A big factory Java Analysis of interview questions + Core summary learning notes + The latest explanation video + Actual project source code 】 Included

本文为[Programmer Xiao Gang]所创,转载请带上原文链接,感谢

  1. L'apprentissage le plus détaillé de springboot sur le Web - day08
  2. Introduction à la page Web de rabbitmq (3)
  3. No Converter found for return value of type: class java.util.arraylist Error Problem
  4. (16) , spring cloud stream message driven
  5. Que faut - il apprendre de l'architecture des microservices Spring Cloud?
  6. Résolution: erreur: Java: distribution cible invalide: 11problème d'erreur
  7. Springboot démarre en une minute et sort de l'enfer de la configuration SSM!
  8. Maven - un outil de gestion essentiel pour les grands projets d'usine, de l'introduction à la maîtrise![️ Collection recommandée]
  9. ️ Push to interview in Large Factory ᥧ - - Spring Boot Automatic Assembly Principle
  10. [️ springboot Template Engine] - thymeleaf
  11. Springboot - MVC Automatic configuration Principle
  12. Mybatis reverse engineering and the use of new version mybatisplus 3.4 reverse engineering
  13. Base de données MySQL - transactions et index
  14. Sécurité du printemps - [authentification, autorisation, déconnexion et contrôle des droits]
  15. Moteur de base de données InnoDB diffère de myisam
  16. Swagger - [springboot Integrated Swagger, configure Swagger, configure scan Interface, configure API Group]
  17. Cadre de sécurité Shiro - [QUICKstart, login Block, User Authentication, request Authorization]
  18. [Introduction à Java] installation de l'environnement de développement - Introduction à Java et construction de l'environnement
  19. 【 linux】 notes d'utilisation tmux
  20. MySQL + mybatis paging query - database series learning notes
  21. Usage relations and differences of count (1), count (*) and count (a field) in MySQL
  22. 2021 Ali Java advanced interview questions sharing, Java Architect interview materials
  23. Mybatis - dynamic SQL statement - if usage - MySQL series learning notes
  24. [go to Dachang series] deeply understand the use of where 1 = 1 in MySQL
  25. [secret room escape game theme ranking list] Based on spring MVC + Spring + mybatis
  26. Redis log: the killer mace of fearless downtime and rapid recovery
  27. 5 minutes to build redis cluster mode and sentinel mode with docker
  28. Java小白入门200例106之遍历ArrayList的几种方式
  29. Java小白入门200例105之Java ArrayList类
  30. Java小白入门200例104之JDK自带记录日志类logging
  31. Practice of high availability architecture of Tongcheng travel network based on rocketmq
  32. Chapter 9 - Linux learning will - file archiving and compression tar --- zip
  33. Java小白入門200例104之JDK自帶記錄日志類logging
  34. JDK avec journalisation de classe dans 200 cas 104
  35. Java ArrayList Class for Introduction to Java LITTLE WHITE 200 example 105
  36. Plusieurs façons de traverser ArrayList à partir de 200 exemples 106
  37. Provectus / Kafka UI: open source Apache Kafka's Web GUI Graphical interface management tool
  38. Design pattern series: Singleton pattern
  39. Java小白入門200例105之Java ArrayList類
  40. Understanding Java record types
  41. Five load balancing algorithms implemented in Java
  42. Data structure must be an example to understand dynamic programming (with universal Python code)
  43. The idea and implementation of recursion in data structure (Python)
  44. The idea and implementation of linked list (Python)
  45. Data structure must be queue and double ended queue (Python)
  46. Idea and implementation of data structure must be able stack (Python)
  47. Data structure | time complexity (with video explanation)
  48. 20 flutter libraries you should know
  49. Case sharing: Online failure caused by Dubbo 2.7.12 bug
  50. Open source | didi open source, general functional components for Java authentication, authentication, management and task scheduling
  51. Flutter multi engine supports platformview and thread merging solution
  52. In depth understanding of netty: viewing netty traffic control from occasional downtime
  53. Spring AOP internal skill cultivation
  54. Interviewer: is Tomcat a symbolic parent delegation mechanism?
  55. Expérimentez la première tablette de consommation Linux. La puce et le système d'origine sont tous faits maison
  56. 2021 summary of the latest Java common open source libraries, Java interview handwritten code
  57. 2021 latest Java factory interview true questions, Kafka introduction video
  58. 01 javase - première connaissance de l'installation de Java et de l'environnement de développement
  59. The sales volume in September broke the record: Weilai and Xiaopeng both exceeded 10000, with an ideal month on month decrease of 24.7%
  60. Redis core principle and practice: implementation principle of hash type and dictionary structure