2011 At the beginning of year , LinkedIn, USA （LinkedIn) Open source an infrastructure software , Take Franz, the Austrian writer · kafka (Franz Kafka） Name of , after LinkedIn Contribute it to Apache The foundation , The software was then used in 2012 year 10 Successful completion of incubation and promotion to Apache Top projects —— This is the famous ApacheKafka. after 7 Year development ,2017 year 11 month ,Apache Kafka Officially evolved to 1.0 Time , This book is based on 1.0.0 Version to introduce Kafka Design principle and actual combat .
With the rapid development of big data era ,Kafka with Its high throughput and low latency 、 High pressure contraction 、 persistence 、 reliability 、 Fault tolerance and high concurrency The advantages of , It's solved “ Accurate collection and analysis of huge data ” The problem of , It is also favored by many large factories and engineers ,
But most people , In the process of study and interview, I found that I didn't fully grasp the meaning of it , Still have a little knowledge of common problems , this The main problem is that there are problems in the process of reading the source code
（1） Too many source files , I don't know the point ;
（2） Too much source code , Countless times from the beginning to giving up ;
（3） Wrong way , encounter “ Technicality ” The code is covered ;
（4） Not systematic enough , Will not classify the source file to read .
Let's introduce a copy of kafka Source code analysis notes .
The source code parsing
Due to the space limitation of this book , There is no detailed introduction to Kafka All the basic knowledge involved in the source code , for example Java NIO、JUC The use of tool classes in packages 、 The use of command line parameter parser, etc , For the convenience of readers , The author only introduces some necessary and important basic knowledge . Before we start source analysis , We hope that the readers can follow the following rules 1 The introduction of chapter is completed Kafka The construction of source code environment , And understand Kafka Core concept of , This can also have better learning effect .
The book consists of five chapters , They don't have a strong connection with each other , Readers can read from scratch , You can also choose the chapters you are interested in .
The first 1 Chapter is Kafka A quick start to , It introduces Kafka The background of 、 Features and application scenarios . After that, the author introduces a new method in practice Kafka The central case , And analyzes the choice of using in this case Kafka The specific reasons and Kafka Play a key role in . Finally, it introduces Kafka The core concepts and Kafka Source . Construction of code debugging environment .
The first 2 The design features and implementation details of producer client are introduced , Dissected KafkaProducer Intercept messages 、 Serialize message 、 Routing message and other functions of the source code implementation , It introduces RecordAccumulator The structure and implementation of . At last, it analyzes KafkaProducer in Sender Source code of thread .
The first 3 Describes the Kafka It also gives some practical suggestions , It also introduces ConsumerGroup Rebalance The principles of the various versions and their operation . Finally, it analyzes in detail KafkaConsumer Operation principle and implementation details of related components .
The first 4 Chapter introduces the composition of Kafka The components on the server side , In order to analyze Kafka The network layer 、API layer 、 The logging stored 、DelayedOperationPurgatory Components 、Kafka Copy mechanism of 、KafkaController、 GroupCoordinator 、Kafka Identity authentication and authority control as well as Kafka Monitoring related implementations . This chapter is Kafka The core content , It involves more design details and programming skills , I hope the readers can get something after reading .
The first 5 Describes the Kafka Provides the use of a number of scripting tools and the specific implementation principle , Understanding these scripts can help managers quickly complete some common management 、 Operation and maintenance 、 Testing capabilities .
Show in detail
kafka actual combat
This book covers Apache Kafka Reference books and reference books with practical guiding significance . The author combines typical usage scenarios , Yes Kafka The whole technical system is explained in a more comprehensive way , So that readers can draw inferences from one instance , Apply it directly to practice . meanwhile , This book is also about Kafka In this paper, the design principle and flow processing component of the system are discussed , And give a detailed case .
This book is divided into 10 Chapter : The first 1 Zhang introduces the message engine system and Kaka The basic concepts and characteristics of , Quickly lead readers into Kafka The world of ; The first 2 Chapter gives a brief review of Apache Kafka The history of development ; The first 3 Chapter details Kafka The construction of cluster environment ; The first 4、5 Chapter discusses in depth Kalka How to use the client ; The first 6 Chapter takes the reader through Kaka Interior design principles ; The first 7-9 Chapter explains by example Kalka Cluster management 、 Monitoring and tuning ; The first 10 Describes the Kafka The newly introduced streaming component .
Adapt to the crowd
This book is suitable for all cloud computing 、 Big data processing technology personnel interested in reading , Especially suitable for message engine 、 Stream processing technology and framework for technical personnel interested in reading .
Kafka Online environment deployment
producer Development \consumer Development
Kafka Design principle 、 management Kafka colony 、 monitor Kafka colony 、 tuning Kafka colony
Kafka Connect And Kafka Streams
You need these two kafka Learning documents , Official account ：Java Architects Alliance , The background to reply Java You can see how to get it