2021 Java back end engineer must know knowledge - (Dubbo, distributed RPC framework)

six miles holy sword 2021-04-08 03:24:06
java end engineer know knowledge


The text has been included in my GitHub Warehouse , welcome Star:github.com/bin39232820…
The best time to plant a tree is ten years ago , The second is now


Interview guide series , In many cases, I don't dig into the details , It's a way for xiaoliuliu to review knowledge in the role of interviewee , So I acquiesce to most things , You as an interviewer , I must understand .


Here's the brain map address


Dubbo What is the framework ? I think it's the distributed technology stack of at least half of the companies with small partners , So today, Xiao Liuliu will review it . There are many questions to ask , ha-ha .

Then the following is a summary of the previous articles

You used dubbo Well , You know what he's doing ?

Apache Dubbo It's a high performance 、 Lightweight open source Java Service Framework

Provides six core competencies : High performance for interface agents RPC call , Intelligent fault tolerance and load balancing , Services automatically register and discover , Highly scalable , Traffic scheduling during operation period , Visual service governance and operation and maintenance . Actually Dubbo It's a RPC frame .


What are you talking about RPC

RPC(Remote Procedure Call)— Remote procedure call , It is a way of requesting services over a network from a remote computer program , You don't need to know the protocols of the underlying network technology . Like two different services A、B Deployed on two different machines , So service A If you want to call a service B What to do with one of the methods in ? Use HTTP Of course, the request can , But it can be a bit more troublesome . RPC This is to make it as easy to call remote methods as local methods .

Actually, as far as I'm concerned RPC It's still very broad , Take a few simple production cases for example

  • First of all, I think it is rarely used by the company fegin Actually, it's a simple one RPC Well ? Because it also blocks the underlying calls , Let's call the remote method as easily as we call the local side .
  • The second is what we are talking about dubbo, Then he and fegin One of the biggest differences , I think the way of network transmission is different , But he's also a RPC frame , And better performance , But he doesn't currently support cross language calls .
  • Third thrift, Why does Xiao Liuliu mention it ? When a company is big , For example, a big Internet company , There must be different languages in different departments , Then you have to get on with him , We have this kind of service on our side ..

The young man can , Then you talk about it. RPC Why not


  • Service consumer (client) The invocation invokes the service as a local invocation ;
  • client stub Responsible for receiving the call 、 The parameters are assembled into a message body capable of network transmission ;
  • client stub Find the service address , And send the message to the server ;
  • server stub Decode the message upon receipt ;
  • server stub The local service is invoked based on the decoding results ;
  • The local service executes and returns the result to server stub;
  • server stub The returned result is packaged into a message and sent to the consumer ;
  • client stub Message received , And decode it ;
  • Service consumers get the end result .

Actually, it's quite simple , If you think about it, you'll know what it is , Hey . You can understand memory , Be sure to understand


Let's talk again Dubbo, You are right about Dubbo I'm familiar with the various modules of ? Do you know what each module does

  • config Configuration layer : External configuration interface , With ServiceConfig, ReferenceConfig Centered , You can initialize the configuration class directly , It can also be done through spring Parse configuration to generate configuration class
  • proxy Service agent layer : Service interface transparent agent , The client that generated the service Stub And the server side Skeleton, With ServiceProxy Centered , The extension interface is ProxyFactory
  • registry Registry level : Encapsulate service address registration and discovery , To serve URL Centered , The extension interface is RegistryFactory, Registry, RegistryService
  • cluster The routing layer : Encapsulate the routing and load balancing of multiple providers , And bridge the registry , With Invoker Centered , The extension interface is Cluster, Directory, Router, LoadBalance
  • monitor Monitoring layer :RPC Call times and call time monitoring , With Statistics Centered , The extension interface is MonitorFactory, Monitor, MonitorService
  • protocol Remote call layer : encapsulation RPC call , With Invocation, Result Centered , The extension interface is Protocol, Invoker, Exporter
  • exchange The information exchange layer : Encapsulate request response patterns , Synchronous to asynchronous , With Request, Response Centered , The extension interface is Exchanger, ExchangeChannel, ExchangeClient, ExchangeServer
  • transport Network transport layer : abstract mina and netty For a unified interface , With Message Centered , The extension interface is Channel, Transporter, Client, Server, Codec
  • serialize Data serialization layer : Some reusable tools , The extension interface is Serialization, ObjectInput, ObjectOutput, ThreadPool

say something Dubbo Let's have a better structure

Dubbo The architecture of


  • Provider: Service providers that expose services
  • Consumer: The service consumer that invokes the remote service
  • Registry: Service registry with discovery registry ( Generally used zk)
  • Monitor: The monitoring center that counts the number of service calls and call time
  • Container: Service run container (Spring The container of )

Call relation description :

  • The service container is responsible for starting , load , Run the service provider .
  • The service provider at startup , Register your services with the registry .
  • Service consumers at startup , Subscribe to the registry for the services you need .
  • The registry returns a list of service provider addresses to the consumer , If there are changes , The registry pushes the change data to the consumer based on the long connection .
  • Serving consumers , From the provider address list , Based on soft load balancing algorithm , Select a provider to invoke , If the call fails , Select another call .
  • Service consumers and providers , Accumulates the number of calls and the call time in memory , Send statistics to the monitoring center every minute on a regular basis .

Chat Dubbo Architecture design method of SPI

For a middleware design principle , Plug in design , Any function can be designed .

  • Protocol extension
  • Call the intercept extension
  • Reference listening extension
  • Exposure monitoring extension
  • Cluster expansion
  • Routing extension
  • Load balancing extension
  • Merge result extension
  • Registry extension
  • Monitoring center expansion
  • Extension point load extension
  • Dynamic agent extension
  • Compiler extensions
  • Dubbo Configuration center extension
  • Message distribution extension
  • Thread pool expansion
  • Serialization extension
  • Network transmission expansion
  • Information exchange extension
  • Networking expansion
  • Telnet Command extension
  • State check extension
  • Container extension
  • Cache extension
  • Validation extension
  • Log adaptation extension

In fact, xiaoliuliu thinks that this is a code design principle , But most of the time , We can do business in this way , High cohesion , Low coupling .

Nice guy , Then you talk about it. Dubbo What are the communication protocols of

dubbo Support different communication protocols

  • dubbo agreement

The default is to go dubbo agreement , Single long connection , What's going on is NIO asynchronous communication , be based on hessian As a serialization protocol . The scene used is : Small amount of data transmitted ( Every request is in 100kb within ), But the concurrency is high .

To support high concurrency scenarios , Generally, the service provider has only a few machines , But there are hundreds of service consumers , Maybe hundreds of millions of calls per day ! At this time, long connection is the most appropriate , That is to maintain a long connection with each service consumer , Maybe all in all 100 A connection . Based on our selector Pattern

  • rmi agreement

go Java Binary serialization , Multiple short connections , Suitable for the same number of consumers and providers , For file transfer , It's less commonly used .

  • hessian agreement

go hessian Serialization protocol , Multiple short connections , It is applicable to cases where the number of providers is more than that of consumers , For file transfer , It's less commonly used .

What are the serialization protocols

dubbo Support hession、Java Binary serialization 、json、SOAP Text serialization multiple serialization protocols ,probuffer. however hessian Is its default serialization protocol , among probuffer It's the fastest .

Why? probuffer The fastest

  • It USES proto compiler , Automatic serialization and deserialization , Very fast , Should compare XML and JSON It's almost there 20~100 times ;
  • Its data compression effect is good , That is to say, the volume of serialized data is small . Because it's small , The bandwidth and speed of transmission will be optimized .

say something Dubbo Load balancing strategy provided

  • Random LoadBalance( Default , Weight based random load balancing mechanism )-> Random , Set random probability by weight . The probability of collision on a section is high , But the larger the dosage, the more evenly distributed , And it's more even after using the weight according to the probability , It is beneficial to dynamically adjust the weight of providers .
  • RoundRobin LoadBalance->( Not recommended , Weight based polling load balancing mechanism )
  • LeastActive LoadBalance-> Minimum number of active calls , Random of the same active number , Active number refers to the difference between counting before and after the call . Make slow providers receive fewer requests , Because the slower the provider, the greater the difference between the count before and after the call .
  • ConsistentHash LoadBalance-> Uniformity Hash, Requests with the same parameters are always sent to the same provider .( If you don't need random load balancing , It requires that all requests of one type go to one node , Let's go for consistency hash Strategy .), When a provider hangs up , The request originally sent to the provider , Based on virtual nodes , Spread out to other providers , It doesn't cause dramatic changes .

Design your own RPC frame , What needs to be considered

ha-ha , In fact, this is also a consideration of the overall situation , It's still a big question , Before Xiao Liuliu wrote a simple rpc wheel , You can see that , It's basically to put duboo Each module in the package is simply implemented by itself . One Web The rear end frame wheels from the handle Http request 【 be based on Netty The request level of Web The server 】 To mvc【 Interface encapsulation forwarding )】, Until then ioc【 Dependency injection 】,aop【 section 】, Until then rpc【 Remote procedure call 】 The last to orm【 Database operation 】 All by themselves ( Simple and easy ) The wheel of .



Actually Dubbo A lot of things , But let's go deeper , Every point can follow the source code , There is a chance for Xiao Liuliu to follow . Actually I think Dubbo It's a great official document , We don't care if we use it or not dubbo Of , I suggest you read , There are many enlightening points for our development , Let me tell you something about it .

Ask for praise everyday

All right, everyone , The above is the whole content of this article , You can see the people here , All are Real powder .

It's not easy to create , Your support and recognition , It's the biggest driving force of my creation , See you in the next article

WeChat search " The procedural life of six pulse sword " reply 888 There are a lot of information I've found for you

本文为[six miles holy sword]所创,转载请带上原文链接,感谢

  1. Java高并发测试框架JCStress
  2. 12000 words, thoroughly explain the optimization principle of MySQL, collect it before you finish
  3. R2dbc, spring data JDBC and Webflux case introduction
  4. Uncle Bob: comparison of class and data structure! Problems that every good software designer and architect should keep in mind
  5. Humor: is functional programming just a few people's pastime? - Mike Hadlow
  6. Java high concurrency testing framework jcstress
  7. Java高并发测试框架JCStress
  8. 基于Spring Boot 2.3以上版本方便创建分层Docker镜像
  9. 使用SpringBoot实现在运行时动态创建Spring Bean
  10. Java high concurrency testing framework jcstress
  11. It is convenient to create hierarchical docker image based on spring boot version 2.3 or above
  12. Using springboot to create spring bean dynamically at runtime
  13. The constructor StringBuilder(String) is undefined 解决方法
  14. The constructor StringBuilder (string) is undefined solution
  15. SpringMVC 九大组件之 ViewResolver 深入分析
  16. SpringMVC 九大组件之 HandlerAdapter 深入分析
  17. SpringMVC 九大组件之 HandlerMapping 深入分析
  18. Spring Boot 定义接口的方法是否可以声明为 private?
  19. SpringMVC 源码分析之 DispatcherServlet
  20. SpringMVC 源码分析之 FrameworkServlet
  21. Spring Boot 中如何统一 API 接口响应格式?
  22. 深入分析 SpringMVC 参数解析器
  23. SpringBoot 中如何自定义参数解析器?
  24. SpringMVC 初始化流程分析
  25. 如何优雅的实现 Spring Boot 接口参数加密解密?
  26. Spring5 里边的新玩法!这种 URL 请求让我涨见识了!
  27. SpringMVC 中的参数还能这么传递?涨姿势了!
  28. 手把手教你开发 MyBatis 插件
  29. 细品 Spring Boot+Thymeleaf,还有这么多好玩的细节!
  30. Spring Boot 日志各种使用姿势,是时候捋清楚了!
  31. Deep analysis of viewresolver of nine components of spring MVC
  32. In depth analysis of handleradapter of nine components of spring MVC
  33. In depth analysis of handlermapping of nine components of spring MVC
  34. Can spring boot define interface methods to be declared private?
  35. Dispatcherservlet of spring MVC source code analysis
  36. Framework servlet for spring MVC source code analysis
  37. How to unify API response format in spring boot?
  38. In depth analysis of spring MVC parameter parser
  39. How to customize parameter parser in springboot?
  40. Analysis of spring MVC initialization process
  41. How to realize encryption and decryption of spring boot interface parameters gracefully?
  42. A new way to play in spring 5! This kind of URL request makes me see better!
  43. Can parameters in spring MVC be passed like this? It's up!
  44. Hand in hand to teach you how to develop mybatis plug-ins
  45. Fine spring boot + thymeleaf, there are so many fun details!
  46. Spring boot logs all kinds of posture, it's time to clear!
  47. Web 3.0踏浪而来,分布式存储举足轻重|时空云邀请您参加Web3.0中国峰会暨分布式存储行业大会
  48. spring-aop 进不了切面方法的解决办法
  49. Web 3.0 is coming, distributed storage is very important | spatiotemporal cloud invites you to attend Web3.0 China Summit and distributed storage industry conference
  50. The solution of spring AOP can't enter the section method
  51. Linux中如何启用root用户
  52. How to enable root in Linux
  53. 踩坑 MySQL 索引,看看你真的会用吗?
  54. Hive优化之配置参数的优化(一)
  55. Step on the MySQL index to see if you really know how to use it?
  56. Optimization of configuration parameters for hive optimization (1)
  57. Linux入门教程资料分享
  58. Introduction to Linux
  59. 外部连接mysql docker容器异常
  60. Exception of external connection MySQL docker container