Hello everyone , I'm Xiaoyu

What I bring you today is about the rabbit RabbitMQ How to cultivate students ,RabbitMQ Medium Rabbit yes The rabbit It means , Running as fast as a rabbit . It's a lightweight model , High availability that supports multiple messaging protocols Message queue .RabbitMQ By Erlang language-written , and Erlang Language is a natural fit for High concurrency Language .

Do they all like rabbits very much , The lovely rabbit plays the role of a big brother in our work , It's not too much to say that it's Ali's favorite now .RabbitMQ I have introduced the rabbit in front , that MQ What does it mean ? In fact, everyone knows ,Message Queue Abbreviation , In official terms RabbitMQ Is a Open source message queuing system . Let's follow Xiaoyu to see how this rabbit is cultivated .

Preface

Now the mainstream in the market MQ There's a lot of , such as ActiveMQ、RabbitMQ、RocketMQ、Kafka、ZeroMQ etc. .

Alibaba also used ActiveMQ , But as the business continues to grow ,ActiveMQ IO There is a bottleneck in the module , Later, Alibaba through a series of optimization, but still can't solve it very well , After that, Alibaba focused on the mainstream message middleware kafka above , however kafka It doesn't meet their demands , In especial Low latency and high reliability .

therefore RocketMQ It's standing on the shoulders of giants (kafka), It has also been optimized to meet the characteristics of Internet companies .

RabbitMQ As a very popular message middleware , It has very rich features and advantages : high reliability 、 Flexible routing 、 Cluster expansion is high 、 High availability 、 Support multiple protocols 、 Support a variety of clients and have a rich plug-in system .

RocketMQ At present, it is widely used in Alibaba group transaction 、 Recharge 、 Flow calculation 、 Message push 、 Log streaming 、binglog distribution Such as the scene .


Concept

RabbitMQ It's a by Erlang Language development AMQP Open source implementation .

AMQP :Advanced Message Queue, Advanced message queue protocol . It is an open standard of application layer protocol , Designed for message-oriented middleware , The client and message middleware based on this protocol can deliver messages , Non-subject product 、 Restrictions on conditions such as development language .

RabbitMQ It originated in the financial system , Used to store forward messages in a distributed system , stay Ease of use 、 Extensibility 、 High availability And so on .

Use scenarios

The application of decoupling

Take what we often say Order system For example , Generally, the order system calls the interface of the inventory system . as follows :

This kind of solution will cause a lot of problems , When the inventory system is not accessible , The inventory reduction of our order system will fail .

When we use it Message queue after , After the user orders , The order system performs Persistent processing , Write message to message queue , Return to user's order successfully placed ; Subscribe to the message of the order , Use pull / Push way , Get order information , Inventory system according to order information , Inventory operation .

In this way, even if there is a problem in the inventory system when placing an order , It does not affect normal order , Because after placing the order , The order system has written to the message queue , I don't care about other operations , Realize the integration of order system and inventory system The application of decoupling .

Asynchronous processing

Take our User registration For example , Users need to send registration email and SMS after registration . as follows :

Serial mode


Parallel mode

When we use message queuing , It's no longer a necessary business logic , Just do it Asynchronous processing . as follows :

Traffic peak clipping

Take what we often mention seckill For example , Everyone must have experienced the double experience of tmall Taobao 11 Well , When this time , We'll be at that particular time , Like in the early hours of the night 0 The demand per second of the point will suddenly increase , If you don't upgrade the system structure , I can't stand so many requests , It's going to crash the system .

When we use message queuing , In this Fastigium When , A lot of users come in , For example, there are 5 Thousands of requests , We just need to put this 5 Thousands of requests in the message queue , The system processes per second 2 Thousands of requests , The corresponding number will be pulled from the message queue , It only processes this per second 2 Thousands of requests . So during the duration of the second kill , There will be hundreds of thousands or more Requests are put on the message queue in . Because after all, second kill will only be in a short period of time , When it's over , Maybe only a few dozen per second , Hundreds of requests enter the message queue . But the system will continue to run at the rate of per second 2 Thousands of requests to process . therefore , End of seckill , The system will take those The rest of the news is consumed .

The main features

reliability (Reliability):RabbitMQ Use mechanisms to ensure reliability , Such as Persistence 、 Transfer confirmation 、 Release confirmation .

Flexible routing (Flexible Routing): Before the message is queued , adopt Exchange To route a message . For typical routing functions ,RabbitMQ Some have been provided built-in Of
Exchange To achieve . For more complex routing functions , Can be Multiple Exchange Bind together , Also through Plug-in mechanism Realize one's own Exchange .

Message cluster (Clustering): Multiple RabbitMQ Servers can form a cluster , Form a logic Broker .

High availability (Highly Available Queues): Queues can be done on machines in the cluster Mirror image , Makes the queue available in the event of a partial node failure .

A variety of protocols (Multi-protocol):RabbitMQ Multiple message queue protocols are supported , such as STOMP、MQTT wait .

Multilingual client (Many Clients):RabbitMQ Support for almost any common language , such as Java、.NET、Ruby wait .

Management interface (Management UI):RabbitMQ Provides an easy-to-use user interface , Enables users to Monitoring and management news Broker Many aspects of .

Tracking mechanism (Tracing): If the message is abnormal ,RabbitMQ Provides Message tracking mechanism , The user can find out what happened .

Plug-in mechanism (Plugin System):RabbitMQ Provides a lot plug-in unit , To expand in many ways , You can also write your own plug-ins .

Architectural model

Message

news , The news is anonymous , It consists of Headers and bodies form . The message body is opaque , The message header consists of a series of optional properties , These attributes include routing-key( Routing key )、priority( Priority over other messages )、delivery-mode( Point out that the message may need persistent storage ) etc. .

Publisher

Message producer , Also a Post a message to the switch Client applications for .

Exchange

exchanger , be used for receive The producer sends messages and routes them to queues in the server .

Binding

binding , Used between message queues and switches relation . A binding is a routing rule that connects a switch to a message queue based on a routing key , So you can think of a switch as a binding Routing table .

Queue

queue , yes RabbitMQ Internal objects of , Used to store messages Containers , It's also the end of the news . A message can be put into One or more queue . The news has been in the queue , Wait for the consumer to connect to the queue and take it away .

Connection

network connections , For example, a TCP Connect .

Channel

channel , A separate... In a multiplexed connection Bidirectional data flow passageway . The channel is built on the real TCP Connect mainland virtual connection ,AMQP Orders are sent by channel , Whether it's publishing news 、 Subscribe to the queue or receive messages , These actions are all done through channels . Because for the operating system to build and destroy TCP It's all very expensive , So the concept of channel is introduced , With Reuse One TCP Connect .

Consumer

Consumers of messages , Represents a... That gets a message from a message queue Client applications .

Virtual Host

Virtual host , A batch of switches 、 Message queues and related objects . The virtual host is share The same authentication and encryption environment Independent server domain .

Broker

Represents a message queuing server Entity .

Exchange type

Exchange There are different distribution strategies according to different types of messages , There are currently four types :direct、fanout、topic、headers .headers matching AMQP News header Instead of routing keys , Besides headers Switches and direct exchanger Exactly the same , But the performance is very poor , At present, it is almost useless , So look directly at the other three types :

Direct Bond distribution

Direct: In the news Routing key (routing key) If and Binding Medium binding key Agreement , The switch sends the message to the corresponding queue . It is perfect match 、 unicast The pattern of .

We use routingKey=”error” Send a message to Exchange , The message will be routed to Queue1(amqp.gen-S9b…, This is from RabbitMQ Automatic generation Of Queue name ) and Queue2(amqp.gen-Agl…); If we routingKey=”info” or routingKey=”warning” To send messages , Then the message will only be routed to Queue2. If we take other routingKey Send a message , Then messages will not be routed to these two Queue in .

Fanout

Fanout: Each sent to fanout The message of type switch will To all bound queues Up . It's very much like a subnet broadcast , The host in each subnet gets a copy of the message .fanout The type of forwarding message is One of the fastest .

producer Publisher Send to Exchange All the news will be route Go to the two in the picture Queue , And finally by two consumers (Consumer1 And Comsumer2) consumption .

topic exchanger

topic exchanger :topic The exchanger passes through Pattern matching Assign the routing key property of the message , Match the routing key to a pattern , At this point, the queue needs binding To a pattern . It will route key and bind key strings segmentation It's a word , These words are separated by dots . It also recognizes two wildcards : Symbol “#” And symbols “” .# matching 0 One or more words , It doesn't match more than one word .

We routingKey=”quick.orange.rabbit” The news will be Simultaneous routing To Q1 And Q2,routingKey=”lazy.orange.fox” Will be routed to Q1,routingKey=”lazy.brown.fox” Will be routed to Q2,routingKey=”lazy.pink.rabbit” Will be routed to Q2( It will only be delivered to Q2 once , Although the routingKey And Q2 Of the two bindingKey all matching );routingKey=”quick.brown.fox”、routingKey=”orange”、routingKey=”quick.orange.male.rabbit” Will be discarded , Because they don't match anything bindingKey .

Installation steps

Generally speaking, installation RabbitMQ Before you install Erlang , You can go to Erlang Download from the official website . Then go RabbitMQ Download the installation package on the official website , Then decompress it . Corresponding installation instructions are provided according to different official websites of the operating system :Windows、Debian / Ubuntu、RPM-based Linux、Mac

Download address

https://www.rabbitmq.com/releases/rabbitmq-server/

download

wget https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.15/rabbitmq-server-generic-unix-3.6.15.tar.xz

install

tar -xvf rabbitmq-server-generic-unix-3.6.15.tar.xz

To configure

vim /etc/profile
Then add  export PATH=$PATH:/opt/rabbitmq/rabbitmq_server-3.6.15/sbin  According to their own installation path to 

Make changes effective

source /etc/profile

modify hosts

vim/etc/hosts

start-up

cd /opt/rabbitmq/rabbitmq_server-3.6.15/sbin/
./rabbitmq-server -detached  Pay attention to root User start 

Check to see if startup succeeded

./rabbitmqctl status

This is a success , If you want remote access

./rabbitmq-plugins enable rabbitmq_management
Then you can access it ( Please close the firewall for non local access )

visit

Springboot Project presentations

springboot Integrate RabbitMQ It's simple , If it's just a simple configuration, it's very few , springboot Provides spring-boot-starter-amqp All kinds of support for messages .

To configure pom file , Mainly to add spring-boot-starter-amqp Of Support

Code demonstration

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

To configure application.properties file , To configure rabbitmq Of Installation address 、 Port and account information

Code demonstration

spring.application.name=spirng-boot-rabbitmq
spring.rabbitmq.host=127.0.0.1
spring.rabbitmq.port=5672
spring.rabbitmq.username=admin
spring.rabbitmq.password=admin

Configure queues

Code demonstration

package com.zpc.rabbitmq
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration
@Configuration
public class RabbitConfig {
    @Bean
    public Queue queue() {
        return new Queue("q_hello");
    }
}

sender

Code demonstration

package com.zpc.rabbitmq
import org.springframework.amqp.core.AmqpTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component
import java.text.SimpleDateFormat;
import java.util.Date
@Component
public class HelloSender {
    @Autowired
    private AmqpTemplate rabbitTemplate
    public void send() {
        String date = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());//24 hourly
        String context = "hello " + date;
        System.out.println("Sender : " + context);
        // In the case of simple column alignment routingKey That is to say Q name
        this.rabbitTemplate.convertAndSend("q_hello", context);
    }
}

The receiver

Code demonstration

package com.zpc.rabbitmq
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component
@Component
@RabbitListener(queues = "q_hello")
public class HelloReceiver 
    @RabbitHandler
    public void process(String hello) {
        System.out.println("Receiver  : " + hello);
    }
}

test

Code demonstration

package com.zpc.rabbitmq
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner
@RunWith(SpringRunner.class)
@SpringBootTest
public class RabbitMqHelloTest 
    @Autowired
    private HelloSender helloSender
    @Test
    public void hello() throws Exception {
        helloSender.send();
    }
}

Built in clusters

RabbitMQ One of the best features is Built in clusters , This function is designed to allow consumers and producers to Continue to run in case of node crash , And by adding more nodes Linear expansion of message communication throughput .RabbitMQ Internal use Erlang The distributed communication framework provided OTP To meet these needs , Make the client lose one RabbitMQ In the case of node connection , Or be able to reconnect to any other node in the cluster Continue to produce 、 News consumption .

RabbitMQ In the cluster Common concepts

Queue metadata : Include Queue name And their attribute , For example, whether it can be persistent , Whether to delete automatically

Exchange metadata : exchanger name type attribute

Binding metadata : Inside is a picture of form Record how messages are routed to queues

vhost Metadata : by vhost Internal queues 、 exchanger 、 Binding provides Namespace and security properties

Technology selection

In our usual development of various message queues MQ Thinking about selection

If the number of user visits is ActiveMQ Within the affordability of , And it's really mainly based on Decoupling and asynchronous To use , You can consider ActiveMQ , It's also close to Java The habits of Engineers , however ActiveMQ Now the maintenance is stopped , meanwhile ActiveMQ Low concurrency , therefore The volume of business must be We can consider using .

RabbitMQ As a pure pedigree of message middleware , with Advanced message protocol AMQP The perfect combination of , There is no substitute in message middleware , however erlang Language prevents us from further study and control , For the company , The underlying technology can't control , But it is Open source Of , There is a comparison Stable Support for , It's also very active .

Have absolute confidence in oneself company technical strength , It can be used RocketMQ , however RocketMQ It was born later , also Update iterations are fast , This means that many pits may be encountered in the process of use , So if your company Java The technology is not very strong , It is not recommended to use .

Small and medium-sized companies , The technical strength is relatively average , The technical challenges are not particularly high , use ActiveMQ、RabbitMQ It's a good choice ; Big companies , Infrastructure research and development strength is strong , use RocketMQ It's a good choice

If it is Big data In real time 、 Log collection and other scenarios , use Kafka It's industry standard , Absolutely no problem , The community is very active , It's a de facto norm in this field almost all over the world .

from performance Come up and see , Message middleware using file system (Kafka、RokcetMq) Performance is the best , So message middleware based on file system storage is the development trend .( from Storage and efficiency Look at the file system >KV Storage > Relational database )

Last

Message queues , In fact, they are very similar , The usage is basically the same , It's just for each open source message queue Focus on not quite the same , We should be based on our own Project requirements To decide which message queue we should choose to serve our project , This project selection work is usually in the early stage of R & D , group leader I've decided which one to use , It's usually not our turn to do it , But the interview may examine the relevant knowledge , So we should check these message queues Understand and be flexible to use .


Recommended reading

I told my girlfriend about encryption algorithm over the weekend , little does one think …
Graphic, :Kafka What are the secrets that make me love it ?
Graphic, : How to explain to your girlfriend what micro service is ?
dried food !MySQL Optimization principle analysis and optimization scheme summary