java Development RocketMQ Message Middleware Principle Basic Explanation

What is RocketMQ

An introduction to RocketMQ on Github:
RcoketMQ is a low latency, highly reliable, scalable, easy-to-use messaging middleware. Has the following characteristics:

Support publish/subscribe (Pub/Sub) and point-to-point (P2P) message models Reliable first-in, first-out (FIFO) and strict sequential delivery in 1 queue Support two message modes: pull (pull) and push (push) Stacking capacity of millions of messages in a single 1 queue Support multiple message protocols, such as JMS, MQTT, etc Distributed highly available deployment architecture satisfying at least one messaging semantics Provide docker images for isolation testing and cluster deployment Provide Dashboard with rich functions such as configuration, indicators and monitoring

For the description of these features, everyone can simply look at them, and they will naturally understand after in-depth study.

Technical terminology

Producer

Message producer, the producer's role is to send messages to MQ, and the producer itself can generate messages, such as reading text messages. It can also provide an interface to the outside, and the external application calls the interface, and then the producer sends the received message to MQ.

Producer Group

Producer group, simply speaking, is a producer group that sends multiple producers with the same type of messages. There is no need to care here, as long as you know that there is such a concept.

Consumer

Message consumer, simply speaking, the application that consumes messages on MQ is the consumer. Whether the messages are processed logically or stored directly in the database depends on business needs.

Consumer Group

A consumer group, similar to a producer, is composed of a plurality of consumer instances that consume the same type of message.

Topic

Topic is a logical classification of messages. For example, if you have order messages and inventory messages, you need to classify them. One is order Topic to store order-related messages, and the other is inventory Topic to store inventory-related messages.

Message

Message is the carrier of the message. 1 Message must specify topic, which is equivalent to the address where the letter is sent. Message also has an optional tag setting so that consumers can filter messages based on tag. Additional key-value pairs can also be added. For example, you need a business key to find messages on broker, which is convenient to diagnose problems during development.

Tag

Tag can be considered as a step further refinement of Topic. 1 in the same business module by introducing tags to mark messages for different purposes.

Broker

Broker is the main role of RocketMQ system, which is actually MQ mentioned in the previous 1. Broker receives messages from producers, stores and prepares consumers for requests to pull messages.

Name Server

Name Server provides routing information for producer and consumer.

RocketMQ Architecture

There are four clusters, namely NameServer cluster, Broker cluster, Producer cluster and Consumer cluster:

NameServer Provides lightweight service discovery and routing. Each NameServer records complete routing information, provides equivalent read and write services, and supports rapid storage expansion.

Broker Handles message storage by providing lightweight Topic and Queue mechanisms, while supporting push (push) and pull (pull) modes and fault-tolerant mechanisms of master-slave architecture.

Producer Producers, instances that generate messages, and Producer with the same Producer Group form a cluster.

Consumer The consumer, the instance that receives the message for consumption, has the same Consumer Group

Consumer form a cluster.

Briefly explain the meaning of arrows in the figure below. Starting from Broker, Broker Master1 and Broker Slave1 are master-slave structures, and data synchronization will be carried out between them, that is, Date Sync. At the same time, each Broker and
All sections in an NameServer cluster
Point to establish a long connection, and regularly register Topic information to all NameServer.

The Producer establishes a long connection with one of the nodes in the NameServer cluster (randomly selected), periodically acquires the Topic routing information from the NameServer, establishes a long connection to the Broker Master providing Topic services, and periodically sends a heartbeat to the Broker. Producer can only send messages to Broker master, but Consumer is not the same as Master and Slave that provide Topic services
Establish a long connection that can subscribe to messages either from Broker Master or from Broker Slave.

RocketMQ Cluster Deployment Mode

Single master mode

That is to say, there is only one master node, which is not a cluster. Once this master node goes down, the whole service will not be available, which is suitable for personal learning.

Multi-master mode

Multiple master nodes form a cluster, and the downtime or restart of a single master node has no impact on the application.

Advantages: Highest performance of all modes
Disadvantages: During the downtime of a single master node, unconsumed messages are not available until the node is restored, and the real-time performance of messages is affected.

Note: Using synchronous brush disk can ensure that messages are not lost. At the same time, queue corresponding to Topic should be distributed in each node in the cluster, not only on a certain node. Otherwise, the downtime of this node will affect the application subscribing to this topic.

Multi-master Multi-slave asynchronous replication mode

On the basis of multi-master mode, each master node has at least one corresponding slave. master
Nodes can be read and written, but slave can only read and not write, similar to mysql's active and standby mode.

Advantages: When master goes down, consumers can read messages from slave, the real-time performance of messages will not be affected, and the performance is almost the same as that of multi-master 1.
Disadvantages: The synchronous method using asynchronous replication may have the problem of message loss.

Multiple master Multiple slave Synchronous Dual Write Mode

Similar to the multi-master multi-slave asynchronous replication pattern, the difference is the way data is synchronized between master and slave.

Advantages: The synchronous mode of synchronous double writing can ensure that data is not lost.
Disadvantages: Sending a single message RT is slightly longer, and its performance is about 10% lower than asynchronous replication.

Brush disk strategy: synchronous brush disk and asynchronous brush disk (referring to whether the node's own data is stored synchronously or asynchronously)

Synchronous mode: Synchronous double write and asynchronous copy (referring to the synchronization of data between 1 set of master and slave)

Note: To ensure data reliability, synchronous brush disk and synchronous double write are needed, but the performance will be lower than other methods.

The above is java development RocketMQ message middleware basic details, more information about RocketMQ message middleware please pay attention to other related articles on this site!