Analysis of new features of redis6.0 shared by dewu Technology

Acquisition technology 2021-01-22 17:34:14
analysis new features redis6.0 redis


Preface

In order to improve the network before and after executing the command I/O performance ,Redis6.0 Introduced Threaded I/O.

Let's learn about the new features Threaded I/O.

Reading order of this article

  • Redis How it works
  • What happened before and after the command was executed
  • Threaded I/O Model
  • Threaded I/O The realization of the system depends on the mechanism
  • Summary and reflection
  • Redis Source code learning ideas

Redis How it works

Loop through events

Redis The function entry of is in server.c in ,main() The method flow is shown in the figure below

stay main() In the method Redis The first thing to do is Initialize various libraries and service configurations . Specific examples :

  • crc64_init() Will initialize a crc For verification Lookup Table
  • getRandomBytes() by hashseed Fill in random elements as initialization values , Used as a hash table seed
  • initServerConfig() A lot of work has been done on server Object property initialization operation :
  • initialization server.runid, Such as 29e05f486b8d41e68234a68c8b77edaff101c194
  • Get the current time zone information , Store in server.timezone in
  • initialization server.next_client_id value , Make the connected client id from 1 Start to grow
  • ACLInit() It's right Redis 6.0 Newly added ACL Initialization of the system , Including initializing the user list 、ACL journal 、 Default user and other information
  • adopt moduleInitModulesSystem() and tlsInit() Initialize the module system and SSL etc.

After initialization , Start Read the user's startup parameters , Similar to most configuration loading processes ,Redis Also through string matching and other analysis of user input argc and argv[], In the process :

  • Get the configuration file path , modify server.configfile Value , Later used to load the configuration file
  • Get the startup option parameter , Such as loadmodule And corresponding Module File path , Save to options variable

After parsing the parameters , perform loadServerConfig(), Read the configuration file and compare with the command line parameters options To merge the contents of , Form a config Variable , And one by one name and value Set in configs In the list . For each config, There are corresponding switch-case Code for .

For example, for loadmodule, Will execute queueLoadModule() Method , To complete the real configuration load :

...
} else if (!strcasecmp(argv[0],"logfile") && argc == 2) {
...
} else if (!strcasecmp(argv[0],"loadmodule") && argc >= 2) {
queueLoadModule(argv[1],&argv[2],argc-2);
} else if (!strcasecmp(argv[0],"sentinel")) {
...

go back to main Method flow ,Redis Will start printing the startup log , perform initServer() Method , Services are based on configuration items , continue by server Object initialization content , for example :

  • Create the event loop structure aeEventLoop( It's defined in ae.h), Assign a value to server.el
  • configurable db number , The allocation size is sizeof(redisDb) * dbnum Of memory space ,server.db Save the address pointer of this space
  • Every db It's all one redisDb structure , Save this structure in key、 The dictionary for saving expiration time, etc. is initialized as empty dict

Then there are some initializations based on different running modes , For example, regular logs are recorded when running in regular mode 、 Load disk persistent data ; And in the sentinel The sentinel log is recorded when the mode is running , Don't load data, etc .

After all the preparatory operations have been completed ,Redis Start falling into aeMain() The event loop of , In this loop, it will be executed continuously aeProcessEvents() Deal with all kinds of events that happen , until Redis End exit .

Event type

Redis There are two types of events in :

  • Time event
  • Document events

Time event is the event that will happen at a certain time , stay Redis They are recorded in a linked list , Every time a new event is created , Will insert a aeTimeEvent node ( The first interpolation ), It stores when the event will occur , What kind of methods need to be called to handle .

Traversing the entire list, we can know how long it is before the most recent time event , Because the nodes in the linked list are self incrementing id Sequential arrangement , But in the dimension of occurrence time, it is out of order .

File events can be seen as I/O The events that caused , Sending a command from the client will cause the server to generate a read I/O, Corresponding to a read event ; Similarly, when the client is waiting for the server message, it needs to be writable , Let the server write content , So there will be a write event .AE_READABLE Event creates a connection on the client 、 Occurs when sending a command or other connection becomes readable , and AE_WRITABLE Events occur when the client connection becomes writable .

The structure of file events is much simpler ,aeFileEvent It records whether this is a readable event or a writable event , The corresponding processing method , And user data .

/* File event structure */
typedef struct aeFileEvent {
int mask; /* one of AE_(READABLE|WRITABLE|BARRIER) */
aeFileProc *rfileProc; /* Read event handling methods */
aeFileProc *wfileProc; /* Write event handling methods */
void *clientData;
} aeFileEvent;

If two things happen at the same time ,Redis Will give priority to AE_READABLE event .

aeProcessEvents

aeProcessEvents() Method to handle all kinds of events that have happened and are about to happen .

stay aeMain() Cycle into aeProcessEvents() after ,Redis First check when the next time event will happen , In the period of time when there is no time event , Can call multiplexed API aeApiPoll() Block and wait for file events to occur . If no file event occurs , Then return after timeout 0, Otherwise, return the number of file events that have occurred numevents.

When there are file events to handle ,Redis Would call AE_READABLE The event rfileProc Methods and AE_WRITABLE The event wfileProc Methods to deal with :

  • Usually, the read event is performed first , Then write the event
  • If set AE_BARRIER, Do the opposite , Reading events is always after writing events .
...
if (!invert && fe->mask & mask & AE_READABLE) {
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
fe = &eventLoop->events[fd];
}
if (fe->mask & mask & AE_WRITABLE) {
if (!fired || fe->wfileProc != fe->rfileProc) {
fe->wfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
if (invert) {
fe = &eventLoop->events[fd]; /* Refresh in case of resize. */
if ((fe->mask & mask & AE_READABLE) &&
(!fired || fe->wfileProc != fe->rfileProc))
{
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
...

After finishing the previous processing ,Redis Will continue to call processTimeEvents() Deal with time events . Traverse the entire time event list , If it's been a while ( Blocking waiting or processing file events takes time ), There are time events happening , Then call the corresponding time event timeProc Method , Get rid of all outdated time events :

...
if (te->when <= now) {
...
retval = te->timeProc(eventLoop, id, te->clientData);
...
processed++;
...
}
...

If the file event has not been executed to the latest time, the event occurrence point , So this time aeMain() No time event will be executed in the loop , Enter next cycle .

Attach source code

/* Process every pending time event, then every pending file event
* (that may be registered by time event callbacks just processed).
* Without special flags the function sleeps until some file event
* fires, or when the next time event occurs (if any).
*
* If flags is 0, the function does nothing and returns.
* if flags has AE_ALL_EVENTS set, all the kind of events are processed.
* if flags has AE_FILE_EVENTS set, file events are processed.
* if flags has AE_TIME_EVENTS set, time events are processed.
* if flags has AE_DONT_WAIT set the function returns ASAP until all
* the events that's possible to process without to wait are processed.
* if flags has AE_CALL_AFTER_SLEEP set, the aftersleep callback is called.
* if flags has AE_CALL_BEFORE_SLEEP set, the beforesleep callback is called.
*
* The function returns the number of events processed. */
int aeProcessEvents(aeEventLoop *eventLoop, int flags)
{
int processed = 0, numevents;
/* Nothing to do? return ASAP */
if (!(flags & AE_TIME_EVENTS) && !(flags & AE_FILE_EVENTS)) return 0;
/* Note that we want to call select() even if there are no
* file events to process as long as we want to process time
* events, in order to sleep until the next time event is ready
* to fire. */
if (eventLoop->maxfd != -1 ||
((flags & AE_TIME_EVENTS) && !(flags & AE_DONT_WAIT))) {
int j;
struct timeval tv, *tvp;
long msUntilTimer = -1;
if (flags & AE_TIME_EVENTS && !(flags & AE_DONT_WAIT))
msUntilTimer = msUntilEarliestTimer(eventLoop);
if (msUntilTimer >= 0) {
tv.tv_sec = msUntilTimer / 1000;
tv.tv_usec = (msUntilTimer % 1000) * 1000;
tvp = &tv;
} else {
/* If we have to check for events but need to return
* ASAP because of AE_DONT_WAIT we need to set the timeout
* to zero */
if (flags & AE_DONT_WAIT) {
tv.tv_sec = tv.tv_usec = 0;
tvp = &tv;
} else {
/* Otherwise we can block */
tvp = NULL; /* wait forever */
}
}
if (eventLoop->flags & AE_DONT_WAIT) {
tv.tv_sec = tv.tv_usec = 0;
tvp = &tv;
}
if (eventLoop->beforesleep != NULL && flags & AE_CALL_BEFORE_SLEEP)
eventLoop->beforesleep(eventLoop);
/* Call the multiplexing API, will return only on timeout or when
* some event fires. */
numevents = aeApiPoll(eventLoop, tvp);
/* After sleep callback. */
if (eventLoop->aftersleep != NULL && flags & AE_CALL_AFTER_SLEEP)
eventLoop->aftersleep(eventLoop);
for (j = 0; j < numevents; j++) {
aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
int mask = eventLoop->fired[j].mask;
int fd = eventLoop->fired[j].fd;
int fired = 0; /* Number of events fired for current fd. */
/* Normally we execute the readable event first, and the writable
* event later. This is useful as sometimes we may be able
* to serve the reply of a query immediately after processing the
* query.
*
* However if AE_BARRIER is set in the mask, our application is
* asking us to do the reverse: never fire the writable event
* after the readable. In such a case, we invert the calls.
* This is useful when, for instance, we want to do things
* in the beforeSleep() hook, like fsyncing a file to disk,
* before replying to a client. */
int invert = fe->mask & AE_BARRIER;
/* Note the "fe->mask & mask & ..." code: maybe an already
* processed event removed an element that fired and we still
* didn't processed, so we check if the event is still valid.
*
* Fire the readable event if the call sequence is not
* inverted. */
if (!invert && fe->mask & mask & AE_READABLE) {
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
fe = &eventLoop->events[fd]; /* Refresh in case of resize. */
}
/* Fire the writable event. */
if (fe->mask & mask & AE_WRITABLE) {
if (!fired || fe->wfileProc != fe->rfileProc) {
fe->wfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
/* If we have to invert the call, fire the readable event now
* after the writable one. */
if (invert) {
fe = &eventLoop->events[fd]; /* Refresh in case of resize. */
if ((fe->mask & mask & AE_READABLE) &&
(!fired || fe->wfileProc != fe->rfileProc))
{
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
processed++;
}
}
/* Check time events */
if (flags & AE_TIME_EVENTS)
processed += processTimeEvents(eventLoop);
return processed; /* return the number of processed file/time events */
}

What happened before and after the command was executed

On the client connection Redis When , Through execution connSetReadHandler(conn, readQueryFromClient), Set when a read event occurs , Use readQueryFromClient() As a way to read Events Handler.

When receiving a command request from the client ,Redis After some checks and Statistics , call read() Method to read the data in the connection into client.querybuf In the message buffer :

void readQueryFromClient(connection *conn) {
...
nread = connRead(c->conn, c->querybuf+qblen, readlen);
...
static inline int connRead(connection *conn, void *buf, size_t buf_len) {
return conn->type->read(conn, buf, buf_len);
}
static int connSocketRead(connection *conn, void *buf, size_t buf_len) {
int ret = read(conn->fd, buf, buf_len);
...
}

Then enter processInputBuffer(c) Start reading message in input buffer , The last to enter processCommand(c) Start processing the input command .

After the result of command execution , First of all, it will be stored in client.buf in , And call addReply(client c, robj obj) Method , Put this client Object is appended to server.clients_pending_write In the list . At this time, the current order , Or say AE_READABLE The incident has been basically handled , Except for some extra Statistics 、 Beyond post-processing , The action of sending a response message will no longer take place .

At present aeProcessEvents() After the method is over , Get into The next cycle , The second loop call I/O The multiplexer interface waits for the file event to occur ,Redis Will check the server.clients_pending_write Whether a client needs to reply , If you have any , Traversal points to each to reply to the client server.clients_pending_write list , adopt listDelNode() Remove clients one by one , And the content to be replied through writeToClient(c,0) Reply out .

int writeToClient(client *c, int handler_installed) {
...
nwritten = connWrite(c->conn,c->buf+c->sentlen,c->bufpos-c->sentlen);
...
static inline int connWrite(connection *conn, const void *data, size_t data_len) {
return conn->type->write(conn, data, data_len);
}
static int connSocketWrite(connection *conn, const void *data, size_t data_len) {
int ret = write(conn->fd, data, data_len);
...
}

Threaded I/O Model

I/O Problems and Threaded I/O The introduction of

If you want to say Redis What performance problems will there be , So from I/O angle , Because it's not like other Database Use the same disk , So there is no disk I/O The problem of .

Before data enters the buffer and writes from the buffer to Socket when , There is a certain network I/O, Especially writing I/O It has a great impact on performance . In the past, we will consider to do pipelining to reduce the network I/O The cost of , Or will Redis Deployed as Redis Cluster to improve performance .

stay Redis 6.0 after , because Threaded I/O The introduction of ,Redis Start to support the threading of network reading and writing , Let more threads participate in this part of the action , While maintaining single thread execution of commands . This kind of change can improve the performance to some extent , But it also avoids the need to introduce locks or other ways to solve the race state problem of parallel execution due to the threading of command execution .

Threaded I/O What are you doing? ?

In the implementation of the old version ,Redis Will be different client The results of command execution are saved in their respective client.buf in , And then put what you want to reply to client Put it in a list , Finally, in the event loop one by one buf Write the content of to the corresponding Socket. In the new version ,Redis Use multiple threads to complete this part of the operation .

Read operations ,Redis Equally for server Object added a clients_pending_read attribute , When the reading event comes , Judge whether the conditions of threaded reading are met , If meet , Then perform a delayed read operation , Put this client Object added to server.clients_pending_read In the list . Just like a write operation , Leave it to the next event loop and use multiple threads to complete the read operation .

Threaded I/O The realization and limitation of

Init Stage

stay Redis Startup time , If the corresponding parameter configuration is met , Will be carried out in I/O Thread initialization operation .

Redis There will be some routine tests , Whether the configuration number meets the requirement of multithreading I/O The requirements of .

/* Initialize the data structures needed for threaded I/O. */
void initThreadedIO(void) {
server.io_threads_active = 0; /* We start with threads not active. */
/* Don't spawn any thread if the user selected a single thread:
* we'll handle I/O directly from the main thread. */
if (server.io_threads_num == 1) return;
if (server.io_threads_num > IO_THREADS_MAX_NUM) {
serverLog(LL_WARNING,"Fatal: too many I/O threads configured. "
"The maximum number is %d.", IO_THREADS_MAX_NUM);
exit(1);
}
...

Create a... With the length of the number of threads io_threads_list list , Each element of a list is another list L,L It will be used to store multiple threads to be processed by the corresponding thread client object .

...
/* Spawn and initialize the I/O threads. */
for (int i = 0; i < server.io_threads_num; i++) {
/* Things we do for all the threads including the main thread. */
io_threads_list[i] = listCreate();
if (i == 0) continue; /* Thread 0 is the main thread. */
...

For the main thread , This is the end of the initialization operation .

...
/* Things we do only for the additional threads. */
pthread_t tid;
pthread_mutex_init(&io_threads_mutex[i],NULL);
setIOPendingCount(i, 0);
pthread_mutex_lock(&io_threads_mutex[i]); /* Thread will be stopped. */
if (pthread_create(&tid,NULL,IOThreadMain,(void*)(long)i) != 0) {
serverLog(LL_WARNING,"Fatal: Can't initialize IO thread.");
exit(1);
}
io_threads[i] = tid;
}
}
...

io_threads_mutex It's a list of mutexes ,io_threads_mutex[i] That is to say i A thread lock , For subsequent blocking I/O Thread operation , After initialization, lock it temporarily . Then create for each thread ,tid That is, its pointer , Save to io_threads In the list . The new thread will always execute IOThreadMain Method .

Reads/Writes

Multithreading read and write mainly in handleClientsWithPendingReadsUsingThreads() and handleClientsWithPendingWritesUsingThreads() Finish in , Because the two are almost symmetrical , So here's just the reading operation .

Again ,Redis There will be a routine examination , Whether to enable threaded read and write and enable threaded read ( If only the former is turned on, only the write operation is threaded ), And whether there is a client waiting to read .

/* When threaded I/O is also enabled for the reading + parsing side, the
* readable handler will just put normal clients into a queue of clients to
* process (instead of serving them synchronously). This function runs
* the queue using the I/O threads, and process them in order to accumulate
* the reads in the buffers, and also parse the first command available
* rendering it in the client structures. */
int handleClientsWithPendingReadsUsingThreads(void) {
if (!server.io_threads_active || !server.io_threads_do_reads) return 0;
int processed = listLength(server.clients_pending_read);
if (processed == 0) return 0;
if (tio_debug) printf("%d TOTAL READ pending clients\n", processed);
...

There will be server.clients_pending_read The list is converted into a linked list for easy traversal , Then each node of the list (*client object ) In a similar Round-Robin( Polling scheduling ) Each thread is allocated in the same way , Threads execute each client There is no need to guarantee the order of reading and writing , The order of arrival of the order has been changed from server.clients_pending_read/write List records , The following will also be executed in this order .

/* Distribute the clients across N different lists. */
listIter li;
listNode *ln;
listRewind(server.clients_pending_read,&li);
int item_id = 0;
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
int target_id = item_id % server.io_threads_num;
listAddNodeTail(io_threads_list[target_id],c);
item_id++;
}
...

Set the status flag , Identifies the current multithreaded read state . Because of the mark ,Redis Of Threaded I/O The instantaneous state can only be read or written , Can't read part of the thread , Partial writing .

...
/* Give the start condition to the waiting threads, by setting the
* start condition atomic var. */
io_threads_op = IO_THREADS_OP_READ;
...

Record the number of clients to be processed for each thread . When different threads read their own pending The length is not 0 when , It will start processing . Be careful j from 1 Start , Means that the main thread pending The length has always been 0, Because the main thread will finish its task synchronously in this method , No need to know the number of tasks waiting .

...
for (int j = 1; j < server.io_threads_num; j++) {
int count = listLength(io_threads_list[j]);
setIOPendingCount(j, count);
}
...

The main thread will handle client processed .

...
/* Also use the main thread to process a slice of clients. */
listRewind(io_threads_list[0],&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
readQueryFromClient(c->conn);
}
listEmpty(io_threads_list[0]);
...

Fall into a cycle and wait ,pending Equal to the sum of the remaining tasks of each thread , When all threads have no tasks , This round I/O End of processing .

...
/* Wait for all the other threads to end their work. */
while(1) {
unsigned long pending = 0;
for (int j = 1; j < server.io_threads_num; j++)
pending += getIOPendingCount(j);
if (pending == 0) break;
}
if (tio_debug) printf("I/O READ All threads finshed\n");
...

We've put... In our respective threads conn Read the contents of to the corresponding client Of client.querybuf In the input buffer , So you can traverse server.clients_pending_read list , Perform command execution operations serially , At the same time client Remove from list .

 ...
/* Run the list of clients again to process the new buffers. */
while(listLength(server.clients_pending_read)) {
ln = listFirst(server.clients_pending_read);
client *c = listNodeValue(ln);
c->flags &= ~CLIENT_PENDING_READ;
listDelNode(server.clients_pending_read,ln);
if (processPendingCommandsAndResetClient(c) == C_ERR) {
/* If the client is no longer valid, we avoid
* processing the client later. So we just go
* to the next. */
continue;
}
processInputBuffer(c);
/* We may have pending replies if a thread readQueryFromClient() produced
* replies and did not install a write handler (it can't).
*/
if (!(c->flags & CLIENT_PENDING_WRITE) && clientHasPendingReplies(c))
clientInstallWriteHandler(c);
}
...

Processing is complete , Add the number of processes to the statistical property , Then return .

...
/* Update processed count on server */
server.stat_io_reads_processed += processed;
return processed;

IOThreadMain

The specific work content of each thread has not been explained before , They will always be trapped in IOThreadMain In the cycle of , Waiting for the time to perform read and write .

Perform some initialization as usual .

void *IOThreadMain(void *myid) {
/* The ID is the thread number (from 0 to server.iothreads_num-1), and is
* used by the thread to just manipulate a single sub-array of clients. */
long id = (unsigned long)myid;
char thdname[16];
snprintf(thdname, sizeof(thdname), "io_thd_%ld", id);
redis_set_thread_title(thdname);
redisSetCpuAffinity(server.server_cpulist);
...

The thread detects its own pending client List length , When the waiting queue length is greater than 0 I'm going to go down , Otherwise, it will be at the beginning of the dead cycle .

Here we use mutex , Give the main thread a chance to lock , bring I/O The thread card is executing pthread_mutex_lock(), To make I/O The effect of thread stopping working .

...
while(1) {
/* Wait for start */
for (int j = 0; j < 1000000; j++) {
if (getIOPendingCount(id) != 0) break;
}
/* Give the main thread a chance to stop this thread. */
if (getIOPendingCount(id) == 0) {
pthread_mutex_lock(&io_threads_mutex[id]);
pthread_mutex_unlock(&io_threads_mutex[id]);
continue;
}
serverAssert(getIOPendingCount(id) != 0);
if (tio_debug) printf("[%ld] %d to handle\n", id, (int)listLength(io_threads_list[id]));
...

take io_threads_list[i] The client list is converted into a linked list for easy traversal , One by one , With the help of io_threads_op Flag to determine whether to execute multithreaded read or write , Complete the operation of the client you want to handle .

...
/* Process: note that the main thread will never touch our list
* before we drop the pending count to 0. */
listIter li;
listNode *ln;
listRewind(io_threads_list[id],&li);
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
if (io_threads_op == IO_THREADS_OP_WRITE) {
writeToClient(c,0);
} else if (io_threads_op == IO_THREADS_OP_READ) {
readQueryFromClient(c->conn);
} else {
serverPanic("io_threads_op value is unknown");
}
}
...

Clear the list of clients you want to process , And change the number to be processed to 0, End the round .

...
listEmpty(io_threads_list[id]);
setIOPendingCount(id, 0);
if (tio_debug) printf("[%ld] Done\n", id);
}
}

Limitation

By viewing the code , On use Threaded I/O The activation of is affected by the following conditions :

  • Configuration item io-threads Need greater than 1, Otherwise, it will continue to use single thread operation to read and write I/O
  • Configuration item io-threads-do-reads Control reading I/O Whether to use threading
  • For delayed reads , from postponeClientRead() Method control . In addition to configuration requirements , It also needs the current client It can't be the role of the master-slave model , It can't be in the state where it has been waiting for the next event loop to thread read CLIENT_PENDING_READ The state of . In this method client Objects are added to the waiting queue , And will client Change the status of CLIENT_PENDING_READ.
  • For multithreaded writing I/O, from handleClientsWithPendingWritesUsingThreads() Medium stopThreadedIOIfNeeded() Methods to limit . In addition to the corresponding configuration items to meet the requirements ,server.clients_pending_write The length of should be greater than or equal to twice the number of configured threads , For example, configuration uses 6 Threads , When the write queue length is less than 12 Will continue to use single thread I/O.
  • I/O The thread is in initThreadedIO() Before it was created , Mutex is locked , So threads can't do actual task processing .server Object's io_threads_active Property is turned off by default , It will not be turned on until the first multithreaded write . This means that read operations after the service is started will still use single thread read , Generate execution results to write pending list in , In the second cycle , The service determines whether the configuration is enabled TIO, take server.io_threads_active Properties to open the , And then multithread write operations , From the next cycle TIO Can be applied to read operations . As I said in the last point, write I/O There will be configuration and queue length determination , It is not necessary to TIO Writing time , I'll put it back server.io_threads_active close , It means that even though you've opened it in the configuration file TIO read , however Redis Still skip using it from time to time depending on the load .

Summary and reflection

Redis 6.0 Introduced Threaded I/O, take Socket Read write latency and threading , On the Internet I/O In the direction of Redis Bring a certain performance improvement , And the threshold is low , Users don't have to make too many changes , You can use idle thread resources without affecting the business .

One side , This part of the ascension may be difficult to make in Redis 5 even to the extent that Redis 3 Users of this version have enough motivation to upgrade , Especially considering many business scenarios Redis It's not bad enough to be a bottleneck , And the new version of benefits has not been massively validated , It is bound to affect the service stability of more users in enterprise applications . meanwhile ,TIO There seems to be a certain gap between the improvement of cluster performance and that of cluster performance , This may make enterprise users who are already in the cluster architecture ignore this function .

In terms of stability , The functionality of the new version has not been massively validated , Whether it's worth upgrading remains to be seen .

This version can be said to be Redis The biggest update since its birth , It's not just Threaded I/O, Include RESP3、ACLs and SSL, If you are interested, you can learn about it by yourself .

Redis Source code learning ideas

Method

README.md It should be that we understand Redis Entrance , Not global search main() Method . Please pay attention to Redis internals What's under the section , Here is the introduction of Redis Code structure of ,Redis Every file is a “general idea”, among server.c and network.c Part of the logic and code for has been introduced in this article , Persistence related aof.c and rdb.c、 Database related db.c、Redis Object related object.c、 Copy related replication.c It's worth noting . Others include Redis In what form is the command encoded , Can also be in README.md Find the answer , In this way, we can quickly locate when we read the code further .

Tools

If it's a tool, I recommend visual code that will do , Install well C/C++ after ,Mac Basically, you can read the source code without pressure .Windows Install well mingw-w64 that will do .

image.png

Other key points of the code , In fact, it also appears in this article :

  • main(), The starting point
  • initServer(), initialization
  • aeMain(), The event loop
  • readQueryFromClient(), Reading events Handler
  • processInputBuffer(), The entry to command processing

If you want to know like this article Network The content of , Can be in aeMain() Place a breakpoint , And then focus on network.c The method in ; If you want to focus on specific commands , Can be in processInputBuffer() Place a breakpoint , Then focus on $command.c Or something like that in a file ,README.md The file also describes the naming format of command methods , Positioning is very easy . The rest of the other actions that often happen , For example, persistence 、 Copy, etc , It will probably appear before and after the execution of the command , Or in time events , Or maybe beforeSleep() in .server.h As defined in redisServer and client yes Redis Two very important structures in , In business, a lot of content is transformed into related operations on their properties , Pay special attention .

Reference documents

https://github.com/redis/redis

https://github.com/redis/redis-doc

https://redis.io/documentation

https://www.youtube.com/c/Redislabs/videos

writing | Night inscription
Focus on Technology , Go hand in hand to the cloud of Technology

版权声明
本文为[Acquisition technology]所创,转载请带上原文链接,感谢
https://javamana.com/2021/01/20210122173312871m.html

  1. 【计算机网络 12(1),尚学堂马士兵Java视频教程
  2. 【程序猿历程,史上最全的Java面试题集锦在这里
  3. 【程序猿历程(1),Javaweb视频教程百度云
  4. Notes on MySQL 45 lectures (1-7)
  5. [computer network 12 (1), Shang Xuetang Ma soldier java video tutorial
  6. The most complete collection of Java interview questions in history is here
  7. [process of program ape (1), JavaWeb video tutorial, baidu cloud
  8. Notes on MySQL 45 lectures (1-7)
  9. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  10. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  11. 精进 Spring Boot 03:Spring Boot 的配置文件和配置管理,以及用三种方式读取配置文件
  12. Refined spring boot 03: spring boot configuration files and configuration management, and reading configuration files in three ways
  13. 【递归,Java传智播客笔记
  14. [recursion, Java intelligence podcast notes
  15. [adhere to painting for 386 days] the beginning of spring of 24 solar terms
  16. K8S系列第八篇(Service、EndPoints以及高可用kubeadm部署)
  17. K8s Series Part 8 (service, endpoints and high availability kubeadm deployment)
  18. 【重识 HTML (3),350道Java面试真题分享
  19. 【重识 HTML (2),Java并发编程必会的多线程你竟然还不会
  20. 【重识 HTML (1),二本Java小菜鸟4面字节跳动被秒成渣渣
  21. [re recognize HTML (3) and share 350 real Java interview questions
  22. [re recognize HTML (2). Multithreading is a must for Java Concurrent Programming. How dare you not
  23. [re recognize HTML (1), two Java rookies' 4-sided bytes beat and become slag in seconds
  24. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  25. RPC 1: how to develop RPC framework from scratch
  26. 造轮子系列之RPC 1:如何从零开始开发RPC框架
  27. RPC 1: how to develop RPC framework from scratch
  28. 一次性捋清楚吧,对乱糟糟的,Spring事务扩展机制
  29. 一文彻底弄懂如何选择抽象类还是接口,连续四年百度Java岗必问面试题
  30. Redis常用命令
  31. 一双拖鞋引发的血案,狂神说Java系列笔记
  32. 一、mysql基础安装
  33. 一位程序员的独白:尽管我一生坎坷,Java框架面试基础
  34. Clear it all at once. For the messy, spring transaction extension mechanism
  35. A thorough understanding of how to choose abstract classes or interfaces, baidu Java post must ask interview questions for four consecutive years
  36. Redis common commands
  37. A pair of slippers triggered the murder, crazy God said java series notes
  38. 1、 MySQL basic installation
  39. Monologue of a programmer: despite my ups and downs in my life, Java framework is the foundation of interview
  40. 【大厂面试】三面三问Spring循环依赖,请一定要把这篇看完(建议收藏)
  41. 一线互联网企业中,springboot入门项目
  42. 一篇文带你入门SSM框架Spring开发,帮你快速拿Offer
  43. 【面试资料】Java全集、微服务、大数据、数据结构与算法、机器学习知识最全总结,283页pdf
  44. 【leetcode刷题】24.数组中重复的数字——Java版
  45. 【leetcode刷题】23.对称二叉树——Java版
  46. 【leetcode刷题】22.二叉树的中序遍历——Java版
  47. 【leetcode刷题】21.三数之和——Java版
  48. 【leetcode刷题】20.最长回文子串——Java版
  49. 【leetcode刷题】19.回文链表——Java版
  50. 【leetcode刷题】18.反转链表——Java版
  51. 【leetcode刷题】17.相交链表——Java&python版
  52. 【leetcode刷题】16.环形链表——Java版
  53. 【leetcode刷题】15.汉明距离——Java版
  54. 【leetcode刷题】14.找到所有数组中消失的数字——Java版
  55. 【leetcode刷题】13.比特位计数——Java版
  56. oracle控制用户权限命令
  57. 三年Java开发,继阿里,鲁班二期Java架构师
  58. Oracle必须要启动的服务
  59. 万字长文!深入剖析HashMap,Java基础笔试题大全带答案
  60. 一问Kafka就心慌?我却凭着这份,图灵学院vip课程百度云