An article to talk about epoll and file events in Redis

This article will introduce you to the file events in Redis. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

Event-driven

Redis server is an event-driven program, divided into file events and Time event

• File event: socket readable and writable event
• Timed task

[Related recommendations:Redis video Tutorial】

They are all encapsulated into the aeEventLoop structure

typedef struct aeEventLoop {
	int stop; // 标识事件是否结束
	aeFileEvent *events; // 文件事件数组，存储已注册的文件事件
	aeFireEvent *fired; // 存储被触发的文件事件
	aeTimeEvent *timteEventHead; // 多个时间事件形成的链表
	void *apidata; // I/O模型的封装
	aeBeforeSleepProc *beforesleep; // 进程阻塞前执行
	aeBeforeSleepProc *aftersleep; // 进程被唤醒后执行
} aeEventLoop;

The event driver actually also loops through while/for , waiting in a loop for the event to occur

while (! eventLoop->stop) {
	if (eventLoop->beforesleep != NULL)
		eventLoop->beforesleep(eventLoop)
	aeProcessEvents(eventLoop, AE_ALL_EVENTS|AE_CALL_AFTER_SLEEP);
}

aeProcessEventsMain function for event processing

epoll

Redis client through TCP socket When interacting with the server, file events refer to readable and writable events of the socket. Generally, non-blocking mode is used. Related I/O multiplexing includes select/epoll/kqueue, etc. Different operating systems have different implementations.

Take epoll as an example. It is a solution proposed by the Linux kernel to handle a large number of concurrent network connections. epollProvides 3 APIs

• epoll_create creates an epoll-specific file descriptor for subsequent epoll-related API calls

int epoll_create(int size)
// size 告知内核程序期望注册的网络连接数目，Linux 2.6.8后改为内核动态分配
// 返回参数是 epoll 专用的文件描述符

• epoll_ctl function registers, modifies or deletes events that need to be monitored with epoll

int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event)
// epfd 函数 epoll_create 返回的 epoll 文件描述符
// op 操作类型 EPOLL_CTL_ADD：注册事件； EPOLL_CTL_MOD：修改网络连接事件； EPOLL_CTL_DEL：删除事件
// fd 网络连接的 socket 文件描述符
// event 需要监控的事件

• epoll_wait function will block the process until monitoring Events occurred on several network connections

int epoll_wait(int epfd, struct epoll_event *event, int maxevents, int timeout)
// epfd 函数epoll_create返回的epoll文件描述符
// epoll_event 作为输出参数使用，用于回传已触发的事件数组
// maxevents 每次能处理的最大事件数目
// timeout epoll_wait 函数阻塞超时时间，如果超过 timeout 时间还没有事件发生，函数就不再阻塞直接返回；当 timeout 等于0是函数立即返回，timeout 等于-1时函数一直阻塞到有事件发生

File events

Reids does not directly use the epoll API, but supports 4 types of I/O at the same time O multiplexing models encapsulate the APIs of these models. Then during the compilation phase, the I/O multiplexing model supported by the operating system is checked, and which model is decided to be reused according to the strategy.

Still taking epoll as an example, Redis is encapsulated as follows

// 对应 epoll_create
static int aeApiCreate(aeEventLoop *eventLoop)

// 对应 epoll_ctl 添加事件
static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask)
// 对应 epoll_ctl 删除事件
static int aeApiDelEvent(aeEventLoop *eventLoop, int fd, int delmask)

// 对应 epoll_wait
static int aeApiPool(aeEventLoop *eventLoop, struct timeval *tvp)

Recall the eventLoop structure mentioned above, its member apidata points to 4 types of I/O multiple Road reuse model object; events stores the event array that needs to be monitored, and uses the socket file descriptor as the array index to access elements; fired stores the triggered event array.

The structure of the file event is defined as follows:

typedef struct aeFileEvent {
	int mask; // 文件事件类型 AE_READABLE 可读事件；AE_WRITEABLE 可写事件
	aeFileProc *rfileProc; // 读事件处理函数指针
	aeFileProc *wfileProc; // 写事件处理函数指针
	void *clientData; // 指向对应的客户端对象
} aeFileEvent;

Look at the implementation of the create file event aeCreateFileEvent

int aeCreateFileEvent (aeEventLoop *eventLoop, int fd, int mask, aeFileProc *proc, void *clientData) {
	aeFileEvent *fe = &eventLoop->evnts[fd];
	if (aeApiAddEvent(eventLoop, fd, mask) == -1)
		return AE_ERR;
	fe->mask |= mask;
	if (mask & AE_READABLE) fe->rfileProc = proc;
	if (mask & AE_WRITABLE) fe->wfileProc = proc;
	fe->clientData = clientData;
	return AE_OK;
}

The Redis server will handle transactions by creating various file events, such as ：

• Create a socket at startup and listen, waiting for the client to connect

aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE, acceptTcpHandler, NULL);

• After the client establishes a socket connection with the server, the server will wait for the client's command request

aeCreateFileEvent(server.el, fd, AE_READABLLE, readQueryFromClient, c);

• After the server processes the client's command request, the command reply will be temporarily cached in the buf buffer of the client structure until the writable event of the client's file descriptor occurs. The command reply will actually be sent to the client

aeCreateFileEvent(server.el, c->fd, AE_READABLLE, sendReplyToClient, c);

Redis The execution of all events is controlled through the aeProcessEvents function. Among them, the execution of file events will cause blocking (epoll_wait). If the blocking event is too long, it will hinder the execution of time events (timing). To avoid this situation, the waiting time passed in when implementing file events, It is obtained by calculating the earliest time event that occurred

int aeProcessEvents(aeEventLoop *eventLoop, int flags) {
	shortest = aeSearchNearestTimer(eventLoop);
	long long ms = (shortest->when_sec - now_sec) * 1000 + \
		shortest->when_ms - now_ms;

	// 阻塞事件发生
	numevents = aeApiPoll(eventLoop, ms);

	for (j=0; j < numevents; j++) {
		aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j]].fd];
		// 处理文件事件，即根据类型执行rfileProc或wfileProc
	}

	// 处理时间事件
	processed += processTimeEvents(eventLoop);
}

Summary

Now let’s take an overall look at the process of the corresponding command of the Redis server

aeMain function schedules and executes file events and time events by calling the aeProcessEvents function. Event-related information is recorded in aeEventLoop. First, obtain the execution time interval n of the shortest time event through the aeSearchNearestTimer function, then call the aeApiPoll function to obtain the monitored socket, and finally execute the event processing functions rfileProc and wfileProc corresponding to the socket, and finally execute the time event function processTimeEvents .

A complete client-server connection event:

• #The server listens for the AE_READABLE event of the package. When the client sends a connection request and generates an AE_READABLE event, the server will Respond to the client's connection request, combine the AE_READABLE event of the client socket with the command request processing function (aeFileProc), and the client can send a command request to the server

• The server sends a command request, and the client socket will generate an AE_READABLE event, triggering the command processor to execute. Executing the command will generate a corresponding command reply. The server will combine the AE_WRITABLE event of the client socket with the command reply processing function ( aeFileProc) When the associated end tries to read the command reply, the client socket will generate an AE_WRITABLE event, triggering the execution of the command reply processor. When the command reply processor writes all the command replies After entering the socket, the server will contact the association between the AE_WRITABLE event of the client socket and the command reply processing function (aeFileProc)

For more programming-related knowledge, please visit: Programming Video! !

The above is the detailed content of An article to talk about epoll and file events in Redis. For more information, please follow other related articles on the PHP Chinese website!