Why does docker kill sometimes fail to kill running containers?

Docker Kill main process

1. The Docker engine uses SIGKILL to send it to the container main process through containerd. After waiting for a period of time, if the container exit message is received from containerd, Then the container Kill is successful.

2. If the wait times out in the previous step, the Docker engine will skip Containerd and send the SIGKILL signal to the container main process through the kill system call. If the kill system call returns that the main process does not exist at this time, then Docker kill is successful. Otherwise, the engine will remain dead until containerd passes the engine and the container exits.

When a problem occurs, a large number of stress processes (actually container processes) are found on the host machine to be in the D state, and the system response slows down. The problem can be explained like this:

1,After Docker kill indirectly sends the SIGKill signal to the main process of the container through containerd, due to the slow system response, the internal child process (stress) of the container is in the D state, so during the timeout period Containerd did not report the container exit. Docker kill has reached the stage of directly sending Sigkill

2,Before this stage, the main process inside the container has exited, so the system call kill and send SIGKILL will soon return that the process does not exist. The engine thinks it has killed the container, and Docker kill returns successfully.

3. After a certain period of time, the container sub-processes recover from the D state. They exit, containerd reports the container exit, and the engine cleans up resources. At this time, Docker ps sees that the container is in the exit state.

Container master/child process is in D state

The process D state indicates that the process is in an uninterruptible sleep state, usually waiting for IO resources. Of course, sometimes if there is a problem with system IO, there will be a large number of processes in the D state. In this state, the signal cannot wake up the process; it can only wait for the process to return from the D state. And in the conventional kernel, if a process has been in D state, theoretically there is no way or means to bring it back from D except restarting the system.

For more related tutorials, please pay attention to thedocker tutorialcolumn on the PHP Chinese website.

The above is the detailed content of Why does docker kill sometimes fail to kill running containers?. For more information, please follow other related articles on the PHP Chinese website!