See you soon! TCP waves twice, have you seen it? What about the four handshakes?

We all know that TCP is a connection-oriented, reliable, byte stream-based transport layercommunication protocol.

Then the "Connection-oriented" mentioned here, It means that you need to establish a connection, use the connection, and release the connection.

Establishing a connection refers to the TCP three-way handshake that we are familiar with.

And using connection, data transmission is carried out in the form of one send and one confirmation.

There is also Release the connection, which is our common TCP four wave gestures.

TCP Four Waves Everyone should be familiar with it, but have you ever seen Three Waves? And twice waving?

Have you seen it? What about the four-way handshake?

Today’s topic is not just about curiosity, nor is it about trivia.

Let’s start with the four waves and get some practical knowledge.

TCP waves four times

Briefly review the four waves of TCP.

Under normal circumstances. As long as the data transmission is completed, both the client and the server can actively initiate four waves to release the connection.

Just like the picture, assuming that the four waves are initiated by the client, then it is the active party. The server passively receives the client's wave request, which is called passive party.

The client and server are in the ESTABLISHED state at the beginning.

Waving for the first time: Under normal circumstances, when the active party executes the close() or shutdown() method, a ## will be sent. The #FIN message comes out, indicating "I will no longer send data".

The second wave: After receiving the FIN message from the active party, the passive party immediately responds with a ACK, meaning " I received your FIN, and I also know that you will no longer send data."

active party no longer sends data. But if at this time, the passive party still has data to send, then continue sending. Note that although the passive side can send data to the active side between the second and third wave, it is not certain whether the active side can receive it normally. This will be discussed later.

The third wave: After the passive party senses the second wave, it will do a series of finishing work, and finally call a close(), At this time, the third wave of FIN-ACK will be issued.

The fourth wave: The active party responds with an ACK, which means it was received.

The first wave and the third wave are both actively triggered by us in the application (such as calling the close() method), which is what we need to pay attention to when writing code. The place.

The second and fourth waving are done automatically by the kernel protocol stack for us. We cannot touch this place when we write code, so we don’t need to care too much.

In addition, regardless of whether it is active or passive, each party sends a FIN and an ACK. Also received a FIN and an ACK. Please pay attention to this point and will mention it later.

Does the program have to execute close() or shutdown() before FIN can be issued?

uncertain. Normally, FIN is emitted by executing the close() or shutdown() method on socket. But in fact, as long as the application exits, whether active exits or passive exits (it is killed for some inexplicable reasons), will Issue FIN.

FIN means "I will no longer send data", so shutdown() will not send FIN to the other party when reading is closed, but FIN will be sent when writing is closed.

If there are too many FIN-WAIT-2 states on the machine, why

According to the above four waves From the figure, it can be seen that FIN-WAIT-2 is the status of active party.

The program in this state has been waiting for the third wave of FIN. The third wave needs to be issued by the passive party by executing close() in the code.

So when there are a lot of FIN-WAIT-2 states on a machine, generally speaking, there will be a lot of CLOSE_WAIT on another machine. You need to check why the machine with a large number of CLOSE_WAIT is reluctant to call close() to close the connection.

So, if there are too many FIN-WAIT-2 states on the machine, it is usually because the peer has not executed the close() method to issue the third wave.

How will the active party handle the data received after close?

An article I wrote before "After the code executes send successfully, will the data be sent out?" 》, from the perspective of source code, it is mentioned that Under normal circumstances, when the program actively executes close();

• if the current connection If the receiving buffer of the corresponding socket has data, RST will be sent.

• If there is data in the sending buffer, it will wait until the sending is completed before sending the first wave FIN.

As we all know, TCP is full-duplex communication, which means that data can be received while sending data.

Close() means that the function of sending and receiving messages must be closed at the same time.

In other words, although theoretically, between the second and third wave, the passive party can transmit data to the active party.

But if the four waves of the active party are triggered by close(), then the active party will not receive this message. And will also reply with RST. Directly end this connection.

Can't data be transferred between the second and third wave of hands?

No. As mentioned earlier, Close() means to close the function of sending and receiving messages at the same time.

If we can only turn off sending messages and not turn off the function of receiving messages , then we can continue to receive messages. This half-close function can be achieved by calling the shutdown() method.

int shutdown(int sock, int howto);

Howto is the disconnection method. The following values ​​are available:

• SHUT_RD: Close reading. At this time, the application layer should no longer try to receive data. Even if the receive buffer receives data in the kernel protocol stack, it will be discarded.

• SHUT_WR: Close writing. If there is still data in the send buffer that has not been sent, the data will be delivered to the target host.

• SHUT_RDWR: Close reading and writing. Equivalent to close().

怎么知道对端socket执行了close还是shutdown

不管主动关闭方调用的是close()还是shutdown()，对于被动方来说，收到的就只有一个FIN。

被动关闭方就懵了，"我怎么知道对方让不让我继续发数据？"

其实，大可不必纠结，该发就发。

第二次挥手和第三次挥手之间，如果被动关闭方想发数据，那么在代码层面上，就是执行了 send() 方法。

int send( SOCKET s,const char* buf,int len,int flags);

send() 会把数据拷贝到本机的发送缓冲区。如果发送缓冲区没出问题，都能拷贝进去，所以正常情况下，send()一般都会返回成功。


然后被动方内核协议栈会把数据发给主动关闭方。

• 如果上一次主动关闭方调用的是shutdown(socket_fd, SHUT_WR)。那此时，主动关闭方不再发送消息，但能接收被动方的消息，一切如常，皆大欢喜。

• If the last active closing party called close(). The active party will directly discard after receiving the data from the passive party, and then reply with a RST.

For the second situation.

Passive sideKernel protocol stack will close the connection after receiving RST. But the kernel connection is closed, and the application layer does not know (unless notified).

At this time, the passive sideapplication layerThe next operation is nothing more than reading or writing.

• If it is read, an error of RST will be returned, which is our common Connection reset by peer.

• If it is written, the program will generate the SIGPIPE signal. The application layer code can capture and process the signal. If not processed, the process will terminate by default. quit unexpectedly.

##To summarize, when the passive closing methodrecv() returns EOF, it means that the active party initiated the first wave through close() or shutdown(fd, SHUT_WR).

If the passive party executes twice at this time send().

• The first timesend() will usually return successfully.

• The second time send(). If the active party initiates the first wave via shutdown(fd, SHUT_WR), then send() will still succeed at this time. If the active party initiates the first wave through close(), then the SIGPIPE signal will be generated at this time, and the process will terminate by default and exit abnormally. If you don't want to exit abnormally, remember to catch and handle this signal.

What will happen if the passive party never waves for the third time

The third time Waving is actively triggered by the passive party, such as calling close().

If due to a code error or some other reasons, the passive party will not execute the third wave.

At this time, the active party will have different behaviors depending on whether it uses close() or shutdown(fd, SHUT_WR) when it waves for the first time. Performance.

• 如果是 shutdown(fd, SHUT_WR) ，说明主动方其实只关闭了写，但还可以读，此时会一直处于 FIN-WAIT-2， 死等被动方的第三次挥手。

• 如果是 close()， 说明主动方读写都关闭了，这时候会处于 FIN-WAIT-2一段时间，这个时间由 net.ipv4.tcp_fin_timeout 控制，一般是 60s，这个值正好跟2MSL一样 。超过这段时间之后，状态不会变成 `TIME-WAIT`，而是直接变成`CLOSED`。

# cat /proc/sys/net/ipv4/tcp_fin_timeout
60

TCP三次挥手

四次挥手聊完了，那有没有可能出现三次挥手？

是可能的。

我们知道，TCP四次挥手里，第二次和第三次挥手之间，是有可能有数据传输的。第三次挥手的目的是为了告诉主动方，"被动方没有数据要发了"。

所以，在第一次挥手之后，如果被动方没有数据要发给主动方。第二和第三次挥手是有可能合并传输的。这样就出现了三次挥手。

If there is data to be sent, can't it be done by waving three times?

The above mentioned is the situation where there is no data to be sent. If second, There is data to be sent between the third wave, so is it impossible to wave three times?

. There is also a feature in TCP called Delayed Confirmation. It can be simply understood as: The receiver does not need to reply with an ACK confirmation packet immediately after receiving the data. On this basis,

ACK confirmation packet will be received, because the receiver can merge the confirmations. And this merge confirmation is placed in the four waves, and the second wave, the third wave, and the data transmission between them can be merged and sent together. So there were three waves.

TCP wave three times delayed confirmation

As mentioned earlier in the four waves, when closing, both parties

. Under normal circumstances, the two ends of the TCP connection are processes with different

. But if the

is the same at both ends of the TCP connection, then when closing the connection, the same thing will happen. One end will send a FIN, and it will also receive An ACK was received, but it happened that the two ends were actually same socket.

TCP waves twiceAnd this kind of connection with the same

at both ends is called TCP self-connection .

是的，你没看错，我也没打错别字。同一个socket确实可以自己连自己，形成一个连接。

一个socket能建立连接？

上面提到了，同一个客户端socket，自己对自己发起连接请求。是可以成功建立连接的。这样的连接，叫TCP自连接。

下面我们尝试下复现。

注意我是在以下系统进行的实验。在mac上多半无法复现。

#  cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"

通过nc命令可以很简单的创建一个TCP自连接

# nc -p 6666 127.0.0.1 6666

上面的 -p 可以指定源端口号。也就是指定了一个端口号为6666的客户端去连接 127.0.0.1:6666 。

# netstat -nt | grep 6666
tcp        0      0 127.0.0.1:6666          127.0.0.1:6666          ESTABLISHED

整个过程中，都没有服务端参与。可以抓个包看下。

可以看到，相同的socket，自己连自己的时候，握手是三次的。挥手是两次的。

上面这张图里，左右都是同一个客户端，把它画成两个是为了方便大家理解状态的迁移。

我们可以拿自连接的握手状态对比下正常情况下的TCP三次握手。

看了自连接的状态图，再看看下面几个问题。

一端发出第一次握手后，如果又收到了第一次握手的SYN包，TCP连接状态会怎么变化？

第一次握手过后，连接状态就变成了SYN_SENT状态。如果此时又收到了第一次握手的SYN包，那么连接状态就会从SYN_SENT状态变成SYN_RCVD。

// net/ipv4/tcp_input.c
static int tcp_rcv_synsent_state_process()
{
    // SYN_SENT状态下，收到SYN包
    if (th->syn) {
        // 状态置为 SYN_RCVD
        tcp_set_state(sk, TCP_SYN_RECV);
    }
}

一端发出第二次握手后，如果又收到第二次握手的SYN+ACK包，TCP连接状态会怎么变化？

第二握手过后，连接状态就变为SYN_RCVD了，此时如果再收到第二次握手的SYN+ACK包。连接状态会变为ESTABLISHED。

// net/ipv4/tcp_input.c
int tcp_rcv_state_process()
{
    // 前面省略很多逻辑，能走到这就认为肯定有ACK
    if (true) {
        // 判断下这个ack是否合法
        int acceptable = tcp_ack(sk, skb, FLAG_SLOWPATH | FLAG_UPDATE_TS_RECENT) > 0;
        switch (sk->sk_state) {
        case TCP_SYN_RECV:
            if (acceptable) {
        // 状态从 SYN_RCVD 转为 ESTABLISHED
                tcp_set_state(sk, TCP_ESTABLISHED);
            } 
        }
    }
}

一端第一次挥手后，又收到第一次挥手的包，TCP连接状态会怎么变化？

第一次挥手过后，一端状态就会变成 FIN-WAIT-1。正常情况下，是要等待第二次挥手的ACK。但实际上却等来了 一个第一次挥手的 FIN包， 这时候连接状态就会变为CLOSING。

// net/
static void tcp_fin(struct sock *sk)
{
    switch (sk->sk_state) {
    case TCP_FIN_WAIT1:
        tcp_send_ack(sk);
    // FIN-WAIT-1状态下，收到了FIN，转为 CLOSING
        tcp_set_state(sk, TCP_CLOSING);
        break;
    }
}

这可以说是隐藏剧情了。

CLOSING 很少见，除了出现在自连接关闭外，一般还会出现在TCP两端同时关闭连接的情况下。

处于CLOSING状态下时，只要再收到一个ACK，就能进入 TIME-WAIT 状态，然后等个2MSL，连接就彻底断开了。这跟正常的四次挥手还是有些差别的。大家可以滑到文章开头的TCP四次挥手再对比下。

代码复现自连接

可能大家会产生怀疑，这是不是nc这个软件本身的bug。

那我们可以尝试下用strace看看它内部都做了啥。

# strace nc -p 6666 127.0.0.1 6666
// ...
socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
fcntl(3, F_GETFL)                       = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(6666), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(6666), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
// ...

无非就是以创建了一个客户端socket句柄，然后对这个句柄执行 bind, 绑定它的端口号是6666，然后再向 127.0.0.1:6666发起connect方法。

我们可以尝试用C语言去复现一遍。

下面的代码，只用于复现问题。直接跳过也完全不影响阅读。

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 


int main()
{
    int lfd, cfd;
    struct sockaddr_in serv_addr, clie_addr;
    socklen_t clie_addr_len;
    char buf[BUFSIZ];
    int n = 0, i = 0, ret = 0 ;
    printf("This is a client \n");

    /*Step 1: 创建客户端端socket描述符cfd*/    
    cfd = socket(AF_INET, SOCK_STREAM, 0);
    if(cfd == -1)
    {
        perror("socket error");
        exit(1);
    }

    int flag=1,len=sizeof(int);
    if( setsockopt(cfd, SOL_SOCKET, SO_REUSEADDR, &flag, len) == -1)
    {
        perror("setsockopt");
        exit(1);
    } 


    bzero(&clie_addr, sizeof(clie_addr));
    clie_addr.sin_family = AF_INET;
    clie_addr.sin_port = htons(6666);
    inet_pton(AF_INET,"127.0.0.1", &clie_addr.sin_addr.s_addr);

    /*Step 2: 客户端使用bind绑定客户端的IP和端口*/  
    ret = bind(cfd, (struct sockaddr* )&clie_addr, sizeof(clie_addr));
    if(ret != 0)
    {
        perror("bind error");
        exit(2);
    }

    /*Step 3: connect链接服务器端的IP和端口号*/    
    bzero(&serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(6666);
    inet_pton(AF_INET,"127.0.0.1", &serv_addr.sin_addr.s_addr);
    ret = connect(cfd,(struct sockaddr *)&serv_addr, sizeof(serv_addr));
    if(ret != 0)
    {
        perror("connect error");
        exit(3);
    }

    /*Step 4: 向服务器端写数据*/
    while(1)
    {
        fgets(buf, sizeof(buf), stdin);
        write(cfd, buf, strlen(buf));
        n = read(cfd, buf, sizeof(buf));
        write(STDOUT_FILENO, buf, n);//写到屏幕上
    }
    /*Step 5: 关闭socket描述符*/
    close(cfd);
    return 0;
}

保存为 client.c 文件，然后执行下面命令，会发现连接成功。

# gcc client.c -o client && ./client
This is a client

# netstat -nt | grep 6666
tcp        0      0 127.0.0.1:6666          127.0.0.1:6666          ESTABLISHED

说明，这不是nc的bug。事实上，这也是内核允许的一种情况。

自连接的解决方案

自连接一般不太常见，但遇到了也不难解决。

解决方案比较简单，只要能保证客户端和服务端的端口不一致就行。

事实上，我们写代码的时候一般不会去指定客户端的端口，系统会随机给客户端分配某个范围内的端口。而这个范围，可以通过下面的命令进行查询

# cat /proc/sys/net/ipv4/ip_local_port_range
32768   60999

也就是只要我们的服务器端口不在32768-60999这个范围内，比如设置为8888。就可以规避掉这个问题。

另外一个解决方案，可以参考golang标准网络库的实现，在连接建立完成之后判断下IP和端口是否一致，如果遇到自连接，则断开重试。

func dialTCP(net string, laddr, raddr *TCPAddr, deadline time.Time) (*TCPConn, error) {
    // 如果是自连接，这里会重试
    for i := 0; i < 2 && (laddr == nil || laddr.Port == 0) && (selfConnect(fd, err) || spuriousENOTAVAIL(err)); i++ {
        if err == nil {
            fd.Close()
        }
        fd, err = internetSocket(net, laddr, raddr, deadline, syscall.SOCK_STREAM, 0, "dial", sockaddrToTCP)
    }
    // ...
}

func selfConnect(fd *netFD, err error) bool {
    // 判断是否端口、IP一致
    return l.Port == r.Port && l.IP.Equal(r.IP)
}

四次握手

前面提到的TCP自连接是一个客户端自己连自己的场景。那不同客户端之间是否可以互联？

答案是可以的，有一种情况叫TCP同时打开。

大家可以对比下，TCP同时打开在握手时的状态变化，跟TCP自连接是非常的像。

比如SYN_SENT状态下，又收到了一个SYN，其实就相当于自连接里，在发出了第一次握手后，又收到了第一次握手的请求。结果都是变成 SYN_RCVD。

在 SYN_RCVD 状态下收到了 SYN+ACK，就相当于自连接里，在发出第二次握手后，又收到第二次握手的请求，结果都是变成 ESTABLISHED。他们的源码其实都是同一块逻辑。

复现TCP同时打开

分别在两个控制台下，分别执行下面两行命令。

while true; do nc -p 2224 127.0.0.1 2223 -v;done

while true; do nc -p 2223 127.0.0.1 2224 -v;done

上面两个命令的含义也比较简单，两个客户端互相请求连接对方的端口号，如果失败了则不停重试。

执行后看到的现象是，一开始会疯狂失败，重试。一段时间后，连接建立完成。

# netstat -an | grep  2223
Proto Recv-Q Send-Q Local Address           Foreign Address         State 
tcp        0      0 127.0.0.1:2224          127.0.0.1:2223          ESTABLISHED
tcp        0      0 127.0.0.1:2223          127.0.0.1:2224          ESTABLISHED

期间抓包获得下面的结果。

可以看到，这里面建立连接用了四次交互。因此可以说这是通过"四次握手"建立的连接。

而且更重要的是，这里面只涉及两个客户端，没有服务端。

看到这里，不知道大家有没有跟我一样，被刷新了一波认知，对socket有了重新的认识。

在以前的观念里，建立连接，必须要有一个客户端和一个服务端，并且服务端还要执行一个listen()和一个accept()。而实际上，这些都不是必须的。

Then next time, the interviewer asks you "Can TCP establish a connection without listen()?", I think everyone should know how to answer.

But the problem comes again. There are only two clients and no listen(). Why can a TCP connection be established?

If you are interested, we will have the opportunity to fill in this hole again in the future.

Summary

• Waving four times, regardless of whether the program is actively executed close(), or the process is killed, it is possible to send the first wave FIN packet. If there are too many FIN-WAIT-2 states on the machine, it is usually because the peer has not executed the close() method to issue the third wave.

• Close() will turn off the function of sending and receiving messages at the same time. shutdown() Can shut down sending or receiving messages individually.

• It is possible that the second and third waves can be combined. So four waves turned into three waves.

• If the same socket connects to itself, TCP self-connection will occur. The waving of the self-connection is twice waving.

• A connection can be established between two clients without listen. This situation is called TCP opens simultaneously, which is caused by four-way handshake.

Finally

mentioned today, regardless of waving twice , or self-connection, or TCP open at the same time or something.

At first glance, it may not be useful for daily brick-moving, and in fact it is indeed useless.

And there is a high probability that it will not be asked during the interview.

After all, most interviewers don’t care how many ways the word fennel can be written.

The purpose of this article is mainly to let everyone re-understand socket from another angle. It turns out that TCP can connect itself, and even two clients can be connected without a server.

This is really, really unexpected.