How to Read Data in Chunks from a `net.Conn` in Go?

Reading Data in Chunks with Go's net.Conn.Read

In Go, you can access raw network connections using net.Conn.Read, which reads incoming data packets into a byte array. However, when you don't know the exact size of the incoming data, reading into a fixed-size array can lead to data truncation or unnecessary buffering.

To handle this, you can employ a more flexible approach using the bufio package or alternative techniques.

Using bufio

The bufio package provides the Reader type, which allows you to read data in chunks. You can create a Reader object from your net.Conn connection, and then use the ReadSlice or ReadBytes methods to read data until a specific delimiter is encountered or the end of the input is reached. For instance, to read data until the end of the packet, you could use the following code:

package main

import (
    "bufio"
    "fmt"
    "net"
)

func main() {
    conn, err := net.Dial("tcp", "google.com:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()

    // Create a bufio.Reader from the net.Conn
    reader := bufio.NewReader(conn)

    // Read data in chunks until the end of the packet
    buf := []byte{}
    for {
        chunk, err := reader.ReadSlice('\n')
        if err != nil {
            if err != io.EOF {
                fmt.Println("read error:", err)
            }
            break
        }
        buf = append(buf, chunk...)
    }
    fmt.Println("total size:", len(buf))
    // fmt.Println(string(buf))
}

Alternative Approach

Alternatively, you can use the bytes.Buffer type to append incoming data chunks and accumulate the total size:

package main

import (
    "bytes"
    "fmt"
    "io"
    "net"
)

func main() {
    conn, err := net.Dial("tcp", "google.com:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()

    // Create a bytes.Buffer to accumulate incoming data
    var buf bytes.Buffer

    // Copy data from the net.Conn to the Buffer
    if _, err = io.Copy(&buf, conn); err != nil {
        fmt.Println("copy error:", err)
    }

    fmt.Println("total size:", buf.Len())
}

By using one of these approaches, you can handle varying data lengths without data truncation or excessive buffering, ensuring efficient data transfer over network connections in Go.

The above is the detailed content of How to Read Data in Chunks from a `net.Conn` in Go?. For more information, please follow other related articles on the PHP Chinese website!