Solution to the problem of Golang TCP sticking and unpacking

What is the sticky wrap problem

I recently wrote the Socket layer using Golang and found that sometimes the receiver reads multiple packets at once. So through looking up information, found that this is the legendary TCP sticky packet problem. Here's how to reproduce the problem by writing code:

Server code server/ main.go

func main() {
	l, err := net.Listen("tcp", ":4044")
	if err != nil {
		panic(err)
	}
	fmt.Println("listen to 4044")
	for {
  //  Listen for a new connection and create a new one  goroutine  to  handleConn function   To deal with 
		conn, err := l.Accept()
		if err != nil {
			fmt.Println("conn err:", err)
		} else {
			go handleConn(conn)
		}
	}
}

func handleConn(conn net.Conn) {
	defer conn.Close()
	defer fmt.Println(" Shut down ")
	fmt.Println(" A new connection: ", conn.RemoteAddr())

	result := bytes.NewBuffer(nil)
	var buf [1024]byte
	for {
		n, err := conn.Read(buf[0:])
		result.Write(buf[0:n])
		if err != nil {
			if err == io.EOF {
				continue
			} else {
				fmt.Println("read err:", err)
				break
			}
		} else {
			fmt.Println("recv:", result.String())
		}
		result.Reset()
	}
}

Client code client/ main.go

func main() {
	data := []byte("[ Here is the 1 A complete packet ]")
	conn, err := net.DialTimeout("tcp", "localhost:4044", time.Second*30)
	if err != nil {
		fmt.Printf("connect failed, err : %v\n", err.Error())
  return
	}
	for i := 0; i <1000; i++ {
		_, err = conn.Write(data)
		if err != nil {
			fmt.Printf("write failed , err : %v\n", err)
			break
		}
	}
}

The results

[

listen to 4044
New connection: [::1]:53079
recv: [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here before Is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete packet] [here is a complete data & # 65533;
recv: & # 65533; ] [Here's a complete packet][here's a complete packet][here's a complete packet][Here's a complete packet][Here's a complete packet]
recv: [here is the complete packet]
recv: [here is the complete packet]
recv: [here's a complete packet][here's a complete packet][here's a complete packet]
recv: [here is the complete packet]
. Omit the rest...

]

As you can see from the console output on the server side, there are three types of output:

One is the normal output of 1 packet. One is that multiple packets "stick" to each other from 1. We define such packets as sticky packets. One is that a packet is "unpacked" to form a broken packet, which is defined as half packet.

Why are there half-packs and sticky packets?

Client 1 sent packets at too much speed in a period of time, and the server did not complete all the processing. The data is then backlogged, creating sticky packets. The defined READ buffer is not large enough, and the packet is too large or due to sticky packets, the server cannot read all the packets at one time, resulting in half packets.

When do you need to consider handling semi-packs and sticky packets?

An TCP connection is a long connection, where data is sent multiple times per connection.
The data sent each time is structured, such as JSON format data or packet protocol defined by ourselves (packet header contains actual data length, protocol magic number, etc.).

solution

Fixed length separation (maximum for each packet, filled with special characters when insufficient), but wasted transmission resources when insufficient data Specific characters are used to split the packet, but Bug occurs if the data contains a split character It is recommended to add length field in the packet to make up for the shortage of the above two ideas

Unpacking the demonstration

Through the above analysis, we had better through the third way of thinking to solve the problem of unpacking sticky package.

Scanner is provided for us in the bufio library to solve this kind of split data problem.

[

type Scanner
Scanner provides a convenient interface for reading data such as a file of newline-delimited lines of text. Successive calls to the Scan method will step through the 'tokens' of a file, skipping the bytes between the tokens. The specification of a token is defined by a split function of type SplitFunc; the default split function breaks the input into lines with line termination stripped. Split functions are defined in this package for scanning a file into lines, bytes, UTF-8-encoded runes, and space-delimited words. The client may instead provide a custom split function.

]

To put it simply:

Scanner provides a convenient interface for reading data. Successive calls to the Scan method get the "tokens" of the file one by one, skipping the bytes between tokens. The specification for token is defined by functions of type SplitFunc. Instead, we can provide custom split functionality.

Let's take a look at what an SplitFunc type of function looks like:

type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)

Examples of Golang's official website documentation 🌰 :

func main() {
	// An artificial input source.
	const input = "1234 5678 1234567901234567890"
	scanner := bufio.NewScanner(strings.NewReader(input))
	// Create a custom split function by wrapping the existing ScanWords function.
	split := func(data []byte, atEOF bool) (advance int, token []byte, err error) {
		advance, token, err = bufio.ScanWords(data, atEOF)
		if err == nil && token != nil {
			_, err = strconv.ParseInt(string(token), 10, 32)
		}
		return
	}
	// Set the split function for the scanning operation.
	scanner.Split(split)
	// Validate the input
	for scanner.Scan() {
		fmt.Printf("%s\n", scanner.Text())
	}

	if err := scanner.Err(); err != nil {
		fmt.Printf("Invalid input: %s", err)
	}
}

So, we can rewrite our program like this:

Server code server/ ES106en.ES107en

func main() {
	l, err := net.Listen("tcp", ":4044")
	if err != nil {
		panic(err)
	}
	fmt.Println("listen to 4044")
	for {
		conn, err := l.Accept()
		if err != nil {
			fmt.Println("conn err:", err)
		} else {
			go handleConn2(conn)
		}
	}
}

func packetSlitFunc(data []byte, atEOF bool) (advance int, token []byte, err error) {
  //  check  atEOF  parameter   and   Packet header 4 Whether one byte   for  0x123456( We define the magic number of the protocol )
	if !atEOF && len(data) > 6 && binary.BigEndian.Uint32(data[:4]) == 0x123456 {
		var l int16
    //  read   In the packet   The actual data   The length of the ( Size of the  0 ~ 2^16)
		binary.Read(bytes.NewReader(data[4:6]), binary.BigEndian, &l)
		pl := int(l) + 6
		if pl <= len(data) {
			return pl, data[:pl], nil
		}
	}
	return
}

func handleConn2(conn net.Conn) {
	defer conn.Close()
	defer fmt.Println(" Shut down ")
	fmt.Println(" A new connection: ", conn.RemoteAddr())
	result := bytes.NewBuffer(nil)
  var buf [65542]byte //  Due to the   Identify packet length   There are only two bytes   Therefore, the packet size is maximum  2^16+4( The magic number )+2( The length of the logo )
	for {
		n, err := conn.Read(buf[0:])
		result.Write(buf[0:n])
		if err != nil {
			if err == io.EOF {
				continue
			} else {
				fmt.Println("read err:", err)
				break
			}
		} else {
			scanner := bufio.NewScanner(result)
			scanner.Split(packetSlitFunc)
			for scanner.Scan() {
				fmt.Println("recv:", string(scanner.Bytes()[6:]))
			}
		}
		result.Reset()
	}
}

Client code client/ ES113en.go

func main() {
	l, err := net.Listen("tcp", ":4044")
	if err != nil {
		panic(err)
	}
	fmt.Println("listen to 4044")
	for {
		conn, err := l.Accept()
		if err != nil {
			fmt.Println("conn err:", err)
		} else {
			go handleConn2(conn)
		}
	}
}

func packetSlitFunc(data []byte, atEOF bool) (advance int, token []byte, err error) {
  //  check  atEOF  parameter   and   Packet header 4 Whether one byte   for  0x123456( We define the magic number of the protocol )
	if !atEOF && len(data) > 6 && binary.BigEndian.Uint32(data[:4]) == 0x123456 {
		var l int16
    //  read   In the packet   The actual data   The length of the ( Size of the  0 ~ 2^16)
		binary.Read(bytes.NewReader(data[4:6]), binary.BigEndian, &l)
		pl := int(l) + 6
		if pl <= len(data) {
			return pl, data[:pl], nil
		}
	}
	return
}

func handleConn2(conn net.Conn) {
	defer conn.Close()
	defer fmt.Println(" Shut down ")
	fmt.Println(" A new connection: ", conn.RemoteAddr())
	result := bytes.NewBuffer(nil)
  var buf [65542]byte //  Due to the   Identify packet length   There are only two bytes   Therefore, the packet size is maximum  2^16+4( The magic number )+2( The length of the logo )
	for {
		n, err := conn.Read(buf[0:])
		result.Write(buf[0:n])
		if err != nil {
			if err == io.EOF {
				continue
			} else {
				fmt.Println("read err:", err)
				break
			}
		} else {
			scanner := bufio.NewScanner(result)
			scanner.Split(packetSlitFunc)
			for scanner.Scan() {
				fmt.Println("recv:", string(scanner.Bytes()[6:]))
			}
		}
		result.Reset()
	}
}

The results

[

listen to 4044
New connection: [::1]:55738
recv: [here is the complete packet]
recv: [here is the complete packet]
recv: [here is the complete packet]
recv: [here is the complete packet]
recv: [here is the complete packet]
recv: [here is a complete packet]
recv: [here is the complete packet]
recv: [here is the complete packet]
. Omit the rest...

]

conclusion