Summary of common methods for golang to read files

A variety of ways to read files using the go language.

Once loaded into memory

// *  The entire file reads into memory and is suitable for smaller files 
// Read a fixed byte at a time 
// The problem is easy to appear disorderly code, because Chinese and Chinese symbols do not occupy 1 A character 
func readAllIntoMemory(filename string) (content []byte, err error) {
 fp, err := os.Open(filename) //  Get file pointer 
 if err != nil {
 return nil, err
 }
 defer fp.Close()
 fileInfo, err := fp.Stat()
 if err != nil {
 return nil, err
 }
 buffer := make([]byte, fileInfo.Size())
 _, err = fp.Read(buffer) //  File contents read to buffer In the 
 if err != nil {
 return nil, err
 }
 return buffer, nil
}

One-time loading into memory is suitable for small files. If the file is too large and memory is tight, the buffer can be used to read multiple times.

reads

// * 1 block 1 Block to read ,  That is to 1 A buffer ,  Read into the buffer multiple times 
// The entire file is read to the buffer by bytes buffer
func readByBlock(filename string) (content []byte, err error) {
 fp, err := os.Open(filename) //  Get file pointer 
 if err != nil {
 return nil, err
 }
 defer fp.Close()
 const bufferSize = 64 //  The buffer size ,  Each read 64 bytes 
 buffer := make([]byte, bufferSize)
 for {
 //  Notice that we're taking theta bytesRead,  Otherwise there's a problem 
 bytesRead, err := fp.Read(buffer) //  File contents read to buffer In the 
 content = append(content, buffer[:bytesRead]...)
 if err != nil {
  if err == io.EOF {
  err = nil
  break
  } else {
  return nil, err
  }
 }
 }
 return
}

Sometimes we also need to do it on a row basis

According to the line read

//  Read line by line , 1 Line is 1 a []byte,  More line is [][]byte
func readByLine(filename string) (lines [][]byte, err error) {
 fp, err := os.Open(filename) //  Get file pointer 
 if err != nil {
 return nil, err
 }
 defer fp.Close()
 bufReader := bufio.NewReader(fp)
 for {
 line, _, err := bufReader.ReadLine() //  According to the line read 
 if err != nil {
  if err == io.EOF {
  err = nil
  break
  }
 } else {
  lines = append(lines, line)
 }
 }
 return
}

Read all the contents of the file using ioutil

func test1() {
 bytes,err := ioutil.ReadFile("filetoread.txt")
 if err != nil {
 log.Fatal(err)
 }
 fmt.Println("total bytes read : ",len(bytes))
 fmt.Println("string read:",string(bytes))
}

conclusion