Communication details between PHP and Go languages

2020-06-03 06:43:39
OfStack

preface

In a recent work, there was a scenario in the php project where you needed a third party function and there happened to be a library written in Golang. So the question is, how do you communicate between different languages? Let's start with 1.

Conventional plan

1. Write an http/TCP service with Golang, and php communicates with Golang via http/TCP

2, Golang through more packaging, as an extension of php.

3. PHP retrieves the executable file of Golang through the system command

Existing problems

1, http request, network I/O will consume a lot of time

2. Need to encapsulate a lot of code

3. Every time PHP calls the Golang program, it needs to be initialized once, which consumes a lot of time

The optimization goal

1. Golang program is initialized only once (because initialization is time-consuming)

2, all requests do not need to go to the network

3. Try not to change the code too much

The solution

1, simple Golang packaging, the third party library compiled into an executable file

2. PHP and Golang communicate through two-way pipeline

The advantage of using two-way pipe communication

1: Minimal encapsulation is required for the existing Golang class library

2: Best performance (IPC communication is the best way to communicate between processes)

3: No need to go to the network request, save a lot of time

4: the program only needs to be initialized once, and 1 is kept in memory

Specific implementation steps

1: The original call to demo from the class library


  package main
  import (
   "fmt"
   "github.com/yanyiwu/gojieba"
   "strings"
  )

  func main() {
   x := gojieba.NewJieba()
   defer x.Free()

   s := " Xiao Ming graduated from institute of Computing Technology, Chinese Academy of Sciences and then studied at Kyoto University in Japan "
   words := x.CutForSearch(s, true)
   fmt.Println(strings.Join(words, "/"))
  }

Save the file as main.go and you are ready to run

2. The adjusted code is:


  package main
  import (
   "bufio"
   "fmt"
   "github.com/yanyiwu/gojieba"
   "io"
   "os"
   "strings"
  )

  func main() {

   x := gojieba.NewJieba(
    "/data/tmp/jiebaDict/jieba.dict.utf8", 
    "/data/tmp/jiebaDict/hmm_model.utf8", 
    "/data/tmp/jiebaDict/user.dict.utf8"
   )
   defer x.Free()

   inputReader := bufio.NewReader(os.Stdin)
   for {
    s, err := inputReader.ReadString('\n')
    if err != nil && err == io.EOF {
     break
    }
    s = strings.TrimSpace(s)

    if s != "" {
     words := x.CutForSearch(s, true)
     fmt.Println(strings.Join(words, " "))
    } else {
     fmt.Println("get empty \n")
    }
   }
  }

With a few simple line adjustments, you receive a string from standard input, then output it by word segmentation

Testing:


 # go build test
 # ./test
 # // Wait for the user to enter, enter "this is 1 A test" 
 #  This is a  1 a   test  // The program

3. Use cat to communicate with Golang for simple test


 // To prepare 1 a title.txt Each line is 1 Words of text 
 # cat title.txt | ./test

Normal output indicates that cat is ready to interact with Golang

4: PHP communicates with Golang

cat, shown above, communicates with Golang using a one-way pipeline. That is: data can only be passed in from cat to Golang, and data output from Golang is not returned to cat, but directly to the screen. But the requirement is that php communicate with Golang. That is, php has to pass data to Golang, and Golang also has to return the execution result to php. Therefore, two-way piping needs to be introduced.

Use of pipes in PHP: popen("/path/test") , specific will not expand to say, because this method can not solve the problem in the text.

Two-way pipeline:


  $descriptorspec = array( 
   0 => array("pipe", "r"), 
   1 => array("pipe", "w")
  );
  $handle = proc_open(
   '/webroot/go/src/test/test', 
   $descriptorspec, 
   $pipes
  );
  fwrite($pipes['0'], " This is a 1 Three test texts \n");
  echo fgets($pipes[1]);

Explanation: use proc_open Open a process and call the Golang program. Also returns an pipes array of two-way pipes, php direction $pipe['0'] Write data from $pipe['1'] Medium read data.

Well, as you may have noticed, I'm the title file, and it's not just about how PHP communicates with Golang. I'm talking about a way to communicate in any language through two-way pipes. (All languages implement pipeline-related content)

Testing:

By comparison test, the time occupied by each process is calculated. The title.txt file mentioned below contains 1 million lines of text, each line of text taken from the b2b platform

1: The whole process takes time


time cat title.txt | ./test > /dev/null

Time: 14.819 seconds, including:

Process cat reads the text

The data is piped into Golang

Golang processes the data and returns the results to the screen

2: It takes time to calculate the segmentation function. Solution: Remove the call of the participle function, i.e. comment out the line of the call participle in the Golang source code


time cat title.txt | ./test > /dev/null

Time: 1.817 seconds, including:

Process cat reads the text

The data is piped into Golang

Golang processes the data and returns the results to the screen

Time spent on segmentation = (time spent on step 1) - (time spent on the above command)

Time: 14.819-1.817 = 13.002 seconds

3: Test the time spent communicating between the cat process and the Golang process


time cat title.txt > /dev/null

Time: 0.015 seconds, including:

Process cat reads the text

The data is piped into Golang

go processes the data and returns the results to the screen

Pipeline communication time :(step 2 time) - (step 3 time)

Pipeline communication time: 1.817-0.015 = 1.802 seconds

4: Time consumption of COMMUNICATION between PHP and Golang

Write a simple php file:


  <?php
   $descriptorspec = array( 
    0 => array("pipe", "r"), 
    1 => array("pipe", "w")
   );

   $handle = proc_open(
    '/webroot/go/src/test/test', 
    $descriptorspec, 
    $pipes
   );

   $fp = fopen("title.txt", "rb");

   while (!feof($fp)) {
    fwrite($pipes['0'], trim(fgets($fp))."\n");
    echo fgets($pipes[1]);
   }

   fclose($pipes['0']);
   fclose($pipes['1']);
   proc_close($handle);

The flow is the same as the above basic 1, read title. txt content, pass the Golang process segmentation through two-way pipe, and then return to php (one more step than the above test: data is returned through the pipe)


time php popen.php > /dev/null

Time: 24.037 seconds, including:

Process PHP reads the text

The data is piped into Golang

Golang processes data

Golang returns the result and writes to the pipe, through which PHP receives the data

Returns the result to the screen

Conclusion:

1. Time distribution in the whole segmentation process

Time consuming using cat control logic: 14.819 seconds

Time consuming using PHP control logic: 24.037 seconds (1 more pipe communication than cat)

One-way pipeline communication time: 1.8 seconds

The participle function in Golang takes 13.002 seconds

2: Performance of word segmentation function: single process, 1 million item heading word segmentation, time consuming 13 seconds

The above time only includes participle time, not dictionary loading time. However, in this scenario, the dictionary is loaded only once, so the loading time can be ignored (about 1 second).

3: PHP is slower than cat.

Language level slow: (24.037-1.8-14.819) / 14.819 = 50%

With single-process comparison testing, no language should be faster than cat.