Go Method steps for implementing the simple RPC framework

This paper aims to describe several core problems in RPC framework design and their solutions, and build a simple RPC framework based on Golang reflection technology.

Project address: ES7en-ES8en

RPC

RPC (Remote Procedure Call), namely remote procedure call, can be understood as, service A wants to call the function of service B which is not in the same memory space, but cannot be called directly because it is not in the same memory space, so it needs to express the semantics of the call and convey the data of the call through the network.

The service side

The RPC server needs to solve two problems:

Since the client is passing the RPC function name, how does the server maintain the mapping between the function name and the function entity How does the server implement the invocation of the corresponding function entity based on the function name

The core processes

Maintains function name to function mappings After receiving the function name and argument list from the client, the argument list is parsed as reflected values and the corresponding function is executed Code the result of the function execution and return it to the client

Methods registration

The server needs to maintain a mapping of RPC function names to RPC function entities, which we can use map Data structures to maintain mapping relationships.

type Server struct {
 addr string
 funcs map[string]reflect.Value
}

// Register a method via name
func (s *Server) Register(name string, f interface{}) {
 if _, ok := s.funcs[name]; ok {
 return
 }
 s.funcs[name] = reflect.ValueOf(f)
}

Execution call

Generally, when a client calls RPC, it sends the function name and argument list as request data to the server.

Because we used it map[string]reflect.Value To maintain the mapping between the function name and the function entity Value.Call() To call the function that corresponds to the function name.

package main

import (
 "fmt"
 "reflect"
)

func main() {
 // Register methods
 funcs := make(map[string]reflect.Value)
 funcs["add"] = reflect.ValueOf(add)

 // When receives client's request
 req := []reflect.Value{reflect.ValueOf(1), reflect.ValueOf(2)}
 vals := funcs["add"].Call(req)
 var rsp []interface{}
 for _, val := range vals {
 rsp = append(rsp, val.Interface())
 }

 fmt.Println(rsp)
}

func add(a, b int) (int, error) {
 return a + b, nil
}

The specific implementation

Due to space limitations, the server implementation of the specific code is not posted here, please see the project address for details.

The client

The RPC client needs to solve one problem:

Since the concrete implementation of the function is on the server side, the client side only has the prototype of the function. How does the client call its function entity through the prototype

The core processes

The function arguments passed by the caller are coded and passed to the server The server response data is decoded and returned to the caller

Generate calls

We can bind 1 function entity to the specified function prototype via ES74en.MakeFunc.

package main

import (
 "fmt"
 "reflect"
)

func main() {
 add := func(args []reflect.Value) []reflect.Value {
 result := args[0].Interface().(int) + args[1].Interface().(int)
 return []reflect.Value{reflect.ValueOf(result)}
 }

 var addptr func(int, int) int
 container := reflect.ValueOf(&addptr).Elem()
 v := reflect.MakeFunc(container.Type(), add)
 container.Set(v)

 fmt.Println(addptr(1, 2))
}

The specific implementation

Due to space limitations, the specific code of the client implementation is not posted here, please see the project address for details.

Data transfer format

We need to define the data format in which the server interacts with the client.

type Data struct {
 Name string    // service name
 Args []interface{} // request's or response's body except error
 Err string    // remote server error
}

Encoding and decoding functions corresponding to interactive data.

func encode(data Data) ([]byte, error) {
 var buf bytes.Buffer
 encoder := gob.NewEncoder(&buf)
 if err := encoder.Encode(data); err != nil {
 return nil, err
 }
 return buf.Bytes(), nil
}

func decode(b []byte) (Data, error) {
 buf := bytes.NewBuffer(b)
 decoder := gob.NewDecoder(buf)
 var data Data
 if err := decoder.Decode(&data); err != nil {
 return Data{}, err
 }
 return data, nil
}

At the same time, we need to define a simple TLV protocol (fixed-length header + variable-length body) to regulate the transmission of data.

// Transport struct
type Transport struct {
 conn net.Conn
}

// NewTransport creates a transport
func NewTransport(conn net.Conn) *Transport {
 return &Transport{conn}
}

// Send data
func (t *Transport) Send(req Data) error {
 b, err := encode(req) // Encode req into bytes
 if err != nil {
 return err
 }
 buf := make([]byte, 4+len(b))
 binary.BigEndian.PutUint32(buf[:4], uint32(len(b))) // Set Header field
 copy(buf[4:], b)                  // Set Data field
 _, err = t.conn.Write(buf)
 return err
}

// Receive data
func (t *Transport) Receive() (Data, error) {
 header := make([]byte, 4)
 _, err := io.ReadFull(t.conn, header)
 if err != nil {
 return Data{}, err
 }
 dataLen := binary.BigEndian.Uint32(header) // Read Header filed
 data := make([]byte, dataLen)       // Read Data Field
 _, err = io.ReadFull(t.conn, data)
 if err != nil {
 return Data{}, err
 }
 rsp, err := decode(data) // Decode rsp from bytes
 return rsp, err
}