How to Design Message Files (Proto) for gRPC Services in. NET Core

Directory How to Design Messages for gRPC Services in. NET Core Messages and C # Classes Arrays and dictionaries Manage change Efficiency and limitations

How to Design Messages for gRPC Services in. NET Core

Defining an gRPC service using the protocol buffer specification is fairly easy, but there are a few things to be aware of when moving from requirements to. NET Core and then managing the evolution of the service.

At the heart of creating the gRPC service is the. proto file, which describes the service in a language-independent format. Using the. proto file, Visual Studio can generate a base class for your service (you only need to write business-specific code), or you can generate client classes for reliable access to the service.

The. proto file must conform to the protocol buffer specification of Google (commonly referred to as ProtoBuf). The contents of the original file allow you to specify the interface of the service. The service interface consists of two parts:

The method provided by your gRPC service Data structure of parameters and return values of these methods

You can use the scalar types defined in [1] of the Protocol Buffers specification to construct these data structures (referred to as "messages" in ProtoBuf). Available types include Boolean values, strings, byte arrays, and various numeric types (floating-point, integer, and long). There is no date or fixed decimal type. In the next column, I will show you how to add timestamp types. For decimals, you can use float... with the precision loss of float.

If you want to start a new project, use the proto3 syntax since 2016. However, you must explicitly specify the proto3 standard on line 1 "non-empty" of the. proto file. Reference specification [2]), otherwise your. proto file will be parsed using the proto2 specification. Specify that your file looks like this using proto3:

syntax = "proto3";

Messages and C # Classes

Using the proto3 specification, the message format for customer information might look like this:

message CustomerResponse {
  int32 custid = 1;
  string firstName = 2;
  string lastName = 3;
  int32 age = 4;
  fixed32 creditLimit = 5;
}

The number after the equal sign specifies the position of the field in the message, starting at Position 1 (in my example, firstName will be the second field in the message). These numbers must be 1-only in the message (that is, you cannot use two fields at the same 1 position). You don't have to list the fields in numeric order, but if you do, you can find duplicate field numbers more easily (although Visual Studio will find any duplicate numbers and report them in the Error List when you build your application). You can also skip positions if necessary. This definition uses only odd numbers, such as:

message CustomerResponse {
  int32 custid = 1;
  string firstName = 3;
  string lastName = 5;
}

In. NET Core, the message format is converted to a class, and each field becomes an attribute of the class with the same name as the message. When naming these properties,. NET Core also converts the first character of the field name to uppercase. So, for example, the custId field in my previous example will become the CustId property on the CustomerResponse class in my code.

In this process, you also remove all underscores in the field name and capitalize the following letters (that is, the Last_name field name becomes the LastName attribute).

The process also involves mapping. NET types to ProtoBuf types (for example, ProtoBuf int32 becomes. NET int, int64 of ProtoBuf becomes long, and fixed32 becomes uint), which requires adding one new class to. NET Core. For example, ProtoBuf supports byte arrays of type bytes. The new. NET data type named ByteString supports this field type. To load ByteString, use the static CopyFrom method of the ByteString class and pass a byte array as follows:

byte[] bytes = new byte[1000];
cr.Valid = ByteString.CopyFrom(bytes);

To retrieve an array of bytes from ByteString, use the object's CopyTo method and pass the array to which you want to copy bytes and the starting position:

cr.Valid.CopyTo(bytes,0);

Arrays and dictionaries

You can also use the keyword "repeated" to include collections in the definition (in ProtoBuf, fields that are not collections are called "singular"). If my customer message requires 1 set of duplicate transaction amounts, you can specify the following fields:

message Customer {
   int32 id = 1;
   repeated fixed32 transactionAmounts = 4;

Duplicate fields also use the new type: Google. Protobuf. RepeatedField when they are converted to properties of the class. For example, my example generates an Google. Protobuf. RepeatedField (unsigned integer) property. You can initialize arrays using the {} syntax, as follows:

CustomerResponse cr = new CustomerResponse
            {
                CreditLimit = {10, 15, 100}    
            };

You are more likely to use its various Add methods to put items into the collection:

cr.CreditLimit.Add ( 200 ) ;

You can use the LINQ method (such as First ()) or access items in RepeatedField by location. It can work normally, for example:

uint tranAmount = cr.CreditLimit [1];

ProtoBuf also supports the Dictionary-type collection called map, which allows you to specify types for dictionary keys and values. My customer messages may use "friendly names" to track the customer's various credit cards to define a dictionary that contains a string of keys ("Petercard", "My Travel Card") and values (credit card numbers):

message CustomerResponse {
  int32 custId = 1;
  map<string, string> cards = 2;

Interestingly, in the Visual Studio 2019 preview, the editor does not highlight map objects like other Type 1 (although it compiles well).

The corresponding property will be of type Google. Protobuf. Collections. MapField, which you can load by passing its Add method to the key and a value, just like any other Dictionary1.

Manage change

It is relatively easy to change the. proto file after going online (the client starts to use it). For example, you can add a field with a new location number to the. proto file used by server-side software without disturbing clients that are still using earlier versions of the file: the client simply ignores fields that are not listed in its. proto file.

Similarly, in the opposite case (when the server. proto file does not have the fields that the client's. proto field has), the client will only find that the server unsent property is set to its default value. Incidentally, fields defined in the server's. proto file that are not defined in the client's. proto file are still sent to the client, but. NET does not provide a convenient way to access it (at least not yet).

Indeed, as the service evolves and modifies its. proto file, you should follow only two rules:

Do not change the location number of an existing field Do not reclaim the position number (i.e. do not replace the outdated field 3 with the new field 3)

However, the property generated from the. proto file cannot be null, so if the property is not set to a value, it will be set to its default value. This means that the number is set to 0; The number is set to 0. Set string to string. Empty (a zero-length string); Boolean becomes false; The ByteString property defaults to an ByteString object, and its IsEmpty property is set to true; And the RepeatedField and MapField properties default to their corresponding objects, each object contains no items, and its Count property is set to 0.

Because of this behavior, there is a risk of deleting fields from the service's. proto file and not updating all clients (or simply not setting properties on objects when the response is generated on the server). The danger is that clients cannot distinguish between unused fields and properties that have been set to their default values. If the My Client Valid property is set to false, the client will not be able to determine whether the client is invalid or whether the server will no longer generate the field.

You may want to consider initializing the property to some "unreasonable" value (for example, the number is-1) so that clients can distinguish between the property set to the default value and the deleted field. Because this is not possible for Boolean values (there is no unreasonable value for Boolean values), you should be especially wary of deleting (or even no longer using) Boolean-type fields.

Efficiency and limitations

As I discussed [4] in my earlier overview [3], one of the features of gRPC services is that their messages are much smaller than those of HTTP-based (RESTful) services. If you really want to take advantage of this efficiency, note that positions 1 to 15 require only one byte of overhead (that is, data beyond the stored value), while positions 16 to 2047 require two bytes. It seems like a good idea to keep the message format below 16 bits.

For additional efficiency tips on the selection of types to package data into the smallest possible space, see the Scalar Type Description in the specification [5].

By the way, you cannot use any of the following as field position numbers: negative numbers, 0, 19,000 to 19,999 (reserved for ProtoBuf) or numbers greater than 536,870,911. May I also suggest that if you want to use these numbers, you will encounter problems that I can't solve in this column.

Really. Don't do that.

These are the details of how to design message files for gRPC services in. NET Core (Proto). For more information about designing message files for gRPC services in. NET Core, please pay attention to other related articles on this site!