Understanding Golang component protobuf

What is protobuf

protocol buffers is a language independent, platform independent and extensible method for serializing structured data. It can be used for (data) communication protocol, data storage, etc. It is a flexible, efficient and automatic structural data serialization method - comparable to XML, but smaller (3-10 times), faster (20-100 times) and simpler than XML.

Difference between protobuf and json

JSON and Protobuf can be used for information exchange. JSON is a simple message format, which is transmitted in text mode, while Protobuf is transmitted in binary mode. Compared with JSON, the message volume will be significantly reduced, so the transmission speed is faster than JSON. In addition, Protobuf is not only a message format for exchange, but also a rule and tool for defining exchange messages. At present, Protobuf basically supports all mainstream languages.


Install from the command line first

go get -u github.com/golang/protobuf/protoc-gen-go

Create another file called test.proto, and type

syntax = "proto3";
package main;

message Test {
    string label = 1;
    int32 type = 2;
    repeated int64 reps = 3;

We take proto 3 as an example, create a message called Test, set three attributes, label, type and int64, and repeat the generated Go language variable type as slice. Now execute protoc on the command line to generate the Go file.

protoc --go_out=./ test.proto

You can see that a file named test.pb.go was generated in the root directory

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: test.proto

package main

import (
	fmt "fmt"
	proto "github.com/golang/protobuf/proto"
	math "math"

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type Test struct {
	Label                string   `protobuf:"bytes,1,opt,name=label,proto3" json:"label,omitempty"`
	Type                 int32    `protobuf:"varint,2,opt,name=type,proto3" json:"type,omitempty"`
	Reps                 []int64  `protobuf:"varint,3,rep,packed,name=reps,proto3" json:"reps,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`

func (m *Test) Reset()         { *m = Test{} }
func (m *Test) String() string { return proto.CompactTextString(m) }
func (*Test) ProtoMessage()    {}
func (*Test) Descriptor() ([]byte, []int) {
	return fileDescriptor_c161fcfdc0c3ff1e, []int{0}

func (m *Test) XXX_Unmarshal(b []byte) error {
	return xxx_messageInfo_Test.Unmarshal(m, b)
func (m *Test) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
	return xxx_messageInfo_Test.Marshal(b, m, deterministic)
func (m *Test) XXX_Merge(src proto.Message) {
	xxx_messageInfo_Test.Merge(m, src)
func (m *Test) XXX_Size() int {
	return xxx_messageInfo_Test.Size(m)
func (m *Test) XXX_DiscardUnknown() {

var xxx_messageInfo_Test proto.InternalMessageInfo

func (m *Test) GetLabel() string {
	if m != nil {
		return m.Label
	return ""

func (m *Test) GetType() int32 {
	if m != nil {
		return m.Type
	return 0

func (m *Test) GetReps() []int64 {
	if m != nil {
		return m.Reps
	return nil

func init() {
	proto.RegisterType((*Test)(nil), "main.Test")

func init() {
	proto.RegisterFile("test.proto", fileDescriptor_c161fcfdc0c3ff1e)

var fileDescriptor_c161fcfdc0c3ff1e = []byte{
	// 104 bytes of a gzipped FileDescriptorProto
	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x2a, 0x49, 0x2d, 0x2e,
	0xd1, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0xc9, 0x4d, 0xcc, 0xcc, 0x53, 0x72, 0xe1, 0x62,
	0x09, 0x49, 0x2d, 0x2e, 0x11, 0x12, 0xe1, 0x62, 0xcd, 0x49, 0x4c, 0x4a, 0xcd, 0x91, 0x60, 0x54,
	0x60, 0xd4, 0xe0, 0x0c, 0x82, 0x70, 0x84, 0x84, 0xb8, 0x58, 0x4a, 0x2a, 0x0b, 0x52, 0x25, 0x98,
	0x14, 0x18, 0x35, 0x58, 0x83, 0xc0, 0x6c, 0x90, 0x58, 0x51, 0x6a, 0x41, 0xb1, 0x04, 0xb3, 0x02,
	0xb3, 0x06, 0x73, 0x10, 0x98, 0x9d, 0xc4, 0x06, 0x36, 0xd2, 0x18, 0x10, 0x00, 0x00, 0xff, 0xff,
	0x7b, 0xa2, 0xc6, 0x01, 0x60, 0x00, 0x00, 0x00,

We do the serialization test in the main file

package main

import (


func main() {
	test := &Test{
		Label: "a",
		Type:  32,
		Reps:  []int64{10, 11},
	resp, err := proto.Marshal(test)
	if err != nil {

protobuf generates a structure named Test, which has three member attributes, which exactly correspond to the test.proto file. Executing the poroto.Marshal method can serialize the structure, and then it can be transmitted with the support of RPC or HTTP.

Golang Component sample code warehouse, welcome star


Tags: Go JSON github xml

Posted on Thu, 26 Mar 2020 12:50:09 -0400 by ethan.whitt