Introduction to protobuf
Protobuf is a binary serialization format, which is smaller in size and faster in transmission than json.
Install protobuf
The purpose of installing protobuf is mainly to compile proto files into python, c, Java callable interfaces.
# 如果gcc版本较低，需要升级gcc
wget https://main.qcloudimg.com/raw/d7810aaf8b3073fbbc9d4049c21532aa/protobuf-2.6.1.tar.gz
tar -zxvf protobuf-2.6.1.tar.gz -C /usr/local/ && cd /usr/local/protobuf-2.6.1
./configure
make && make install
# 可以在/etc/profile或者~/.bash_profile末尾设置永久有效
export PATH=$PATH:/usr/local/protobuf-2.6.1/bin
Use the following command to check whether the installation is successful.
[root@CodeOnTheRoad ~]# protoc --version
libprotoc 2.6.1
build python interface
Create a cls.proto file to define the serialization structure:
package cls;
message Log
{
message Content
{
required string key = 1; // 每组字段的 key
required string value = 2; // 每组字段的 value
}
required int64 time = 1; // 时间戳，UNIX时间格式
repeated Content contents = 2; // 一条日志里的多个kv组合
}
message LogTag
{
required string key = 1;
required string value = 2;
}
message LogGroup
{
repeated Log logs = 1; // 多条日志合成的日志数组
optional string contextFlow = 2; // 目前暂无效用
optional string filename = 3; // 日志文件名
optional string source = 4; // 日志来源，一般使用机器IP
repeated LogTag logTags = 5;
}
message LogGroupList
{
repeated LogGroup logGroupList = 1; // 日志组列表
}
Just use the following command to convert the proto file to a python callable interface.
protoc cls.proto --python_out=./
After execution, cls_pb2.py will be generated in this directory .
Serialization
import cls_pb2 as cls
import time
# 构建protoBuf日志内容
LogLogGroupList = cls.LogGroupList()
LogGroup = LogLogGroupList.logGroupList.add()
LogGroup.contextFlow = "1"
LogGroup.filename = "python.log"
LogGroup.source = "localhost"
LogTag = LogGroup.logTags.add()
LogTag.key = "key"
LogTag.value = "value"
Log = LogGroup.logs.add()
Log.time = int(round(time.time() * 1000000))
Content = Log.contents.add()
Content.key = "Hello"
Content.value = "World"
print(LogLogGroupList)
# 序列化
data = LogLogGroupList.SerializeToString()
print(data)
In fact, it means that a protobuf structure text is serialized into binary form.
deserialization
Deserialization is to convert binary into protobuf structure.
# 反序列化
LogLogGroupList = cls.LogGroupList()
LogLogGroupList.ParseFromString(data)
print(LogLogGroupList)
operation result
The above serialization and deserialization code results run as follows:
Post-95 young programmers, write about personal practice in daily work, from the perspective of beginners, write from 0 to 1, detailed and serious. The article will be published on the public account [ Getting Started to Give Up Road ], looking forward to your attention.
