In the repository, I demo how to use protocol buffers. The protocol buffers is an efficient storing approach. It basically consists of three parts.

• The protocol is defined in a .proto file.
• The file stores the data in the binary format.
• The script can be generated by the tool protoc for different targets, like Python, Java, C++, etc.

It is a standard defined and released by Google. It is common to compare between protocol buffers (protobuf) and JSON for the reason of transmission and storing data.

For the definition, you can simply create a file with extension .proto. The following content is an example.

syntax = "proto2"; package tutorial; message Person { optional string name = 1; optional int32 id = 2; optional string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { optional string number = 1; optional PhoneType type = 2 [default = HOME]; } repeated PhoneNumber phones = 4; } message AddressBook { repeated Person people = 1; } 

• The version of protobuf is necessary, and it is defined by syntax.
• Use message to define the data object.
• Use enum to define the similiar type.
• Use repeated to note this field can be repeated many times. The optional represents the field can be set or not. The required means the field is necessary. Notice that the reqired fields are necessary even if you edit the protocol definition.
• The number is the identity for the attribute, it should not be resued while editing the protocol file.

Next you can generate the script or code based on your platforms, like Python, C++, etc. In the first you have to install the protobuf compiler. Generally, you can install it by one of the following approaches.

• Building from the scratch.

sudo apt-get install libprotobuf-dev git clone https://github.com/google/protobuf cd protobuf ./autogen.sh ./configure make check sudo make install sudo ldconfig

• Installing the pre-compiled version

sudo apt update sudo apt install protobuf-compiler

After that, you can check the installation by protoc --version. If the installation is complete, you can now generate the script or code. Here I will demo generating the Python script.

protoc --python_out=. *.proto

After that, you will get the *.py script. You can use them in your script like import *.py in Python.

After you import the module generated, you can manipulate it to generate the binary file. Before generating the binary file, you have to install necessary Python packages.

python3 -m virtualenv -p python3 env source ./en/bin/activate pip3 install --no-cache-dir -r ./requirements.txt

You now can start programming.

# for example import addressbook_pb2

You can edit the main.py and run it. The basic flow is like the following script.

def AddPerson(address_book_path): # use the pre-generated Python script ADDRESS_BOOK = addressbook_pb2.AddressBook() try: # read the existing address book serialized data with open(address_book_path, "rb") as fin: ADDRESS_BOOK.ParseFromString(fin.read()) except Exception as err: print("Failed in loading the address book {}.".format(address_book_path)) print("Can't load the address book. Create a new now.") PromptForPerson(ADDRESS_BOOK.people.add()) # write the serialized data to the local file with open(address_book_path, "wb") as fout: fout.write(ADDRESS_BOOK.SerializeToString())

After programming, you can run the command to generate the binary file.

""" Usage: python3 main.py <binary_file_path> <operation>  operation: add|list """ # add the data python3 main.py address_book add # retrieve the data python3 main.py address_book list