Protobuf messages are gaining popularity across many domains. There are libraries like nanopb in C to support the protobuf encoding/decoding. However they are slow and also pose significant challenge in implementation. GoLang protobuf encoding/decoding is fast and provides support of unmarshalling data from JSON to proto format.

We will be creating a C shared library containing functions that read from json file, protobuf encode and decode.

Full Code can be found here:

GoLang Code

Lets start with Student.proto. I am using proto3 for this example:

syntax="proto3"; package main; message Parents { string dad=1; string mom=2; } enum Gender { Female = 0; Male = 1; } message Student { string name = 1; int32 age = 2; Parents parents=3; Gender gender=4; } message Students { repeated bytes StudentEntry=1; } 

As you can see above proto includes enum, nested proto and repeated bytes. Student represents a student's data, each student's data will be encoded and added to Students StudentEntry and all the encoded entries will be encoded again.

Below is input JSON file student_list.json which is used to populate proto fields. We have to use the same names and nested levels for the unmarshal of Json to proto.

{ "0" : { "Name": "Elliot", "Age" : 15, "Parents" : { "Dad" : "George", "Mom" : "Elena" }, "Gender" : 0 }, "1" : { "Name": "David", "Age" : 14, "Parents" : { "Dad" : "Keith", "Mom" : "Sarah" }, "Gender" : 1 } } 

Encoding Protobuf from JSON

To integrate with C, library C is used to convert GoData to C datatypes. Comment export is used to export functions to C lib.
We will create a new GoLang program file student_en_dc.go and add encode and decode functions to it.
Json file is input parameter of this function get_student_enbuf passed from C. This function returns encoded buffer and its length to C.

//export get_student_enbuf func get_student_enbuf(student_json_file *C.char) (unsafe.Pointer,C.int) { . . return C.CBytes(students_en),C.int(int32(len(students_en))) 

First step in encoding is to Unmarshall JSON to proto. As there are more than one student entries, I have added entries using number keys. This cannot be directly unmarshalled, we would have to extract json data of each student and then unmarshal. I am first unmarshalling json file data to an interface. Below is the code:

jsonbytes, _ := ioutil.ReadAll(jsonFile) var parsed_proto map[string]interface{} json.Unmarshal(jsonbytes, &parsed_proto) 

Then parse each interface to obtain student json data and unmarshall it into our proto. Aftermath proto is encoded as shown.

 for i=0;i<len(parsed_proto);i++ { selectedjb, err := json.Marshal(parsed_proto[strconv.Itoa(i)]) if err != nil { log.Println("Error in reading json index",err) return nil,0 } var parsed_jf map[string]interface{} //Each student data to proto json.Unmarshal(selectedjb, &parsed_jf) student := &Student{} //Proto Encoding of student json.Unmarshal(selectedjb, student) 

Encoded data is added to StudentEntry of Students. Then Students data is encoded. We have the encoded buffer.

 students.StudentEntry=append(students.StudentEntry,student_en) } students_en, err := proto.Marshal(students) 

Decoding protobuf

Students is decoded first following all the student entries.

Function decode_student_enbuf takes encoded data and its length as input passed from C and returns decoded buffer to C.

//export decode_student_enbuf func decode_student_enbuf(data *C.char,leng C.int) *C.char { . . return C.CString(decoded_student) } 

Before decode, *char is converted to GoBytes and then unmarshalled to proto.

new_students := &Students{} student_en:=C.GoBytes(unsafe.Pointer(data),leng) err := proto.Unmarshal(student_en, new_students) 

Decoded students data is used to obtain each StudentEntry and is decoded. Decoded student is appended to a string and is returned to C.

for i=0;i<len(new_students.StudentEntry);i++ { new_student := &Student{} err = proto.Unmarshal(new_students.StudentEntry[i], new_student) decoded_student=decoded_student+"\n"+proto.MarshalTextString(new_student) 

Building C-Shared file

Lets compile proto to generate Student.pb.go.Then build c-shared file.

protoc --go_out=. Student.proto go build -o student.so -buildmode=c-shared Student.pb.go student_en_dc.go 

student.so and student.h files are generated.
Following lines would be present in the generated header file. Golang function get_student_enbuf returns tuple(more than one return parameters), this is converted to struct when header files are generated as seen below.

/* Return type for get_student_enbuf */ struct get_student_enbuf_return { void* r0; int r1; }; extern struct get_student_enbuf_return get_student_enbuf(char* p0); extern char* decode_student_enbuf(char* p0, int p1); 

C Code

From C side, generated header file is included. Students list JSON file is passed to function get_student_enbuf, returned encoded buffer and its length is populated in struct stu. The same are passed to decode function decode_student_enbuf, the returned decoded string is printed.
Finally both encoded and decoded buffers are freed.

#include <stdio.h> #include "student.h" #include <string.h> #include <stdlib.h> #include <stdbool.h> struct get_student_enbuf_return stu; int main() { stu = get_student_enbuf("student_list.json"); int stu_list_len = stu.r1; if(stu.r0!=NULL) { char *decoded_stu = decode_student_enbuf((char*)stu.r0,stu.r1); if(decoded_stu != NULL){ printf("%s",decoded_stu); free((char*)decoded_stu); } free((char*)stu.r0); } return 0; } 

Compiling and executing C

Link with the created student.so, create object student and run the object.

gcc -o student student_cgo.c ./student.so ./student name: "Elliot" age: 15 parents: < dad: "George" mom: "Elena" > name: "David" age: 14 parents: < dad: "Keith" mom: "Sarah" > gender: Male 

Full Code can be found here: