DEV Community

Blacksmoke16
Blacksmoke16

Posted on

oq - A portable/performant jq wrapper Part 2

#jq #json #python #go

OQ Revisited

Over a year ago I published a blog post introducing my jq wrapper oq. Given it has been quite a while since then I decided to rerun the benchmarks I did in the original post with the goal of seeing if things changed, both in regards to oq itself and the greater ecosystem.

Methodology

I tested the latest versions of the packages in the first post. I also added the Go version of yq.

The tested packages, and versions, for the benchmarks include:

  • jq (1.6) - Installed via pacman -S jq
  • yq (go) (4.6.1) - Downloaded latest binary from the latest GH Release
  • yq (python) (2.12.0) - Installed via pip3 install yq
  • oq (1.2.0) - Built locally with Crystal 0.36.1 via shards build --production --release

Each package was tested against the same 3 benchmarks in my original post, plus an additional benchmark for XML => JSON, mainly since yq (python) has a dedicated command for handling XML input/output that I didn't account for in the first post. The data is gathered via the GNU Time command. E.g. /usr/bin/time -v.

Setup

OS: 5.10.15-1-MANJARO x86_64
CPU: Intel i7-7700k
Memory: 32GB @ 3,000 MHz
SSD: Samsung 970 EVO Plus - 1TB

Results

This section summarizes the results, with key metrics in table form. See the appendix for the raw benchmark data.

Simple

Simply doing a pass through JSON => JSON then counting the lines on a pretty small input file:

{ "guests": [ { "name": "Jim", "age": 17, "numbers": [ 1, 2, 3 ] }, { "name": "Bob", "age": 51, "numbers": [ 4, 5, 6 ] }, { "name": "Susan", "age": 85, "numbers": [ 7, 8, 9 ] } ] }
Enter fullscreen mode Exit fullscreen mode
Name Max Memory (MB) Wall Clock Time (seconds)
jq 3.04 0.02
yq (go) 6.02 0.01
yq (python) 14.728 0.07
oq 6.536 0.07

All packages performed relatively equally, nothing noteworthy.

Jeopardy.json

Getting the length of a 53MB JSON file.

The file used: jeopardy.json.

Name Max Memory (MB) Wall Clock Time (seconds)
jq 230.32 0.66
yq (go) 705.292 2.54
yq (python) 2,922.996 108.89
oq 230.304 0.74

This test resulted in some more useful data, causing the true performance of each package to be more obvious. jq and oq fared the best, being pretty comparable in both memory usage and execution time. yq (go) was the next best, using 3x the memory and took ~3.5x longer than oq. yq (python) was by far the worse; using 12.7x the memory and taking 147x longer than oq.

YAML => XML/JSON

Consuming a ~57MB YAML file, and outputting XML and JSON.

The file used: sde/bsd/invItems.yaml from the EVE Online SDE Export.

Name Max Memory (MB) Wall Clock Time (seconds)
jq N/A N/A
yq (go) - JSON 5,202.032 32.77
yq (python) - JSON 5,742.736 206.2
yq (python) - XML 5,742.704 217.17
oq - JSON 575.6 14.01
oq - JSON - SimpleYAML 334.632 14.16
oq - XML 649.984 15.77
oq - XML - SimpleYAML 334.504 15.83

The final test showed similar metrics as the previous one. jq doesn't support non JSON formats so it was excluded. oq did the best with the execution time being fairly stable, while the memory usage was halved when using the new SimpleYAML format for the input. Interestingly yq (go) fared better execution wise in this test, being only 2.34x slower, but using ~15.5x more memory. yq (python) once again was the worse, with memory usage slightly worse than yq (go), but with an execution time of 15.5x slower than oq.

XML => JSON

Consume the output of the last benchmark, and convert it back to JSON. This test is mainly to not be biased as yq (python) includes a dedicated sub-package xq for handling XML input/output.

Name Max Memory (MB) Wall Clock Time (seconds)
jq N/A N/A
yq (go) N/A N/A
yq (python) 795.06 15.59
oq 2600.964 20.28

Neither jq or yq (go) support XML input, so they were excluded. The dedicated sub-package paid off for yq (python), beating oq by a fair margin in both memory consumption and execution time. It looks like it's able to stream the XML input, while XML input for oq is the only input format that cannot be streamed (yet).

Conclusion

It seems the performance of yq (python) has gotten a bit better since last time. Shaving off over 2GB of memory and 2 minutes of execution time in the jeopardy.json benchmark. However, it still is by far the slowest, and most memory hungry package (when not using XML as the input format). yq (go) is in a better spot, but still has pretty high memory usage depending on the exact use case. oq seems to still be the most efficient overall, with both memory and execution time being on par with jq. Thanks to its focus on streaming of data when possible (all but XML input at the moment); oq is able to use much less memory comparatively.

Appendix

Simple

jq 

Command being timed: "jq . data.json | wc -l" User time (seconds): 0.02 System time (seconds): 0.00 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 3040 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 288 Voluntary context switches: 1 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 31
Enter fullscreen mode Exit fullscreen mode

yq (go) 

Command being timed: "./go-yq -Pj e . data.json | wc -l" User time (seconds): 0.01 System time (seconds): 0.00 Percent of CPU this job got: 105% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.01 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 6020 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 617 Voluntary context switches: 76 Involuntary context switches: 2 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 31
Enter fullscreen mode Exit fullscreen mode

Using the -j and -P options to replicate the default behavior of the other binaries.

yq (python) 

Command being timed: "yq . data.json | wc -l" User time (seconds): 0.06 System time (seconds): 0.00 Percent of CPU this job got: 87% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.07 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 14728 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 4 Minor (reclaiming a frame) page faults: 3637 Voluntary context switches: 43 Involuntary context switches: 1 Swaps: 0 File system inputs: 1328 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 31
Enter fullscreen mode Exit fullscreen mode

oq 

Command being timed: "oq . data.json | wc -l" User time (seconds): 0.06 System time (seconds): 0.01 Percent of CPU this job got: 104% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.07 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 6536 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 808 Voluntary context switches: 58 Involuntary context switches: 2 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 31
Enter fullscreen mode Exit fullscreen mode

Jeopardy.json

jq 

216930 Command being timed: "jq length jeopardy.json" User time (seconds): 0.59 System time (seconds): 0.06 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.66 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 230320 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 59454 Voluntary context switches: 1 Involuntary context switches: 26 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

yq (go) 

216930 Command being timed: "./go-yq e length jeopardy.json" User time (seconds): 3.41 System time (seconds): 0.22 Percent of CPU this job got: 143% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.54 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 705292 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 176999 Voluntary context switches: 787 Involuntary context switches: 163 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

yq (python) 

216930 Command being timed: "yq length jeopardy.json" User time (seconds): 108.62 System time (seconds): 0.86 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 1:48.89 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 2922996 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 816903 Voluntary context switches: 13512 Involuntary context switches: 586 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

oq 

216930 Command being timed: "oq length jeopardy.json" User time (seconds): 0.70 System time (seconds): 0.12 Percent of CPU this job got: 110% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.74 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 230304 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 59966 Voluntary context switches: 13569 Involuntary context switches: 7 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

YAML => XML/JSON

jq

N/A

yq (go)

JSON 
Command being timed: "./go-yq -Pj e . invItems.yaml > invItems.go-yq.json" User time (seconds): 47.66 System time (seconds): 1.46 Percent of CPU this job got: 149% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:32.77 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 5202032 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 1395040 Voluntary context switches: 37244 Involuntary context switches: 1922 Swaps: 0 File system inputs: 0 File system outputs: 138984 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

yq (python)

JSON 
Command being timed: "yq . invItems.yaml > invItems.python-yq.json" User time (seconds): 205.35 System time (seconds): 1.82 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 3:26.20 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 5742736 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 1587436 Voluntary context switches: 13384 Involuntary context switches: 1836 Swaps: 0 File system inputs: 0 File system outputs: 138984 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode
XML 
Command being timed: "yq -s -x --xml-root items --xml-dtd {"item": .[] | .} invItems.yaml > invItems.python-yq.xml" User time (seconds): 215.17 System time (seconds): 2.03 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 3:37.17 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 5742704 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 1683522 Voluntary context switches: 32842 Involuntary context switches: 1031 Swaps: 0 File system inputs: 0 File system outputs: 195072 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

oq

JSON 
Command being timed: "./oq -i yaml . invItems.yaml > invItems.oq.json" User time (seconds): 12.90 System time (seconds): 12.24 Percent of CPU this job got: 179% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:14.01 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 575600 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 231391 Voluntary context switches: 289799 Involuntary context switches: 166 Swaps: 0 File system inputs: 0 File system outputs: 138984 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode 
Command being timed: "./oq -i simpleyaml . invItems.yaml > invItems.oq.simple.json" User time (seconds): 12.27 System time (seconds): 12.18 Percent of CPU this job got: 172% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:14.16 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 334632 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 89229 Voluntary context switches: 2981745 Involuntary context switches: 246 Swaps: 0 File system inputs: 0 File system outputs: 138984 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode
XML 
Command being timed: "./oq -i yaml -o xml --xml-root items . invItems.yaml" User time (seconds): 16.00 System time (seconds): 12.04 Percent of CPU this job got: 177% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:15.77 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 649984 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 249933 Voluntary context switches: 381174 Involuntary context switches: 266 Swaps: 0 File system inputs: 0 File system outputs: 195072 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

Example output:

<?xml version="1.0" encoding="utf-8"?> <items> <item> <flagID>0</flagID> <itemID>0</itemID> <locationID>0</locationID> <ownerID>0</ownerID> <quantity>-1</quantity> <typeID>0</typeID> </item> <item> <flagID>0</flagID> <itemID>1</itemID> <locationID>0</locationID> <ownerID>0</ownerID> <quantity>-1</quantity> <typeID>0</typeID> </item> ... </items>
Enter fullscreen mode Exit fullscreen mode 
Command being timed: "oq -i simpleyaml -o xml --xml-root items . invItems.yaml" User time (seconds): 15.27 System time (seconds): 11.99 Percent of CPU this job got: 172% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:15.83 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 334504 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 89233 Voluntary context switches: 3029485 Involuntary context switches: 298 Swaps: 0 File system inputs: 0 File system outputs: 195072 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

XML => JSON

jq

N/A

yq (go)

N/A

yq (python) 

Command being timed: "xq .items invItems.python-yq.xml > invItems.python-yq.xml.json" User time (seconds): 16.37 System time (seconds): 0.39 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:15.59 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 795060 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 341577 Voluntary context switches: 14927 Involuntary context switches: 87 Swaps: 0 File system inputs: 0 File system outputs: 168064 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

oq 

Command being timed: "./oq -i xml .items invItems.oq.xml > invItems.oq.xml.json" User time (seconds): 20.25 System time (seconds): 16.87 Percent of CPU this job got: 183% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:20.28 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 2600964 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 766666 Voluntary context switches: 2117666 Involuntary context switches: 1249 Swaps: 0 File system inputs: 0 File system outputs: 168064 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
Enter fullscreen mode Exit fullscreen mode

Top comments (0)