Python Serialization Benchmark

This repository maintains a set of benchmarks for python serialization frameworks.

You can find the latest benchmarks on this page.

Currently the following projects are benchmarked.

Along with a baseline custom function that doesn't use a framework.

Running the test suite

A Docker container is bundled with the repository which you can use to run the benchmarks. Firstly make sure you have Docker installed.

  1. Install Docker

  2. Build the container $ docker-compose build

  3. Run the tests. $ docker-compose run --rm tests

Library Many Objects (seconds) One Object (seconds) Relative --------------------- ------------------------ ---------------------- ---------- serpyco 0.00796103 0.00404453 1 Custom 0.009902 0.00483012 1.22711 lima 0.012424 0.00625205 1.55562 Pickle 0.0180094 0.0179398 2.99438 serpy 0.0356748 0.0189779 4.55228 Strainer 0.0529356 0.0261641 6.58858 Toasted Marshmallow 0.0865085 0.0465157 11.0802 Colander 0.238111 0.116309 29.5214 Lollipop 0.32454 0.156491 40.0673 Avro 0.442609 0.220602 55.2419 Marshmallow 0.496984 0.242603 61.6036 kim 0.616623 0.307922 77.0097 Django REST Framework 0.761326 0.527056 107.315

The Benchmark

Each framework is asked to serialize a list of 2 objects a 1000 times, and then 1 object a 1000 times.

This is the current object that is being serialized.

class ChildTestObject(object): def __init__(self, multiplier=None): self.w = 1000 * multiplier if multiplier else 100 self.x = 20 * multiplier if multiplier else 20 self.y = 'hello' * multiplier if multiplier else 'hello' self.z = 10 * multiplier if multiplier else 10 class ParentTestObject(object): def __init__(self): self.foo = 'bar' self.sub = ChildTestObject() self.subs = [ChildTestObject(i) for i in xrange(10)] def bar(self): return 5 benchmark_object = ParentTestObject()

Discussion

Serialization from python objects to JSON, XML, or other transmission formats is a common task for many web related projects. In order to fill that need a number of frameworks have arised. While their aims are similar, they don't all share the same attributes. Here are how some of the features comapre.

Project Serialization Encoding Deserialization Validation
Django REST Framework Yes Yes Yes Yes
serpy Yes No No No
Marshmallow Yes Yes Yes Yes
Lollipop Yes No Yes Yes
Strainer Yes No Yes Yes
Kim Yes No Yes Yes
serpyco Yes Yes Yes Yes
Toasted Marshmallow Yes Yes Yes Yes
Colander Yes No Yes< Yes
Lima Yes No No No
Avro Yes Yes Yes No