Add documentation on how to run benchmarks locally

[UnknownPlatypus]

Hi !

First thanks for the project, this is really impressive, I'm really enjoying it and really like the design decisions you made.

I was curious and wanted to run the benchmarks locally. On the way I added a README.md with small instruction on how to do so. I figured it might be useful to others, hence this PR.

I've also moved the CONTRIBUTING.md file to the top level to make it more discoverable, I almost missed it. Let me know if it's fine for you.

For what it's worth, here are the results I got running them on my dell laptop (XPS 15 9530 - 13th Gen Intel i7-13700H @ 4.800GHz) using CPython 3.11.

python -m benchmarks.bench_validation

• msgspec: 0.19.0
• mashumaro: 3.15
• cattrs: 24.1.2
• pydantic: 2.10.4

+-------------+-------------+------+-------------+-----+------------+-----+--------------+------+ | | encode (μs) | vs. | decode (μs) | vs. | total (μs) | vs. | memory (MiB) | vs. | +=============+=============+======+=============+=====+============+=====+==============+======+ | msgspec | 113.2 | 1.0 | 310.3 | 1.0 | 423.5 | 1.0 | 1.1 | 1.0 | +-------------+-------------+------+-------------+-----+------------+-----+--------------+------+ | mashumaro | 581.2 | 5.1 | 2129.3 | 6.9 | 2710.5 | 6.4 | 3.1 | 2.8 | +-------------+-------------+------+-------------+-----+------------+-----+--------------+------+ | pydantic v2 | 1595.9 | 14.1 | 1359.9 | 4.4 | 2955.8 | 7.0 | 16.9 | 15.4 | +-------------+-------------+------+-------------+-----+------------+-----+--------------+------+ | cattrs | 783.9 | 6.9 | 2624.4 | 8.5 | 3408.3 | 8.0 | 3.3 | 3.0 | +-------------+-------------+------+-------------+-----+------------+-----+--------------+------+ 

python benchmarks/bench_encodings.py --protocol json

• msgspec: 0.19.0
• orjson: 3.10.13
• ujson: 5.10.0
• rapidjson: 1.20
• simdjson: 6.0.2

+-----------------+-------------+-----+-------------+-----+------------+-----+ | | encode (μs) | vs. | decode (μs) | vs. | total (μs) | vs. | +=================+=============+=====+=============+=====+============+=====+ | msgspec structs | 100.9 | 1.0 | 279.5 | 1.0 | 380.4 | 1.0 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | orjson | 119.6 | 1.2 | 355.1 | 1.3 | 474.7 | 1.2 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | msgspec | 131.4 | 1.3 | 353.1 | 1.3 | 484.5 | 1.3 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | ujson | 471.5 | 4.7 | 618.8 | 2.2 | 1090.3 | 2.9 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | rapidjson | 382.8 | 3.8 | 758.1 | 2.7 | 1140.9 | 3.0 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | simdjson | 813.2 | 8.1 | 532.7 | 1.9 | 1345.9 | 3.5 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | json | 809.4 | 8.0 | 619.0 | 2.2 | 1428.3 | 3.8 | +-----------------+-------------+-----+-------------+-----+------------+-----+ 

python benchmarks/bench_encodings.py --protocol msgpack

• msgspec: 0.19.0
• msgpack: 1.1.0
• ormsgpack: 1.7.0

+-----------------+-------------+-----+-------------+-----+------------+-----+ | | encode (μs) | vs. | decode (μs) | vs. | total (μs) | vs. | +=================+=============+=====+=============+=====+============+=====+ | msgspec structs | 103.8 | 1.0 | 245.6 | 1.0 | 349.4 | 1.0 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | msgspec | 98.1 | 0.9 | 358.9 | 1.5 | 457.0 | 1.3 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | ormsgpack | 99.8 | 1.0 | 382.9 | 1.6 | 482.8 | 1.4 | +-----------------+-------------+-----+-------------+-----+------------+-----+ | msgpack | 285.2 | 2.7 | 562.2 | 2.3 | 847.4 | 2.4 | +-----------------+-------------+-----+-------------+-----+------------+-----+ 

python benchmarks/bench_large_json.py

• msgspec: 0.19.0
• ujson: 5.10.0
• orjson: 3.10.13
• python-rapidjson: 1.20
• pysimdjson: 6.0.2

+---------------------+--------------+-------+-----------+------+ | | memory (MiB) | vs. | time (ms) | vs. | +=====================+==============+=======+===========+======+ | **msgspec structs** | 43.7 | 1.0x | 160.4 | 1.0x | +---------------------+--------------+-------+-----------+------+ | **msgspec** | 326.4 | 7.5x | 760.1 | 4.7x | +---------------------+--------------+-------+-----------+------+ | **json** | 443.3 | 10.1x | 973.3 | 6.1x | +---------------------+--------------+-------+-----------+------+ | **ujson** | 515.8 | 11.8x | 1140.0 | 7.1x | +---------------------+--------------+-------+-----------+------+ | **rapidjson** | 555.7 | 12.7x | 1192.7 | 7.4x | +---------------------+--------------+-------+-----------+------+ | **orjson** | 595.7 | 13.6x | 814.6 | 5.1x | +---------------------+--------------+-------+-----------+------+ | **simdjson** | 875.8 | 20.0x | 1167.9 | 7.3x | +---------------------+--------------+-------+-----------+------+ 

python benchmarks/bench_structs.py

• msgspec: 0.19.0
• attrs: 24.3.0
• pydantic: 2.10.4

+----------------------+-------------+-------------+---------------+------------+ | | import (μs) | create (μs) | equality (μs) | order (μs) | +======================+=============+=============+===============+============+ | **msgspec** | 9.23 | 0.06 | 0.01 | 0.02 | +----------------------+-------------+-------------+---------------+------------+ | **standard classes** | 5.98 | 0.25 | 0.05 | 0.11 | +----------------------+-------------+-------------+---------------+------------+ | **attrs** | 371.38 | 0.25 | 0.04 | 1.51 | +----------------------+-------------+-------------+---------------+------------+ | **dataclasses** | 342.43 | 0.25 | 0.09 | 0.10 | +----------------------+-------------+-------------+---------------+------------+ | **pydantic** | 335.05 | 0.96 | 0.86 | N/A | +----------------------+-------------+-------------+---------------+------------+ 

python benchmarks/bench_gc.py

+-----------------------------------+--------------+-------------------+ | | GC time (ms) | Memory Used (MiB) | +===================================+==============+===================+ | **standard class** | 56.75 | 211.66 | +-----------------------------------+--------------+-------------------+ | **standard class with __slots__** | 51.63 | 120.11 | +-----------------------------------+--------------+-------------------+ | **msgspec struct** | 10.93 | 120.11 | +-----------------------------------+--------------+-------------------+ | **msgspec struct with gc=False** | 1.99 | 104.85 | +-----------------------------------+--------------+-------------------+ 

python benchmarks/bench_library_size.py

+--------------+---------+------------+-------------+ | | version | size (MiB) | vs. msgspec | +==============+=========+============+=============+ | **msgspec** | 0.19.0 | 0.48 | 1.00x | +--------------+---------+------------+-------------+ | **pydantic** | 2.10.4 | 6.38 | 13.28x | +--------------+---------+------------+-------------+ 

[UnknownPlatypus]

Doest not make sense anymore now that #919 landed