This is a set of memory and time benchmarks for the various Haskell effects libraries, including fused-effects, polysemy, freer-simple, and extensible-effects. Benchmarks for equivalent mtl and handwritten computations are included for comparison purposes. It uses Cabal's new Backpack feature to save a great deal of boilerplate code.
fused-effects and mtl achieve roughly commensurate performance. polysemy is within one to three orders of magnitude from fused-effects and mtl. freer-simple and extensible-effects are considerably slower.
On a 15-inch 2017 MacBook Pro (3.1GHz Intel i7, 16GB RAM), here are results for space benchmarks:
Countdown Case Allocated fused-effects 40 mtl 40 polysemy 1,304 freer-simple 1,096 extensible-effects 1,304 shallow 232 Countdown + exc Case Allocated GCs fused-effects 336 mtl 336 polysemy 792 freer-simple 864 extensible-effects 984 shallow 320 (The fact that fused-effects and mtl have the same allocations is not an error; fused-effects was designed to have almost identical performance characteristics to mtl.)
Time (State-based countdown):
benchmarked Countdown/Put/fused-effects time 5.692 μs (5.570 μs .. 5.835 μs) benchmarked Countdown/Put/mtl time 5.500 μs (5.386 μs .. 5.719 μs) benchmarked Countdown/Put/polysemy time 1.903 ms (1.845 ms .. 1.954 ms) benchmarked Countdown/Put/freer-simple time 1.324 ms (1.282 ms .. 1.361 ms) benchmarked Countdown/Put/extensible-effects time 1.383 ms (1.343 ms .. 1.422 ms) benchmarked Countdown/Put/shallow time 275.1 μs (270.1 μs .. 281.6 μs) Time (State + Error countdown):
benchmarked Countdown/Put+Exc/fused-effects time 9.829 μs (9.645 μs .. 10.04 μs) benchmarked Countdown/Put+Exc/mtl time 9.968 μs (9.779 μs .. 10.15 μs) benchmarked Countdown/Put+Exc/polysemy time 418.1 μs (408.9 μs .. 427.1 μs) benchmarked Countdown/Put+Exc/freer-simple time 1.157 ms (1.120 ms .. 1.199 ms) benchmarked Countdown/Put+Exc/extensible-effects time 1.232 ms (1.212 ms .. 1.262 ms) benchmarked Countdown/Put+Exc/shallow time 55.12 μs (54.34 μs .. 56.29 μs) Time (custom HTTP DSL):
benchmarked HTTP/fused-effects time 8.311 μs (8.060 μs .. 8.592 μs) benchmarked HTTP/polysemy time 14.99 ms (14.26 ms .. 15.80 ms) benchmarked HTTP/extensible-effects time 1.612 ms (1.578 ms .. 1.654 ms) benchmarked HTTP/Deep embedding time 2.027 ms (1.876 ms .. 2.184 ms) benchmarked HTTP/Shallow embedding time 15.94 μs (15.49 μs .. 16.46 μs) benchmarked HTTP/freer-simple time 1.478 ms (1.443 ms .. 1.515 ms) - More benchmarks:
Writer, logging,grammar,pipes. - I am going to make this emit pretty graphs, or at least graphable .csv files, someday.