Skip to content

A micro-benchmark that compares some Haskell serialization libraries

License

Notifications You must be signed in to change notification settings

haskell-perf/serialization

Repository files navigation

Benchmarks for Haskell serialization libraries.

For every library, we measure the serialization time, deserialization time and size of the encoded output for a set of test datasets.

Summary Results

The summary tables list, for every test, all libraries that are within 30% of the best result (with best library listed first).

Transfer Time

To merge all measures in a concrete use case, we estimate the total transfer time at different transmission speeds, that's to say the the time that it takes to serialize, move across a network (with no compression and no protocol overheads) and deserialize a given dataset.

Dataset\Measure transfer [10 MBits] transfer [100 MBits] transfer [1000 MBits]
BinTree Direction flat flat flat,persist,store
BinTree Int flat flat flat,store,persist
Cars flat flat flat,store,persist
Iris flat,store,packman,persist,cereal,serialise store,flat,persist store,flat
[Direction] flat flat,persist flat,persist,store

When the network speed is low, transfer times are dominated by the size of the encoded dataset. At higher speeds, by the (de)serialisation times.

Tests are executed on an Intel Xeon W3520 @ 2.67GHz. On a slower machine, de(serialization) times would matter more, on a faster machine size would be more significant.

(De)serialization Times and Size

Dataset\Measure deserialization serialization size
BinTree Direction flat,persist,store persist,store flat
BinTree Int store,persist,flat,cereal flat,persist,store flat
Cars persist,flat,store store flat
Iris store,persist,flat store packman,flat,cereal,persist,store,serialise
[Direction] flat,persist,cereal,store persist,store,flat flat

Full Results

Full Results

Tested Libraries

Performance is not the only relevant property, depending on your needs you should also consider other features like laziness and compatibility.

Package Laziness Compatibility
store-0.5.0.1 Strict Haskell-Same-Architecture
packman-0.5.0 Lazy Haskell-Same-Architecture
cereal-0.5.7.0 Strict Haskell
binary-0.8.5.1 Lazy Haskell
serialise-0.2.1.0 Lazy Multi-Language
flat-0.3.4 Strict Multi-Language
persist-0.1 Strict Haskell

Compatibility Levels (lowest to highest):

  • Haskell-Same-Architecture
    • Compatible across Haskell systems sharing the same CPU and endianness
  • Haskell
    • Compatible across Haskell systems
  • Multi-Language
    • Compatible across different programming languages

Test Data

Test Description
BinTree Direction Binary Tree of a simple enumeration data type
BinTree Int Binary Tree of Ints
[Direction] A List of a simple enumeration data type
Cars A dataset of Cars descriptions (mostly lists, enumerations and Ints)
Iris A dataset of Iris descriptions (mostly lists and floats)

Shout if you would like other tests to be added!

Running the Benchmarks

Run the benchmarks with:

stack bench :all

If you get this error:

...<stdout>: commitBuffer: invalid argument (invalid character)

Try:

export LC_ALL=C.UTF-8

The executable will write the report.md, report.html and report.json files containing the full test results and will print out the two summary results tables in markdown format.

Tests will be compiled with GHC 8.4.4.

About

A micro-benchmark that compares some Haskell serialization libraries

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published