Aeson Object Design (Part 4)
I had to get one more task done before I can pop this Aeson issue off of the stack of topics that are occupying my thoughts: some benchmarks. In Aeson Object Design (Part 2), I explored the cost of representing objects with various data structures. Today, I created the benchmarks for the normal scenario. Note that I do not plan on benchmarking the attack scenario since the length of the list of pairs can be used to mitigate attacks, as described in Aeson Object Design (Part 3).
The benchmark code can be found on GitHub.
Update: The initial code used stack script
in an attempt
to put everything in a single file. Thank you very much to Taylor Fausak for pointing
out that stack script
executes code with
runghc
, which affects the benchmarks! I am correcting the
mistake here, but be sure to also check out Taylor’s benchmarks.
- File
bench.hs
runs the benchmarks and creates two files with the results:report.html
andreport.csv
. - File
stats.hs
reads thereport.csv
file and createsstats.csv
, which lists the mean processing time of each data type for various sizes.
Note that I am not committing the report files to the repository
because they are large, but I am committing the stats.csv
results from the benchmarks that I ran.
As can be seen in the chart below, I measured very similar performance for all types with small numbers of pairs.
- Lists marginally outperformed
Vector
for objects with 1 pair. - Lists marginally outperformed
Map
for objects with 6 or fewer pairs. - Lists marginally outperformed
HashMap
for objects with 8 or fewer pairs. Vector
marginally outperformedHashMap
for objects with 17 or fewerVector
marginally outperformedMap
for objects with 35 or fewer pairs.Map
marginally outperformedHashMap
for objects with 13 or fewer pairs.
Based on this data, I would get optimal performance on my laptop by
using Vector (Text, Value)
for objects with 17 or fewer
pairs and using HashMap
for objects with 18 or more
pairs.