Aeson Object Design (Part 4)

I had to get one more task done before I can pop this Aeson issue off of the stack of topics that are occupying my thoughts: some benchmarks. In Aeson Object Design (Part 2), I explored the cost of representing objects with various data structures. Today, I created the benchmarks for the normal scenario. Note that I do not plan on benchmarking the attack scenario since the length of the list of pairs can be used to mitigate attacks, as described in Aeson Object Design (Part 3).

The benchmark code can be found on GitHub.

Update: The initial code used stack script in an attempt to put everything in a single file. Thank you very much to Taylor Fausak for pointing out that stack script executes code with runghc, which affects the benchmarks! I am correcting the mistake here, but be sure to also check out Taylor’s benchmarks.

• File bench.hs runs the benchmarks and creates two files with the results: report.html and report.csv.
• File stats.hs reads the report.csv file and creates stats.csv, which lists the mean processing time of each data type for various sizes.

Note that I am not committing the report files to the repository because they are large, but I am committing the stats.csv results from the benchmarks that I ran.

As can be seen in the chart below, I measured very similar performance for all types with small numbers of pairs.

• Lists marginally outperformed Vector for objects with 1 pair.
• Lists marginally outperformed Map for objects with 6 or fewer pairs.
• Lists marginally outperformed HashMap for objects with 8 or fewer pairs.
• Vector marginally outperformed HashMap for objects with 17 or fewer
• Vector marginally outperformed Map for objects with 35 or fewer pairs.
• Map marginally outperformed HashMap for objects with 13 or fewer pairs.

Based on this data, I would get optimal performance on my laptop by using Vector (Text, Value) for objects with 17 or fewer pairs and using HashMap for objects with 18 or more pairs.