[BuildStream] Weekly performance update

[tristan maat codethink co uk (Tristan Daniël Maat)]

Hi,

A little late this week - I had to wait for some benchmarking to finish
and figure out what was going on; We had some trouble with timing
consistency and moving the repository. Note that this week's results may
not perfectly match up with last weeks', they were re-generated from
scratch due to a slight change to the runners' OS.

James won't be contributing (or doing these mails) for a long while, so
I'll take the time to thank him for his contributions here, good luck
wherever things take you :)

Off to the weekly results:

---

Ipython notebook with the results:
https://mybinder.org/v2/gl/BuildStream%2Fbenchmarking%2Fbst-benchmarks-notebooks/master?filepath=all_results.ipynb

the results are hosted at:
https://hub.gke.mybinder.org/user/buildstream-ben-marks-notebooks-in380isy/notebooks/all_results.ipynb

Don't forget to run "Kernel" -> "Restart and Run all" to actually
generate the graphs.

Note: If the link has expired, you'll need to go to
https://mybinder.org/ and enter:
https://gitlab.com/BuildStream/benchmarking/bst-benchmarks-notebooks into the url
field, with branch "master" and file "all_results.ipynb". Then "launch".

---

All results were obtained on the same hardware (a Codethink dev
machine configured as a GitLab runner - not being used for anything
else), and the results files are being appended to by a CI job. Specs
as follows:

* Linux (Debian stretch)
* x86_64
* 16G RAM
* 500Gb SSD
* Intel i7-3770
* 8 cores @ 3.40 GHz

---

Here are the times and max-rss (peak memory usage) results for a `bst
show` of base-files/base-files.bst in the Debian-like project (branch:
jennis/add_all_files [1]) for the last 5 weeks:

┌────────────────┬─────────────────────────────────┬─────────────────────────────────┐
│                │              Show               │         Show once built         │
├────────────────┼────────────────┬────────────────┼────────────────┬────────────────┤
│     Dates      │    Time (s)    │  Memory (MiB)  │    Time (s)    │  Memory (MiB)  │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 12/08 -> 18/08 │ 6.67           │ 201.0          │ 6.75           │ 227.0          │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 19/08 -> 25/08 │ 6.33           │ 200.0          │ 6.56           │ 229.0          │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 26/08 -> 01/09 │ 6.28           │ 200.0          │ 6.64           │ 229.0          │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 02/09 -> 08/09 │ 6.63           │ 201.0          │ 6.84           │ 230.0          │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 11/09 -> 15/09 │ 6.49           │ 202.0          │ 6.81           │ 230.0          │
└────────────────┴────────────────┴────────────────┴────────────────┴────────────────┘

---

Here are the times and max-rss results for a `bst build` (4, 8 and 12
builders) of base-files/base-files.bst in the Debian-like project
(branch: jennis/add_all_files [1]):

┌────────────────┬─────────────────────────────────┬─────────────────────────────────┬─────────────────────────────────┐
│                │           4 Builders            │           8 Builders            │           12 Builders  
         │
├────────────────┼────────────────┬────────────────┼────────────────┬────────────────┼────────────────┬────────────────┤
│     Dates      │    Time (s)    │  Memory (MiB)  │    Time (s)    │  Memory (MiB)  │    Time (s)    │  
Memory (MiB)  │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 12/08 -> 18/08 │ 121.72         │ 224.0          │ 127.67         │ 224.0          │ 124.18         │ 224.0 
         │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 19/08 -> 25/08 │ 130.44         │ 227.0          │ 133.12         │ 227.0          │ 131.54         │ 227.0 
         │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 26/08 -> 01/09 │ 131.58         │ 227.0          │ 133.74         │ 227.0          │ 131.95         │ 227.0 
         │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 02/09 -> 08/09 │ 132.97         │ 228.0          │ 139.49         │ 228.0          │ 133.31         │ 228.0 
         │
├────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┼────────────────┤
│ 11/09 -> 15/09 │ 134.35         │ 228.0          │ 139.27         │ 228.0          │ 133.14         │ 228.0 
         │
└────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┴────────────────┘

---

Please see [2] if you're unable to open the weekly notebook locally.

[1] https://gitlab.com/BuildStream/benchmarking/debian-stretch-bst/tree/jennis/add_all_files
[2] https://mail.gnome.org/archives/buildstream-list/2019-February/msg00046.html

--

Regards,

Tristan Maat