[BuildStream] Weekly performance update - REBOOT

[James Ennis <james ennis codethink co uk>]

Hello,

Apologies for not sending these emails out over the last few weeks.
The CI started failing when I was away on holiday and in addition to
this the benchmarks machine was updated which skewed the results
slightly.

However, I saw this as a good opportunity to rewrite a lot of the
benchmarking repo and it is now much more reliable. Also, I've
back-generated the data since June and I intend to back-generate
further.

Also, rather than taking an average of the commits from the week, I will
just display the results from the last commit of the week in the weekly
tables.

-----

As before, a notebook containing *all* of the results should be
available via following link (note that this may take a seconds to
"build"):

https://mybinder.org/v2/gh/james-ennis/bst-benchmarks-notebooks/master?filepath=all_results.ipynb

When the notebook has loaded please click on
"Kernel" -> "Restart and Run all" to generate the graphs.

Note: If the link has expired, you'll need to go to
https://mybinder.org/ and enter:
https://github.com/james-ennis/bst-benchmarks-notebooks into the url
field, with branch "master" and file "all_results.ipynb". Then "launch".

-----

All results were obtained on the same hardware (a Codethink dev machine
configured as a GitLab runner - not being used for anything else), and
the results files are being appended to by a CI job. Specs as follows:

* Linux (Debian stretch)
* x86_64
* 16G RAM
* 500Gb SSD
* Intel i7-3770
* 8 cores @ 3.40 GHz

-----

Here are the times and max-rss (peak memory usage) results for a
`bst show` of base-files/base-files.bst in the Debian-like project
(branch: jennis/add_all_files [1]) for the last 5 weeks:

┌─────────┬─────────────────┬─────────────────┐
│         │      Show       │ Show once built │
├─────────┼───────┬─────────┼───────┬─────────┤
│Dates    │Time   │Memory   │Time   │Memory   │
│         │(s)    │(M)      │(s)    │(M)      │
├─────────┼───────┼─────────┼───────┼─────────┤
│15/07 -> │6.25   │198      │6.11   │223      │
│21/07    │       │         │       │         │
├─────────┼───────┼─────────┼───────┼─────────┤
│22/07 -> │6.51   │201      │6.39   │225      │
│28/07    │       │         │       │         │
├─────────┼───────┼─────────┼───────┼─────────┤
│29/07 -> │6.58   │201      │6.33   │225      │
│04/08    │       │         │       │         │
├─────────┼───────┼─────────┼───────┼─────────┤
│05/08 -> │6.57   │201      │6.79   │226      │
│11/08    │       │         │       │         │
├─────────┼───────┼─────────┼───────┼─────────┤
│12/08    │6.58   │201      │6.77   │226      │
│18/08    │       │         │       │         │
└─────────┴───────┴─────────┴───────┴─────────┘

-----

Here are the times and max-rss results for a `bst build`
(4, 8 and 12 builders) of base-files/base-files.bst in the Debian-like
project (branch: jennis/add_all_files [1]).

┌─────────┬─────────────────┬─────────────────┬──────────────────┐
│         │   4 Builders    │   8 Builders    │   12 Builders    │
├─────────┼───────┬─────────┼───────┬─────────┼───────┬──────────┤
│Date     │Time   │Memory   │Time   │Memory   │Time   │Memory    │
│         │(s)    │(M)      │(s)    │(M)      │(s)    │(M)       │
├─────────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤
│15/07 -> │128.20 │221      │130.18 │221      │126.27 │221       │
│21/07    │       │         │       │         │       │          │
├─────────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤
│22/07 -> │129.61 │223      │131.63 │223      │128.31 │223       │
│28/07    │       │         │       │         │       │          │
├─────────┼───────┼─────────┼───────┼─────────┼───────┼──────────│
│29/07 -> │130.06 │223      │133.03 │224      │129.37 │224       │
│04/08    │       │         │       │         │       │          │
├─────────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤
│05/08 -> │129.05 │223      │134.05 │224      │129.83 │224       │
│11/08    │       │         │       │         │       │          │
├─────────┼───────┼─────────┼───────┼─────────┼───────┼──────────┤
│24/06 -> │134.24 │223      │135.14 │224      │132.45 │224       │
│30/06    │       │         │       │         │       │          │
└─────────┴───────┴─────────┴───────┴─────────┴───────┴──────────┘

-----

Please see [2] if you're unable to open the weekly notebook locally.

Thanks,
James

[1] 
https://gitlab.com/jennis/debian-stretch-bst/tree/jennis/add_all_files
[2] 
https://mail.gnome.org/archives/buildstream-list/2019-February/msg00046.html