gnome-continuous: triple-buffering deployments

[Owen Taylor <otaylor redhat com>]

[ Adding gnome-continuous-list to this, I forgot that we had a specific
list. See
https://mail.gnome.org/archives/gnome-os-list/2016-August/msg00021.html
 and https://mail.gnome.org/archives/gnome-os-list/2016-September/msg00
000.html for a related older thread. gnome-os-list removed from the
Reply-To: ]

By improving the builddisks step of gnome-continuous, we can go from
having an OSTree tree to having an updated VM of that OSTree in less
than a minute. But my end goal is using gnome-continuous for
development, not just for continuous integration/delivery. For that, a
minute is still a very long time. Is there some way that we can reduce
the time to just a few seconds? The biggest component (taking almost
half the time) of the update is the deployment step - just walking the
repository and writing out a new tree of hardlinks. The idea I had for
improving this is that if we have an old deployment that we assume is
correct and the new and old OSTree information, we can efficiently
convert that old deployment into a new deployment without any need to
look at the parts of the tree that haven't changed.

In detail, the way this would work is:

 * VM contains deployments of the last two versions with the
   full OSTree trees
 * We pull the next version
 * We take the older (non-booted) deploymentand clean it up by
   removing /etc
 * We apply the diff between version N-2 and version N to the old tree,
   creating a tree of version N
 * We finish the deployment with a new 3-way merge of /etc

Trying this out would take some non-trivial coding within ostree so I
wanted to run this by the list first. Has this been thought about?
Would it work? Did I miss something and it already works like this? ;-)

Thanks!
Owen

P.S. - in the development scenario, you could just do this within the
VM before rebooting, cutting out most of the remaining overhead and
hopefully achieving the "just a few seconds" goal. Generating the new
OSTree commit still would take an unacceptably long amount of time, but
I have some tricks in mind there too...