[BuildStream] Cache keys

[William Salmon <will salmon codethink co uk>]

Hi All

After some recent experiences I believe we still haven't got this right!

Here are some thought I have had after talking to Tom Mewett and Thomas 
Coldrick.

Some time a plugin changes in a fully braking way and we need to bump 
the version and change the cache key of every element built with or from 
the plugin. I have come to believe this to be true for bst unless we 
make some huge changes.

But often we add a feature to a plugin, (I have just added git-lfs 
support to the git_tag plugin)

This plugin will only change its behavior if its cache key is changed.

ie. The plugin only changes its behavior if you set `use-lfs: True` and 
if you set that then its unique key will change.

If we left the plugin version the same then only plugins who's behavior 
has changed will have there cache keys changed. But users of projects 
using the plugin but who do not have the new version will get unhelpful 
error messages when the plugin can not validate the `use-lfs: True` 
propriety

So I must bump the plugin version so that users get useful messages but 
I am also going to cause a huge amount of compute work as the project 
invalidates almost all of of its cache keys.

The reality is that compute is relatively cheap so this is not the end 
of the world and is unavoidable for many plugin changes.

But its still not inconsiderable.

My Idea...

Could we add `features` to our plugins cache keys as well as versions?

Plugin authors can already do this to some extent, by having a

source:
        kind: bob
        url: bob.com
        features:
            use-shinnys: True

Then just validate for features being a dict and if any of the keys are 
not recognized then raise a error.

and then

add 'use-shinnys' to the unique key only if it is used.

This way if a user uses `use-shinnys: True` but they have a old plugin 
then an error message can be generated at configure time saying that a 
newer plugin is needed. But this is not as helpful as the version number 
as the version number tells the user exactly which version of the plugin 
they need.

Another issue with this is that in order to deal with bugs in 
use-shinnys we really need some thing like:

source:
        kind: bob
        url: bob.com
        features:
            use-shinnys:
                use: True
                version: 1

This seems to be getting out of hand again.

Dose anyone else have a better idea, i can see we need to do better but 
its a tricky problem, (We would have solved it if it wasnt :) )

Will