[BuildStream] Proposal: bst source delete command

[Chandan Singh <chandan chandansingh net>]

Hi all,

BuildStream currently offers no way of systematically deleting locally cached
sources. I think it should however do so - not only to maintain symmetry with
the `artifact delete` command but also because sources can be substantial in
size in many cases.

In this message, I propose to add a new `source delete` command.

Unlike its `artifact delete` counterpart, this is slightly more nuanced as we
need to delete the sources from two places:

1. From the local CAS cache
2. From the plugin's cache directories

(1) is simple, but (2) would require cooperation from source plugins. To that
end, I propose to add a new abstract method for source plugins, like
`Source.delete()`.

Plugins can choose to do nothing if they don't maintain an extra local cache,
like local and workspace sources.

Plugins that do maintain a local cache can implement this method and delete the
corresponding directories from their cache directories. For example, the `tar`
plugin would delete something like `~/.cache/buildstream/tar/URL_FOR_TARBALL`.

Open questions
--------------

1. Since source plugins do not know about how many times a a given source is
   used, they can't do reference counting of sources. So, imagine a scenario
   where elements A and B both use a git repository G. Using this approach,
   `bst source delete A` would end up deleting it, as we have no way of knowing
   that B is also using it. This may not be too bad but the user may have
   expected the source to persist if another element is still using it.

   I think this is probably fine but we should make this quite clear in the
   relevant documentation.

2. What to do with unreachable sources? How do we delete them? It is not
   inconceivable to imagine that certain URLs are no longer referenced. But,
   again, we don't have any way of ascertaining that.

I don't have answers to either of these questions so I'd be curious to hear if
you have any ideas. Nevertheless, I don't think they are blockers for this
feature and we can put them on the back burner if can't find a solution for them
right now.

Let me know what you think.

Thanks!
Chandan