External Plugin Repository

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi all,

So today we landed a patch which allows BuildStream to load third party
plugins from third party packages that are installed on the host and
discoverable with python's setuptools.

We are only using setuptools for package discovery and the policy is
fairly lose right now (just load the plugin if it's installed, it's up
to the plugin to explicitly require a recent enough buildstream
version), but the way it is setup we can add other methods of plugin
discovery without breaking existing configuration API, if that ends up
being desirable.

Right now we're looking into creating a secondary repository where we
can develop both fringe, heavily use case specific element plugins, and
or plugins which we're less inclined to commit to an API contract yet
(so for instance, this can also serve for an incubator repository for
things we may later add to the main set of BuildStream plugins).

This is mostly a means to reduce maintenance friction in release cycles
and to avoid an ever growing huge plugins repository being hosted in
the main repo, but also of course as a means to ensure that plugin
sharing mechanisms exist and get tested early.

So before 1.0, I'd like to migrate the following elements outside of
the BuildStream git repo and into a separate auxiliary plugins package:

  o dpkg_build & dpkg_deploy

    Because I feel these are quite use case specific; accepting these
    in the core plugins would open the door to many similar ones;
    probably best to keep these separate.

  o x86image

    Arguably this can be split out for the same reasons, but this one
    may turn out to be so useful that we might want to move it back.

    I'm a bit concerned about how the configuration API is going to
    evolve here, because there are a lot of possible options and
    features, currently the element is a very rigid.

    Essentially if we add this to core BuildStream, it would be nice to
    have another chance to correct potential bad API design choices,
    before promoting it to the core repo.


I realize that these reasonings dont necessarily match up with
eachother, as in; it doesnt necessarily make sense to lump these two
categories together into the same repo.

The reason I hesitate to create two separate repos (one as a sort of
incubation repo, and another as a stable repo but for relatively fringe
features), is only because we would end up starting with a repo with 2
dpkg elements and another repo with one x86 specific image deployment
element.

Any objections to starting with this repo setup for now ?


Cheers,
    -Tristan