Re: [BuildStream] [Proposal] Plugin fragmentation / Treating Plugins as Sources

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Sun, 2019-04-14 at 08:49 +0100, Angelos Evripiotis wrote:
> On Sun, Apr 14, 2019 at 6:45 AM Tristan Van Berkom
> > <tristan vanberkom codethink co uk> wrote:
> > It seems a bit more challenging, if we do go ahead with this; could I
> > assume to get assistance from you to solve the challenge of making this
> > part actually work ?
>
> Sure thing! I can pick it up on Tuesday if Chandan doesn't beat me to it :)

That sounds great.

If we can get some prototype which demonstrates the base concept of:

  * Installing two separate venvs with different dependency versions
  * Running an interpretor which loads a plugin in both separate
    venvs (it could even be the same plugin, and need not be a
    "BuildStream" plugin, but some python module loaded on demand)
  * Prove that we infact have separation (perhaps by having the plugin
    just print the versions of it's dependencies).

That would be enough for me to start working with (still sounds quite
challenging).

If this does actually work, I would recommend that we adjust the
proposal to obsolete the pip origin completely in favor of the new venv
origin.

Basically a venv origin as describe here would be pretty much the same,
except it would provide additional safety, so I don't think it makes
sense to continue using the `pip` origin.

I would still suggest that we aim for maximum fragmentation of plugins
into mini plugin repos, as this grants us more flexibility for upstream
plugin development, and avoids any controversy around what plugin
should be allowed into what repo (which were the main drivers of my
proposal in the first place).

Cheers,
    -Tristan