Re: [BuildStream] min-version approach for plugins

[Chandan Singh <chandan chandansingh net>]

Hi Tristan,

Thanks for writing this up. I think the idea is a win-win for everyone - plugin
authors who don't have to bump format versions, and users who won't have to
match format version to package version to fix errors. So +1 from my side.

>   * For pip source plugins, we should allow the project.conf to specify
>     semantic version specifiers as defined by pep 0440[4], and require
>     that plugin packages to be loaded via the pip plugin source follow
>     the same pep 440 for its version identifiers.
>
>     Rationale:
>
>       These are much easier for users to express and understand than
>       the BST_FORMAT_VERSION numbers we were using previously, it is
>       also less work for plugin developers as they do not have to
>       remember to bump a version digit on their plugin classes as they
>       evolve.
>
>       Users will have errors saying "project required foo >= 2.4,
>       but only foo 2.2 was installed" (or similar), rather than
>       messages saying "project required foo plugin format version
>       26, please go figure out what package version you need for
>       yourself", making life easier all around.
>
> To implement this for pip plugin sources, we can use distlib[5] and do
> not need overly complex code to do it.

I looked into this a bit, and I think we already support this - kind of :)

While declaring the plugins, we can actually already supply version strings as
requirements in the `package-name` field. So, this works today (intentionally or
not): `package-name: bst-plugins-experimental >= 1.91`.

If, however, the requirement is not met, we will get an ugly error like so:

        # long stack trace here
        pkg_resources.VersionConflict: (bst-plugins-experimental
1.93.1 (/.venvs/bst/lib/python3.7/site-packages),
Requirement.parse('bst-plugins-experimental>=2'))

I think we probably just need to catch this and display it more sensibly. And,
we won't need to parse any requirement strings ourselves (yay!) since this
already handles complex requirements like
"bst-plugins-experimental <2, >1, <1.95, >1.91, !=1.93.0".

> I think this will be a net improvement over BuildStream 1, where I have
> been seeing lot's of breaking/changing plugin API surfaces evolve, and
> have yet to see the BST_FORMAT_VERSION of a plugin set to anything
> other than 0.

I agree that it's not common but I've noticed it a few times. Actually our
experimental plugin repositories has two plugins which have non-zero
BST_FORMAT_VERSION. git_tag in on version 1 and collect_manifest is on
version 2 already.

(Having said that, I haven't looked much into the history to see whether these
version bumps were really necessary or not.)

At any rate, I think semantic versioning is a better solution all around.

Cheers!
Chandan