Re: [BuildStream] Dropping support for `setup.py test`

[Mathieu Bridon <bochecha daitauha fr>]

Hi,

Speaking as the Buildstream Fedora packager…

On Fri, 2019-12-06 at 11:19 +0000, Chandan Singh wrote:
> On Fri, Dec 6, 2019 at 3:09 AM Tristan Van Berkom
> <tristan vanberkom codethink co uk> wrote:
> ...
> > The `setup.py test` invocation is important precisely *because* it
> > does not install any specific set of dependencies automatically or
> > run in any virtual environment.
>
> I don't think this is entirely correct. `setup.py test` will not
> install any build requirements (like Cython) the way we have defined
> it in pyproject.toml. But, it will absolutely install any and all of
> BuildStream's dependencies and testing dependencies, if they aren't
> already installed. Compared to tox, the only difference being that
> they will not be installed in a virtual environment, but as eggs in
> current directory.
>
> > This is the correct testing path we need to have for the integrator
> > workflow (not for the developer workflow), as this allows the
> > integrator to properly test how BuildStream will be have once it is
> > integrated with your system and installed on your machine.
>
> Thanks for stating this. This is what I suspected but wasn't 100%
> sure. I'd agree that this is a valid use-case. What I fail to
> understand is how `setup.py test` is solving this. As mentioned
> above, it will install any missing dependencies so there's no
> guarantee that the integrator is testing how BuildStream will
> actually be installed.

For example in Fedora, the build system doesn't have access to the
network, so if a dependency is missing when running `setup.py test`
then the build will fail.

So if the tests pass, then all the required dependencies were already
present in the buildroot, and none were downloaded.

(I currently don't run the tests when building the Fedora package
because too many test deps are missing from the distro though, but the
above still stands)

> If the use case is that the integrators are already expected to have
> all the dependencies installed in some environment, then why do we
> need this additional inderiction of `setup.py test`? Can't we just
> call pytest at that point?

That would be perfectly fine indeed.


-- 
Mathieu