Re: [BuildStream] A reboot of the testing discussion

[Angelos Evripiotis <angelos evripiotis gmail com>]

Hi Jim!

Thanks for the write-up, that sounds a good summary to me.

I’d also like to reveal the identity of this mysterious ‘Antoine’ - in fact it was I, Angelos, all along. (I think? :)

> So, given the 'separate test element' strategy, we still have two approaches to testing:

I think examples might help make sure we’re in sync, here is my jet-lagged attempt:

- - -

> 1) Have the finished artifact depend on all its usual dependencies and on all the tests. This allows tests of one element and the builds of its dependents to occur in parallel.

e.g. one possible arrangement of this would be:
- Finished artefact “app.bst” depends on “build/lib1.bst” and “alltest/lib1.bst”.
- “build/lib1.bst” depends on “build/lib2.bst”.
- “test/lib1.bst” depends on “build/lib1.bst”, it does the actual testing.
- “alltest/lib1.bst” depends on “test/lib1.bst” and “alltest/lib2.bst”. It doesn’t perform any action, it’s just a grouping. This provides the check that all transitive deps of lib1 are tested.
- There are no dependencies from “build/*” to “test/*”, this separates those concerns.
- There are no dependencies from “test/*” to “test/*”, only to “build/*”. This means all tests can run in parallel.

2) Make each BuildElement depend on the tested result of its build dependencies. This will avoid wasted time spent building things whose dependencies fail tests.

e.g. following the conventions of the first example:
- Finished artefact “app.bst” depends on “test/lib1.bst”.
- “build/lib1.bst” depends on “test/lib2.bst”.
- “test/lib1.bst” depends on “build/lib1.bst”, it does the testing and passes through the built artefacts.

- - -

Does that agree with your understanding Jim?

I apologise in advance for any formatting weirdness- I’m doing this from my phone.

Cheers!

Angelos