Re: test locations



On Mon, Jul 13, 2009 at 3:55 PM, Martin Nordholts<enselic gmail com> wrote:
> Always building the tests ensures that they are kept up do date with changes
> in the rest of the code base. If they are not built along with the rest of
> the code base then they will eventually become out of sync and someone has
> to fix them later, possibly months after the change that introduced the
> compilation errors were made.
>
That argument has essentially no value at all to me, for lots of reasons.

First of all, compiling all tests while I'm experimenting on the code
and nowhere near a checkin is just annoying.

Next, just compiling the tests and not actually running them is almost
useless, as it only finds API breakage. You need to actually run the
tests to find the bugs. And in the context of glib there's a
multiplied effect, as API breakage is found almost immediately.

Then, I expect someone to run make check before doing checkins anyway.
And if people forget it, there should be a build farm that catches it
and complains loudly (either on IRC or on mailing lists), flagging the
commit that broke.

And finally, it's trivial to figure out which checkin broke a test
with git bisect run.


Note that I'm not opposed to having a useful test suite in glib and
gtk - in fact, I'd welcome it. But it's important that the test suite
not just aids in finding bugs but also has a very limited impact on
hackers, so they don't outright reject it.
The current setup seems to not be very good at that, as the number of
tests does not seem to grow and people seem to never run it (yes, that
includes me). I'd even say they are annoyed by it (see this mail
thread), but that might just be me.
If you wanted to improve the testsuite situation, you should have a
look at other projects that successfully make heavy use of testsuites
and copy what they do. Examples of such projects I've contributed to
are Mozilla, Webkit or Cairo. (If a jibe is allowed: None of them
build the testsuite on make, as it would at least double build times.)

I've attached a patch that seems to work for me.

Cheers,
Benjamin

Attachment: diff
Description: Binary data



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]