Re: build.gnome.org

[Martin Pitt <martin pitt ubuntu com>]

Hello Colin,

Colin Walters [2013-01-15 15:34 -0500]:
> On Tue, 2013-01-15 at 11:07 +0100, Martin Pitt wrote:
> 
> > We have experimented with that a bit, by building 
> > 
> >   https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/
> 
> Interesting!  Looks quite useful.  Are you doing anything with
> respect to the "jhbuild sysdeps --install" infrastructure or is
> the system package set maintained manually?

Right now in our Juju charm it's a manual list:

  http://bazaar.launchpad.net/~jibel/charms/quantal/jhbuild/trunk/view/head:/files/jhbuild.config/gnome-core.sysdeps

I'm not quite sure why; Jean-Baptiste, did jhbuild sysdeps not work
well enough in principle?

> The gnome-control-center failure appears to be something in the
> jenkins/jhbuild glue not understanding how to properly do git
> submodules.

Yeah, there are several modules which show a problem with the test
environment; that, and e. g. g-settings-daemon is stumbling over
"PYTHONPATH for python3"
(https://bugzilla.gnome.org/show_bug.cgi?id=688353) etc.

As I said, this needs some caretaking. Yesterday we got some fixes to
a couple of modules; we now have some time to devote to this.

> http://git.gnome.org/browse/gnome-ostree/tree/src/ostbuild/js/builtins/qa_smoketest.js
> 
> This should all be pretty fast - I want this test running within 5
> minutes after I push a commit to glib/gjs/whatever.  And 5 minutes is
> slow - with some optimizations, that could be 1 minute.
> 
> *After* that test has completed, we can do stuff like run selected tests
> from glib/pygobject/gjs etc.  In the installed test model, we can choose
> exactly which one of those tests to run.
> 
> I know this installed tests model is orthogonal (somewhat complementary,
> somewhat conflicting) to the work you've been doing for the current
> "distcheck" model of nondestructive per-user tests.  It's true that
> those types of tests can be very easy and fast for developers to run.  
> 
> But the qemu-based testing model is very powerful, and there's no excuse
> for not having a suite of qemu-based tests run automatically build
> server side.

I fully agree; we need both unit and these system integration tests.
I'd like to think that adding "make check" tests isn't conflicting to
that; when writing tests with above in mind, you can usually set them
up in a way that they can run both against the built source tree as
well as the installed system; for example, when submitting the tests
for gvfs I added both a "make check" as well as a "make installcheck"
rule, and you can just run e. g.  tests/gvfs-test.py out of a pristine
git checkout. In many cases that's usually a matter of not writing the
tests with hardcoded paths against the source and then setting
variables like $PATH, $GST_REGISTRY_1_0, etc. to the build tree when
running the tests against the build tree.

We also use those tests in our "autopkgtest" model where we install a
proposed package update into a standard distro install and then run
the package's tests against that system. Architecture wise this has
the same requirements as for above "system-wide smoke tests".

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)