Re: Announcement/RFC: jhbuild continuous integration testing -- mystery of recent flood of failures solved

[Simon McVittie <simon mcvittie collabora co uk>]

On 19/02/13 13:54, Tristan Van Berkom wrote:
> On Tue, Feb 19, 2013 at 10:13 PM, Simon McVittie
> <simon mcvittie collabora co uk> wrote:
>      GTestDBus is IMO ideal for regression testing (or in-tree unit tests),
>      I made a short write-up on this not long ago in my blog[0].

GTestDBus combines two things: setting the D-Bus service activation path
to a private one, and running a new D-Bus session.

It does some of what self-contained regression tests should do, but by
no means all of it - I think regression tests should also consider
setting XDG_RUNTIME_DIR, XDG_DATA_*, XDG_CONFIG_*, DISPLAY (if used),
and HOME.

with-session-bus.sh (and dbus-run-session, if reviewed) specifically
only does the "new D-Bus session" part, although you can supply a D-Bus
configuration file via --config if you want to change the service
activation path too, and the rest can be done via env(1).

>     For this case (why I thought about Colin's ideas on installed tests), it
>     suddenly becomes interesting to have tests which do not use GTestDBus
>     in a controlled environment, but instead to test the system as a whole,
>     running only services from ${build_prefix}/share/dbus-1 but certainly
>     avoiding anything from /usr/share/dbus-1/ (or at least properly prioritizing
>     the build prefix services over the system installed ones, if we can't avoid
>     using those).

This should still use a new D-Bus session (and probably XDG_RUNTIME_DIR
and HOME) for the tests, but could set XDG_DATA_*, XDG_CONFIG_* to look
in ${build_prefix} before /usr.

>     Perhaps some additional features added to GTestDBus could allow us
>     to run the same tests in both contexts (i.e. the installed test context
>     with a "full gnome environment" vs. the isolated in-tree context).

In my experience, regression tests for D-Bus components usually need
controllable "mock" versions of various D-Bus services: only a small
subset of these tests (those that test "normal" behaviour, and aren't
too picky about implementation details) will work with the real version
of those services.

    S