Re: Gtk+ unit tests (brainstorming)

[mathieu lacage <Mathieu Lacage sophia inria fr>]

On Mon, 2006-10-30 at 15:34 +0100, Tim Janik wrote:
> On Wed, 25 Oct 2006, Havoc Pennington wrote:
> 
> > Hi,
> >
> > When coding dbus I thought I'd try a project with a focus on unit tests.
> > It has (or at least had for a while) exceptionally high test coverage,
> > around 75% of basic blocks executed in make check. The coverage-analyzer
> > has been busted for a couple years though.
> >
> > Here are my thoughts from dbus:
> >
> >  - the "make coverage-report" was by far the biggest win I spent time
> >    on.
> 
> ah, interesting. could you please explain why you consider it
> such a big win?

I use a similar setting in a project of mine and the big win is, for me,
the ability of knowing what needs to be done: it is easy to spot the
locations which are never tested and thus likely to be buggy.

> 
> >  - frequently I needed to add extra interfaces or levels of
> >    abstraction to be able to test effectively. For example,
> >    allowing "dummy" implementations of dependency
> >    module to be inserted underneath a module I was testing.
> >
> >    dbus is heavily conditionalized on a DBUS_BUILD_TESTS
> >    parameter, which allows adding all kinds of test-only code
> >    without fear of bloating the production version. One price
> >    of this is that the tested lib is slightly different from the
> >    production lib.
> 
> ah, good to know. though i'd consider that price considerably high for a
> project of the size and build time of Gtk+, and where we'd really benefit
> from having *many* developers and contributors run make check.
> especially, when you have a quite large legacy code base, instead of
> developing with conditionalized test hooks from the start.

Usually, these test hooks are always ON for developer builds and only
OFF for release builds. 

> >  - based on how nautilus does unit tests, I put the tests in the file
> >    with the code being tested. The rationale is similar to the
> >    rationale for inline documentation. I think it's a good approach,
> >    but it does require a distinct "test build" (DBUS_BUILD_TESTS).
> 
> sounds interesting as well. the downsize is of course the assorted
> file growth, and gtk+ already isn't a particularly good citizen in
> terms of loc per file ;)
> 
> $ wc -l *.c | sort -r | head
>    380899 total
>     14841 gtktreeview.c
>     11360 gtkaliasdef.c
>      9154 gtkiconview.c
>      8764 gtkfilechooserdefault.c
>      8632 gtktextview.c
>      8060 gtkwidget.c
> 
> >    Another advantage of this is that internal code can be tested, while
> >    it may not be possible to fully exercise internal code using the
> >    public API.
> 
> thanks for your insight havoc. i'll definitely look into the coverage
> report generation at some later point.

I think havoc used his own report generation tool. You might want to
give lcov a try: it generates some nice-looking html.

Mathieu