Re: Gtk+ unit tests (brainstorming)

[Havoc Pennington <hp redhat com>]

Hi,

When coding dbus I thought I'd try a project with a focus on unit tests.
It has (or at least had for a while) exceptionally high test coverage, 
around 75% of basic blocks executed in make check. The coverage-analyzer 
has been busted for a couple years though.

Here are my thoughts from dbus:

 - the "make coverage-report" was by far the biggest win I spent time
   on. It gave percentage basic blocks tested, globally, by module,
   and by file. Then for each file, it did the gcov-style annotation
   to show what was not covered.

   So when coding, you could make coverage report, then see what you
   had failed to test. Also, if you just wanted to work on improving
   test coverage, you could use make coverage-report to find
   areas to work on.

   the coverage-report target simply depended on make check, so it
   was only a single command to run all the tests and look at
   their coverage in a helpful format.

 - frequently I needed to add extra interfaces or levels of
   abstraction to be able to test effectively. For example,
   allowing "dummy" implementations of dependency
   module to be inserted underneath a module I was testing.

   dbus is heavily conditionalized on a DBUS_BUILD_TESTS
   parameter, which allows adding all kinds of test-only code
   without fear of bloating the production version. One price
   of this is that the tested lib is slightly different from the
   production lib.

 - based on how nautilus does unit tests, I put the tests in the file
   with the code being tested. The rationale is similar to the
   rationale for inline documentation. I think it's a good approach,
   but it does require a distinct "test build" (DBUS_BUILD_TESTS).

   Another advantage of this is that internal code can be tested, while
   it may not be possible to fully exercise internal code using the
   public API.

Havoc