Re: GNOME's testing strategy for GUIs

[David Bolter <dtb gnome org>]

To my mind, adding regression tests is like spending 1 hour to save 100. 
Perhaps what we need is a regression test 
evangelist/documentation/mentor? Will, I really appreciated your phone 
call last year answering my questions about the Orca test harness. It 
sounds like a really great setup, although I'm sure there are other 
great options. Will and the Orca team have a handy wiki on their 
approach here: http://live.gnome.org/Orca/RegressionTesting

I'd like to see documentation on 'GNOME recommended automated testing' 
for all the kinds of projects we see in GNOME (including for the various 
languages). I think this thread is a great way to try and get community 
consensus and to collect information on what various projects use. I 
suspect a lot of projects use none or very little (sadly, including GOK).

IMHO this needs to change.

cheers,
David
Willie Walker wrote:
> Hey All:
>
> Great discussion so far.  I just want to add something about why 
> regression testing is important to me.  In the Orca project, we have a 
> regression test suite that we use regularly.  It helps give us great 
> sanity checking that our changes and bug fixes don't introduce new 
> problems.  It has caught issues introduced by those infamous 
> simple-one-liner-cannot-hurt-a-thing fixes, and it has also been a great 
> tool to help us with performance analysis.
>
> It took a while to ingrain it into the Orca culture, but regression 
> testing is just so useful to us that I cannot imagine NOT doing it. 
> This is not a call to use Orca's stuff, BTW.  It's just amplifying the 
> need for and usefulness of regression across all of GNOME.
>
> An interesting scenario might be to hook up with the build brigade 
> folks.  We might have a set of systems that just continually build 
> GNOME, but we might dedicate another set of systems to continually build 
> GNOME followed by a complete set of regression tests.  At the same time, 
> individual component developers should also be able to set up their own 
> machines to focus testing just on their components.  For example, as 
> soon as http://bugzilla.gnome.org/show_bug.cgi?id=508147 is fixed, I 
> hope to work on a cronjob to automatically test Orca against nightly 
> builds of Firefox.
>
> Finally, IMO the focus on the regression tests should be more about 
> coverage of the important stuff and less about the time it takes to run 
> them.  For example, even if they took 7 days to run, that would still 
> give us tons better coverage and would detect integration problems far 
> sooner.
>
> Will
>
> Nagappan A wrote:
>   
> > Hi Vincent,
> >
> > Currently the scripts are not compatible. We are ready to support it, to 
> > an extent !
> >
> > Thanks
> > Nagappan
> >
> > On Fri, Feb 15, 2008 at 6:10 AM, Vincent Untz <vuntz gnome org 
> > <mailto:vuntz gnome org>> wrote:
> >
> >     Le jeudi 14 février 2008, à 21:48 -0800, Nagappan A a écrit :
> >      > LDTP team is ready to implement what ever the features required
> >     by GNOME
> >      > community.
> >
> >     Is it possible to have some script to convert dogtail scripts to LDTP
> >     ones (and vice versa)? Or have a common format?
> >
> >     I think one thing that is currently blocking all this is that we have
> >     both LDTP and dogtail, and we don't choose one of the two solutions.
> >     Being able to use both will just make things easier and will enable to
> >     move to the next steps.
> >
> >     Vincent
> >
> >     --
> >     Les gens heureux ne sont pas pressés.
> >
> >
> >
> >
> > --
> > Linux Desktop (GUI Application) Testing Project - 
> > http://ldtp.freedesktop.org
> > http://nagappanal.blogspot.com
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > desktop-devel-list mailing list
> > desktop-devel-list gnome org
> > http://mail.gnome.org/mailman/listinfo/desktop-devel-list
> >     
>
> _______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/desktop-devel-list
>