Re: [gnome-desktop-testing] Mago discovery method + make check



Hello,

This is caused by the test cases layout that is assumed by mago:
<MAGO_PATH>/<application>/<test_suite>

If they don't have an subdirectory for each application, then setting MAGO_PATH
to 'dirname $(pwd)' and application parameter to 'basename $(pwd)' should work.

However if the layout above no longer suites our needs, then we can indeed
discuss if it should be changed (this will entail changes in the directory
structure for the logs also).

Best regards,
    Javier

Ara Pulido wrote:
> Hi!
> 
> On 07/10/09 16:14, Javier Collado wrote:
>> Hello,
>>
>> The problem regarding the folders should be solved by setting MAGO_PATH
>> environment variable properly.
>>    
> MAGO_PATH was properly set to ".", but it skips the current folder, and
> start looking in subfolders.
> 
> Regards,
> Ara.
> 
>> Regarding the return value, it's ok to return an error response if
>> some test
>> case has failed. Anyway, as far as I know, this doesn't affect checkbox
>> integration since the goal is to parse log files directly to get all
>> test case
>> results.
>>
>> Best regards,
>>      Javier
>>
>> Ara Pulido wrote:
>>   
>>> Hello all!
>>>
>>> The Ayatana team [1] are using mago for their acceptance tests as part
>>> of their "make check" tests. They were using a version of mago that
>>> didn't include the discovery methods.
>>> When they upgraded to the newest mago, the tests didn't run anymore.
>>>
>>> The problem is that discovery only searches in the subfolders, but not
>>> the current folder. That won't work to integrate in a make check, as
>>> mago will be called directly from the tests folder. I have reported this
>>> as bug 445216 [2].
>>>
>>> The other problem that they are having, is that mago always returns 0
>>> (success). They would like it to return non-zero if no tests were found
>>> or a test case failed. I have reported this as bug 444608 [3].
>>>
>>> As these are important changes I wanted to start a discussion about them
>>> on the mailing list, to see what people think. I am specially interested
>>> in the return 0 one, as it may affect some of the integration with other
>>> test runners (checkbox???)
>>>
>>> Thanks,
>>> Ara.
>>>
>>>
>>> [1] https://launchpad.net/~ayatana
>>> [2] https://bugs.launchpad.net/mago/+bug/445216
>>> [3] https://bugs.launchpad.net/mago/+bug/444608
>>> _______________________________________________
>>> http://live.gnome.org/DesktopTesting
>>> desktop-testing-list mailing list
>>> desktop-testing-list gnome org
>>> http://mail.gnome.org/mailman/listinfo/desktop-testing-list
>>>      
>>    
> 

Attachment: signature.asc
Description: OpenPGP digital signature



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]