[Automated-testing] Executing gcc and gdb test suites

[Mark Hatle]

On 4/13/18 11:58 AM, Sair, Umair wrote:
> Thanks Mark, I'll explore dejagnu on my side.
> 
> I am currently keen to know yocto project test coverage for toolchain and whether it is enough. I would appreciate if anyone can help me out.
> 
> What about toolchain test coverage in windows sdk? Juro, I guess you can answer this better as you are the maintainer of meta-mingw layer.

(Not Juro, but...)
Dejagnu works with mingw as well to help test.  I don't have details how we
build this, but I know it's our primary test.  We compare the dejagnu results
between the meta-mingw and regular builds to verify the windows SDK is not faulty.

--Mark

> Regards,
> Umair Sair
> 
> ________________________________________
> From: Mark Hatle <mark.hatle at windriver.com>
> Sent: Friday, April 13, 2018 8:30 PM
> To: Sair, Umair; automated-testing at yoctoproject.org; yocto-builds at yoctoproject.org
> Subject: Re: [Automated-testing] Executing gcc and gdb test suites
> 
> On 4/13/18 10:23 AM, Mark Hatle wrote:
>> On 4/9/18 10:53 AM, Sair, Umair wrote:
>>> Hi folks,
>>>
>>>
>>> I am trying to find what sort of testing we perform for cross toolchain. I have
>>> visited Yocto 2.4 test plan wiki [1] and testopia dashboard [2]. So far, I am
>>> able to find simple test cases that just confirm that gcc and g++ successfully
>>> build a test file.
>>
>> I don't know what, if anything this Yocto Project does.
>>
>> But at Wind River we run dejagnu (included with the toolchain) as part of our
>> our toolchain validation.  This will cover most regression situations and is a
>> really good sanity test that things are working in a reasonable way.
>>
>> This is something that could be extended into the Yocto Project (open embedded)
>> environment.  It would be a natural fit to the 'ptest' infrastructure.  The only
>> real issue is that dejagnu is known to now pass with a 100% rate, so you really
> (hate replying to myself, that should have been)
> 
> real issue is that dejagnu is known to -NOT- pass with a 100% rate, so you really
> 
>> need multiple runs to watch for regressions.
> 
> --Mark
> 
>> --Mark
>>
>>>
>>> Do we have support for running gcc test suites [3] that are available in gcc
>>> source? The same question about gdb test suites.
>>>
>>>
>>> [1] https://wiki.yoctoproject.org/wiki/Yocto_Project_2.4_Release_Test_Plan
>>>
>>> [2] https://bugzilla.yoctoproject.org/tr_show_product.cgi
>>>
>>> [3] https://gcc.gnu.org/onlinedocs/gccint/Testsuites.html
>>>
>>>
>>> Regards,
>>> *Umair Sair*
>>>
>>>
>>
>