[yocto] LTP status for Yocto 1.0/M2

[Stewart, David C]

Kevin - this is quite valuable analysis into the test failures - thank you!

The one which surprises me is the cron failure. But sure enough, the Sato image I have running on my desk doesn't have cron anything installed.  So I'm wondering, which build profile do we test against?  LSB?

Also, do we have bugzillas for the failures?

Dave

From: poky-bounces at yoctoproject.org [mailto:poky-bounces at yoctoproject.org] On Behalf Of Tian, Kevin
Sent: Wednesday, January 12, 2011 3:24 AM
To: yocto at yoctoproject.org; poky at pokylinux.org
Subject: [poky] LTP status for Yocto 1.0/M2

Hi, all,

I'd like to give a quick update about current LTP status. For detail, please check out:
       https://wiki.yoctoproject.org/wiki/LTP_result

Generally LTP is for desktop compliance and not strictly apply to Yocto when customized for specific purpose.

So the main purpose of this task is:

a)    Track ongoing trend to avoid regression

b)    Understand existing failures and category whether they simply come from customization

The stretch goal is:

c)    Reduce the failures as much as possible

Now I'd like to say that:

a)    is done by abstracting data from our QA results

b)    is largely done and most failures falling into that category have been recorded

c)    is on-going

the summary as below:

[cid:image001.png at 01CBB234.F2AF7660]

The majority of the failures are common to all the targets, and thus I now focus on qemux86 for major analysis. The "similarity" row above shows how much common failures exist on other targets compared to qemux86. Because current round of QA test uses a "quiet" option when doing LTP test, lots of debug information are lost. So the similarity is simply done by compare the name of the failed cases, instead of checking its actual error output. I'll confirm them later manually, but current ratio still makes lots of sense.

Beagleboard and routerstation may require recollecting data. On beagleboard, the low similarity is caused by missing an option when doing LTP test. On routerstation, it looks that kernel config options for IPC (sem, shm, msq, ...) are problematic and the disk space is also not enough.

You can click the target name like "qemux86" to get detail progress for each target. For example, for qemux86:
https://wiki.yoctoproject.org/wiki/Qemux86-ltp

[cid:image005.png at 01CBB234.F2AF7660]

For other targets, there's a similarity line to show the difference with qemux86. Take qemux86-64 for example:
       https://wiki.yoctoproject.org/wiki/Qemux86_64-ltp

[cid:image006.png at 01CBB234.F2AF7660]

Let me know if you have any comments.

Thanks
Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110112/1e87c359/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 70058 bytes
Desc: image001.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110112/1e87c359/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 24525 bytes
Desc: image003.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110112/1e87c359/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 17423 bytes
Desc: image005.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110112/1e87c359/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 15400 bytes
Desc: image006.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110112/1e87c359/attachment-0003.png>