[yocto] Fullpass Test Report for Yocto 1.3_M1 20120606 Build

Wed Jun 13 19:23:39 PDT 2012

Hi all
    First, I am so sorry for my mistake for the only one new found hob bug in the report I sent out yesterday, it should be Bug 2576 [Hob2] wrong image profile selected after template load. And now this report is more complete , cause I integrated the non-x86 report into it.
This is the Fullpass test report for Yocto 1.3 20120606 build. There are 16 bugs found in this test circle. We find some meta-intel BSPs failed to build with "Weekly build 1.3_M1 Week 4", one is for connman-gnome and another is for gst-ffmpeg, Due to the issue, QA testing against Sugarbay/eMenlow/Crownbay/FRI2/Huron River is blocked. The 2 bugs we reported for these build error are bug 2561 and 2562.

For Hob, a  minor issue is that Incorrect image shown when loading template file, bug 2576.

For Core Build System, there are 10 issues, Mulitilb could not work with lsb , sato build and lib32-connman-gnome build;  two bugs for Yocto-bsp tools, X can't start in sato image built by yocto-bsp tool and fail to build core-image-sato when add patch into the kernel; qemu-native build failed in self-hosted-image with vmdk and gtk-directfb image failed.; there is some dispute with the issue lib64-core-image-sato build failed due to sanity check , QA will focus on it whether the case design issue or not.

For Distribution testing, we tested on three distributions, they are Fedora 16,OpenSuse 12.1 and Ubuntu 11.10, and no issue found with Yocto build.

For ADT, we found three issues that are Segmentation fault on qemuarm , Yocto-bsp tool fail to create Yocto BSP and ADT-Installer installation issue. Other features perform well.

For Build performance, one qemux86 sato build on a Core i7 machine costs 108 minutes, which is almost the same as the result in 1.2 M4 RC5 build(111 minutes)
Note: The test report is archived @ https://wiki.yoctoproject.org/wiki/Yocto_1.3_Milestone_Test_Report

Test Summary:
---------------------------------------

Test Result Summary

Component

Target

Status

Comments

BSP

eMenlow

BLOCK

connman-gnome do_configure fail on autobuilder for several BSPs

Blacksand

GOOD

Everything runs well

Crownbay-emgd

BLOCK

connman-gnome do_configure fail on autobuilder for several BSPs

Sugarbay

BLOCK

connman-gnome do_configure fail on autobuilder for several BSPs

HuronRiver-sato-sdk

BLOCK

connman-gnome do_configure fail on autobuilder for several BSPs

Jasperforest-lsb

GOOD

Everything runs well

Beagleboard

GOOD

Everything runs well

Routerstationpro

GOOD

Everything runs well

Mpc8315e-rdb

GOOD

Everything runs well

QEMU

qemux86

GOOD

Everything runs well

qemux86-64

GOOD

Glib-Gobject-warning when launch connman-properties

qemuarm

GOOD

Everything runs well

qemuppc

GOOD

Everything runs well

qemumips

GOOD

Everything runs well

ADT

BUGGY

Glxgears failed on qemux86/x86-64/arm/mips/ppc on Ubuntu 11.10 64bit,ADT-Installer failed if the folder /opt/poky/1.2+snapshot exist;Segmentation fault on qemuarm from /usr/lib/libust.so.0; 32bit kernel generated oprofile data file can not be read by remote 64bit opreport

Distribution Support

GOOD

yocto build on Fedora 16,OpenSuse 12.1 and Ubuntu 11.10 successful

Compliance

BLOCK

LTP and POSIX on Blacksand could pass; Crownbay,Sugarbay and HuronRiver image blocked by connman-gnome do_configure fail

Core build system

BLOCK

Mulitilb could not work with lsb and sato build; X can't start in sato image built by yocto-bsp tool; fail to build core-image-sato when add patch into the kernel;qemu-native build failed in self-hosted-image with vmdk;do_rootfs failed for lib32-connman-gnome with ipk;lib64-core-image-sato build failed due to sanity check

Power/Performance

BLOCK

connman-gnome do_configure fail on autobuilder for several BSPs

Stress

GOOD

Both Crashme and Helltest on Jasperforest, Crashme and LTP stress could pass 24 hours testing;

HOB

GOOD

Incorrect image shown when loading template file

Critical bugs, more than 50% test cases are blocked

Only Normal, Minor or Enhancement bugs, or less than 10% test cases failed

Normal, Major and Critical bugs, and more than 10% test cases failed

Detailed Test Result for each component

Target

Total TCs

Not Run

Passed

Failed

Not testable (Blocked)

eMenlow Sato-SDK

66

0

0

0

66(bug 2561)

n450 Sato-SDK

64

0

63

1(bug 2535)

0

Crownbay-Sato-SDK

66

0

0

0

66(bug 2561)

Sugarbay Sato-SDK

69

0

0

0

69(bug 2561)

HuronRiver Sato-SDK

70

0

0

0

70(bug 2561)

Jasperforest lsb-SDK

32

0

32

0

0

Beagleboard Sato-SDK

40

0

40

0

0

Routerstationpro Sato-SDK

33

0

33

0

0

Mpc8315e-rdb Sato-SDK

36

0

36

0

0

Qemux86-64 Sato

30

0

30

0

0

Qemux86-64 Sato-SDK

35

0

35

0

0

Qemux86 Sato

30

0

30

0

0

Qemux86 Sato-SDK

35

0

35

0

0

Qemuarm Sato

30

0

30

0

0

Qemuarm Sato-SDK

35

0

35

0

0

Qemumips Sato

30

0

30

0

0

Qemumips Sato-SDK

35

0

35

0

0

Qemuppc Sato

30

0

30

0

0

Qemuppc Sato-SDK

35

0

35

0

0

ADT

49

0

42

7(bug 1452 2068 2069)

0

Core build system

68

0

55

12(bug 2571 2572 2559 2589 2586 2590 2560 2580)

1(bug 2589)

Power & Performance

4

0

0

0

4(bug 2561)

Compliance

2

0

0

0

2(bug 2561)

Stress

3

0

3

0

0

HOB

38

0

38

0

0

Total

965

0

667

20

278

* You can check the detailed test result in attachment for each target.
** The failed/blocked case number is listed with failed cases’ bug number.

Commit Information
---------------------------------------
Tree/Branch: poky/master
Commit id: df8f55a919b3cc602ce1f1c51630c7edf6e36b55

Meta branch: master

Commit id: 8bc2fa135060e941bcbb4dcfb664e175a3eb871d

Issue summary
--------------------------------------
Component

Bug Number

Target Milestone

System & Core OS

System Usage

Bug 2508 Fotowall icons appear abnormal on qemumips sato-sdk images with nightly build (20120523-2)

1.3 M2

Bug 2509 The 'About' option with connman-properties doesn't display the picture.

1.3 M2

Other

Bug 2535 syslog is not started by default in background on bsps(Crownbay,N450 etc)

1.3 M2

New! Bug 2561 connman-gnome do_configure fail on autobuilder for several BSPs

New! Bug 2562 gst-ffmpeg do_configure failed on autobuilder for lsb build

Core build system

New! Bug 2559 [yocto-bsp]X can't start in sato image built by yocto-bsp tool

New! Bug 2560 [multilib]lib64-core-image-sato build failed due to sanity check

New! Bug 2571 [multilib] no provider for lib32/lib64-lsbinitscripts

New! Bug 2572 [multilib] failed dependency for util-linux with multilib build

New! Bug 2580 Build gtk-directfb image failed

New! Bug 2585 [wiki]The wiki using yocto-bsp create qemu BSP should sync with our source code

New! Bug 2586 qemu-native build failed in self-hosted-image with vmdk

New! Bug 2587 [yocto-bsp]the default branch is not in remote branch

New! Bug 2589 [yocto-kernel] fail to build core-image-sato when add patch into the kernel

New! Bug 2590 [multilib]do_rootfs failed for lib32-connman-gnome with ipk

ADT

Bug 1452 32bit kernel generated oprofile data file can not be read by remote 64bit operation

Bug 2592 Yocto-bsp tool fail to create Yocto BSP with the same build&output location

Bug 2534 [Regression]Glxgears failed on qemux86/x86-64/arm/mips/ppc on Ubuntu 11.10 64bit

New! Bug 2577 Segmentation fault on qemuarm from /usr/lib/libust.so.0

New! Bug 2592 Yocto-bsp tool fail to create Yocto BSP with the same build&output location

New! Bug 2553 ADT-Installer failed if the folder /opt/poky/1.2+snapshot exist

Power & Performance

Power consumption of crownbay is block by bug 2561.

Build time for a sato image on Core i7 860 machine is 109 minutes. Detailed result on

https://wiki.yoctoproject.org/wiki/Performance_Test

Compliance

LTP result: https://wiki.pokylinux.org/wiki/LTP_result<https://wiki.pokylinux.org/wiki/LTP_result>

POSIX result: https://wiki.pokylinux.org/wiki/Posix_result<https://wiki.pokylinux.org/wiki/Posix_result>

Stress

Both Crashme and Helltest on Jasperforest could pass 24 hours stress testing.LTP stress could pass 24 hours testing;Crushme on Beagleboard is not test.

Distribution Support

yocto build on Fedora 16,OpenSuse 12.1 and Ubuntu 11.10 without issue

HOB

New! Bug 2576 [Hob2] wrong image profile selected after template load

Best Regards,
Hongna

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120614/8683d028/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Yocto 20120606 Build_ detail_testreport_Fullpass.ods
Type: application/oleobject
Size: 31291 bytes
Desc: Yocto 20120606 Build_ detail_testreport_Fullpass.ods
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120614/8683d028/attachment.bin>