[yocto] Fullpass Test Report for Yocto 1.2 M4 RC5 Build

Fri Apr 27 08:34:42 PDT 2012

Hi all
This is the Fullpass test result for 1.2 M4 RC5. In last test circle, the 2 critical bugs(pseudo-nativesdk issue and Hob can’t launch when sanity network check fail) in RC4 are got fixed. There are only 3 new minor issues reported, one for Hob(bug 2371), which is exceptional situation such as out of memory, we should handle the internal state for hob, one is for ADT that users can't import or create Bitbake Commander Project if sanity network check fail(bug 2372), another is for Build Appliance that no xdg-open in self-hosted image(bug 2370).

All features on BPSs and QEMUs work well, with 4 old open bugs, they are i2c_transfer error on e-Menlow; Xorg eats lots of CPU time after standby on Crownbay-emgd; Fotowall doesn't run on qemumips and Glib-Gobject-warning when launch connman-properties on qemux86-64.

For ADT, most features perform well, except that clutter run failed on qemux86/x86-64, 32bit kernel generated oprofile data file can’t be read by remote 64bit operation and can't import or create BC Project if sanity network check fail.
For Hob, QA test it against latest poky/denzil branch with commit d20a24310eb43fed9a3f3e75752c9ad45a18cbe4, with only one minor issue found that we should handle the exception out of memory.

For Core build features, the only new issue reported is that no xdg-open in Build Appliance. For the other features(including x32, non-GPLv3, multilib, yocto-bsp/kernel tools, etc), their status are same as RC4 test result - the major issue is that multilib could not work with -sdk and ipk build.
For Distribution testing, N+1 testing is performed against CentOS/Ubuntu/Opensuse/Fedora. The build issues on Opensuse 12.2 Milestone 2 and Fedora 17 Alpha still exist, world build on all distributions is still on-going. We will double check bug 2363 against Fedora 17 Beta.
For Build performance, one qemux86 sato build on a Core i7 machine costs 111 minutes, which is almost the same as the result in 1.2 M4 RC4 build(110 minutes)
Note: The test report is archived @ https://wiki.yoctoproject.org/wiki/Yocto_1.2_Milestone_Test_Report.

Test Summary:
---------------------------------------
Test Result Summary

Component

Target

Status

Comments

BSP

eMenlow

GOOD

I2C_transfer error

Blacksand

GOOD

Everything runs well

Crownbay-emgd

GOOD

Xorg eat lots of cup's resource after standby;

Sugarbay

GOOD

Everything runs well

Jasperforest-lsb

GOOD

Everything runs well

Beagleboard

GOOD

Everything runs well

Routerstationpro

GOOD

Everything runs well

Mpc8315e-rdb

GOOD

Everything runs well

QEMU

qemux86

GOOD

Everything runs well

qemux86-64

GOOD

Glib-Gobject-warning when launch connman-properties

qemuarm

GOOD

Everything runs well

qemuppc

GOOD

Everything runs well

qemumips

GOOD

Everything runs well

ADT

BUGGY

Clutter failed on qemux86/x86-64,32bit kernel generated oprofile data file can not be read by remote 64bit opreport

Distribution Support

On going

m4-native do_compile task failed on Opensuse 12.2 Milestone 2;do_rootfs fail on fedora17 64bit; world building is running.

Compliance

GOOD

No regression issue with LTP and POSIX

Core build system

GOOD

lib64 core-sato-sdk build against qemux86 fails at qt4-x11-free; fail to find bblayers if TOPDIR is not "build";fonts corrupted with ipk build for 32bit connman against qemux86-64;no xdg-open for "View Files"

Power/Performance

GOOD

Power consumption of crownbay is 23.837W

Stress

GOOD

Both Crashme and Helltest on Jasperforest; LTP on Beagleboard could pass 24 hours stress testing.

HOB

GOOD

Handle out of memory exception

Critical bugs, more than 50% test cases are blocked

Only Normal, Minor or Enhancement bugs, or less than 10% test cases failed

Normal, Major and Critical bugs, and more than 10% test cases failed

Detailed Test Result for each component

Target

Total TCs

Not Run

Passed

Failed

Not testable (Blocked)

eMenlow Sato-SDK

65

0

64

1(bug 310)

0

n450 Sato-SDK

64

0

64

0

0

Crownbay-Sato-SDK

65

0

64

1(bug 1258)

0

Sugarbay Sato-SDK

69

0

69

0

0

Jasperforest lsb-SDK

32

0

32

0

0

Beagleboard Sato-SDK

40

0

40

0

0

Routerstationpro Sato-SDK

33

0

33

0

0

Mpc8315e-rdb Sato-SDK

36

0

36

0

0

Qemux86-64 Sato

30

0

29

1(bug 2282)

0

Qemux86-64 Sato-SDK

35

0

34

1(bug 2282)

0

Qemux86 Sato

30

0

30

0

0

Qemux86 Sato-SDK

35

0

35

0

0

Qemuarm Sato

30

0

30

0

0

Qemuarm Sato-SDK

35

0

35

0

0

Qemumips Sato

30

0

30

0

0

Qemumips Sato-SDK

35

0

35

0

0

Qemuppc Sato

30

0

30

0

0

Qemuppc Sato-SDK

35

0

35

0

0

ADT

49

0

42

7(bug 1452 2068 2069)

0

Core build system

68

0

66

2(bug 2355 )

0

Power & Performance

4

0

4

0

0

Compliance

2

0

2

0

0

Stress

3

0

3

0

0

HOB

38

0

38

0

0

Total

893

0

880

13

6

* You can check the detailed test result in attachment for each target.
** The failed/blocked case number is listed with failed cases’ bug number.

Distribution test
--------------------------------------
Distribution

Host Arch

Target Machine

world build

core-sato-sdk build

Fedora 16

x86_64

qemuppc

Running

　PASS

Fedora 17 Alpha

x86_64

qemuarm

Running

Failed(bug2363)

Ubuntu 11.10

x86_64

qemumips

Running

　PASS

Ubuntu 12.04 Beta 1

x86_64

qemuppc

Running

　PASS

Opensuse 12.1

x86_64

qemumips

Running

　PASS

Opensuse 12.2 Milestone 2

x86_64

qemux86-64

Running

Failed(bug2201)

Centos 5.6

x86

qemux86

Running

　PASS

Centos 6.2

x86_64

qemuppc

Running

　PASS

Commit Information
---------------------------------------
Tree/Branch: denzil
Commit id: 35196ff703726addf7f4051ee09d70bf9ee0c5d0
Meta branch :master
Commit id: 8024d571900fb1244316ec96dd867fcb4cedfc89

For Hob
Tree/Branch: denzil
Commit id: d20a24310eb43fed9a3f3e75752c9ad45a18cbe4

Issue summary
--------------------------------------
Component

Bug Number

Target Milestone

System & Core OS

System Usage

Bug 310 - [eMenlow] i2c_transfer error on e-Menlow

1.1

Bug 1258 -[crownbay] Xorg eats lots of CPU time after standby

1.1 M4

Other

Bug 1477 Fotowall doesn't run on qemumips

1.3

Bug 2282 - Glib-Gobject-warning when launch connman-properties on qemux86-64 sato/sato-sdk image

1.2 M4

Core build system

Bug 2219 - [yocto-kernel] fail to find bblayers if TOPDIR is not "build"

1.3

Bug 2355 [multilib] lib64 core-sato-sdk build against qemux86 fails at qt4-x11-free

1.2.1

Bug 2356 [multilib] fonts corrupted with ipk build for 32bit connman against qemux86-64

1.2.1

Bug 2360 PACKAGE_ARCHS cause wrong packages installed into image

1.2.1

New! Bug 2370 - [Build Appliance] no xdg-open for "View Files"

ADT

Bug 1452 32bit kernel generated oprofile data file can not be read by remote 64bit operation

1.3

Bug 2068 Clutter run failed in qemux86/x86-64 using toolchain

1.3

Bug 2069 Clutter run failed in qemux86/qemux86_64 on ubuntu/opensuse/fedora.

1.2.1

New! Bug 2372 [eclipse]Can't import or create Bitbake Commander Project if sanity network check fail

Power & Performance

Power consumption of crownbay is 24.015W.

Build time for a sato image on Core i7 860 machine is 111 minutes. Detailed result on

https://wiki.yoctoproject.org/wiki/Performance_Test

Compliance

LTP result: https://wiki.pokylinux.org/wiki/LTP_result<https://wiki.pokylinux.org/wiki/LTP_result>

POSIX result: https://wiki.pokylinux.org/wiki/Posix_result<https://wiki.pokylinux.org/wiki/Posix_result>

Stress

Both Crashme and Helltest on Jasperforest could pass 24 hours stress testing.LTP stress could pass 24 hours testing;Crushme on Beagleboard is not test.

Distribution Support

Bug 2201 m4-native do_compile task failed when building image on opensuse12.2 Milestone2

--

Bug 2363 core-image-sato-sdk failed to do do_rootfs on fedora17 64bit

1.2.1

HOB

New! Bug 2371 [Hob]Handle out of memory exception in HOB

Verified Bugs
-----------------------------------
1. Bug 2221<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2221> Glxgears failed in qemux86-64 on host FC16 x86.
2. Bug 2299<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2299> pseudo-nativesdk: libpseudo.so can not find GLIBC_2.14 .
3. Bug 2309<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2309> [Regression]Glxgears failed in qemux86/x86-64 on Fedora/opensuse/ubuntu
4. Bug 2351<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2351> pseudo-nativesdk not working correctly on FC16 32bit or OpenSuse 12.1 64bit
5. Bug 2361<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2361> User could not launch Hob or run bitbake if sanity network check fail

6. Bug 2362<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2362> pseudo-native failed to compile on opensuse12.1 64bit

Best Regards,

Jiajun

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120427/29c53273/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Yocto 1.2M4  RC5 detail_testreport_Fullpass.ods
Type: application/oleobject
Size: 28049 bytes
Desc: Yocto 1.2M4  RC5 detail_testreport_Fullpass.ods
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120427/29c53273/attachment.bin>