[yocto] Fullpass Test Report for Yocto 1.2 M4 RC3 Build

Wed Apr 18 20:08:37 PDT 2012

Hi all,
This is the Fullpass Test Report for Yocto 1.2 M4 RC3 build. There are 6 new bugs found. The most critical one is for routerstationpro which blocked all cases on routerstationpro , reported as bug 2292; There are 3 new minor bugs reported for build appliance, x32 and Hob, as bug 2324, 2300 and 2325; 2 bugs are new reported for ADT that libpseudo.so can’t find GLIBC_2.14 and glxgears fail in qemux86/x86-64.
Except routerstationpro , all BSPs and QEMU targets work well without major issue, there are 4 open bugs , they are: unable to set the static IP on all BSPs and qemu target, CPU utilization is high level after standby on crownbay; I2C_transfer error on eMenlow and connman-property launch with Glib-Gobject-warning on qemux86-64 image.
For Hob testing, it is based on master with commit 720fe9e371eef16da19df0885188aa0341ad2090, there is only minor issue found with it that “View files” does not work on Ubuntu 11.04. All the other functions could work well.
For ADT, except clutter, all features work well except the old bug 32bit kernel generated oprofile data file can’t be read by remote 64bit operation.
For distribution testing, N+1 testing is performed against CentOS/Ubuntu/Opensuse/Fedora. The build issues on Fedora 17 Alpha and Opensuse 12.2 Milestone 2 still exist.
For Core build feature, most functions could work, except for multilib. There are some new failures with multilib, which we do not submit bugs for them since developers already have fixing patches. Except for build appliance, all the other components are based on RC3. Build appliance is based on master with commit 720fe9e371eef16da19df0885188aa0341ad2090. Yocto-bsp tool fails to create layer against json file since Tom’s fixing patch is not in RC3 yet.
For build performance, one qemux86 sato build on a Core i7 machine costs 110 minutes, which is almost the same as the result in 1.2 M4 RC2 build(112 minutes)
Note: The test report is archived @ https://wiki.yoctoproject.org/wiki/Yocto_1.2_Milestone_Test_Report.

Test Summary:
---------------------------------------
Test Result Summary

Component

Target

Status

Comments

BSP

eMenlow

GOOD

I2C_transfer error; No connman icon

Blacksand

GOOD

No connman icon

Crownbay-emgd

GOOD

Xorg eat lots of cup's resource after standby;
No connman icon

Sugarbay

GOOD

No connman icon

Jasperforest-lsb

GOOD

Everything runs well

Beagleboard

GOOD

Can not login via serial port

Routerstationpro

GOOD

Everything runs well

Mpc8315e-rdb

GOOD

Everything runs well

QEMU

qemux86

GOOD

No connman icon on the toolbar

qemux86-64

GOOD

No connman icon on the toolbar;Glib-Gobject-warning when launch connman-properties

qemuarm

GOOD

No connman icon on the toolbar

qemuppc

GOOD

No connman icon on the toolbar

qemumips

GOOD

Fotowall can not be launched;No connman icon

ADT

BUGGY

libpseudo.so can not find GLIBC_2.14; Glxgears run fail on qemux86/x86-64 block all cases about clutter;32bit kernel generated oprofile data file can not be read by remote 64bit operation

Distribution Support

GOOD

m4-native do_compile task failed on Opensuse 12.2 Milestone 2;mklibs-native failed to compile fedora17 64bit

Compliance

GOOD

No regression issue with LTP and POSIX

Core build system

GOOD

yocto-bsp fails to create bsp with json file;"Run image" not workable in Hob;gcc-cross-initial and libgcc fails to do_patch with x32 build

Power/Performance

GOOD

Power consumption of crownbay is 24.109W

Stress

GOOD

Both Crashme and Helltest on Jasperforest and both Crashme and LTP on Beagleboard could pass 24 hours stress testing.

HOB

GOOD

Everything runs well

Critical bugs, more than 50% test cases are blocked

Only Normal, Minor or Enhancement bugs, or less than 10% test cases failed

Normal, Major and Critical bugs, and more than 10% test cases failed

Detailed Test Result for each component

Target

Total TCs

Not Run

Passed

Failed

Not testable (Blocked)

eMenlow Sato-SDK

65

0

63

2(bug 310,2202)

0

n450 Sato-SDK

64

0

63

1 (bug 2202)

0

Crownbay-Sato-SDK

65

0

63

2(bug 2202,1258)

0

Sugarbay Sato-SDK

69

0

68

1(bug 2202)

0

Jasperforest lsb-SDK

32

0

32

0

0

Beagleboard Sato-SDK

40

0

40

0

0

Routerstationpro Sato-SDK

33

0

0

0

33(bug 2292)

Mpc8315e-rdb Sato-SDK

36

0

36

0

0

Qemux86-64 Sato

31

0

30

1(bug 2202 2282)

0

Qemux86-64 Sato-SDK

35

0

34

1(bug 2202 2282)

0

Qemux86 Sato

31

0

30

1(bug 2202)

0

Qemux86 Sato-SDK

35

0

34

1(bug 2202)

0

Qemuarm Sato

30

0

29

1(bug 2202)

0

Qemuarm Sato-SDK

35

0

34

1(bug 2202)

0

Qemumips Sato

30

0

29

1(bug 2202)

0

Qemumips Sato-SDK

35

0

34

1(bug 2202)

0

Qemuppc Sato

30

0

29

1(bug 2202)

0

Qemuppc Sato-SDK

35

0

34

1(bug 2202)

0

ADT

49

0

39

4(bug 1452, 2299)

6(bug 2068 2069 2221 2309)

Core build system

63

0

49

4(bug 2222)

0

Power & Performance

4

0

4

0

0

Compliance

2

0

2

0

0

Stress

3

0

3

0

0

HOB

38

0

38

0

0

Total

890

0

817

24

39

* You can check the detailed test result in attachment for each target.
** The failed/blocked case number is listed with failed cases’ bug number.

Distribution test
--------------------------------------
Distribution

Host Arch

1.2 M4 rc3Build

Fedora 16

x86_64

PASS

Fedora 17 Alpha

x86_64

Failed(bug2278)

Ubuntu 11.10

x86_64

PASS

Ubuntu 12.04 Beta 1

x86_64

PASS

Opensuse 12.1

x86_64

PASS

Opensuse 12.2 Milestone 2

x86_64

Failed(bug2201)

Centos 5.6

x86

PASS

Centos 6.2

x86_64

PASS

Commit Information
---------------------------------------
For Blacksand/Crownbay/Sugarbay/Jasperforest/eMenlow
Tree/Branch: Poky/1.2_M4
Commit id: 14286fe71e037ce895f01b51ac66a8986100b18f
Meta branch: master
Commit id: b669b9363197a6af2014c5d54b2c651894e62a5e

For qemuarm/qemumips/qemuppc/qemux86/qemux86-64/ADT/eclipse-plugin/toolchain/Beagleboard/ Routerstationpro/ Mpc8315e-rdb
Tree/Branch: Poky/1.2_M4
Commit id: fae1c7a5fdcea199338b5f91efaafd7c72aec5dd

For HOB/Build appliance:
Tree/Branch: Poky/master
Commit id: 720fe9e371eef16da19df0885188aa0341ad2090

Issue summary
--------------------------------------
Component

Bug Number

Target Milestone

System & Core OS

System Usage

Bug 310 - [eMenlow] i2c_transfer error on e-Menlow

1.1

Bug 1258 -[crownbay] Xorg eats lots of CPU time after standby

1.1 M4

Other

Bug 2282 - Glib-Gobject-warning when launch connman-properties on qemux86-64 sato/sato-sdk image

1.2 M4

Bug 2202 No connman icon on top right corner

1.2 M4

New! Bug 2292 [routerstationpro] routerstationpro boot failed with 1.2 rc3

Core build system

Bug 2222 [yocto-bsp] fails to create bsp with json file

1.2 M4

New! 2324 [Build Appliance] "Run image" not workable in Hob

1.2 M4

New! 2300 gcc-cross-initial and libgcc fails to do_patch with x32 build

1.2

ADT

Bug 1452 32bit kernel generated oprofile data file can not be read by remote 64bit operation

1.3

Bug 2068 Clutter run failed in qemux86/x86-64 using toolchain

1.3

Bug 2069 Clutter run failed in qemux86/qemux86_64 on ubuntu 11.10 x86-64.

1.2 M4

Bug 2221 Glxgears failed in qemux86-64 on host FC16 x86

--

New! Bug 2299 pseudo-nativesdk: libpseudo.so can not find GLIBC_2.14

New! Bug 2309 [Regression]Glxgears failed in qemux86/x86-64 on Fedora/opensuse/ubuntu

Power & Performance

Power consumption of crownbay is 24.109W.

Build time for a sato image on Core i7 860 machine is 110 minutes. Detailed result on

https://wiki.yoctoproject.org/wiki/Performance_Test

Compliance

LTP result: https://wiki.pokylinux.org/wiki/LTP_result<https://wiki.pokylinux.org/wiki/LTP_result>

POSIX result: https://wiki.pokylinux.org/wiki/Posix_result<https://wiki.pokylinux.org/wiki/Posix_result>

Stress

Both Crashme and Helltest on Jasperforest could pass 24 hours stress testing.Crashme and LTP on Beagleboard are still running.

Distribution Support

Bug 2201 m4-native do_compile task failed when building image on opensuse12.2 Milestone2

--

Bug 2278 mklibs-native failed to compile fedora17 64bit

1.2 M4

HOB

New! 2325 [Hob] "View files" does not work on my 32bit Ubuntu 11.04

Verified Bugs
-----------------------------------
ADT:

1.     Bug 2263<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2063> Fail to download qemuppc's kernel using ADT-installer

2.     Bug 1968<https://bugzilla.yoctoproject.org/show_bug.cgi?id=1968> "runqemu" can't launch a qemu target image using toolchainSDK on FC16/Opensuse x86_64

3.     Bug 2059<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2059> Fail to build project based on Clutter with taget arch arm/mips/ppc

4.     Bug 2066<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2066> Glxgears failed in qemuarm/qemuppc/qemumips.

5.     Bug 2301<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2301> ADT repo for 1.2M4rc3 fail to install qemux86-64 image

6.     Bug 2270<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2270> Cross configure fail using sysroot forder of poky tree and meta-toolchain

HOB:

1.     Bug 2210<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2210> [Hob] BB_NUMBER_THREADS and PARALLEL_MAKE do not work

2.     Bug 2211<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2211> [Hob] removed package still shown as selected after image build finished

3.     Bug 2254<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2254> [Hob2]Removing meta-yocto layer meet with OE-core's config fail

4.     Bug 2255<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2255> [Hob2] Inconformity recipes when change machine

5.     Bug 2274<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2274> [Hob2]ToochainSDK depends on image profile

6.     Bug 2285<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2285> [Hob] multilib recipes selected in View Recipes with PREFERRED_PROVIDER set

Others:

1. Bug 2192<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2192> [beagleboard C4] No login prompt in serial port console with 1.2 M4 20120328 build

2. Bug 2209<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2209> [self-hosted] genext2fs fails to generate rootfs without enough memory

Best Regards,
Hongna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120419/a8d771e4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Yocto 1.2M4  RC3 detail_testreport_Fullpass.ods
Type: application/oleobject
Size: 27887 bytes
Desc: Yocto 1.2M4  RC3 detail_testreport_Fullpass.ods
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120419/a8d771e4/attachment.bin>