[yocto] Fullpass Test Report for Yocto 1.2 M4 RC2 Build

Wed Apr 11 23:58:33 PDT 2012

Hi all
         This is the Fullpass Test Report for Yocto 1.2 M4 RC2 build. There are totally 46 bugs found in this testing, with 9 new bugs reported. All M4 testable features in 1.2 are covered in M4 RC2 testing, including yocto-bsp/kernel, multilib, Hob, etc.
BSP and QEMU targets work well without major issue. The only 2 new issues on BSPs are connman-property launch failure and and Glib-Gobject-warning, no login via serial console on beagleboard.
For Hob testing against poky/master branch with commint 190f6d791d51aaa4cfb9f1cf932bc205ff674fb5, there are 3 new bugs reported. Most functions could work well in Hob, hob-toochain may build pass but it depends on image profile. Inconformity recipes when change machine; uses can’t remove meta-yocto layer.
For Core build system, there are 1 new bugs reported, Archive source failed when build image. Other issues like disk monitor and image build with PAM could not work, Multilib build appliance could not work at all.
For ADT, except for Clutter template creation, most functions could work well and glxgears can work well on most host machine. A new minor issue is reported for ADT installer with ppc kernel.
     For distribution testing, N+1 testing is performed against CentOS/Ubuntu/Opensuse/Fedora. Two new bugs are CentOS 5.6 x86 failed to compile gdb and Fedora 17 alpha x86-64 failed to compile mklibs-native.
For build performance, one qemux86 sato build on a Core i7 machine costs 112 minutes, which saves 13.8% time than the result in 1.2 M4 RC1 build(130 minutes)
     Note: The test report is archived @ https://wiki.yoctoproject.org/wiki/Yocto_1.2_Milestone_Test_Report.

Test Summary:
---------------------------------------

Test Result Summary

Component

Target

Status

Comments

BSP

eMenlow

GOOD

I2C_transfer error; No connman icon

Blacksand

GOOD

No connman icon

Crownbay-emgd

GOOD

Xorg eat lots of cup's resource after standby;
No connman icon

Sugarbay

GOOD

No connman icon

Jasperforest-lsb

GOOD

Everything runs well

Beagleboard

GOOD

Can not login via serial port

Routerstationpro

GOOD

Everything runs well

Mpc8315e-rdb

GOOD

Everything runs well

QEMU

qemux86

GOOD

No connman icon on the toolbar;

qemux86-64

GOOD

No connman icon on the toolbar; Glib-Gobject-warning when launch connman-properties

qemuarm

GOOD

No connman icon on the toolbar

qemuppc

GOOD

No connman icon on the toolbar

qemumips

GOOD

Fotowall can not be launched;No connman icon

ADT

BUGGY

Fail to download qemuppc's kernel; runqemu cannot launch with toochainSDK on FC16/opensuse x86-64 host; Clutter failed run

Compliance

GOOD

No regression issue with LTP and POSIX

Core build system

BUGGY

Useradd command fails after 10 retries; Archive source failed when build image; Fail to build image with PAM support; Self-hosted-image testing is blocked due to build failure;Multilib has different build/boot issues with different settings;Disk monitor doesn't work sometimes; yocto-bsp fail to create bsp with json file

Power/Performance

GOOD

Power consumption of crownbay is 24.28W

Stress

GOOD

Both Crashme and Helltest on Jasperforest and both Crashme and LTP on Beagleboard could pass 24 hours stress testing.

HOB

BUGGY

Removing meta-yocto layer fail,inconformity recipes when change machine, BB_NUMBER_THREADS and PARALLEL_MAKE could not work;template file and hob-image .bb file has incomplete package information.

Critical bugs, more than 50% test cases are blocked

Only Normal, Minor or Enhancement bugs, or less than 10% test cases failed

Normal, Major and Critical bugs, and more than 10% test cases failed

Detailed Test Result for each component

Target

Total TCs

Not Run

Passed

Failed

Not testable (Blocked)

eMenlow Sato-SDK

65

0

63

2(bug 310,2202)

0

n450 Sato-SDK

64

0

63

1 (bug 2202)

0

Crownbay-Sato-SDK

65

0

63

2(bug 2202,1258)

0

Sugarbay Sato-SDK

69

0

68

1(bug 2202)

0

Jasperforest lsb-SDK

32

0

32

0

0

Beagleboard Sato-SDK

40

0

40

0

0

Routerstationpro Sato-SDK

33

0

33

0

0

Mpc8315e-rdb Sato-SDK

36

0

36

0

0

Qemux86-64 Sato

31

0

30

1(bug 2202 2282)

0

Qemux86-64 Sato-SDK

35

0

34

1(bug 2202 2282)

0

Qemux86 Sato

31

0

30

1(bug 2202)

0

Qemux86 Sato-SDK

35

0

34

1(bug 2202)

0

Qemuarm Sato

30

0

29

1(bug 2202)

0

Qemuarm Sato-SDK

35

0

34

1(bug 2202)

0

Qemumips Sato

30

0

29

1(bug 2202)

0

Qemumips Sato-SDK

35

0

34

1(bug 2202)

0

Qemuppc Sato

30

0

29

1(bug 2202)

0

Qemuppc Sato-SDK

35

0

34

1(bug 2202)

0

ADT

48

0

39

9(bug 1452, 1968,2059,2068,2068,2263,2221)

0

Core build system

69

0

52

15(bug 2214,2218,2212,2224,2209,2216,2272)

2(bug 2209)

Power & Performance

4

0

4

0

0

Compliance

2

0

2

0

0

Stress

3

0

3

0

0

HOB

38

0

32

6(bug2210,2211,2217,2245,2255,2274)

0

Total

895

0

847

46

2

   Distribution test
--------------------------------------
  Distribution

Host Arch

1.2 M4 rc1Build

Fedora 16

x86_64

PASS

Fedora 17 Alpha

x86_64

Failed(bug2278)

Ubuntu 11.10

x86_64

PASS

Ubuntu 12.04 Beta 1

x86_64

PASS

Opensuse 12.1

x86_64

PASS

Opensuse 12.2 Milestone 2

x86_64

Failed(bug2201)

Centos 5.6

x86

Failed(bug2264)

Centos 6.2

x86_64

PASS

Commit Information
---------------------------------------
Tree/Branch:1.2_M4
Commit id: 27aa5489fb40d99c58bb2a4bbf2311e2c502a091

Meta branch: master
Commit id: f89405e115d73426c8a6450b6e795b5885d6bdf3

HOB Information:
Tree/Branch: Poky/master
Commint id: 190f6d791d51aaa4cfb9f1cf932bc205ff674fb5

Issue summary
--------------------------------------

Component

Bug Number

Target Milestone

System & Core OS

System Usage

Bug 503 - [emenlow] system cannot enter S3 standby mode

1.2

Bug 310 - [eMenlow] i2c_transfer error on e-Menlow

1.1

Other

Bug 1258 -[crownbay] Xorg eats lots of CPU time after standby

1.1 M4

Bug 2202 No connman icon on top right corner

1.2 M4

Bug 2192 [beagleboard C4] No login prompt in serial port console with 1.2 M4 20120328 build

1.2 M4

Bug 2190 [autobuilder] dtb file for mpc8315e-rdb should be copied to the sharing directory

1.2 M4

Bug 1477 Fotowall doesn't run on qemumips

1.3

New! Bug 2265 [autobuilder] version field in kernel name mismatched with real version

Core build system

Bug 2212 Disk monitor doesn't work if not set disk_inode_interval

1.2 M4

Bug 2218 Fail to build image with PAM support

1.2 M4

Bug 2209 [self-hosted] genext2fs fails to generate rootfs without enough memory

1.2 M4

Bug 2214 [multilib] lib32/lib64 build failed due to lack of libc6

1.2 M4

Bug 2216 [multilib] core-image-sato ipk build failed due to dependency of lib32-ofono

1.2 M4

Bug 2224 [multilib] lib32/lib64 lsb build failed due to pam.conf missing for recipe at

1.2 M4

New! Bug 2282 - Glib-Gobject-warning when launch connman-properties on qemux86-64 sato/sato-sdk image

New! Bug 2272 - Archive source failed when build image

ADT

Bug 1452 32bit kernel generated oprofile data file can not be read by remote 64bit opreport

1.3

Bug 2059 Fail to build project based on Clutter with taget arch arm/mips/ppc

1.2 M3

Bug 2068 Clutter run failed in qemux86/x86-64 using toolchain

1.3

Bug 2069 Clutter run failed in qemux86/qemux86_64.

1.2 M4

Bug 1968 "runqemu" can't launch a qemu target image using toolchainSDK on FC16/Opensuse x86_64

1.2 M2

Bug 2115 Add prompt for Perf/LatencyTop when eclipse connect to target (low property)

1.3

Bug 2116 Setting Kernel Module as a required field for SystemTap (low property)

1.3

Bug 2118 No response from SystemTap after connection timed out (low property)

1.3

Bug 2221 Glxgears failed in qemux86-64 on host FC16 x86

--

New! Bug 2263 Fail to download qemuppc's kernel using ADT-installer

Power & Performance

Power consumption of crownbay is 24.28W.

Build time for a sato image on Core i7 860 machine is 112 minutes. Detailed result on

https://wiki.yoctoproject.org/wiki/Performance_Test

Compliance

LTP result: https://wiki.pokylinux.org/wiki/LTP_result<https://wiki.pokylinux.org/wiki/LTP_result>

POSIX result: https://wiki.pokylinux.org/wiki/Posix_result<https://wiki.pokylinux.org/wiki/Posix_result>

Stress

Both Crashme and Helltest on Jasperforest and both Crashme and LTP on Beagleboard could pass 24 hours stress testing

Distribution Support

Bug 2201 m4-native do_compile task failed when building image on opensuse12.2 Milestone2

--

New! Bug 2264 gdb failed to compile on 32bit host

New! Bug 2278 mklibs-native failed to compile fedora17 64bit

HOB

Bug 2210 [Hob] BB_NUMBER_THREADS and PARALLEL_MAKE do not work

1.2 M4

Bug 2211 [Hob] removed package still shown as selected after image build finished

1.2 M4

Bug 2217 [Hob] fail to build hob-image-hob.bb from command line

1.2 M4

New! Bug 2254 [Hob2]Removing meta-yocto layer meet with OE-core's config fail

New! Bug 2255[Hob2]Inconformity recipes when change machine

New! Bug 2274 [Hob2]ToochainSDK depends on image profile

Verified Bugs
-----------------------------------

ADT:
1.2055 ADT-installer failed to install PPC Target architectures

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2055
2.2066 Glxgears failed in qemuarm/qemuppc/qemumips.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2066
3.Remove the popup of "Canceled" in Oprofile/lttng/powertop (low property)

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2117
4.Bug 1927 Glxgears failed in qemux86-64
https://bugzilla.yoctoproject.org/show_bug.cgi?id=1927
5.Glxgears failed in qemux86/x86-64 on ubuntu 11.10 x86-64 with nVidia GLX driver
https://bugzilla.yoctoproject.org/show_bug.cgi?id=1886

Core Build system:

1. [multilib] core-image-sato build pass but X not up

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2215

Best Regards,
Hongna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120412/d0c292f1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Yocto 1.2M4  RC2 detail_testreport_fullpass.ods
Type: application/oleobject
Size: 28585 bytes
Desc: Yocto 1.2M4  RC2 detail_testreport_fullpass.ods
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120412/d0c292f1/attachment.bin>