[yocto] Fullpass Test Report for Yocto 1.2 M4 RC1 Build

Mon Apr 2 00:32:53 PDT 2012

Hi all,
         This is the Fullpass Test Report for Yocto 1.2 M4 RC1 build. There are totally 36 bugs found in this testing, with 20 new bugs reported. All M4 testable features in 1.2 are covered in M4 RC1 testing, including yocto-bsp/kernel, multilib, Hob, etc. More distributions are added into distro testing.
         BSP and QEMU targets work well without major issue. The only 2 new issues on BSPs are connman-property launch failure and no login via serial console on beagleboard. For Hob, there are 4 new bugs reported. Most functions could work well in Hob, except that toolchain build could not work; BB_NUMBER_THREADS and PARALLEL_MAKE could not work; template file and hob-image .bb file has incomplete package information. For Core build system, there are 10 new bugs reported. Disk monitor and image build with PAM could not work. multilib, x32 and build appliance could not work at all. For ADT, except for Clutter template creation, most functions could work well. A new minor issue is reported for ADT installer with ppc.
         For distribution testing, N+1 testing is performed against CentOS/Ubuntu/Opensuse/Fedora. CentOS 5.x could not run bitbake due to old version of python. Opensuse 12.2 M2 failed to compile m4-native.
         For build performance, one qemux86 sato build on a Core i7 machine costs 130 minutes, which saves 2% time than the result in 1.2 M3 RC1 build(133 minutes).

      Note: The test report is archived @ https://wiki.yoctoproject.org/wiki/Yocto_1.2_Milestone_Test_Report.

Test Summary:
---------------------------------------
Test Result Summary

Component

Target

Status

Comments

BSP

eMenlow

GOOD

I2C_transfer error; No connman icon

Blacksand

GOOD

No connman icon

Crownbay-emgd

GOOD

Xorg eat lots of cup's resource after standby;
No connman icon

Sugarbay

GOOD

No connman icon

Jasperforest-lsb

GOOD

I/OAT DMA engine init failed

Beagleboard

GOOD

Can not login via serial port

Routerstationpro

GOOD

Everything runs well

Mpc8315e-rdb

GOOD

Everything runs well

QEMU

qemux86

GOOD

No connman icon

qemux86-64

GOOD

No connman icon

qemuarm

GOOD

No connman icon

qemuppc

GOOD

No connman icon

qemumips

GOOD

Fotowall can not be launched;No connman icon

ADT

BUGGY

ADT-installer failed insatll qemuppc; All Clutter relatet testing failed

Distribution Support

GOOD

m4-native do_compile task failed on opensuse12.2 M2

Compliance

GOOD

No regression issue with LTP and POSIX

Yocto-BSP/Kernel

GOOD

Fails to create bsp with json file;fail to find bblayers if TOPDIR is not "build"

Core build system

BUGGY

build applicance, multilib and x32 could not pass build; fail to build image with PAM support; Disk monitor doesn't work sometimes;

Power/Performance

GOOD

Power consumption of sugarbay and crownbay are 50.05W and 25.1W respectively

Stress

GOOD

Both Crashme and Helltest on Jasperforest and both Crashme and LTP on Beagleboard could pass 24 hours stress testing.

HOB

GOOD

Toolchain build, BB_NUMBER_THREADS and PARALLEL_MAKE could not work;template file and hob-image .bb file has incomplete package information.

Critical bugs, more than 50% test cases are blocked

Only Normal, Minor or Enhancement bugs, or less than 10% test cases failed

Normal, Major and Critical bugs, and more than 10% test cases failed

Detailed Test Result for each component

Target

Total TCs

Not Run

Passed

Failed

Not testable (Blocked)

eMenlow Sato-SDK

65

0

63

2(bug 310,2202)

0

n450 Sato-SDK

64

0

63

1 (bug 2202)

0

Crownbay-Sato-SDK

65

0

63

2(bug 2199,1258)

0

Sugarbay Sato-SDK

69

0

68

2(bug 2202)

0

Jasperforest lsb-SDK

32

0

32

0

0

Beagleboard Sato-SDK

40

0

40

0

0

Routerstationpro Sato-SDK

33

0

33

0

0

Mpc8315e-rdb Sato-SDK

36

0

36

0

0

Qemux86-64 Sato

31

0

31

0

0

Qemux86-64 Sato-SDK

35

0

35

0

0

Qemux86 Sato

31

0

31

0

0

Qemux86 Sato-SDK

35

0

35

0

0

Qemuarm Sato

30

0

30

0

0

Qemuarm Sato-SDK

35

0

35

0

0

Qemumips Sato

30

0

30

0

0

Qemumips Sato-SDK

35

0

35

0

0

Qemuppc Sato

30

0

30

0

0

Qemuppc Sato-SDK

35

0

35

0

0

ADT

47

0

38

9(bug 1968,2055,2059,2066,2068,2069)

0

Core build system

69

0

49

18(bug 2209,2212,2214,2215,2216,2218,2222, 2223, 2224)

2(bug 2209)

Power & Performance

4

0

4

0

0

Compliance

2

0

2

0

0

Stress

2

0

2

0

0

HOB

38

0

34

4(bug2210,2211,2217,2285)

0

Total

893

0

854

37

2

Distribution test
--------------------------------------
Distribution

Host Arch

1.2 M4 RC1 Build

Fedora 16

x86_64

PASS

Fedora 17 Alpha

x86_64

PASS

Ubuntu 11.04

x86_64

PASS

Ubuntu 11.10

x86_64

PASS

Ubuntu 12.04 Beta 1

x86_64

PASS

Opensuse 12.1

x86_64

PASS

Opensuse 12.2 Mileston 2

x86_64

Failed(bug2201)

Centos 5.6

x86

PASS(bug 2220)

Centos 5.7

x86

PASS(bug 2220)

Centos 6.2

x86_64

PASS

Glxgears&Clutter test
--------------------------------------
Case

Host

Target

qemuarm/ppc/mips

qemux86

qemux86-64

glxgear

32b Fedora 16

Bug 2066

PASS

Bug 1927

64b Ubuntu 11.10

Bug 2066

Bug 1886

Bug 1886

64b Opensuse 12.1

Bug 2066

PASS

PASS

Clutter

32b Fedora 16

Bug 2059

Bug 2068

Bug 1927

64b Ubuntu 11.10

Bug 2059

Bug 2069

Bug 2069

64b Opensuse 12.1

Bug 2059

Bug 2068

Bug 2068

Commit Information
---------------------------------------

Tree/Branch: Poky/1.2_M4

Commit id: 4d9f4d6ac25f39fe6d5491d05c7a26195bab648b

Meta branch: master

Commit id: f89405e115d73426c8a6450b6e795b5885d6bdf3

Issue summary
--------------------------------------
Component

Bug Number

Target Milestone

System & Core OS

System Usage

Bug 503 - [emenlow] system cannot enter S3 standby mode

1.2

Bug 1840 - glxgears fail to run on Blacksand

1.2

Bug 1477 - Fotowall doesn't run on qemumips

Bug 310 - [eMenlow] i2c_transfer error on e-Menlow

1.1

Other

Bug 1258 -[crownbay] Xorg eats lots of CPU time after standby

1.1 M4

New! Bug 2202 No connman icon on top right corner

New! Bug 2192 [beagleboard C4] No login prompt in serial port console with 1.2 M4 20120328 build

New! Bug 2190 [autobuilder] dtb file for mpc8315e-rdb should be copied to the sharing directory

New! Bug 2191 [autobuilder] i686-x86_64 toolchain is copied to the incorrect sharing directory

Core Build System

New! Bug 2212 Disk monitor doesn't work if not set disk_inode_interval

1.2 M4

New! Bug 2218 Fail to build image with PAM support

New! Bug 2209 [self-hosted] genext2fs fails to generate rootfs without enough memory

New! Bug 2214 [multilib] lib32/lib64 build failed due to lack of libc6

New! Bug 2215 [multilib] core-image-sato build pass but X not up

New! Bug 2216 [multilib] core-image-sato build pass but X not up

New! Bug 2219 [yocto-kernel] fail to find bblayers if TOPDIR is not "build"

New! Bug 2223 - [x32] gcc-cross-intermediate do_install fail for x32 build

New! Bug 2224 - [multilib] lib32/lib64 lsb build failed due to pam.conf missing for recipe at

New! Bug 2222 [yocto-bsp] fails to create bsp with json file

ADT

Bug 2055 ADT-installer failed to install PPC Target architectures

1.2 M3

Bug 2058 Keyboard no response on qemuppc when using NFS

Bug 2059 Fail to build project based on Clutter with taget arch arm/mips/ppc

1.2 M3

Bug 2066 Glxgears failed in qemuarm/qemuppc/qemumips.

1.2 M3

Bug 2068 Clutter run failed in qemux86/x86-64 using toolchain

1.2 M3

Bug 2069 Clutter run failed in qemux86/qemux86_64.

1.2 M3

Bug 1968 "runqemu" can't launch a qemu target image using toolchainSDK on FC16/Opensuse x86_64

1.2 M2

Bug 2115 Add prompt for Perf/LatencyTop when eclipse connect to target (low property)

1.2 M3

Bug 2116 Setting Kernel Module as a required field for SystemTap (low property)

1.2 M3

Bug 2117 Remove the popup of "Canceled" in Oprofile/lttng/powertop (low property)

1.2 M3

Bug 2118 No response from SystemTap after connection timed out (low property)

1.2 M3

Power & Performance

Power consumption of sugarbay and crownbay are 50.05W and 25.1W respectively.
Build time for a sato image on Core i7 860 machine is 130 minutes. Detailed result on
https://wiki.yoctoproject.org/wiki/Performance_Test

Compliance

LTP result: https://wiki.pokylinux.org/wiki/LTP_result<https://wiki.pokylinux.org/wiki/LTP_result>

POSIX result: https://wiki.pokylinux.org/wiki/Posix_result<https://wiki.pokylinux.org/wiki/Posix_result>

Stress

Both Crashme and Helltest on Jasperforest and both Crashme and LTP on Beagleboard could pass 24 hours stress testing.

Distribution Support

New! Bug 2220 [CentOS 5.x] bitbake fails to run due to old python

New! Bug 2201 m4-native do_compile task failed when building image on opensuse12.2 Milestone2

Hob

New! Bug 2185 [Hob2] hob-toolchain build failed

New! Bug 2210 [Hob] BB_NUMBER_THREADS and PARALLEL_MAKE do not work

New! Bug 2211 [Hob] removed package still shown as selected after image build finished

New! Bug 2217 [Hob] fail to build hob-image-hob.bb from command line

Best Regards,

Jiajun

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120402/9895196f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Yocto 1.2M4  RC1 detail_testreport_fullpass.ods
Type: application/oleobject
Size: 26650 bytes
Desc: Yocto 1.2M4  RC1 detail_testreport_fullpass.ods
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20120402/9895196f/attachment.bin>