[yocto] Yocto 1.5.3 Full Pass test report

Georgescu, Alexandru C alexandru.c.georgescu at intel.com
Mon Jul 21 05:43:01 PDT 2014 

Hello,
Here is the full pass test report for Yocto 1.5.3 release build.

New Issue Summary:
                - 6535 - ADT installation failed for x86_64 with build 1.5.3 rc1
                - 6528 - Check if AUTOPR could be export/import for package build
                - 5128 - [Beagleboard xM]X can not startup with 3.10 kernel - this is a known issue marked as wontfix since support for beagleboard is dropped. 1.5.2 and previous point releases inherits this issue too.
                - 5473 - During this test run, we've discovered that Eclipse Kepler plugin on Dora branch cannot create a new Bitbake Commander project using Git Clone, therefore we've used Master instead which preserves the same functionality and compatibility for Dora.
                - On the Distro Testing side, all tests are green on all distributions excepting nightly-world on OpenSuse 13.1 which is giving constant timeouts and on Ubuntu 14.04 which is failing at do_compile for gtk+. We are currently investigating the both issues but they are not release blockers.
            - there were other failures during the build, like wrong Build Appliance built or missing toolchains for ADT-installers, issues ` have been fixed and retested on time.

The full pass report is available at this link: https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3 and below, you can find the summary report.


Testing data

 Test type: Full Pass Release

 Branch: Dora

 Build name: 1.5.3

 poky commit: dc743744d8e3a9caef577aeab6b7f51bc08ded17

 Eclipse Plugin commit Kepler:  master:4a167fd662262ebdaa3cf0d332ac0debb52d7904

 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-1.5.3.rc1/

 Distributions tested: Ubuntu 14.04 64bit, Fedora 20 64bit, Opensuse 13.1 64bit, CentOS 6.5 64 bit



Summary Report
Testing Summary Report

Test Run

Test Plan

Environment

Passed

Other issues

Failed

Failing bugs

Blocked

Blocking bugs

Complete

2655<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2655>2657<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2657>

ADT: dora branch

Fedora 20 i686

82.8%

none

1.7%

6535<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6535>

15.5%

6536<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6536>6535<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6535>

100.0%

2654<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2654>2656<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2656>

ADT: dora branch

Ubuntu 14.04 x86_64

100.0%

none

0.0%

none

0.0%

none

100.0%

2641<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2641>2645<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2645>

BSP/QEMU: dora branch

Beagleboard

71.0%

none

4.3%

5182<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5182>

24.6%

5182<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5182>

100.0%

2628<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2628>2629<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2629>

BSP/QEMU: dora branch

generic-x86 on AtomPC

97.1%

none

2.9%

4005<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4005>5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

0.0%

none

100.0%

2619<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2619>2620<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2620>

BSP/QEMU: dora branch

genericx86-64 on Sugarbay

98.6%

none

1.4%

5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

0.0%

none

100.0%

2650<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2650>2651<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2651>

BSP/QEMU: dora branch

MPC8315e-rdb

100.0%

none

0.0%

none

0.0%

none

100.0%

2646<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2646>2647<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2647>

BSP/QEMU: dora branch

P1022ds

100.0%

none

0.0%

none

0.0%

none

100.0%

2630<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2630>2631<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2631>

BSP/QEMU: dora branch

qemu-x86

100.0%

none

0.0%

none

0.0%

none

100.0%

2638<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2638>2639<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2639>

BSP/QEMU: dora branch

qemuarm

100.0%

none

0.0%

none

0.0%

none

100.0%

2643<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2643>2644<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2644>

BSP/QEMU: dora branch

qemumips

100.0%

none

0.0%

none

0.0%

none

100.0%

2640<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2640>2642<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2642>

BSP/QEMU: dora branch

qemuppc

100.0%

none

0.0%

none

0.0%

none

100.0%

2652<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2652>2653<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2653>

BSP/QEMU: dora branch

qemux86-64

100.0%

none

0.0%

none

0.0%

none

100.0%

2648<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2648>2649<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2649>

BSP/QEMU: dora branch

Routerstationpro

100.0%

none

0.0%

none

0.0%

none

100.0%

2661<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2661>2662<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2662>2663<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2663>

Eclipse plugin: Dora branch

Fedora 20 x86_64

96.4%

none

0.0%

none

3.6%

5640<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5640>

100.0%

2621<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2621>2622<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2622>2623<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2623>

Eclipse plugin: Dora branch

Ubuntu 14.04 x86_64

100.0%

5473<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5473>5850<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5850>

0.0%

none

0.0%

none

100.0%

2636<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2636>2637<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2637>

Meta-yocto: dora branch

Ubuntu 14.04 x86_64

100.0%

6527<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6527>

0.0%

none

0.0%

none

100.0%

2633<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2633>2634<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2634>

Hob: dora branch

Ubuntu 14.04 x86_64

93.5%

none

6.5%

6008<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6008>5892<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5892>5995<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5995>

0.0%

none

100.0%

2626<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2626>2627<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2627>

OE-Core: dora branch

Ubuntu 14.04 x86_64

100.0%

none

0.0%

none

0.0%

none

100.0%

2624<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2624>2625<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2625>

Bitbake: dora branch

Ubuntu 14.04 x86_64

95.5%

none

4.5%

6528<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6528>

0.0%

none

100.0%

2658<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2658>

Runtime: dora branch

Multiple Environments

100.0%

none

0.0%

none

0.0%

none

100.0%

2635<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2635>

Build Appliance: dora branch

qemu-kvm on Ubuntu 12.04 libvirt

100.0%

6525<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6525>

0.0%

none

0.0%

none

100.0%

Total: 21



Test Case Status Report

Status

Count

BLOCKED

36

FAILED

12

PASSED

985

Total: 1033


[http://chart.apis.google.com/chart?cht=p3&chs=432x120&chd=t:&chd=t:36,12,985&chco=FFFF66|FF9933|00FF00&chl=BLOCKED(36)+3%25|FAILED(12)+1%25|PASSED(985)+95%25]



Bug Status Report

[Bug Status]

ID[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Priority[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Status[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Resolution[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Test Runs[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Description[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Version[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Target Milestone[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

Age (days)[?]<https://wiki.yoctoproject.org/wiki/WW28_-_2014-07-09_-_Full_Pass_Release_Dora_1.5.3>

4005<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4005>

Low

NEW

2628<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2628>

dmesg log errors in Atom-PC

1.3.1

1.7

485

5182<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5182>

Medium+

RESOLVED

WONTFIX

2645<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2645> 2641<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2641>

[Beagleboard xM]X can not startup with 3.10 kernel

1.5

1.5.1

299

5473<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5473>

Medium

REOPENED

2621<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2621>

[KEPLER] unable to create new bitbake commander project

1.5

1.5.2

246

5640<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5640>

Medium

RESOLVED

WONTFIX

2661<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2661>

systemtap output is useless in Juno

1.6

1.6

215

5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

Medium

NEW

2620<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2620> 2629<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2629>

Cannot wake from suspend state using the USB mouse or keyboard

1.6

1.6

142

5850<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5850>

Low

NEW

2622<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2622>

Application name not listed in run configurations when installing the ADT using Bitbake

1.6

Future

140

5892<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5892>

Medium

RESOLVED

FIXED

2634<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2634>

Deploy image does not always write correctly to USB

1.6

1.6 M4

132

5995<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5995>

Medium

RESOLVED

WORKSFORME

2634<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2634>

xinit:giving up on sato image

1.6

1.6 M4

113

6008<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6008>

Medium

RESOLVED

FIXED

2633<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2633>

All layers can be removed

1.4.3

1.4.3

112

6525<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6525>

Medium+

RESOLVED

FIXED

2635<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2635>

Wrong build-appliance image for 1.5.3 rc1

1.5.3

1.5.3

NEW

6527<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6527>

Medium

NEW

2637<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2637>

[1.5.3] yocto-bsp tool: Wrong tune file for qemuppc

1.5.3

1.5.3

NEW

6528<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6528>

Low

NEW

2624<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2624>

Case 147 - Check if AUTOPR could be export/import for package build (pre 1.6)

1.5.3

1.5.3

NEW

6535<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6535>

Medium

NEW

2657<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2657> 2655<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2655>

ADT installation failed for x86_64 with build 1.5.3 rc1

1.5.3

1.7

NEW

6536<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6536>

Medium

VERIFIED

FIXED

2655<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2655> 2657<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=2657>

autobuilder: for 1.5.3rc1 build, no x86-64 toolchain exists for i686 type host

1.5.3

1.5.3

NEW

Total: 14


Build Performance Test

Detailed description on how performance test are run can be found on Performance Test<https://wiki.yoctoproject.org/wiki/Performance_Test> wiki.

Performance Charts

On the build performance side, 1.5.3 overall the build time increased. Building the rootfs increased with about 1:30 minutes on Ubuntu host (from 4:50 to 6:21 on average) and about 50 seconds on Fedora host (from 4:03 to 4:56 on average). Baking a sato image with rm_work increased with about 2 minutes (from 1:09:07 hours to 1:11:) and 20 seconds (from 1:13:56 to 1:14:16). The kernel remained on average the same (14:25 minutes on Ubuntu and 12:26 minutes on Fedora).

Below you can see some snapshots of the charts, but you can view more details on this link<https://wiki.yoctoproject.org/charts/charts_15/performance_test.html>.

For Bitbake core-image-sato with rm_work

[Yocto 153 - build core-image-sato with rm work.png]<https://wiki.yoctoproject.org/wiki/File:Yocto_153_-_build_core-image-sato_with_rm_work.png>

Building the Virtual kernel

[Yocto 153 - bitbake virtual kernel.png]<https://wiki.yoctoproject.org/wiki/File:Yocto_153_-_bitbake_virtual_kernel.png>

Building the rootfs

[Yocto 153 - creating rootfs for core-image-sato.png]<https://wiki.yoctoproject.org/wiki/File:Yocto_153_-_creating_rootfs_for_core-image-sato.png>

Distro Testing

Distro Testing is intended to catch bugs that are distribution specific using the yocto-autobuilder. The tests are all run on identical hardware and with all OS-es updated. The distributions used will be Fedora, Ubuntu, CentOS, OpenSuse with their latest update. More details on the coverage for the Distro testing is available here at Distribution_Support#Distro_Testing<https://wiki.yoctoproject.org/wiki/Distribution_Support#Distro_Testing> wiki.

Summary

All tests are green on all distributions excepting nightly-world on OpenSuse 13.1 which is giving constant timeouts and on Ubuntu 14.04 which is failing at do_compile for gtk+. We are currently investigating the both issues.

Distro Results


BRANCH

master

COMMIT

Poky: dc743744d8e3a9caef577aeab6b7f51bc08ded17

RELEASE

1.5.3 RC1

Ubuntu 14.04 LTS

* nighlty-world

FAIL

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



Fedora 20

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



CentOS 6.5

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



OpenSuse 13.1

* nighlty-world

FAIL

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS


Regards,
--
Alexandru Georgescu
Yocto QA Engineer
SSG/SSD Open Source Technology Center Romania
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 10437 bytes
Desc: image001.jpg
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 12608 bytes
Desc: image002.jpg
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 458 bytes
Desc: image003.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 94351 bytes
Desc: image004.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 86433 bytes
Desc: image005.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 91620 bytes
Desc: image006.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20140721/e0822a0a/attachment-0003.png>

More information about the yocto mailing list