[yocto] Yocto 1.7_M4.rc5 Full Pass test report

Georgescu, Alexandru C alexandru.c.georgescu at intel.com
Fri Oct 24 00:39:12 PDT 2014 

Hello,
Here is the Full Pass test report for the Yocto 1.7_M4.rc5 build, which is also the last release candidate for Yocto 1.7 known also as Dizzy.

Overall there were no regressions found during this release candidate full pass test which could block the release. One of the areas which shows an increase in the no of bugs is Toaster but for Toaster there will be a separate release so having this in mind it is expected for them to be fixed until its separate release. Also, Distro testing is all green and there are no degradations in the build performance tests.
Below you can see the summary report for our Full Pass, along with Distro and build performance charts. The full pass test report on the wiki can be accessed here: https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5.

Testing data

 Test type:  Full Pass Release Test

 Branch: master

 Build name: 1.7_M4.rc5

 poky commit: 924da20c6f2f183fa7e930e4c3c6b14c090acbe5.tar

 Eclipse Plugin commit Kepler: 4a167fd662262ebdaa3cf0d332ac0debb52d7904

 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-1.7.rc5/

 Distributions tested: Ubuntu 14.04 64bit, Fedora 20 64bit, Opensuse 13.1 64bit, CentOS 7 64 bit

Summary Report
Testing Summary Report

Test Run

Test Plan

Environment

Passed

Other issues

Failed

Failing bugs

Blocked

Blocking bugs

Complete

3038<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3038>3040<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3040>

ADT: master branch

Fedora 20 i686

100.0%

none

0.0%

none

0.0%

none

100.0%

3037<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3037>3039<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3039>

ADT: master branch

Ubuntu 14.04 x86_64

100.0%

none

0.0%

none

0.0%

none

100.0%

3053<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3053>3054<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3054>

BSP/QEMU: master branch

Beaglebone Black

98.7%

none

1.3%

6665<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6665>

0.0%

none

100.0%

3055<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3055>3058<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3058>

BSP/QEMU: master branch

EdgeRouter

100.0%

none

0.0%

none

0.0%

none

100.0%

3075<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3075>

BSP/QEMU: master branch

generic-x86 on Atom N2600

100.0%

none

0.0%

none

0.0%

none

100.0%

3076<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3076>

BSP/QEMU: master branch

generic-x86 on AtomPC

100.0%

none

0.0%

none

0.0%

none

100.0%

3071<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3071>

BSP/QEMU: master branch

genericx86-64 on NUC

100.0%

none

0.0%

none

0.0%

none

100.0%

3072<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3072>

BSP/QEMU: master branch

genericx86-64 on Sugarbay

97.4%

none

2.6%

5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

0.0%

none

100.0%

3061<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3061>3064<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3064>

BSP/QEMU: master branch

MPC8315e-rdb

100.0%

none

0.0%

none

0.0%

none

100.0%

3051<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3051>3052<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3052>

BSP/QEMU: master branch

P1022ds

100.0%

none

0.0%

none

0.0%

none

100.0%

3043<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3043>3045<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3045>

BSP/QEMU: master branch

qemu-x86

100.0%

none

0.0%

none

0.0%

none

100.0%

3056<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3056>3057<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3057>

BSP/QEMU: master branch

qemuarm

100.0%

none

0.0%

none

0.0%

none

100.0%

3062<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3062>3063<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3063>

BSP/QEMU: master branch

qemumips

100.0%

none

0.0%

none

0.0%

none

100.0%

3059<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3059>3060<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3060>

BSP/QEMU: master branch

qemuppc

100.0%

none

0.0%

none

0.0%

none

100.0%

3042<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3042>3044<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3044>

BSP/QEMU: master branch

qemux86-64

100.0%

none

0.0%

none

0.0%

none

100.0%

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

Eclipse Plugin: master branch

Ubuntu 14.04 x86_64

93.9%

5769<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5769> 5163<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5163>

6.1%

5285<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5285> 4403<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4403> 5713<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5713>

0.0%

none

100.0%

3067<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3067>3068<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3068>

Meta-yocto: master branch

Ubuntu 14.04 x86_64

100.0%

6156<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6156>

0.0%

none

0.0%

none

100.0%

3070<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3070>3073<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3073>

Hob: master branch

Ubuntu 14.04 x86_64

97.8%

none

0.0%

none

2.2%

none

100.0%

3048<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3048>

Toaster: master branch

Fedora 20 x86_64

96.7%

none

3.3%

6684<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6684>

0.0%

none

100.0%

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

Toaster: master branch

Ubuntu 14.04 LTS x86_64

90.5%

none

9.5%

6832<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6832> 6685<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6685> 6833<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6833> 5255<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255> 5266<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5266> 5811<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5811>6688<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6688>

0.0%

none

100.0%

3049<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3049>3050<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3050>

OE-Core: master branch

Ubuntu 14.04 x86_64

96.6%

none

3.4%

6804<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6804>

0.0%

none

100.0%

3046<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3046>3047<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3047>

BitBake: master branch

Ubuntu 14.04 x86_64

91.9%

none

5.4%

6752<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6752>

2.7%

none

100.0%

3077<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3077>

Runtime: master branch

Multiple Environments

100.0%

none

0.0%

none

0.0%

none

100.0%

3066<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3066>

Build Appliance: master branch

qemu-kvm on Ubuntu 12.04 libvirt

100.0%

none

0.0%

none

0.0%

none

100.0%

Total: 24



Test Case Status Report

Status

Count

BLOCKED

2

FAILED

12

PASSED

1195

RUNNING

1

Total: 1210


[http://chart.apis.google.com/chart?cht=p3&chs=432x120&chd=t:&chd=t:2,12,1195,1&chco=FFFF66|FF9933|00FF00|0000FF&chl=BLOCKED(2)+0%25|FAILED(12)+1%25|PASSED(1195)+99%25|RUNNING(1)+0%25]



Bug Status Report

[Bug Status]

ID[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Priority[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Status[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Resolution[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Test Runs[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Description[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Version[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Target Milestone[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

Age (days)[?]<https://wiki.yoctoproject.org/wiki/WW43_-_2014-10-18_-_Full_Pass_-_1.7_M4.rc5>

4403<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4403>

Medium

ACCEPTED

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

stap fails on core-image-sato x86-64

1.4

Future

543

5163<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5163>

Medium

ACCEPTED

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

no external tools displayed when first trying to run QEMU - test case 48 failed on Ubuntu 12.04 x86-64 host(build 1.5 M4 rc2)

1.5

1.7

402

5255<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255>

Medium

ACCEPTED

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

source_url in the tasks table is empty

unspecified

Future

390

5266<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5266>

Medium

RESOLVED

FIXED

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

quagga/ripd: two unexpected behavior after received SIGHUP signal

unspecified

1.5.1

388

5285<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5285>

Medium

NEW

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

powertop: Output is useless

unspecified

Future

386

5713<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5713>

Medium

NEW

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

systemtap displays blank pop-up error window

1.5

1.7

281

5769<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5769>

Low

NEW

3065<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3065>

ssh connection doesn't disconnect when qemu is restarted/shut down

1.6

1.7

262

5811<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5811>

Medium

IN PROGRESS IMPLEMENTATION

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

the orm_variablehistory table should collect the variable value for each file

unspecified

Future

249

5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

Medium

NEW

3072<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3072>

Cannot wake from suspend state using the USB mouse or keyboard

1.6

1.7

243

6156<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6156>

Medium

ACCEPTED

3068<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3068>

[crosstap] no trace message output on qemuarm

1.6

1.7

187

6665<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6665>

Medium

NEW

3054<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3054>

[Beaglebone] Xserver segmentation fault when reboot the system

1.7

1.7

51

6684<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6684>

Medium

NEW

3048<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3048>

The content of the license column shouldn't be 4.0k.

1.7

1.7.1

43

6685<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6685>

Medium

REOPENED

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

Elapsed_time value null for tasks with outcome=0

1.7

1.7.1

43

6688<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6688>

Medium

VERIFIED

FIXED

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

Build image using managed mode - Toaster web interface tries to load from address http://0.0.0.0:8000

1.7

1.8

43

6752<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6752>

Medium

NEW

3047<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3047>

bitbake-prserv-tool has issues when it comes to exporting the PR database

1.7

1.7 M4

25

6804<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6804>

Medium

NEW

3050<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3050>

do_rootfs fails for btrfs image build

1.7

1.7

12

6832<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6832>

Medium

NEW

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

disk_io value in database doesn't match sometime with the value in the task buildstats file

1.7

1.7.1

4 (Recent)

6833<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6833>

Medium

NEW

3069<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3069>

elapsed_time value in database doesn't match sometime the value in the task stats file

1.7

1.7.1

4 (Recent)

Total: 18


Bug Trend

An overview of this week's Yocto Bug Trend<https://wiki.yoctoproject.org/wiki/Yocto_Bug_Trend> charts can be accessed on the following page<https://wiki.yoctoproject.org/charts/combo.html>

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test<https://wiki.yoctoproject.org/wiki/Performance_Test> wiki. The below charts are displaying the build performance trend for 1.7 by milestone starting 1.6 on Master. The charts by Milestone can be accessed at this link<https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html> where you can view more detailed information or by clicking on the images which will open a new window with the interactive chart.

As a status, building the rootfs remained the same as rc4, with 4:16 minutes against 4:20 (1.7_M1.rc1) on Ubuntu 12.04 and 4:14 against 4:20(1.7_M1.rc1) on fedora 19 so overall we can consider that there are no notable degradations in building the kernel but it maintained about the same comparing with M1. The kernel build time actually improved by average with few mentions that some values are far from the average (shorted and longer too). Comparing the build time for the kernel we can see that it takes 12:16 minutes (1.7_M4.rc5) against 12:44 (1.7_M1) on Ubuntu 12 and 11:18 minutes (1.7_M4.rc5) against 11:38 (M1) on Fedora. Also, for building a sato image with rm_work shows an improvement trend with 1:08:36 hours (1.7_M4.rc5) against 1:10:08(1.7_M1) on Ubuntu 12 and 1:10:32 hours (1.7_M4.rc5) against 1:13:10 (1.7_M1) on Fedora 19.

Overall there are improvements against beginning of 1.7, but also no degradations against the last release candidate, 1.7_M4.rc5, which can be seen in the charts below.

Performance Charts

For Bitbake core-image-sato with rm_work

[Ww43 - rmwork.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-core-image-sato-rmwork.html>

Building the Virtual kernel

[Ww43 - kernel.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-virtual-kernel.html>

Building the rootfs

[Ww43 - rootfs.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-core-image-sato-rootfs.html>

Distro Testing

Distro Testing is intended to catch bugs that are distribution specific using the yocto-autobuilder. The tests are all run on identical hardware and with all OS-es updated. The distributions used will be Fedora, Ubuntu, CentOS, OpenSuse with their latest update. More details on the coverage for the Distro testing is available here at Distribution_Support#Distro_Testing<https://wiki.yoctoproject.org/wiki/Distribution_Support#Distro_Testing> wiki.

Summary

At this point all buildsets are fine.

Distro Results
BRANCH

master

COMMIT

Poky: 924da20c6f2f183fa7e930e4c3c6b14c090acbe5

RELEASE

1.7_M4.rc5

Ubuntu 14.04 LTS

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



Fedora 20

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



CentOS 7

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



OpenSuse 13.1

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS


Regards,
--
Alexandru Georgescu
Yocto QA Engineer
SSG/SSD Open Source Technology Center Romania
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 11582 bytes
Desc: image001.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 10769 bytes
Desc: image002.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.gif
Type: image/gif
Size: 462 bytes
Desc: image003.gif
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 75017 bytes
Desc: image004.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 85315 bytes
Desc: image005.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 83674 bytes
Desc: image006.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141024/46dddb27/attachment-0004.png>

More information about the yocto mailing list