[yocto] Yocto 1.8_M1.rc1 Full Pass Test report

Georgescu, Alexandru C alexandru.c.georgescu at intel.com
Mon Dec 15 09:19:58 PST 2014 

Hello,
Here is the Yocto 1.8_M1.rc1 Full Pass test report.
New issue summary:

  *   There are many Toaster bugs due to the many changes during 1.8_M1 on Toaster side and they appeared in both UI and Backend after new changes.
  *   Due to the oprofile upgrade which now uses perf interface to control the data collection (dropping opcontrol) OprofileUI which was used by Eclipse as a profiling tool needs to be upgraded #7054<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7054>
  *   Psplash was found not working when booting from runlevel 3 #7057<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7057>
For more detailed report you can access the full pass test report here<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1#Performance_Charts>. The Full list of bugs found during this Full Pass release, build performance and distro results summary data can be found below.

Testing data

 Test type:  Full Pass Release Test

 Branch: master

 Build name: 1.8_M1.rc1

 poky commit: b813bdebb36501500e86fea5f7e15b4b15ea0902

 Autobuilder images repository: http://autobuilder.yoctoproject.org/pub/releases/yocto-1.8M1

 Distributions tested: Ubuntu 14.04 64bit, Fedora 20 64bit, Opensuse 13.2 64bit, CentOS 7 64 bit

Summary Report
Testing Summary Report

Test Run

Test Plan

Environment

Passed

Other issues

Failed

Failing bugs

Blocked

Blocking bugs

Complete

3293<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3293>3295<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3295>

ADT: master branch

Fedora 20 i686

100.0%

none

0.0%

none

0.0%

none

100.0%

3292<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3292>3294<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3294>

ADT: master branch

Ubuntu 14.04 x86_64

100.0%

none

0.0%

none

0.0%

none

100.0%

3284<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3284>3285<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3285>

BSP/QEMU: master branch

Beaglebone Black

98.7%

none

1.3%

6665<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6665>

0.0%

none

100.0%

3278<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3278>3279<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3279>

BSP/QEMU: master branch

EdgeRouter

100.0%

none

0.0%

none

0.0%

none

100.0%

3256<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3256>3257<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3257>

BSP/QEMU: master branch

generic-x86 on AtomPC

97.4%

none

2.6%

7057<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7057>

0.0%

none

100.0%

3268<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3268>3269<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3269>

BSP/QEMU: master branch

genericx86-64 on Sugarbay

97.6%

none

2.4%

6850<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6850>6584<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6584>5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

0.0%

none

100.0%

3280<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3280>3281<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3281>

BSP/QEMU: master branch

MPC8315e-rdb

100.0%

none

0.0%

none

0.0%

none

100.0%

3282<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3282>3283<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3283>

BSP/QEMU: master branch

P1022ds

100.0%

none

0.0%

none

0.0%

none

100.0%

3258<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3258>3260<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3260>

BSP/QEMU: master branch

qemu-x86

100.0%

none

0.0%

none

0.0%

none

100.0%

3286<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3286>3287<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3287>

BSP/QEMU: master branch

qemuarm

98.4%

none

1.6%

4024<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4024>

0.0%

none

100.0%

3288<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3288>3289<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3289>

BSP/QEMU: master branch

qemumips

100.0%

none

0.0%

none

0.0%

none

100.0%

3290<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3290>3291<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3291>

BSP/QEMU: master branch

qemuppc

100.0%

none

0.0%

none

0.0%

none

100.0%

3259<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3259>3261<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3261>

BSP/QEMU: master branch

qemux86-64

100.0%

none

0.0%

none

0.0%

none

100.0%

3266<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3266>

Eclipse Plugin: master branch

Fedora 20 x86_64

97.0%

5769<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5769>5285<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5285>4403<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4403>

3.0%

7054<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7054>

0.0%

none

100.0%

3297<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3297>3298<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3298>

Meta-yocto: master branch

Ubuntu 14.04 x86_64

100.0%

6156<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6156>

0.0%

none

0.0%

none

100.0%

3262<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3262>3264<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3264>

Hob: master branch

Ubuntu 14.10 x86_64

97.8%

none

0.0%

none

2.2%

none

100.0%

3267<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3267>3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

Toaster: master branch

Fedora 20 x86_64

82.5%

none

17.5%

7023<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7023>7022<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7022>7026<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7026>7025<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7025>7004<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7004>7052<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7052>7060<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7060>

0.0%

none

100.0%

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

Toaster: master branch

Ubuntu 14.10 x86_64

92.7%

none

7.3%

6832<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6832>6833<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6833>5226<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5226>5255<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255>5811<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5811>

0.0%

none

100.0%

3253<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3253>3254<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3254>

OE-Core: master branch

Ubuntu 14.10 x86_64

96.6%

none

3.4%

6804<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6804>

0.0%

none

100.0%

3255<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3255>3277<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3277>

BitBake: master branch

Ubuntu 14.10 x86_64

97.2%

none

2.8%

6752<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6752>

0.0%

none

100.0%

3271<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3271>

Runtime: master branch

Multiple Environments

100.0%

none

0.0%

none

0.0%

none

100.0%

3272<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3272>

Build Appliance: master branch

qemu-kvm on Ubuntu 12.04 libvirt

100.0%

none

0.0%

none

0.0%

none

100.0%

Total: 22



Test Case Status Report

Status

Count

BLOCKED

1

ERROR

1

FAILED

19

PASSED

1210

Total: 1231


[http://chart.apis.google.com/chart?cht=p3&chs=432x120&chd=t:&chd=t:1,1,19,1210&chco=FFFF66|FF0000|FF9933|00FF00&chl=BLOCKED(1)+0%25|ERROR(1)+0%25|FAILED(19)+2%25|PASSED(1210)+98%25]



Bug Status Report

[Bug Status]

ID[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Priority[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Status[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Resolution[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Test Runs[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Description[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Version[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Target Milestone[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

Age (days)[?]<https://wiki.yoctoproject.org/wiki/WW51_-_2014-12-07_-_Full_Pass_-_1.8_M1.rc1>

4024<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4024>

Low

REOPENED

3286<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3286>

start qemuarm with "core-image-sato", use "dmesg | grep -i error" shows 2 errors

1.4

1.8 M1

632

4403<https://bugzilla.yoctoproject.org/show_bug.cgi?id=4403>

Medium

ACCEPTED

3266<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3266>

stap fails on core-image-sato x86-64

1.4

Future

593

5226<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5226>

Medium

NEW

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

in 'variables' API, the 'changed' field seems to return always 'FALSE'

1.5

Future

445

5255<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255>

Medium

ACCEPTED

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

source_url in the tasks table is empty

unspecified

Future

440

5285<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5285>

Medium

NEW

3266<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3266>

powertop: Output is useless

unspecified

Future

436

5769<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5769>

Low

NEW

3266<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3266>

ssh connection doesn't disconnect when qemu is restarted/shut down

1.6

1.8

312

5811<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5811>

Medium

IN PROGRESS IMPLEMENTATION

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

the orm_variablehistory table should collect the variable value for each file

unspecified

Future

299

5834<https://bugzilla.yoctoproject.org/show_bug.cgi?id=5834>

Medium

NEW

3269<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3269>

Cannot wake from suspend state using the USB mouse or keyboard

1.6

1.8

293

6156<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6156>

Medium

ACCEPTED

3297<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3297>

[crosstap] no trace message output on qemuarm

1.6

1.8

237

6584<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6584>

Medium

NEW

3268<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3268>

genericx86-64 on Sugarbay dmesg uvesafb: probe of uvesafb.0 failed with error -22

1.7

1.8

132

6665<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6665>

Medium

NEW

3285<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3285>

[Beaglebone] Xserver segmentation fault when reboot the system

1.7

1.8

101

6752<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6752>

Medium

NEW

3255<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3255>

bitbake-prserv-tool has issues when it comes to exporting the PR database

1.7

1.8

75

6804<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6804>

Medium

REOPENED

3254<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3254>

do_rootfs fails for btrfs image build

1.7

1.8

62

6832<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6832>

Medium

ACCEPTED

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

disk_io value in database doesn't match sometime with the value in the task buildstats file

1.7

1.7.1

54

6833<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6833>

Medium

RESOLVED

FIXED

3270<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3270>

elapsed_time value in database doesn't match sometime the value in the task stats file

1.7

1.7.1

54

6850<https://bugzilla.yoctoproject.org/show_bug.cgi?id=6850>

Low

NEW

3268<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3268>

Minnowboard Xorg Open ACPI failed error

1.7

Future

52

7004<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7004>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

Toaster UI - wrong ETA displayed for current running build

1.8

1.8

5(Recent)

7022<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7022>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

All layers page: sorting does not reset if sorted column is removed

1.8

1.8

4(Recent)

7023<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7023>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

All layers page: column head misspelled

1.8

1.8

4(Recent)

7025<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7025>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

View all targets: graphical overlap in table

1.8

1.8

4(Recent)

7026<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7026>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

All layers page: Sorting does not work

1.8

1.8

4(Recent)

7052<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7052>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

Filters in the all layers page return only the information shown in the page

1.7

1.8

NEW

7054<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7054>

Medium

ACCEPTED

3266<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3266>

OprofileUI needs to be updated to reflect new oprofile functionality

1.8

1.8

NEW

7057<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7057>

Medium+

NEEDINFO

3256<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3256>

psplash not available when booting in runlevel 3/ runlevel 5

1.8

1.8 M3

NEW

7060<https://bugzilla.yoctoproject.org/show_bug.cgi?id=7060>

Medium

NEW

3276<https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=3276>

Filters in "All targets" page not working

1.8

1.8 M2

NEW

Total: 25


Bug Trend

An overview of this week's Yocto Bug Trend<https://wiki.yoctoproject.org/wiki/Yocto_Bug_Trend> charts can be accessed on the following page<https://wiki.yoctoproject.org/charts/combo.html>

Build Performance Test

Detailed description on how performance test are run can be found on Performance Test<https://wiki.yoctoproject.org/wiki/Performance_Test> wiki. The below charts are displaying the build performance trend for 1.7 by milestone starting 1.6 on Master. The charts by Milestone can be accessed at this link<https://wiki.yoctoproject.org/charts/perf_milestone/performance_test.html> where you can view more detailed information or by clicking on the images which will open a new window with the interactive chart.
§  The rootfs takes a little longer, with 4:20 against 4:16 minutes on ubuntu and 4:17 against 4:14 on Fedora.
§  Building the Kernel takes less, with 12:11 against 12:16 on Ubuntu and 11:12 against 11:18 on Fedora.
§  Building a core-image-sato with rm_work shows some degradation by taking 1:09:39 hours against 1:08:36 on Ubuntu and 1:13:02 hours against 1:10:32 on Fedora.

Overall we can see improvements in building the Kernel, the rootfs is stable but building the image takes more with an 1 minute average on Ubuntu and 2:30 minutes on Fedora, a normal consequence of the many changes between 1.7 release and 1.8_M1. Depending on the future changes in master, we might see improvements or degradations in this test.



Performance Charts

For Bitbake core-image-sato with rm_work

[Ww51 - rmwork.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-core-image-sato-rmwork.html>

Building the Virtual kernel

[Ww51 - kernel.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-virtual-kernel.html>

Building the rootfs

[Ww51 - rootfs.png]<https://wiki.yoctoproject.org/charts/perf_milestone/bitbake-core-image-sato-rootfs.html>

Distro Testing

Distro Testing is intended to catch bugs that are distribution specific using the yocto-autobuilder. The tests are all run on identical hardware and with all OS-es updated. The distributions used will be Fedora, Ubuntu, CentOS, OpenSuse with their latest update. More details on the coverage for the Distro testing is available here at Distribution_Support#Distro_Testing<https://wiki.yoctoproject.org/wiki/Distribution_Support#Distro_Testing> wiki.

Summary

At this point all buildsets are fine.

Distro Results
BRANCH

master

COMMIT

Poky: b813bdebb36501500e86fea5f7e15b4b15ea0902

RELEASE

1.8_M1.rc1

Ubuntu 14.10

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



Fedora 20

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



CentOS 7

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



OpenSuse 13.2

* nighlty-world

PASS

* buildtools

PASS

* nightly-arm

PASS

* nightly-multilib

PASS

* nightly-qa-systemd

PASS

* nightly-x86

PASS

* nightly-x86-64

PASS

* nightly-x86-64-lsb

PASS



Regards,
--
Alexandru Georgescu
Yocto QA Engineer
SSG/SSD Open Source Technology Center Romania
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 84533 bytes
Desc: image007.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.png
Type: image/png
Size: 82542 bytes
Desc: image008.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.png
Type: image/png
Size: 78783 bytes
Desc: image009.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 10933 bytes
Desc: image002.jpg
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 13237 bytes
Desc: image004.jpg
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 458 bytes
Desc: image006.png
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20141215/b4106338/attachment-0003.png>

More information about the yocto mailing list