[yocto] QA Test Report for Yocto Project Release 2.3 M3 rc2

[Jair Gonzalez]

Hello All,

 

Here is the report for the Full QA Cycle on Release 2.3 M3 rc2

 

Full Report :
https://wiki.yoctoproject.org/wiki/WW14_-_2017-03-28_-_Full_Test_Cycle_2.3_M
3_rc2

 

 

======== Summary ========

 

The QA cycle for release 2.3 M3 rc2 is complete. There are 3 new issues
found, 2 of them are High and one is M+.  Bug 11284 was not allowing
Selftest execution so a workaround was provided (read more details within
the bug description). Although it does not block M3 rc2, this issue will
need to get addressed before the mayor release. In relation to Performance,
all of the tasks measurements remained relatively without changes compared
to the previous version with small percentage variations between -1.35% to
3.31% on both machines. pTest shows improvements on strace and valgrind, but
small regressions on tcl and util-linux.

 

QA Hints

       - There are 5 high bugs that are still open, so we should keep focus
on this for M4.

 

=========================

 

 

======== Details ========

Bugs

      * New

          - High

            o   11267         X can not launch on beaglebone [1]

            o   11284         selftest: tests that capture output fail on
GDC Autobuilders [2]

          - M+

            o   11266         runqemu: fails when rm_work is active [3]

      * High / M+ Not New

            o   11119         runqemu-extract-sdk is failing when extracting
.tar.bz file [4]

            o   11222         [eclipse] C compiler cannot create executables
when reconfiguring project [5]

            o   10477         yocto-bsp: tool should output a conf file to
be consume by the qemu runner [6]

            o   11193         runqemu can not launch image if enable
"rm_work" [7]

            o   11223         devtool runqemu doesn't work in eSDK [8]

 

Full Bug Report :
https://wiki.yoctoproject.org/wiki/WW14_-_2017-03-28_-_Full_Test_Cycle_2.3_M
3_rc2#Bugs_Found_during_QA_Test

 

=========================

 

 

====== Performance ======

 

Performance measurements on both Fedora and Ubuntu machines were relatively
stable compared to the previous release, with percentage variations between
-1.35% to 3.31%. The Fedora machine showed improvements in all tasks except
a small regression on rmwork. Below are the tables with the times of the
tests:

 

-        Ubuntu

 

Test         2.3 M3 rc1        2.3 M3 rc2        %

sato         1:10:15           1:10:00           -0.36

rootfs       2:28              2:26              -1.35

rmwork       1:05:28           1:04:47           -1.04

kernel       5:11              5:12              0.32

eSDK         3:01              3:07              3.31

 

-        Fedora

 

Test       2.3 M3 rc1         2.3 M3 rc2        %

sato       1:12:34            1:14:57           3.28

rootfs     2:40               2:44              2.50

rmwork     1:07:10            1:06:45           -0.62

kernel     6:29               6:38              2.31

eSDK       3:17               3:20              1.52

 

 

Performance Charts :
https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.htm
l

 

========================

 

 

========= pTest ========

 

    -  There were improvements on pass rates for the strace and valgrind
modules

    -  Modules that decreased pass rate were tcl and util-linux

    -  No new bugs were added, although there are 12 issues still open

 

pTest full report :
https://wiki.yoctoproject.org/wiki/WW14_-_2017-03-28_-_Full_Test_Cycle_2.3_M
3_rc2#pTest_for_genericx86-64_on_NUC

 

========================

 

 

Direct links to the Bugs

1.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11267

2.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11284

3.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11266

4.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11119

5.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11222

6.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477

7.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11193

8.      https://bugzilla.yoctoproject.org/show_bug.cgi?id=11223

 

 

Regards,

Jair Gonzalez

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170405/c41bb9b1/attachment.html>