[yocto] QA cycle report for 2.2.2 rc2

[Cruz, Libertad]

Hello All,



Here is the report for the 2.2.2 point release test cycle.

Full Report :  https://wiki.yoctoproject.org/wiki/WW37_-_2017-09-13_-_Full_Point_Release_Test_Cycle_2.2.2_rc2

======= Summary ========

The QA cycle for point release 2.2.2 rc2 is complete.  There are 3 new bugs: bug 12080[2]  blocks testing to the eSDK component since eSDK functionalities cannot be executed. Bugs 12075[1], 12073[3], 12081[4] don't block full functionalities.



LPT results are waiting to be updated by Windriver.


Performance

Compared with 2.2.2 rc1 there was a regression of 14.71 while building rootfs on Ubuntu, there was also a regression on Fedora while building sato of 7.46% and a regression on kernel of 12.12%.





Ubuntu                 Test       2.2.2_rc1             2.2.2_rc2             %

                              sato       1:00:42              1:01:16               0.93

                              rootfs    2:16                    2:36                     14.71

                              rmwork 0:59:29              0:59:51               0.62

                              kernel    5:01                    5:00                     -0.33

                              eSDK      3:22                    3:19                     -1.49





Fedora                 Test       2.2.2_rc1             2.2.2_rc2             %

                              sato       1:01:55              1:06:32               7.46

                              rootfs    2:49                    2:51                     1.18

                              rmwork 0:59:52              1:01:32               2.78

                              kernel    6:03                    6:47                     12.12

                              eSDK      3:44                    3:54                     4.46





ptest

There was an improvement on the pass rate in the following packages in comparison with 2.2.2 rc1:

gdk-pixbuf with 5%

libxml2 with 4.20%

lttng-tools with 3.20%



On the other side there was a regression on the pass rate in comparison with 2.2.2 rc1:

acl with 6.80%

e2fsprogs with 100% regression

gawk with .10%

quilt with 1.80%

valgrind of 53.10%





Compliance

Results are yet to be documented by Windriver.



======= QA-Hints========



There is only one concern on the eSDK component it cannot execute. QA recommends an 2.2.2 rc3 to only verify the eSDK component.



======= Bugs ========



       New Bugs

            -12075[1]  [morty] testimage quemux86/qemux86-64 lsb tcgetattr: Input/output error^M

            -12080[2] [Morty][Test Case 1605] TCTEMP_2.3_AUTO_sdkext_eSDK_devtool_build_make

            -12073[3] selftest: setUpClass failed failing on ubuntu 17.04 (oeqa.selftest.signing.Signing)

            -12081[4] [morty] PTEST: Package Name flex, gzip, slang, do not appear in 2.2.2 rc2



Full Bug Report : https://wiki.yoctoproject.org/wiki/WW37_-_2017-09-13_-_Full_Point_Release_Test_Cycle_2.2.2_rc2#Bugs_Found_during_QA_Test





======== Links =========



    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12075 [1]

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12080 [2]

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12073 [3]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12081 [4]







Regards

Libertad G.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170914/11dea8e5/attachment.html>