[yocto] QA cycle report for 2.4 RC2

[Cruz, Libertad]

Hello All,

Enjoy viewing the full Report for 2.4 RC2:  https://wiki.yoctoproject.org/wiki/WW41_-_2017-10-13_-_Full_Test_Cycle_2.4_RC2



======= Summary ========



The QA cycle for release 2.4 RC2 is complete.  There are 5 new bugs from which so far none of them are high. There seems to be no blocking issues



ptest

There was a significant increase in pass rate for e2fsprogs from 0% to 85.76%. Python incremented it's pass rate by 99.61%. There was a significant decrease of the pass rate for libpcre from 100% to 66.67%, bug 12215 was opened, which was given medium priority and targeted to be fixed on milestone 2.5.



Toaster

There was a significant issue in licensing for toaster which led to the testing of Toaster in commit 96967261993f8b583b51cdeccbb1fd06042d2192.  QA sees no greater issue with the toaster functionalities.

Posix

2.4 RC1 Posix results are very similar to last QA cycle 2.4 M3 rc1. They only show that Edgerouter had one more failure if compared with the previous run.



                  Failures                  2.4 RC1           2.4 RC

                  Generic86-64           46                      46

                  Beaglebone              45                      45

                  Edgerouter               47

                  Mpc8315e-rdb         45                      45



For more details visit:  https://wiki.yoctoproject.org/wiki/Posix_result and https://wiki.yoctoproject.org/wiki/POSIX-results



LTP

2.4 RC2 LTP results show that, Edgerouter was not able to run Windriver will update the results soon. For currently executed LTP test,  QA has not found bugs.



                                                     2.4 RC1                  |        2.4 RC2

                                                Total  |  Failures      |      Total   |  Failures

               Genericx86-64         1391 |   27             |      1391   |     27

               Beaglebone              1353 |   18             |      1353   |     17

               Edgerouter               1353 |   34             |      1353   |

               Mpc8315e-rdb        1353  |   31            |      1353   |     30



For more details: https://wiki.yoctoproject.org/wiki/LTP_result



Performance


Measurements  form 2.4rc2 are almost the same as observed on previous rc1, no noticeable changes are observed.


Ubuntu                Test       2.4_rc1     2.4_rc2            %
                              sato       1:15:24     1:16:32            1.50
                              rootfs    2:45           2:45                0.00
                              rmwork 1:08:19     1:08:20            0.02
                              kernel    5:41           5:45                1.17
                              eSDK      2:40           2:42                1.25


Fedora                 Test       2.4_rc1     2.4_rc2            %
                              sato       1:17:02     1:17:23            0.45
                              rootfs    2:44           2:40               -2.44
                              rmwork 1:10:25    1:10:30            0.12
                              kernel    6:04           6:05                0.27
                              eSDK      2:42           2:39               -1.85



======= QA-Hints========



>From the results documented on the test cycle wiki there is no blockers for this commit to be released. Please also view the report for 2.4 RC1[8]  for a review of the still existing bugs.



======= Bugs ========



       New Bugs

            -12213[1] [TC 1059] test_parselogs failed on MMAX64 (Turbot) with direct firmware load for rtl_nic/rtl8168g-2.fw error

            -12214[2] [TC 1059] test_parselogs failed om MMAX64 (Turbot, WIC image) : usb 1-3: device descriptor read/64, error -71

            -12218[3] devtool modify failing in fedora 26

            -12220[4] [OE-seltest] Some automated test cases are being missed in the test run.

            -12215[5] ptest-runner libpcre one failed



       Not new M+ bugs

            -11994[6]  Graphical qemu doesn't work over remote X on Fedora 26

            -12131[7]  Kernel traceback when reboot beaglebone black









Full Bug Report:



======== Links =========

    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12213[1<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12213%5b1>]



    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12214[2<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12214%5b2>]



    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12218[3<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12218%5b3>]



    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12220[4<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12220%5b4>]



    5.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12215 5<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12215%205>]



    6.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994[6<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994%5b6>]



    7.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12131[7<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12131%5b7>]



    8.    https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1#Bugs_Found_during_QA_Test[<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994%5b19>8]







Regards

Libertad G.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20171014/a79dd264/attachment.html>