[yocto] QA cycle report for 2.4 M3 rc1

[Cruz, Libertad]

Hello All,



Here is the report for the 2.4 M3 rc1 full release test cycle.

Full Report :  https://wiki.yoctoproject.org/wiki/WW35_-_2017-08-30_-_Full_Test_Cycle_2.4_M3



======= Summary ========

The QA cycle for release 2.4 M3 rc1 is complete.  There are seven new bugs: bug 12013[7]  blocks testing to poky-tiny image since it always asks password when booting the image. Other medium+ bug 11984[1]   cross tap does not work on 2.4 M3 RC1 and bug 11994[2]   X can not launch on Fedora 26 for 2.4 M3 RC1, do not block full functionality nor a full testrun.  Bugs 12004[4], 12005[5], 12006[6]   are toaster related.



Neon Eclipse-Plugin under Debian 8 is having project building problems, this is still under investigation I suspect it is user configuration errors. Mars and Neon Eclipse-Plugin ran perfectly on Ubuntu.



LPT results are waiting to be updated by Windriver.


Performance
There were no relevant changes on measurements compared to  2.4 M2 rc3, just a small regression of 6% on fedora machine when building rootfs.


Ubuntu    Test       2.4 M2 rc3    2.4 M3 rc1    %
                 sato       1:14:08         1:15:58         2.47
                 rootfs    2:31              2:34              1.99
                 rmwork 1:06:40         1:08:15         2.37
                 kernel    5:32              5:36             1.20
                 eSDK      2:32              2:39             4.61

Fedora    Test       2.4 M2 rc3    2.4 M3 rc1    %
                sato       1:12:25         1:14:13         2.49
                rootfs     2:25             2:35              6.90
               rmwork   1:07:48        1:09:06         1.92
               kernel     5:58             5:58              0.00
               eSDK       2:34             2:37              1.95




ptest

There was a better pass rate of 5% on gdk-pixbuf  in comparison with 2.3 M2 rc3.



Compliance

For posix test there is no changes in the number of failures when comparing  2.4 m2 rc3 and 2.4 m3 rc1.



                    Failures            2.4 M3 RC1      2.4 M2 RC3

                  Generic86-64           46                      46

                  Beaglebone              45                      45

                  Edgerouter               46                      46

                  Mpc8315e-rdb        45                      45



Results are found here: https://wiki.yoctoproject.org/wiki/Posix_result

Detail analysis found here:  https://wiki.yoctoproject.org/wiki/POSIX-results



The LPT results are found: https://wiki.yoctoproject.org/wiki/LTP_result .





======= QA-Hints========



There are no important fully blocked functionalities QA does not think an rc2 is necessary.



======= Bugs ========



       New Bugs
            -11984[1]  crosstap doesn't work on 2.4 M3 RC1
            -11994[2] X can not launch on Fedora 26 for 2.4 M3 RC1
            -11995[3] Update yocto-bsp script for kernel 4.12
            -12004[4] Views: Edit Columns button is not alphabetically ordered
            -12005[5]  Cant download other artifacts when modifying built recipe
            -12006[6]  Image recipes view: Recipe file link not redirecting correctly
            -12013[7]  Poky tiny image doesn't boot, it ask for password
       Not new M+ bugs
            -11879[8]  Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64



Full Bug Report : https://wiki.yoctoproject.org/wiki/WW35_-_2017-08-30_-_Full_Test_Cycle_2.4_M3#Bugs_Found_during_QA_Test



======== Links =========



    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11984<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11917>[1]

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11923%20>[2]

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11995<https://bugzilla.yoctoproject.org/show_bug.cgi?id=10832%20>[3]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12004<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11833%20>[4]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12005<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12005%20>[5]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=12006<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12006%20>[6]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11833%20>12013[7]

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11879<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11833%20>[8]







Regards

Libertad G.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170830/9ed67b87/attachment.html>