[yocto] QA cycle report for 2.1.3 rc1

[Cruz, Libertad]

Hello All,



Here is the report for the 2.1.3 rc1 full point release test cycle.



Full Report : https://wiki.yoctoproject.org/wiki/WW24_-_2017-14-06_-_Full_Pass_2.1.3_rc1





======== Summary ========



The QA cycle for release 2.1.3 rc1 is complete, there were 4 new issues related to the configuration of the new autobuilder server that provided the images.



Performance measurements showed a regression of 20% on Ubuntu and 11% on Fedora compared with 2.1.2, 11655[3] bug was raised.



======== Details ========



QA Hints



The 2.1.3 QA test cycle had difficulties with the new autobuilder server, because it was the first time this server provided images it was missing configurations therefore some images were not created bugs 11644[2] and 11637[1]. There will be a 2.1.3 rc2 where we will test the integrity of the images and the test cases that were blocked or failed.



Bugs

       * New

            - 11637 [krogoth] No WIC images present in autobuilder[1]

            - 11644 [Yocto 2.1.3] missing toolchian sdk for armv5e[2]

            - 11655 [krogoth] build-perf:time for build "rootfs" is increased on 2.1.3 compared to 2.1.2[3]

            - 11646 [krogoth] WIC image does not load in graphical environment[5]

            - 11654 [krogoth] ERROR: setUpClass (oeqa.selftest.signing.Signing) [4]

            - 11658 [krogoth] testimage multiple errors and skipped test present in WIC image (Minnowmax64/NUC) [6]

            - 11659 [krogoth] genericx86 fails to boot after install from USB on MMAX32[7]







Full Bug Report : https://wiki.yoctoproject.org/wiki/WW24_-_2017-14-06_-_Full_Pass_2.1.3_rc1#Bugs_Found_during_QA_Test





====== Performance ======



Performance measurements showed a regression compared with 2.1.2 when building rootfs of 20% (around 20 seconds) on Ubuntu machine and 11% (around 15 seconds) on Fedora, all other measurements are on the same levels without showing any important change.



Ubuntu    Test    2.1.2_rc1    2.1.3_rc1    %

          sato    0:56:22      0:55:57      -0.74

          rootfs  1:59         2:26         22.69

          rmwork  0:55:10      0:55:43      1.00

          kernel  4:18         4:26         3.10

          eSDK    2:36         2:28         -5.13





Fedora    Test    2.1.2_rc1    2.1.3_rc1    %

          sato    0:55:48      0:55:59      0.33

          rootfs  2:01         2:15         11.57

          rmwork  0:55:23      0:56:18      1.66

          kernel  4:41         4:41         0.00

          eSDK    2:35         2:41         3.87



Performance Charts :  https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html





======== Links =========



    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11637

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11644

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11655

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11654

    5.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11646

    6.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11658

    7.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11659





Regards

Libertad G.