[yocto] QA Test Report for Yocto Project 2.2 rc4

[Perez Carranza, Jose]

Hi

Here is the report for QA Cycle on 2.2 rc4

Full Report :  https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4

Summary

There were found 10 new bugs, 3 of them are already triaged as M+ 1 as Medium and the rest are not yet triaged, no one of these bugs was a blocker for QA cycle hence all components were fully executed, there are 12 bugs for pTest that were out of the focus in past milestones due some issues on reports, all of them are medium. New scripts of performance is giving correct data and this time the charts has a comparison between old and new script results , in general all the measurements on performance are stable. All high and M+ bugs found on previous QA cycle (M3 rc1) were verified, none of them is present on current release.

Bugs

           *New bugs found
               10455 - [Test Case 1059] Parselogs is failing on MM32 giving error: blk_update_request: I/O error, dev loop0 [1]
10456 - X can not launch on qemumips64 [2]
10481 - [Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) WIC giving error: GPT: Use GNU Parted to correct GPT errors. [3]
10490 - Error on oeqa.runtime.rpm.RpmInstallRemoveTest when running testimage [4]
10494 - [Test Case 1059] Parselogs is failing on genericx86-64-WIC on NUC giving error: Failed to load DMC firmware [5]
10434 - runqemu can not launch qemu image in Build Appliance [6]
10447 - eSDK: runqemu fails unless unless DEPLOY_DIR_IMAGE is set [7]
10477 - yocto-bsp: tool should output a conf file to be consume by the qemu runner [8]
10463 - [Test Case 910] "Recipe file" column isn't sortable [9]
10492 - [Test Case 1398] Build log cannot be downloaded [10]

Full List of bugs :  https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#New_and_notable_issues_Found_during_QA_Test_.26_Community

          *Old New High and M+

-        NA

         *Old Medium Low
            Full list of bugs: https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#Other_Bugs_found_during_QA_Test_.28NOT_NEW.29

          *pTest bugs
- These bugs were out of the focus due a issues with automatic report on previous milestones, there is a new table showing the failures and all the bugs were commented with the results for this milestone.
- Full log of the execution is now loaded to the wiki along with the results [11]

             Full list of bugs: https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#pTest_open_bugs


Performance

-        New script is now running on the machines and is giving correct a precise data.

-        After resolution of bug 10285 correct measurement were retrieved for "core-image-sato -c rootfs" , numbers reported by logs before the failure were added to this measurement to have a better readably on the chart and a note was added to the wiki page.

-        This time the charts contains the measurement for old and new script in same milestone to have a comparison, the times a pretty similar hence new script will be fully enabled on machines for upcoming measurements.

-        In all the measurements  there is shown a slight improvement on times in new script mostly due there is a more precise collection of times

-        Full list of charts [12]

Links

1 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10455
2 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10456
3 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10481
4 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10490
5 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10494
6 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10434
7 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10447
8 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477
9 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10463
10 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10492
11 - https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#pTest_results_for_genericx86-64_on_NUC
12 - https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#Build_Performance_Test

Regards,
José

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20161025/337bed05/attachment.html>