[yocto] QA Test Report for Yocto Project 2.2 rc4

Tue Oct 25 12:56:06 PDT 2016

Jose,

This is a good report.  Seems that we are doing well in terms of closing out m+/high priority bugs without a lot of new ones appearing.

--David C

From: Perez Carranza, Jose
Sent: Tuesday, October 25, 2016 10:48 AM
To: yocto at yoctoproject.org; Jolley, Stephen K <stephen.k.jolley at intel.com>
Cc: Lock, Joshua G <joshua.g.lock at intel.com>; Wold, Saul <saul.wold at intel.com>; Avery, Brian <brian.avery at intel.com>; Graydon, Tracy <tracy.graydon at intel.com>; Richard Purdie <richard.purdie at linuxfoundation.org>; Cobbley, David A <david.a.cobbley at intel.com>; Esquivel, Benjamin <benjamin.esquivel at intel.com>; Delgado, Sonia <sonia.delgado at intel.com>; Burton, Ross <ross.burton at intel.com>; Eggleton, Paul <paul.eggleton at intel.com>; Zhang, Jessica <jessica.zhang at intel.com>
Subject: QA Test Report for Yocto Project 2.2 rc4

Hi

Here is the report for QA Cycle on 2.2 rc4

Full Report :  https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4

Summary

There were found 10 new bugs, 3 of them are already triaged as M+ 1 as Medium and the rest are not yet triaged, no one of these bugs was a blocker for QA cycle hence all components were fully executed, there are 12 bugs for pTest that were out of the focus in past milestones due some issues on reports, all of them are medium. New scripts of performance is giving correct data and this time the charts has a comparison between old and new script results , in general all the measurements on performance are stable. All high and M+ bugs found on previous QA cycle (M3 rc1) were verified, none of them is present on current release.

Bugs

           *New bugs found
               10455 - [Test Case 1059] Parselogs is failing on MM32 giving error: blk_update_request: I/O error, dev loop0 [1]
10456 - X can not launch on qemumips64 [2]
10481 - [Test Case 1059] Parselogs is failing on genericx86 (32 and 64 bits) WIC giving error: GPT: Use GNU Parted to correct GPT errors. [3]
10490 - Error on oeqa.runtime.rpm.RpmInstallRemoveTest when running testimage [4]
10494 - [Test Case 1059] Parselogs is failing on genericx86-64-WIC on NUC giving error: Failed to load DMC firmware [5]
10434 - runqemu can not launch qemu image in Build Appliance [6]
10447 - eSDK: runqemu fails unless unless DEPLOY_DIR_IMAGE is set [7]
10477 - yocto-bsp: tool should output a conf file to be consume by the qemu runner [8]
10463 - [Test Case 910] "Recipe file" column isn't sortable [9]
10492 - [Test Case 1398] Build log cannot be downloaded [10]

Full List of bugs :  https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#New_and_notable_issues_Found_during_QA_Test_.26_Community

          *Old New High and M+

-          NA

         *Old Medium Low
            Full list of bugs: https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#Other_Bugs_found_during_QA_Test_.28NOT_NEW.29

          *pTest bugs
- These bugs were out of the focus due a issues with automatic report on previous milestones, there is a new table showing the failures and all the bugs were commented with the results for this milestone.
- Full log of the execution is now loaded to the wiki along with the results [11]

             Full list of bugs: https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#pTest_open_bugs

Performance

-          New script is now running on the machines and is giving correct a precise data.

-          After resolution of bug 10285 correct measurement were retrieved for "core-image-sato -c rootfs" , numbers reported by logs before the failure were added to this measurement to have a better readably on the chart and a note was added to the wiki page.

-          This time the charts contains the measurement for old and new script in same milestone to have a comparison, the times a pretty similar hence new script will be fully enabled on machines for upcoming measurements.

-          In all the measurements  there is shown a slight improvement on times in new script mostly due there is a more precise collection of times

-          Full list of charts [12]

Links

1 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10455
2 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10456
3 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10481
4 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10490
5 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10494
6 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10434
7 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10447
8 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477
9 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10463
10 - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10492
11 - https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#pTest_results_for_genericx86-64_on_NUC
12 - https://wiki.yoctoproject.org/wiki/WW43_-_2016-10-18_-_Full_Test_Cycle_2.2_rc4#Build_Performance_Test

Regards,
José

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20161025/0e6a2215/attachment.html>