[yocto] QA cycle report for 2.5 M3 RC1

[Yeoh, Ee Peng]

Hello All,
This is the full report for 2.5 M3 RC1:  
https://wiki.yoctoproject.org/wiki/WW15_-_2018-04-09-_Full_Test_Cycle_-_2.5_M3_rc1

======= Summary ========

The QA cycle for release 2.5 M3 RC1 was completed.  Team had discovered 7 new bugs.

There were 3 bugs related to runtime [4] [6] [7], where core-image-lsb-sdk unable to boot on Minnowboard Turbot, runtime tests failed on OpenSUSE42.3 host machine, & connect host with serial console failed to display login prompt for core-image-sato-sdk-genericx86-64 on Minnowboard Turbot.  There were 3 bugs related to selftest [1] [2] and kernel development [5]. 

This QA cycle were executed with additional manual testing on the images/artifacts created by the new Autobuilder codebase (http://autobuilder.yoctoproject.org/pub/yocto-2.5_M3.rc1/), where the images/artifacts were downloaded and manually tested.  There was 1 bug discovered from new Autobuilder codebase where Build Appliance failed [3] to build image.  

Testcase#202 (syslog) were skipped in multiple BSP automated runtime tests where it cannot run with sysklogd installed.  Testcase#1550 (bitbake dependency explorer) was failing and it was still undergoing regression testing. 

Performance tests shown that build virtual/kernel time for this cycle increased by ~6% compared to 2.5 M2 RC1. 

This QA cycle had automated package management runtime tests as well as most of the meta-ide-support and SDK toolchain using both existing and new automated tests available. 

======= QA-Hints========

No significant bugs were discovered. 

======= New Bugs ========

[1] Bug 12647 - [2.5 M3 rc1][OE-Core:Debian9] recipe xcursor-transport-theme went backwards, breaks package feeds
[2] Bug 12655 - [2.5 M3 rc1][OE Core:Debian9] layerappend.LayerAppendTests.test_layer_appends - Testcase 1196: ERROR
[3] Bug 12637 - [2.5 M3 RC1] Build appliance failed on glibc-initial do_configure when bitbake core-image-minimal
[4] Bug 12633 - [2.5 M3 RC1] BSP-QEMU:core-image-lsb-sdk-x86.wic unable to boot on Minnowboard Turbot
[5] Bug 12641 - [2.5 M3 RC1] Kernel Development Build external modules (hello-mod) test case: Function failed do_rootfs
[6] Bug 12665 - [2.5 M3 rc1] OpenSUSE42.3 runtime test fail at runtime_test.TextExport, runtime_test.TestImage, runtime_test.TestImage and oe_runmake
[7] Bug 12666 - [2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard Turbot

======== Links =========
1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12647 [1] 
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12655 [2]
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12637 [3]
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12633 [4]
5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12641 [5]
6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12665 [6]
7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12666 [7]

Regards
Ee Peng