[yocto] QA cycle report for 2.6 M4 RC1

[Jain, Sangeeta]

Hello All,



This is the full report for 2.6 M4 RC1:

https://wiki.yoctoproject.org/wiki/WW44_-_2018-10-30_-_Full_Test_Cycle_2.6_M4_RC1





Summary



All planned tests were executed.



Total Test Executed - 3339

Passed Test - 3322

Failed Test - 7

Blocked Test - 4



There were zero high priority defect.  Team had found 4 new defects.

ptest for 1 module failed in current release but passed in previous 2.6 M3 rc1.  For Openssl no test was executed in current release as well as previous release. Present status of bugs for respective modules is as  follows:



ModuleName  -  BugId  -  Present Status



gstreamer        - 12990     - New



Note: For busybox, pass rate is lower than previous release, but no test cases which passes in previous release failed in current release. Lower pass rate is due to new test cases added in this release. No bug filed.



Performance test



 rootfs performance on ubuntu1604 upgraded by  13.04% in 2.6 M3 RC1 w.r.t. 2.6 M2 RC1



QA-Hints



For performance test, in this release, QA team has performed analysis on results from "yocto-perf" mailing list, rather than on results from Linux foundation machines.
We observed two major difference between machines used to run Performance test by "yocto-perf" mailing list and Linux Foundation machines:

1) Mailing List data points was not constant, sometime more, sometime less
2) Mailing List used machine with different spec



New Bugs



[1] Bug 12974 -  [2.6 M4 RC1] [OE-Core] Crosstap doesn't work on 2.6 M4 RC1

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12974



 [2] Bug 129<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12952>79 - [2.6 M4 rc1] test_recipetool_create_cmake failed on Fedora 27

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12979


[3] Bug 129<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12946>91 - [2.6 M4 RC1][Build-Appliance] Bitbake build-appliance-image getting failed during building image due to webkitgtk package
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991

[4] Bug 129<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12954>92 - [2.6 M4 rc1] test_devtool_add_fetch_git failed on Fedora 27
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12992

Thanks & Regards,
Sangeeta Jain

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20181109/66e158b4/attachment.html>