[yocto] QA cycle report for 2.6.2 RC3

[Jain, Sangeeta]

>-----Original Message-----
>From: richard.purdie at linuxfoundation.org <richard.purdie at linuxfoundation.org>
>Sent: Friday, 12 April, 2019 6:58 PM
>To: Jain, Sangeeta <sangeeta.jain at intel.com>; 'yocto at yoctoproject.org'
><yocto at yoctoproject.org>; Eggleton, Paul <paul.eggleton at intel.com>; 'Michael
>Halstead' <mhalstead at linuxfoundation.org>; Erway, Tracey M
><tracey.m.erway at intel.com>; 'sjolley.yp.pm at gmail.com'
><sjolley.yp.pm at gmail.com>; openembedded-core-
>request at lists.openembedded.org
>Cc: Sangal, Apoorv <apoorv.sangal at intel.com>; Yeoh, Ee Peng
><ee.peng.yeoh at intel.com>
>Subject: Re: QA cycle report for 2.6.2 RC3
>
>On Wed, 2019-04-10 at 02:38 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.6.2 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> ·        For results of all automated tests, refer to results at
>> public AB [1].
>> ·        For other test results, refer to attachment [2].
>> ·        For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> ·        For full test report, refer attachment [4]
>> ·        For ptest results, please refer to results at public AB [5]
>> ·        For ptest report, refer to attachment [6]
>> 1 new defects are found in this cycle, Beaglebone [7].
>> QA hint:  Similar issue for sysemtap failure (bug 13153) was found in
>> 2.7 M2 RC1 which is now resolved for master.
>> Number of existing issues observed in this release is 3- toaster [8],
>> Build-appliance [9] and qemu-shutdown [10] For ptest, regression data
>> is not available for this release. 3 packages are facing timeout
>> issues: lttng-tools [11], openssh [12] and strace [13].
>> Test result report on Public AB shows following failures:
>> buildgalculator.GalculatorTest.test_galculator
>> QA Comment: Expected failure for hw limitation
>> python.PythonTest.test_python3
>> QA Comment: Failure due to “python3” test case not backported to Thud.
>> No bug filed as not real Yocto failure.
>
>Thanks for running QA on this. Firstly, I can comment on the bugs:
>
>> [7] Bug 13273 - [2.6.2 RC3] Systemtap doesn't work on beaglebone
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13273
>
>We've root caused this on master and believe we know how to fix this, its a
>kernel makefile configuration issue. We can likely document this as a known issue
>and fix in 2.6.3.
>
>> Previous Bugs observed in this release:
>>
>> [8] Bug 13191 – [Test Case 1439] Build Failure after adding or
>> removing packages with and without dependencies
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13191
>>
>> [9] Bug 12991 - [Bug] [2.6 M4 RC1][Build-Appliance] Bitbake build-
>> appliance-image getting failed during building image due to webkitgtk
>> package.
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991
>
>This is a long running issue we've been "ignoring", its likely resource issues on the
>machine/VM running the test.
>
>> [10] Bug 13234 - [2.7 M3 RC1] qemumips & qemumips64: failed to
>> shutdown
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13234
>
>This is an issue root caused to be from 4.19 and 5.0 kernels. I thought
>2.6.2 had a 4.18 kernel so shouldn't have this issue? The bug wasn't reopened?

2.6.2 had shutdown issue with qemuarm. Which is a known issue for 2.6 release and marked as won't fix.
Correct bug id for Qemu arm shutdown issue is:

Bug 12499 - [2.5 M1 RC3] qemuarm: failed to shutdown
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12499

Thanks for pointing out!

>
>> ptest Bugs:
>>
>> [11] Bug 13255 - [2.7 M3 rc1] lttng-tools ptest facing timeout issue
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13255
>>
>> [12] Bug 13256 - [2.7 M3 rc1] openssh ptest facing timeout issue
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13256
>>
>> [13] Bug 13274 - [2.6.2 RC3] strace ptest facing timeout issue
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13274
>
>ptest has been a bit tricky and we've been actively working these bugs on master.
>Once we have them resolved there we can backport to older releases.
>
>There is one other critical issue we have with 2.6.2 which is the revert of the
>boost upgrade. I'm very reluctant to release without that revert as it caused
>problems for a lot of people.
>
>I'm wondering whether we should rebuild 2.6.2 one commit later and then
>release that assuming it passes a rerun of the automated QA (but not the manual
>QA). Opinions on that?
>
>I'd propose fixing the systemtap and ptest issues for 2.6.3.
>
>Cheers,
>
>Richard
>
>


Thanks & Regards,
Sangeeta Jain