[yocto] QA cycle report for 2.6 M4 RC1

[richard.purdie at linuxfoundation.org]

On Sat, 2018-11-10 at 10:02 -0800, akuster808 wrote:
> 
> On 11/10/18 8:25 AM, richard.purdie at linuxfoundation.org wrote:
> > On Fri, 2018-11-09 at 09:24 +0000, Jain, Sangeeta wrote:
> > > This is the full report for 2.6 M4 RC1: 
> > > https://wiki.yoctoproject.org/wiki/WW44_-_2018-10-30_-_Full_Test_Cycle_2.6_M4_RC1
> > 
> > Thanks Sangeeta and team!
> > 
> > Now we have the QA report for YP 2.6 M4 rc1 (Final 2.6) we need to make
> > a release go or nogo decision. To do this we have the following:
> >  
> > QA Report: https://wiki.yoctoproject.org/wiki/WW44_-_2018-10-30_-_Full_Test_Cycle_2.6_M4_RC1
> > Release Criteria: https://wiki.yoctoproject.org/wiki/Yocto_Project_v2.6_Status#Milestone_4.2FFinal_-_Target_Oct._26.2C_2018
> > 
> > We'd be happy to take representations from members and the community to
> > help reach that decision.
> 
> Regarding. 
> Bug 12991 - [2.6 M4 RC1][Build-Appliance] Bitbake build-appliance-
> image getting failed during building image due to webkitgtk package
> Does it mean the Build-Appliance is non functioning ?  It was broken
> at the Sumo release time as well. Should it be dropped as the release
> criteria?

Build-appliance is a tricky test as it tests multiple things, roughly:

* vmdk images under vmware
* the web browser
* toaster
* whether we can self host (build poky within poky)

The fact the webkit recipe failed to build may be due to several
reasons:

* random race type condition
* lack of memory in the VM
* phase of the moon
* other things

I'm not convinced its a release blocker, or that it invalidates the b-a 
test, or that it would even reproduce. If it does reproduce that would
be more interesting and easier to debug.

I do think we're going to have to split up the b-a test in 2.7 so that
even if its not manual QA'd, we can automatically test some pieces of
what it covers.

Cheers,

Richard