[yocto] QA cycle report for 2.6 M2 RC1

[richard.purdie at linuxfoundation.org]

On Thu, 2018-08-09 at 05:57 +0000, Yeoh, Ee Peng wrote:
> ======= Summary ========
>  
> All planned tests were executed except a few SDK automated tests were
> skipped in Autobuilder (testrun# 9830-9836, 9796-9797).  Team found
> that these SDK automated tests were running successfully in core-
> image-sato-sdk image but not the existing core-image-sato used by
> Autobuilder, investigating why these automated tests were skipped in
> core-image-sato and no message available in testsdk logfile to
> explain why it was skipped.

I think Ross just sent a patch for this, assuming it was the galculator
test?

>   Team also found that eSDK devtool automated testcase(s) were
> skipped unexpectedly due to the unknown error on testcase dependency
> logic, temporary removing the dependency logic and successfully
> executed the devtool tests.

We should probably have a bug to track and resolve this issue as it
hasn't been resolved yet as far as I know. I believe its a problem
introduced by the parallelism changes. We may have to rework the
dependency logic.

> New Bugs
> [1] Bug 12866 - [2.6 M2 rc1] wic test_fix_size failed
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12866
>  
> [2] Bug 12864 - [2.6 M2 rc1] buildhistorydifftest failed
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12864
>  
> [3] Bug 12869 - [2.6 M2 rc1] libxml test cases failed
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12869

The good news is that these have all already been fixed in master and
are understood issues with good resolutions. I don't believe any of
them are enough to warrant an rc2. My recommendation is we release rc1
subject to my question below.

Why does the QA report not show 100% of tests were completed in all
cases?

We do need to take the list of bugs in the QA report itself and see if
we can resolve some of them for the final release, Stephen, your help
in driving/tracking that would be appreciated.

Cheers,

Richard