[yocto] QA cycle report for 2.5.3 RC3

richard.purdie at linuxfoundation.org richard.purdie at linuxfoundation.org
Tue Apr 23 07:35:30 PDT 2019


On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>  
> QA cycle report for 2.5.3 RC3:
>  
> No high milestone defects. 
> Test results are available at following location:
> ·        For results of all automated tests, refer to results at
> public AB [1].
> ·        For other test results, refer to attachment [2].
> ·        For test report for test cases run by Intel and WR team,
> refer attachment [3]
> ·        For full test report, refer attachment [4]
> ·        For ptest results, please refer to results at public AB [5]
> ·        For ptest report, refer to attachment [6]
> No new defects are found in this cycle.
> Number of existing issues observed in this release is 2- toaster [7]
> and Build-appliance [8]
> For ptest, regression data is not available for this release. No
> timeout issues.
> Test result report on Public AB shows no failures.

I've been discussing with Tracy and Vineela how to best handle the test
results for the release for 2.5.3. In the end I:

a) Copied in the ptest results to the right place. This will happen
automagically in all future builds:

https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-ptest/

b) Copied in the intel test results to their own directory:

https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/

c) Manually generated an updated test report for the combined results
with the commands:

$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-
2.5.3
$resulttool report . > testreport.txt

d) Added a header to the report which consisted of the report from QA
with details of the bugs etc.

This means we have a top level testreport file which contains all the
test information about the release.

I'd like to make this the standard procedure for release. The release
notes and announcement can refer to the test report included with the
release and the test results as its there all together.

Ultimately I'd like to improve the formatting of the report (separate
out ptest regressions from the other regressions, maybe html, maybe
graphs, better regression information) but that is something for the
future.

Does that work for everyone?

Cheers,

Richard




More information about the yocto mailing list