[yocto] QA cycle report for 2.4 M2 rc2
Cruz, Libertad
libertad.cruz at intel.com
Wed Aug 2 17:55:15 PDT 2017
Hello All,
Here is the report for the 2.4 M2 rc2 full point release test cycle.
Full Report : https://wiki.yoctoproject.org/wiki/WW31_-_2017-07-31_-_Full_Test_Cycle_2.4_M2_rc2
======= Summary ========
The QA cycle for release 2.4 M2 rc2 is complete.
Performance
Measurements were mostly the same levels as of 2.4 M1 rc1, there was a little improvement of 5% on rootfs build time for Fedora machine.
Ubuntu Test 2.4 M1 rc1 2.4 M2 rc2 %
sato 1:12:11 1:13:58 2.47
rootfs 2:24 2:31 4.86
rmwork 1:08:27 1:07:22 -1.58
kernel 5:26 5:39 3.99
eSDK 2:30 2:30 0.00
Fedora Test 2.4 M1 rc1 2.4 M2 rc2 %
sato 1:13:49 1:13:52 0.07
rootfs 2:38 2:29 -5.70
rmwork 1:08:22 1:08:42 0.49
kernel 6:10 6:04 -1.62
eSDK 2:36 2:34 -1.28
ptest
There was an improvement of the pass rate for gdk-pixbuf of 5% , an improvement of 1.30% on glib-2.0, an improvement of 2.9% on openssh, an improvement of 15.2% on util-linux. There was a regression on the pass rate for libxml2 of 4.20% .
Compliance
For posix test there is no mayor change in the number of failures on 2.4m2 rc2 and 2.4 m1 rc1.
Failures 2.4 M2 RC2 2.4 M1 RC1
Generic86-64 46 45
Beaglebone 45 45
Edgerouter 46 46
Mpc8315e-rdb 45 45
Results are found here: https://wiki.yoctoproject.org/wiki/Posix_result
Detail analysis found here: https://wiki.yoctoproject.org/wiki/POSIX-results
For LTP testing we had 2 additional failures in comparison to 2.4 M1 rc1
Genericx86-64 mpc8315e-rdb
2.4 M2 rc2 Total 1384 1348
Failures 24 30
Genericx86-64 mpc8315e-rdb
2.4 M1 rc1 Total 1384 1348
Failures 22 30
The LPT results are found: https://wiki.yoctoproject.org/wiki/LTP_result .
======= QA-Hints========
QA does not approve that the workarounds provided for 2.4 m2 rc2 were on different commits. Testing was not limited to 2.4 m2 rc2 commit "8e15e9b6e478f6368034519b2a8fd3c7ea71d23b" it was performed by following each commit on the provided workarounds described below.
--11850[1] was opened to track the fact that Eclipse plugins were not publish under 2.4 m2 rc2 so, the plugins were taken from previous build 2.4 m2 rc1. These files need to be copied in to 2.4m2 rc2 if and before release is approved.
--11834[10] describes that if you add a bad layer to bblayers bitbake server will hang in connecting and although is resolved & verified this bug was linked to this release candidate because the commit is not integrated to 2.4 m2 rc2. By linking this issue in the test report users can then find the correct commit that will fix it.
--11857[9] was opened to track that there are some problems creating wic images on CentOS is also resolved & verified, but it is still linked to 2.4 m2 rc2 because the commit is not integrated to this release candidate. Users can then find the correct commit that will fix it.
--11855[4] this bug reports the fact that libgcc-initial can't build on fedora i686, but again when trying with latest master commit it passes. Users can then know which commit to work on.
--11889[11] Toaster does not build on 2.4 m2 rc2 but when changing to latest master commit it works fine.
This testing cycle report does not reflects the results exclusively of 2.4 m2 rc2 but of all the other commits that were applied so that eclipse, toaster, bitbake and ADT on Fedora i686 would not be blocked. We recommend a 2.4 m2 rc3 where all the commits are integrated.
On the wiki a new section was created for Compliance results. For LTP testing there is currently an issue with the hardware infrastructure of Wind River hence beaglebone and edgerouter have not been tested.
======= Bugs ========
New Bugs
-11850[1] [Autobuilder] Eclipse-plugin files are not found in autobuilder
-11873[2] These TCs need to be added to testopia Distrodata.test_checkpkg and Wic.test_mkfs_extraopts
-11879[3] Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64
-11855[4] Failed to build libgcc-initial on Fedora26 i686
-11859[5] [Test Case 1883] TestImage.test_testimage_dnf Centos7
-11868[6] wic.Wic.test_mkfs_extraopts - Testcase -1 opensuse422
-11869[7] FAILED runqemu.RunqemuTests.test_boot_deploy - Testcase 2007: Ubuntu1604
-11871[8] distrodata.Distrodata.test_checkpkg - Testcase -1: ubuntu1604
-11857[9] FAIL [23.106s]: test_mkfs_extraopts (wic.Wic) centos7
-11834[10] if you have a bad layer in bblayers, bitbake server connection hangs
Full Bug Report : https://wiki.yoctoproject.org/wiki/WW31_-_2017-07-31_-_Full_Test_Cycle_2.4_M2_rc2#Bugs_Found_during_QA_Test
======== Links =========
1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11850 [1]
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11873 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477%20> [2]
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11879 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=10832%20> [3]
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11855 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11811%20> [4]
5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11859 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11812%20> [5]
6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11868<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11813> [6]
7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11869 [7]
8. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11871 [8]
9. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11857 [9]
10. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11834 [10]
11. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11889 [11]
Regards
Libertad G.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170803/7014f0aa/attachment.html>
More information about the yocto
mailing list