[yocto] FW: QA cycle report for 2.4 RC1
Cruz, Libertad
libertad.cruz at intel.com
Fri Oct 6 16:38:26 PDT 2017
Hello All,
Enjoy viewing the full Report for 2.4 RC1: https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1
======= Summary ========
The QA cycle for release 2.4 RC1 is complete. There are 18 new bugs from which 5 are high priority and are targeted to be fixed in 2.4 M4. Four of the High Priority bugs (12143[1], 12144[2], 12145[3], 12146[4]) are ptest related and one (12194[5]) is Toaster related. An additional medium+ priority (12131[6] Kernel traceback when reboot beaglebone) is targeted to be fixed by 2.4 M4. As an extra note on this report QA wants to highlight that, there are a total of 46 Bugs targeted to be fixed in 2.4 M4 (link [20]).
ptest
Results show that python and e2fsprogs are not executing, hence bugs 12143[1] and 12146[4], respectively, were opened and given a high priority.
Bash (bug 12145[3]) had a decrement in the pass rate of 1.27% and busybox (bug 12144[2]) had a decrement in the pass rate of 0.19%. Although they are high priority, QA sees that the decrement is not very significant, hence this bugs can be lowered in priority.
Eclipse
Some good news: testing for Eclipse-Oxygen was added for the first time to the QA test cycle.
Posix
2.4 RC1 Posix results are very similar to last QA cycle 2.4 M3 rc1. They only show that Edgerouter had one more failure if compared with the previous run.
Failures 2.4 RC1 2.4 M3 RC1
Generic86-64 46 46
Beaglebone 45 45
Edgerouter 47 46
Mpc8315e-rdb 45 45
For more details visit: https://wiki.yoctoproject.org/wiki/Posix_result and https://wiki.yoctoproject.org/wiki/POSIX-results
LTP
2.4 RC1 LTP results show that, for Genericx86-64 and Beaglebone, the failures incremented by 1 test. On Edgerouter we have an increment of 2 failures if compared with QA cycle 2.4 M3 rc1.
2.4 RC1 | 2.4 M3 RC1
Total | Failures | Total | Failures
Genericx86-64 1391 | 27 | 1391 | 26
Beaglebone 1353 | 18 | 1353 | 17
Edgerouter 1353 | 34 | 1353 | 32
Mpc8315e-rdb 1353 | 31 | 1353 | 31
For more details: https://wiki.yoctoproject.org/wiki/LTP_result
Performance
In general, the measurements were on the same levels showed up on 2.4_M3. The only noticeable variation was on "rootfs" that is showing an increase of 5% on Ubuntu and 7% on Fedora, although this does not represent an important regression.
Ubuntu Test 2.4 M3 rc1 2.4_rc1 %
sato 1:15:58 1:15:24 -0.75
rootfs 2:34 2:45 7.14
rmwork 1:08:15 1:08:19 0.10
kernel 5:36 5:41 1.49
eSDK 2:39 2:40 0.63
Fedora Test 2.4 M3 rc1 2.4_rc1 %
sato 1:14:13 1:17:02 3.80
rootfs 2:35 2:44 5.81
rmwork 1:09:06 1:10:25 1.91
kernel 5:58 6:04 1.68
eSDK 2:37 2:42 3.18
======= QA-Hints========
Due to the fact that, there are 5 high priority bugs and 46 bugs targeted to be fixed and solved by 2.4 M4, QA requests an RC2 so that all the commits/fixes relevant to 2.4 release are integrated.
======= Bugs ========
New Bugs
-12143[1] Ptest-runner does not run python, in 2.4rc1
-12144[2] ptest-busybox cases failed in 2.4rc1
-12145[3] ptest-bash cases failed in 2.4rc1
-12146[4] ptest-e2fsprog cases failed in 2.4rc1
-12194[5] Toaster server lost "toaster.conf" environment, user settings lost
-12131[6] Kernel traceback when reboot beaglebone black Medium+
-12170[7] Add mkfs.ext support to busybox on poky-tiny
-12138[8] [master] the icon on the connection manager appears without connection in Eclipse Plugin
-12168[9] media player - must be warned that it is not possible to play MPEG-1 and MP3 formats
-12150[10] bug for QA to create Testcases for this new test on testopia AUTO_eSDK_sdkext
-12133[11] The search results for numbers are incorrect.
-12140[12] The link and the x button from the search's result did not work as expected
-12142[13] The color from the table from the project build changed when you use an special character.
-12164[14] The field from time's task is empty
-12171[15] After building an image, checking the packages there's nothing.
-12175[16] [Test Case 946] View detailed configuration information for a build
-12200[17] The button from New custom image is not working as expected
-12201[18] The button from New custom image is letting you press it and it's supposed to be disabled.
Not new M+ bugs
-11994[19] Graphical qemu doesn't work over remote X on Fedora 26
Full Bug Report: https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1#Bugs_Found_during_QA_Test
======== Links =========
1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12143[1]
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12144[2]
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12145[3]
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12146[4]
5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12194[5]
6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12131[6]
7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12170[7]
8. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12138[8]
9. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12168[9]
10. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12150[10]
11. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12133[11]
12. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12140[12]
13. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12142[13]
14. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12164[14]
15. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12171[15]
16. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12175[16]
17. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12200[17]
18. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12201[18]
19. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994[19]
20. https://wiki.yoctoproject.org/wiki/WW40_-_2017-10-06_-_Full_Test_Cycle_2.4_RC1#Current_community_Open_Bugs_-_Medium.2B.2FHigh[20]
Regards
Libertad G.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20171006/a93c2e60/attachment.html>
More information about the yocto
mailing list