[yocto] QA cycle report for 2.4 RC3

Tue Oct 24 12:24:25 PDT 2017

Hello All, I'm sending the report on behalf of Libertad G.

Enjoy reading the full Report for 2.4 RC3:  https://wiki.yoctoproject.org/wiki/WW43_-_2017-10-23_-_Full_Test_Cycle_2.4_RC3   

======= Summary ========

The QA cycle for release 2.4 RC3 is complete.  There are 7 new bugs, 5 Toaster related; other one is related with LSB test to update to use a newest binary.  Also we have an eclipse plugin bug about not able to launch qemu. None of the 7 bugs are blocking functionality.

ptest 
Results are same as RC2, there are no changes in the pass rates,  and no high priority bugs. 

Posix
2.4 RC3 Posix results are very similar to last QA cycle 2.4 RC2. Now the Edgerouter test were executed.

                  Failures                  2.4 RC3           2.4 RC2
                  Generic86-64           46                      46
                  Beaglebone              45                      45
                  Edgerouter               47                      
                  Mpc8315e-rdb         45                      45

For more details visit:  https://wiki.yoctoproject.org/wiki/Posix_result and https://wiki.yoctoproject.org/wiki/POSIX-results

LTP
2.4 RC3 LTP results show that, Edgerouter was not able to run Windriver will update the results soon. For currently executed LTP test,  QA has not found bugs. 

                                                        2.4 RC2		   2.4 RC3
                                                    Total   |  Failures	Total	| Failures
               Genericx86-64              1391   |     27		1391	|    26
               Beaglebone                  1353   |     17		1353	|    18
               Edgerouter                   1353   |     		1353	|
               Mpc8315e-rdb             1353   |     30		1353	|    29

For more details: https://wiki.yoctoproject.org/wiki/LTP_result

Performance
Measurements  form 2.4rc3 are the same as observed on previous rc2.  

  Ubuntu           Test        2.4_rc2                2.4_rc3                %
                          sato       1:16:32                1:15:28                -1.39
                          rootfs    2:45                     2:45                      0.00
                          rmwork 1:08:20                1:08:58                0.93
                          kernel    5:45                     5:44                      -0.29
                          eSDK      2:42                     2:37                      -3.09                     

Fedora		Test        2.4_rc2                2.4_rc3                %
                          sato       1:17:23                1:17:22                -0.02
                          rootfs    2:40                     2:40                      0.00
                          rmwork 1:10:30                1:10:36                0.14
                          kernel    6:05                     6:01                      -1.10
                          eSDK      2:39                     2:41                      1.26

======= QA-Hints========

The QA cycle is complete and the team hasn't found blocking issues. The release candidate is stable hence QA does not opposed for candidate to be released, however there are still 2 bugs on Current community Open bugs targeted to be fixed for 2.4 M4 (11385[1]  and 11757[2]) , they need to be moved. 

======= Bugs ========

       New Bugs
	- 12270	[3] Cannot launch QEMU from Eclipse using eSDK
	- 12240	[4] LSB-test: package list should be updated to use latest binary available
	- 12259	[5] when building 3 or more images left hand summary is not shown correctly
	- 12260	[6] why arent the meta-qt3 and meta-qt4 present in the compatible layers for poky
	- 12261	[7] Importing layers, confirmation label is misleading
	- 12262	[8] Layer details page: optional layers can't be blank when importing a layer
	- 12263	[9] project layers can be duplicated

       Not new M+ bugs
	- 12131	[10] Kernel traceback when reboot beaglebone black
	- 12185	[11] devtool_modify_native test failure in ross/rocko
	- 11994	[12] Graphical qemu doesn't work over remote X on Fedora 26

Full Bug Report: 
	- https://wiki.yoctoproject.org/wiki/WW43_-_2017-10-23_-_Full_Test_Cycle_2.4_RC3#Bugs_Found_during_QA_Test

======== Links =========
1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11385	[1]
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11757	[2]
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12270	[3]
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12240	[4]
5. Https://bugzilla.yoctoproject.org/show_bug.cgi?id=12259	[5]
6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12260	[6]
7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12261	[7]
8. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12262	[8]
9. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12263	[9]
10. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12131	[10]
11. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12185	[11]
12. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11994	[12]

Regards
Sonia Leon