[yocto] QA Tests in Yocto development environment block diagram

[Stoicescu, CorneliuX]

Hello Dilip,

I will try and answer as much as I can on behalf of Alex:

We have 2 types of tests:

1)      Buildtime

2)      Runtime

Each type of test has 2 main categories(from a QA point of view):

-          Automated

-          Manual

Here is where you can find each:


-          Automated Buildtime:

o   Sanity tests(run for every build):

§  In the Yocto sources: meta/classes/sanity.bbclass

§  In the Yocto sources: meta-yocto/classes/poky-sanity.bbclass

§  There are other many various sanity checks done explicitly throughout poky. I do not have the knowledge to get in more detail on this though.

o   We are working on a new feature that will add automated "gray box" buildtime tests: https://bugzilla.yoctoproject.org/show_bug.cgi?id=4740


-          Automated Runtime:

o   Automated image tests:

§  Covers only QEMU(virtual) targets

§  After building an image, if the correct configuration is set, it will boot up and tests will be run on it consisting of commands sent over ssh

§  The tests can be found in the Yocto sources: meta/lib/oeqa/runtime/

§  This uses the 'testme.bbclass' from /meta/classes/testme.bbclass

§  More documentation on this feature can be found here: https://wiki.yoctoproject.org/wiki/Image_tests

§  The test cases are also described in Testopia: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=18

·         Note that this source contains both manual and automated tests

§  We are working on a new feature to expand these tests to BSP(hardware) targets: https://bugzilla.yoctoproject.org/show_bug.cgi?id=1596

o   Package testing

§  Documentation: https://wiki.yoctoproject.org/wiki/Ptest


-          Manual Buildtime:

o   Can be found in Testopia: https://bugzilla.yoctoproject.org/tr_show_product.cgi

§  At the moment the following buildtime products in Testopia contain tests:

·         ADT: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=1

·         Eclipse Plugin: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=5

·         HOB: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=8

·         OE-CORE: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=25

·         Meta-Yocto: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=26

·         Bitbake: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=9

·         Build Appliance: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=28


-          Manual Runtime:

o   Covers BSP(hardware) and QEMU(virtual) targets

o   Can be found in Testopia: https://bugzilla.yoctoproject.org/tr_show_product.cgi

§  At the moment the following runtime products in Testopia contain tests:

·         BSP: https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=18

·         Runtime(mostly performance and compliance tests): https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=34

o   Note that this source contains both manual and automated tests

I hope I did not miss anything and that I was clear in my description. If you need any more information, please don't hesitate to ask!

Regards,
Corneliu
Romania Yocto QA

From: yocto-bounces at yoctoproject.org [mailto:yocto-bounces at yoctoproject.org] On Behalf Of Dilip Kumar
Sent: Tuesday, November 19, 2013 8:08 PM
To: Georgescu, Alexandru C
Cc: yocto at yoctoproject.org
Subject: Re: [yocto] QA Tests in Yocto development environment block diagram


Dear Alex,

Thank you for your response.

And I need to know one more thing whether these QA tests are done manually or it's automated during the build or its done separately by Yocto for particular distribution?

Is there any QA tests done during the build procedure locally?

regards
dilip kumar b
On 14-Nov-2013 12:15 am, "Georgescu, Alexandru C" <alexandru.c.georgescu at intel.com<mailto:alexandru.c.georgescu at intel.com>> wrote:
Hi,
Our overall testing plan is available here: https://wiki.yoctoproject.org/wiki/Yocto_1.6_Overall_Test_Plan. Since we are in the last phase of planning for 1.6, this is not the final version yet.

Regarding the test cases, all you have to do is to have an account in our bug tracking website https://bugzilla.yoctoproject.org/ and follow the instructions in the Testopia wiki, our test case platform management system: https://wiki.yoctoproject.org/wiki/Testopia. There you can find the tests performed against the Yocto Project components.

Regards,
--
Alexandru Georgescu

From: yocto-bounces at yoctoproject.org<mailto:yocto-bounces at yoctoproject.org> [mailto:yocto-bounces at yoctoproject.org<mailto:yocto-bounces at yoctoproject.org>] On Behalf Of Dilip Kumar
Sent: Wednesday, November 13, 2013 06:27
To: yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: [yocto] QA Tests in Yocto development environment block diagram


Dear all,

I need know what is QA tests and what all the tests it will do and need to know more information about QA please help me out in this.

regards
dilip kumar b
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20131121/66304472/attachment.html>