[Automated-testing] CKI Test Stack Survey (was RE: CKI project introduction)

[Veronika Kabatova]

----- Original Message -----
> From: "Veronika Kabatova" <vkabatov at redhat.com>
> To: "Tim Bird" <Tim.Bird at sony.com>
> Cc: automated-testing at yoctoproject.org
> Sent: Thursday, August 8, 2019 2:26:17 PM
> Subject: Re: [Automated-testing] CKI Test Stack Survey (was RE: CKI project	introduction)
> 
> 
> 
> ----- Original Message -----
> > From: "Tim Bird" <Tim.Bird at sony.com>
> > To: vkabatov at redhat.com
> > Cc: automated-testing at yoctoproject.org
> > Sent: Wednesday, August 7, 2019 9:10:08 PM
> > Subject: RE: CKI Test Stack Survey (was RE: CKI project introduction)
> > 
> > Thanks!
> > 
> > Sorry this took so long for me to process (only 5 months!).
> > 
> > I've created the following wiki page for the survey response:
> > https://elinux.org/CKI_survey_response
> > 
> > And it's linked from the main test stack survey page here:
> > https://elinux.org/Test_Stack_Survey
> > 
> 
> Thanks! A lot of things changed since I filled out the initial info. I think
> I already have access to wiki editing so I'll just go ahead and update the
> page so I don't have to give you more work :)
> 

Updated! Hopefully I got everything.


Veronika

> 
> > It's quite interesting to see how CKI compares with other systems.
> > I'm not sure if it's a one-to-one correspondence, but I get the impression
> > that CKI/Beaker is comparable in division of responsibilities to LKFT/LAVA.
> > 
> > It would be interesting to do a comparison between Beaker and LAVA.
> > They are quite different in implementation, although at a high level
> > I think they do similar things.
> > 
> 
> I'm not familiar with LAVA except "it runs tests" but if you want any info
> about Beaker feel free to ask and I'll do my best to answer everything.
> 
> > I added an "additional information" section.  Can you take a look and
> > make sure that what I have there is correct.  For example, I think your
> > main project web site is at:
> > https://cki-project.org
> > 
> > with project source repositories at:
> > https://github.com/CKI-project
> > 
> > and I think your open source tests are at:
> > https://github.com/CKI-project/tests-beaker
> > 
> > Can you recommend some good example tests from your collection, to examine
> > to get a feel for your system?
> > 
> > Specifically, if you have a very simple test, and a "characteristic" test
> > that you
> > could point to (give the repository URL for each test), that would help me
> > understand
> > your test definition (which is what I'm working on now, in preparation for
> > the
> > meetings in Lisbon).
> > 
> 
> The tests we have are maintained by kernel QE and the interface they use to
> talk to Beaker varies a lot from team to team. However we do actually have
> a super simple example test -- basically "exit 0" wrapped as Beaker task [0].
> 
> A test that is still very simple but uses more Beaker features (result
> reporting, log uploads etc.) is [1]. A more complicated one is the LTP Lite
> wrapper which includes a knownissue detection and spans into a different
> include directory.
> 
> Most test definition information can be found in "Makefile" and "metadata"
> (if present) files. We also have an internal test definition "DB" which we
> are looking into open sourcing but didn't get that far yet. This one
> contains information about test maintainers, test waiving, which trees and
> architectures the test should be executed, which modified files in kernel
> should trigger the test etc. Some of these information is also present in
> the data/code of the tests themselves but not all.
> 
> 
> Let me know if this helps or if you need more info. I'll try to update the
> info page later today.
> 
> Veronika
> 
> 
> [0] https://github.com/CKI-project/tests-beaker/tree/master/example
> [1] https://github.com/CKI-project/tests-beaker/tree/master/acpi/acpitable
> [2]
> https://github.com/CKI-project/tests-beaker/tree/master/distribution/ltp/lite
> 
> > Thanks,
> >  -- Tim
> > 
> 
> <snipped the long survey part>
> --
> _______________________________________________
> automated-testing mailing list
> automated-testing at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/automated-testing
>