[Automated-testing] [RFC] Analysis of the Yocto Test System

[Markus Boos]

Hi Claudius

We experienced the same critical assumptions as you did. 

For our testing we use:
- Jenkins (CI and trigger and display RF tests)
- Robot Framework (system and acceptance test framework)
- Serial Connection to DUT (good 'old test interface)
- Two real ethernet interfaces, where one provides a test network for the DUT
Everything runs on a single linux host

Our setup gives us flexibility, as we have devices built with other buildsystems.

BR
Markus




> -----Ursprüngliche Nachricht-----
> Von: automated-testing-bounces at yoctoproject.org [mailto:automated-testing-
> bounces at yoctoproject.org] Im Auftrag von Claudius Heine
> Gesendet: Montag, 9. Februar 2015 10:48
> An: automated-testing at yoctoproject.org
> Cc: Wolfgang Denk; Roger Meier
> Betreff: [Automated-testing] [RFC] Analysis of the Yocto Test System
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi,
> 
> in the last week I analyzed Yoctos automated Runtime testing System to find out if it
> could be used within DENX and Simens and, if not, what needs to be additionally
> implemented.
> 
> We at DENX currently use DUTS[1] for our testing of u-boot, Linux kernel and its
> drivers. But DUTS is rather old and implemented with Tcl/expect scripts and
> therefore has become problematic to work with.
> 
> My investigation of Yoctos Test System revealed that it is currently more focused on
> application and package testing and for example doesn't even provide convenient
> ways to test systems without network access (ssh required) or limited storage
> resources (Master Image Deployment).
> 
> There also doesn't seems to be a generic method of controlling u-boot with its
> command line interface. That would be useful for us, because for many of our test
> cases a running Linux isn't even necessary.
> 
> Also the deployment options are currently rather limited. NFS and TFTP would be
> nice to have.
> 
> I uploaded a draft of my report[2] and request you comments, ideas or suggestions.
> 
> Thanks and best regards,
> Claudius
> 
> [1] http://www.denx.de/wiki/DUTS/DUTSDocs
> [2]
> https://owncloud.denx.de/public.php?service=files&t=ed2e2aa62dbc71e11e1da3c80fe
> 1a77a
> 
> - --
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch at denx.de -----
> BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> 
> iQIbBAEBAgAGBQJU2IJVAAoJEBFzyxmYCLFTxHQP+LI3VTVJYVBB/G+OvAcXmau
> i
> kJNoWnhGGpEJDbQqY3m9Dp5olFo31vet7o6MWmzLctq8gDpSeZwvKRM0o+3L9E4
> E
> uDLmlrdfdrzbCcTtybT/PB9JKSL/K6WKSILJHPnLzxCBmwlo5OnugGeuMeBW5SPF
> ccBbgS1nvVSRPUhJFZPSlpYrgtwhDvkpioXcay6KJd226I4veVwkVQbxaQnD2QoL
> 4RTDbB1ZfoM2KiEpHiRVFUoIibMsXYRehuyTnUgTvn5FYREsdh49dozqBob3tcCw
> 90ITSTVo1RCai7RXvz0jFWeJcOu4dDt6FAKf0sxdh8UeA7O0EUKH6O83k3gWmzi4
> MODsbI1BxWP61h20o36ipnUn56TXpVkOJscXeKHBBQO+UKq9QkYZC871oWK38PL
> +
> FnYjP8eVkifQ/sGPTz/dad6l6iTnvkapGUfglI81rjXqIjitXuk7FsiS46lvIqDb
> YvZEkLk2BPXWYqc3O4s5UcXMqT+PvYVOFWqb4WbYif/VXGPFGsSrPl8v0fpMcVS
> q
> a+Y/O+Uiynu8xKnxmhKhKAvxRhF/oP0FLDUXXXbppJOjXBaLHQK+dIvGvv8+pXIn
> 0bIZSpsZquEUMgL3mQWWtnTQLUfK3Sj5xC470YdJQwfEt8UuDIcKSe4UaR1U8x75
> 8WGHtQ7P0xinSAtxsYE=
> =1lcd
> -----END PGP SIGNATURE-----
> --
> _______________________________________________
> automated-testing mailing list
> automated-testing at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/automated-testing