[Automated-testing] [RFC] Analysis of the Yocto Test System

[Otavio Salvador]

Hell Claudius,

On Mon, Feb 9, 2015 at 7:48 AM, Claudius Heine <ch at denx.de> wrote:
> in the last week I analyzed Yoctos automated Runtime testing System to
> find out if it could be used within DENX and Simens and, if not, what
> needs to be additionally implemented.
>
> We at DENX currently use DUTS[1] for our testing of u-boot, Linux
> kernel and its drivers. But DUTS is rather old and implemented with
> Tcl/expect scripts and therefore has become problematic to work with.

I know how bad it can be for maintenance :-) I worked with expect a
lot in the 'digress'  (Debian Installer Regression Test System) while
in charge of the Debian Installer, few years ago ...

> My investigation of Yoctos Test System revealed that it is currently
> more focused on application and package testing and for example
> doesn't even provide convenient ways to test systems without network
> access (ssh required) or limited storage resources (Master Image
> Deployment).
>
> There also doesn't seems to be a generic method of controlling u-boot
> with its command line interface. That would be useful for us, because
> for many of our test cases a running Linux isn't even necessary.
>
> Also the deployment options are currently rather limited. NFS and TFTP
> would be nice to have.

I fully agree with your conclusions here. I would be interested in
know if you have thought about possible solutions for those
limitations and how it could be done? Also LAVA seems to have some of
those features already implemented. Did you look at it?

> I uploaded a draft of my report[2] and request you comments, ideas or
> suggestions.

Very good. By the way is it done in which documentation framework?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750