[yocto] [Announcement] Yocto Project 1.5 Milestone 3 now available.

[Björn Stenberg]

Paul Eggleton wrote:
> Right, that's what we would be aiming for. Perhaps you could talk a little
> bit about how you manage your hardware testing and any additional software
> and scripts you make use of there?

We run our tests from buildbot, using an expect script to allocate the right serial port (and wait if it is busy), fetch the built files, unpack the rootfs to an NFS disk, reset the board, tftp the kernel and dtb, boot the kernel and start the test suite.

A line-based timeout is used for for the test suites, meaning that the timeout is reset for each new line of output. This allows us to handle very long-running test cases while still being able detect hung tests rather quickly.

Multiple images and test suites are built and ran in parallell, which is why the script waits for the serial port. One of the test suites we run is ptest-runner.

The output of all tests is parsed and stored in a database, cross-referenced with the buildbot build info. We have written an extension to the buildbot web interface presenting the test results as they relate to each build.

-- 
Björn