[yocto] Automating imaging building and testing, what aproach to use!?

[Paul Eggleton]

Hi Daniel,

On Tue, 30 Aug 2016 17:18:44 Daniel. wrote:
> While writing software we're used to delivery packages, libraries and
> stacks. There are out there a lot of continuos integration solutions
> to automaticaly build and test these kinds of software. But when
> dealing to images the thing is more tricky.
> 
> I can't run the tests at the same machine that build the image because
> crosscompilation take place on 99% of the cases. What is aproach that
> you guys are using to automate and increase the quality of your
> images?
> 
> Automating the build is the easy part my concert is about automating
> the runtime tests that need the target board to run. In my case I
> depend on hardware to fully test the image features. Is there any
> reliable way to automate image installation and boot!?

There are some folks here working on automated hardware tests (on CC), perhaps
they can expand on what we're currently doing in that area. At least in the
existing code we do have basic support for running tests on real hardware that
may be worth looking into - at the moment though it's pretty rudimentary when
it comes to interacting directly with the hardware though. You can see what
we've currently got here:

  http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#hardware-image-enabling-tests

We've looked at LAVA several times, and I'm sorry to say the conclusion each
time is that its a mess - both from a usage perspective and looking at the
code. It was disappointing to us because initially it looked like it was going
to solve a lot of our problems. Maybe others have had different experiences -
I'd love to hear details if anyone is prepared to share.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre