[yocto] Automating imaging building and testing, what aproach to use!?

[Daniel.]

Thanks Randy and Mark for the replies!

In my case we have a two boards running basically the same distribuition,
diverging only on some peripherals. I was planing to install/wire
everything up and then use https://mender.io/ to automate update process.
After that I can create post installations tests and see they run at every
image update. I'll have to design the tests so that the output of then goes
to some web interface or pdf+email but this is not a problem.

Another option is using buildbot to automate recipe builing and using it to
feed some intranet rpm/debi/ipk repo. Configure my testing targets with
something like "auto-update". After that is the same mater of creating post
install tests. The pro here is that this way I'll be testing by recipe
builds too, and that updates are faster. The con is that mender updates are
safer ...

The major complexity comes from the fact that we're using distribuited
hardware as may be the case of other people here. We have a "master"
hardware that connects to "slave" hardware, where master is the Linux and
slave are some embedded boards. To test everything up I need to wire
everything up before testing. I would like to hear people solutions for
that case since mulitple "setups" are possible testing every possible setup
is not feasible.

By the way, that LAVA seems promissing! I'll take a closer look on it!
Thanks for sharing :)

Regards,


2016-08-31 2:59 GMT-03:00 Mike Looijmans <mike.looijmans at topic.nl>:

> On 30-08-16 22:18, Daniel. wrote:
> > Hey everybody!
> >
> > While writing software we're used to delivery packages, libraries and
> > stacks. There are out there a lot of continuos integration solutions
> > to automaticaly build and test these kinds of software. But when
> > dealing to images the thing is more tricky.
> >
> > I can't run the tests at the same machine that build the image because
> > crosscompilation take place on 99% of the cases. What is aproach that
> > you guys are using to automate and increase the quality of your
> > images?
> >
> > Automating the build is the easy part my concert is about automating
> > the runtime tests that need the target board to run. In my case I
> > depend on hardware to fully test the image features. Is there any
> > reliable way to automate image installation and boot!?
>
> It helps a lot if you take testing into account when designing the
> hardware.
>
> We have a bunch of hardware targets in a corner. The board can be powered
> on/off forcibly by controlling/monitoring the RTS/DTR/CTS/DSR lines of the
> serial debug port (FTDI chip).
>
> A python script on a PC boots a board, puts it into USB "DFU" mode by
> "typing"
> commands into u-boot, and then sends an image over USB into RAM. The board
> then configures its USB OTG port as network, and the Python script
> connects to
> the board via SSH (paramiko) over that USB connection when it sees the
> "network" come up. Using the SSH connection, it can run whatever tests it
> needs doing (send data and even executables, run shell commands, and get
> reliable feedback on process completion).
>
> One test PC can control multiple boards (each board needs two USB
> connectors
> in this setup).
>
> I'd also be interested to know what other projects are doing.
>
>
>
> Kind regards,
>
>
>
> Mike Looijmans
>
> System Expert
>
>
>
>
>
> *TOPIC Products*
>
>
>
>
>
> Materiaalweg 4
>
>
>
>
>
> 5681 RJ Best
>
> T:
>
> +31 (0) 499 33 69 69
>
> Postbus 440
>
> E:
>
> mike.looijmans at topicproducts.com
>
> 5680 AK Best
>
> W:
>
> www.topicproducts.com
> The Netherlands
>
> <https://www.facebook.com/TopicProducts>
> <https://twitter.com/TopicProducts>
> <https://www.linkedin.com/company/topic-embedded-products>
> Please consider the environment before printing this e-mail
>
> Topic zoekt gedreven (embedded) software specialisten!
> <http://topic.nl/vacancy/topic-zoekt-technische-software-engineers/>
>
>


-- 
*"Do or do not. There is no try"*
  *Yoda Master*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160831/8a7ffc91/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: imagec5944f.PNG
Type: image/png
Size: 9075 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160831/8a7ffc91/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image32a64f.JPG
Type: image/jpeg
Size: 1087 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160831/8a7ffc91/attachment.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image312119.JPG
Type: image/jpeg
Size: 1060 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160831/8a7ffc91/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: imageab7767.JPG
Type: image/jpeg
Size: 1088 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160831/8a7ffc91/attachment-0002.jpe>