[yocto] Discussion: Package testing

[Björn Stenberg]

Frans Meulenbroeks wrote:
> a run-all-tests.sh script is somewhat problematic as
> different systems have different packages installed so different tests.

The idea is that the top level run-all script looks in the /opt/ptest directory and runs all package tests that are installed there. It has no built-in knowledge of any tests.

> I think testers are mainly interested in success (all tests give ok is
> good, if a test fails, it would be fine with me if I would have to go to a
> package specific place to find log files etc (or it could be written to a
> common log file)

While failing tests are the most interesting during development, keeping track of both failed and successful test cases is important for regression tracking and test development over time. Therefore we want to produce an output that clearly identifies each test case ran and its result.

> The only specific point I can think of is cleaning up after a test is run.

Good point, I forgot to mention that. It is the responsibility of the per-package "run-ptest" script to clean up after the test.

-- 
Björn