[Automated-testing] LAVA Intro presentation

[Philip Balister]

On 02/04/2014 07:19 PM, William Mills wrote:
> On 02/04/2014 05:11 AM, Philip Balister wrote:
>> On 01/30/2014 05:51 PM, William Mills wrote:
>>>
>>> I will try to write up my vision of the use cases.  I will also take a
>>> crack at capturing an outline on the YP wiki.  However, I also think we
>>> need to hear from Paul or Darren.
>>>
>>
>> I've got a coupe of use cases :
>>
>> 1) Given an image built with OE, I need to load test case and exercise
>> the installed software for proper operation.
>>
>> 2) Given a "standard" image, have tests that load additional packages
>> and run tests on them. Possibly from build server package feeds.
>>
>> In both cases I think we can use existing ptest packages to test the
>> installed software.
> 
> This is good.
> 
> I think I understand #2 above but I am trying to understand where #1 is
> different.
> 
> Are the test assets (ptest packages etc) already in the image in case
> #1?  Or are the being "loaded" by some other means than package manager
> operations (such as a scp push, nfs mount, etc)?  Something else?

One image is complete on load onto the test hw, the other loads a base
image and the test process installs packages. This way you do not need
to create new image to test a new package.

Philip

> 
>>
>> What I am looking for is a way to look for regressions i packages as
>> they are updated. This would be useful for distro maintainers who want
>> confidence a package upgrade will not break current users and developers
>> how want to make sure their software updates do not break on archs they
>> are not actively developing/testing on.
>>
>> All of this should be completely transparent to the operator, so that
>> boards are not bricked and require manual intervention to revocer from
>> hard failures.
>>
>> Board farm scaling is important so the system can test large numbers of
>> updates in parallel.
>>
>> Philip
>>