[Automated-testing] Farming together - areas of collobration

[Steve McIntyre]

On Mon, Feb 12, 2018 at 09:51:37PM +0000, Tim.Bird at sony.com wrote:
>> From: Steve McIntyre on Monday, February 12, 2018 10:33 AM
>> On Tue, Nov 14, 2017 at 09:16:01PM +0000, Bird, Timothy wrote:
>> >
>> >To be a bit more blunt about it, as a concrete example: would LAVA
>> >adopt a new system for low-level board control, if it presented
>> >itself?  I kind of doubt it. Who would do this work?
>> 
>> That's the big question, yes. In the LAVA team, we've already solved a
>> lot of the issues that we've seen, in our own ways. We want to be good
>> Open Source citizens (of course!), but it's difficult to justify
>> spending much engineering time on ripping things out and moving to
>> different underpinnings unless we can see concrete benefit. I'd expect
>> most teams to be the same. Chicken and egg, as you said earlier. The
>> key thing to make it all worthwhile will be to show the value of doing
>> it, while minimising the cost. Let's see where we can take that.

,,,

>One thing I've observed is that there are substantial differences
>in the level of abstraction and the board management architecture
>for these different systems.  I previously had hoped we could do a simple
>survey of the different systems, and come up with a set of verbs
>that everyone could use - making it possible for, say, tests written
>in LAVA to run in a farm that used labgrid or r4d.  I would certainly like
>to easily run Fuego tests in LAVA-based farms.  Actually, I'd like to make
>Fuego independent of the board farm management software, so the
>user can choose.

Right. :-)

>However, it has turned out to not be that simple.  Different systems
>impose different hardware requirements, or make assumptions about
>the nature of system deployment, that seem incompatible with
>other systems.  Or, at least that's my understanding.
>
>I'm not sure where to start, but maybe it would be good to have a
>discussion about the basic requirements and operation of each
>system, to see where the compatibilities and incompatibilities are.
>(Or, maybe start with the requirements that the upper-level
>software or user have, that are using the different systems).

Nod, that sounds like the best place to start. Get people in a room
and swap what we know and expect.

>Would it be worth putting together some kind of "board farm summit"
>at the next plumbers?  I would suggest ELC, but it's already programmed,
>and coming up soon, and Linaro has Connect going on the same month, so
>I expect travel would be an issue.

Plumbers sounds like a good plan for me and other Linaro
folks. Looking at the ELC dates, it's not *quite* clashing with our
next Connect in Hong Kong, but it's not likely to work with only a few
days between.

Cheers,
-- 
Steve McIntyre                                steve.mcintyre at linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs