[Automated-testing] Call for agenda items for the Open Automated Testing Standard (OATS) call this Thursday Nov, 14th

[Tim.Bird at sony.com]

> -----Original Message-----
> From: Jan Lübbe
> 
> On Wed, 2019-11-13 at 09:13 +0000, Milosz Wasilewski wrote:
...
> 
> So far I've not thought of test frameworks (where I'd include labgrid)
> and individual engineers of being at the same level. Rather engineers
> would be users for LAVA (via hacking sessions) or labgrid (i.e. via
> labgrid-client).

I'll just inject some notions here.  ttc (which is the provisioning and
board management layer I'm currently using  with Fuego) has support for reserving
boards, but it is focused on resolving contention between engineers.
The basic idea is that engineer reservations take priority.  An engineer can
do an manual reservation, which holds off the CI loop until
the engineer releases the reservation. 

The intended method of use is oriented towards time-division multiplexing
at a large granularity (reservations are expected to be multiple hours).
An engineer is supposed to release the board when done with interactive
use, but a lab manager can break a reservation if the engineer forgets
(or can set a cron job to do this at the start of a CI time period or workday
in their timezone).

The basic idea is that dev boards for products are scarce, and need to
be shared between engineers in different time zones, and by the CI
system.

CI operations are run when the board is not reserved by an engineer.
If the CI system starts to get backlogged, a lab manager has to manually
break a reservation (and add a reservation for the CI system to keep
engineers off a board during CI execution).  None of this
is currently automated.

> 
> My current idea is: The "master scheduler" prototype would talk to each
> test framework and get their internal queue status. Then it would look
> for queued jobs where the necessary boards/resources are currently
> allocated to a different test framework. If they are idle there, it
> would deallocate them there (whether via locking them or placing them
> into a maintainance mode depends on the framework). Then it would
> enable them in the framework that currently needs that board and let
> the job progress to execution.

This sounds like it would work for what I have envisioned for sharing
boards between multiple frameworks (and engineers) in a single lab.

Fuego has board scheduling at 2 layers: 1 in Jenkins for jobs triggered on
the local host, and 1 at the command line level, for jobs that are requested
from federated labs.  This second level of reservation is not very well
implemented at the moment, but is a feature under development for
the next release.

With regard to your implementation idea, Jenkins provides a REST
API to inspect the job queue for a particular node (corresponding to
a board in Fuego), and there's a different REST API provided by Fuego's
fserver for inspecting requests from federated labs.

Figuring out how to wrapper or standardize this API would be useful, IMHO.

> 
> There would probably need to be some mapping of boards names between
> the different framesworks.
> 
> I think that concept should work with the current implementations of
> lava, fuego and labgrid. Then one could experiment with sharing boards
> between frameworks, although the scheduling would probably be less
> efficient than in a single framework.

Agreed.
 -- Ti m