[Automated-testing] Board management API discussion at ATS - my ideas

[Geert Uytterhoeven]

Hi Jan,

On Tue, Oct 22, 2019 at 11:47 AM Jan Lübbe <jlu at pengutronix.de> wrote:
> On Sun, 2019-10-20 at 08:41 +0000, Tim.Bird at sony.com wrote:
> > > > reserving a board:
> > > > verbs: reserve and release
> > >
> > > This touches a critical point: Many existing frameworks have some
> > > scheduler/queuing component, which expects to be the only instance
> > > making decisions on which client can use which board. When sharing a
> > > lab between multiple test frameworks (each with it's own scheduler),
> > > there will be cases where i.e. Lava wants to run a test while the board
> > > is already in use by a developer.
> > >
> > > The minimal interface could be a blocking 'reserve' verb. Would
> > > potentially long waiting times be acceptable for the test frameworks?
> >
> > I don't think so.  I'd rather have the 'reserve' verb fail immediately if
> > the board is already reserved, maybe returning
> > data on who has the reservation and some estimate of the reservation
> > duration.
>
> Generating such an estimate can be difficult, but could be done based
> on previous sessions from the same client.
>
> > And maybe also (optionally) queue a reservation for the caller.
> > I think the decision of whether to wait for a board to be available
> > or do something else should be up to the scheduler, and not the
> > reservation manager (part of the BM).
>
> Without queuing you can't really have priorities, at at that point the
> 'reservation manager' is already some sort of scheduler.
>
> > I would think that any reservation needs to have a time limit
> > associated with it.  Possibly we also need a mechanism to break a reservation,
> > if needed.
>
> In labgrid, you need to poll your reservation status regularly or it
> will expire. And you can cancel it explicitly, of course.
>
> > It might also be useful to support reservation priorities.
> > But I would want to start with something simple and grow from there.
>
> Without priorities, a developer may have to wait for multiple Jenkins
> background jobs to finish, before he can access the board
> interactively. That would annoy me pretty quickly. ;)

Alternatively (or additionally), a reservation could include an estimate
of the run time, to allow the scheduler to make informed decisions (e.g.
prioritize the 5-minute developer boot test over the weekly full-day
regression testsuite).

To avoid gaming the system, you want to enforce time limits based on
those estimates.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds