[Automated-testing] Farming together - areas of collobration

[Andrew MURRAY]

Thanks for joining  in : )

> -----Original Message-----
> From: Kieran Bingham [mailto:kbingham at kernel.org]
> On 11/11/17 00:23, Andrew Murray wrote:
> > My vision of a farm is that it would consist of three distinct layers...
> >
> > The Grass - This is the bottom layer which provides the bare minimum
> > software abstraction to the fundamental capabilities of the farm. This
> > consists of physical devices and suitable drivers for them. The
> > interface to higher layers is of the language of turn power port 2 on,
> > bridge relay 5 on, etc. The goal here is for us to be able pick up
> > some hardware off the shelf (e.g. a APC power switch) and to have a
> > 'driver' already available from this community that fits into this
> > view of the world. In order to achieve this it would be necessary to
> > define categories of devices (power, relay, serial), the verbs (power
> > cycle, power off, etc) and a way of addressing physical devices. I
> > believe the LabGrid stack has a good model in this respect.
> 
> I had started looking at this part - and then having come across labgrid - I
> believe I will start adapting this rather than rewrite a competing system.
> 
> My requirement here is abstracting the physical hardware out to a description
> (like YAML) that is easy to update when the hardware is moved. And I think
> labgrid covers this.

I'm encouraged that everyone seems happy to try LabGrid.

> 
> 
> 
> > The Cow - This is the middle layer which uses the bottom layer and
> > provides an abstraction of a board, this is where boards are defined
> > and related to the physical devices. This layer would manage exclusive
> > access to boards and self-tests. Users of this layer could find out
> > what boards are available (access control), what their capabilities
> > are and access to those capabilities. The language used here may be
> > turn board on, get serial port, press BOOT switch, etc. The goal here
> > is that we can work as a community to create definition of boards - a
> > bit like device tree at a high physical level.
> 
> 
> I think this layer is quite intrinsically linked with the lower layer, so it's hard for
> me to visualise them as separate.

I guess it converts verbs to actions, and seemingly provides access control, scheduling, etc.

> 
> I guess in reality - it is this layer that I had already started looking at, and I had
> a vision to implement an access system over SSH similar to gitlab.

Yes we adopted this approach for authentication - works really well : )

> 
> I started prototyping (though don't expect much here yet):
> 	https://github.com/kbingham/lab.git
> 
> 
> My design goals here are that an individual should be able to access targets
> with name based resolution such as:
> 
>   lab beaglebone serial # Load up a shared serial console over tmux/screen
>   lab beaglebone on
>   lab beaglebone reboot
>   lab beaglebone upload Image board.dtb   # Upload boot files using rsync/scp
> 
> 
> By using the authorized_keys file of SSH to redirect all connections through a
> parsing script we can have some fun in manipulating what the request actually
> does. It also provides authentication, and security. ( I hope SSH is still
> considered secure )

Yes this is our exact approach - we call this script 'operator' it marshalls all requests from users.

> 
> Anyway, I envisage this being backed by some database storage or such which
> will maintain connection states, and permissions, and maybe a web-front end
> too.

Ours is all file based : |

> > In my view it's the bottom two layers that need the most work. I have
> > a particular vision for how part of the 'Grass' layer should work..
> 
> I think perhaps we should meet up for a beer to discuss this sometime!
> (farmers-con anyone?)

At least you and I are local to each other.

> 
> 
> > I find that when managing a farm, it would be nice to add new boards
> > without having to change some integral farm configuration or have
> > particular access rights to 'the farm' to plumb in a new board. I.e.
> > so that anyone can add a farm without risk of breaking anything else.
> > In our farm we install boards on rack-mount shelves - I think it would
> > be a fantastic idea if we could define a standard that:
> >
> > - Standardises connectivity between a shelf and the farm PC (perhaps
> > just USB, Ethernet and power)
> > - Requires each shelf to contain a USB stick with a file that
> > describes the board, the physical USB topology between the shelf and
> > its components (USB serial adaptors etc)
> >
> 
> Intriguing - this is an extra abstraction layer that I had not thought of (the
> 'shelf')

The motivation is to avoid spaghetti junction. 

> 
> > This immediately opens up some benefits:
> >
> > - You can now make a new piece of hardware available to farm users by
> > creating a shelf and plugging it into the farm - no need to go and
> > configure the farm - this makes it really easy.
> > - Likewise you can plug your shelf into any shelf of the farm - i.e.
> > makes maintenance really easy
> > - You could plug your shelf into your PC and farm software on your PC
> > would detect the hardware and give you access.
> > - You could plug your shelf into anyone elses farm.
> 
> Ok - well this is just 'moving' the configuration onto a USB stick... which will still
> need updating as the shelf evolves...

Yes, as Jan points out this would be tedious.

> 
> 
> > As a slightly more concrete example, imagine you have a beagle bone
> > black, there would be a community definition file available that
> > describes that it is a board named 'Beagle Bone Black' which describes
> > that it has a power jack, a power button, a reset button, serial, USB
> > and an Ethernet cable. Perhaps the definition file has additional
> > information for example that to power on the board you have to apply
> > power and then press the power button. You could then create a shelf
> > with the power connected, a USB relay for the power button and serial.
> > You could then create a mapping file on a USB stick that describes:
> 
> Isn't the information that a board has a power connection implicit?

Maybe for the simple case. But there could be more complex cases, e.g. a board with multiple serial.

> For me the information on *how* to power that board is more important:
> 
> # power.yaml
> power:
>   acme1:
>     driver: baylibre-acme	# provides 'on/off/reboot/status'
>     ports: 8
> 
>   energenie:
>     driver: sispmctl
>     device: 0
>     host: raspberrypi.local 	# Objects on a different 'server'
> 
> # Boards.yaml (or support a file per 'shelf'?)
> boards:
>   beaglebone:
>     serial: /dev/serial/by-path/platform-3f980000.usb-usb-0:1.4:1.0-port0
>     power:
>      - acme1 port-3      # Not sure how to 'cross reference' in yaml yet ...
>      - energenie port-4  # I have a scenario that needs two switches enabled
> 
> 
> I assume (I need to check still) that the above is something like what labgrid is
> providing.

Similar (see my response to Jan) - I think it's this we need to all work and align on.

> 
> 
> > - That this relates to a community definition file called BBB.
> > - BBB Power is connected to the shelf power supply
> > - BBB power button is connected to a USB relay of type XXX on the
> > shelf with a specific relative physical path (from the end of the
> > shelf) using port 1
> > - BBB serial is connected to a USB serial on the shelf with a specific
> > relative physical path (from the end of the shelf)
> > - The remainder of the other connections are not made.
> >
> > When the farm detects the USB it can present a BBB to the user with
> > information about what capabilities are available. The user can then
> > interact using BBB terminology - e.g. press BBB power button, or more
> > generally - e.g. turn board on.
> >
> > Of course I've made many assumptions here about shelves, the hardware
> > within it etc - but this is just an idea for discussion.
> >
> > Would this split of layers be beneficial?
> >
> > Would such a standard be helpful?
> >
> > What areas of farming need to be addressed?
> 
> Boot methods? Abstracting data storage?
>  NFS Root, TFTP server for Kernel Images  Boards that boot from SD card (I see a
> lot of people are looking at SDMux'es)  USB DFU / Android / 96boards style
> boot?

I don't know if we have anything for this to sit on yet though.

> 
> 
> > Is this the right place to start?
> 
> Of course - we need to start somewhere!
> 
> 
> > Keen to all feedback. (Apologies for the rambling and terrible farm
> > analogies :D)
> 
> 
> Well - I think cows are a bit big. I was thinking of farming things like chickens,
> but it seems the reality is that it's more like rabbits based on the speed at
> which the boards seem to breed and multiply when you start a farm. ;-)

😊

Andrew Murray

> 
> 
> > [1] - BoF Slides -
> > https://www.elinux.org/images/5/52/BoF-FarmingTogether.pdf
> > [2] - Farming Together Wiki - https://www.elinux.org/Board_Farm
> >
> > Thanks,
> >
> > Andrew Murray
> >
> 
> Cheers
> 
> --
> Kieran