[meta-intel] [PATCH 6/6] rmc: document and examples for rmc distro feature

[Tom Zanussi]

Hi Jianxun,

Overall this looks like a good start, some specific comments inline.

There also seems to be some documentation in the recipes themselves that
would be good to have in this document.  The recipes refer to this file
for 'more information' and there is more information here, but there's
also useful background information and descriptions of how things are
meant to work together in the recipes that isn't here.

For the most part, the documentation in the recipes that doesn't
specifically talk about the recipe details should be a subset of what's
here, I think.

On 07/12/2016 12:59 PM, Jianxun Zhang wrote:
> Provide a README.rmc.distro for rmc-distro.
> 
> Also check in fingerprints and configuration data for several
> boards as examples for users. They can be used for validation too.
> 
> Signed-off-by: Jianxun Zhang <jianxun.zhang at linux.intel.com>
> ---
>  README.rmc.distro                                  | 261 +++++++++++++++++++++
>  .../rmc/boards/T100-32bit/BOOTENTRY.CONFIG         |   2 +
>  .../rmc/boards/T100-32bit/T100-32bit.fp            | Bin 0 -> 116 bytes
>  common/recipes-bsp/rmc/boards/T100-32bit/boot.conf |   4 +
>  .../recipes-bsp/rmc/boards/T100-32bit/install.conf |   4 +
>  .../rmc/boards/minnowmax/BOOTENTRY.CONFIG          |   1 +
>  common/recipes-bsp/rmc/boards/minnowmax/boot.conf  |   4 +
>  .../recipes-bsp/rmc/boards/minnowmax/minnowmax.fp  | Bin 0 -> 143 bytes
>  .../rmc/boards/minnowmaxB3/BOOTENTRY.CONFIG        |   1 +
>  .../recipes-bsp/rmc/boards/minnowmaxB3/boot.conf   |   4 +
>  .../rmc/boards/minnowmaxB3/minnowmaxB3.fp          | Bin 0 -> 148 bytes
>  .../rmc/boards/nucgen6/BOOTENTRY.CONFIG            |   2 +
>  .../rmc/boards/nucgen6/INSTALLER.CONFIG            |   6 +
>  common/recipes-bsp/rmc/boards/nucgen6/KBOOTPARAM   |   1 +
>  common/recipes-bsp/rmc/boards/nucgen6/boot.conf    |   4 +
>  common/recipes-bsp/rmc/boards/nucgen6/install.conf |   4 +
>  common/recipes-bsp/rmc/boards/nucgen6/mylib.conf   |   7 +
>  common/recipes-bsp/rmc/boards/nucgen6/nuc6.fp      | Bin 0 -> 149 bytes
>  common/recipes-bsp/rmc/rmc-distro.bb               |   2 +-
>  19 files changed, 306 insertions(+), 1 deletion(-)
>  create mode 100644 README.rmc.distro
>  create mode 100644 common/recipes-bsp/rmc/boards/T100-32bit/BOOTENTRY.CONFIG
>  create mode 100644 common/recipes-bsp/rmc/boards/T100-32bit/T100-32bit.fp
>  create mode 100644 common/recipes-bsp/rmc/boards/T100-32bit/boot.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/T100-32bit/install.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmax/BOOTENTRY.CONFIG
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmax/boot.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmax/minnowmax.fp
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmaxB3/BOOTENTRY.CONFIG
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmaxB3/boot.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/minnowmaxB3/minnowmaxB3.fp
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/BOOTENTRY.CONFIG
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/INSTALLER.CONFIG
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/KBOOTPARAM
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/boot.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/install.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/mylib.conf
>  create mode 100644 common/recipes-bsp/rmc/boards/nucgen6/nuc6.fp
> 
> diff --git a/README.rmc.distro b/README.rmc.distro
> new file mode 100644
> index 0000000..f32d856
> --- /dev/null
> +++ b/README.rmc.distro
> @@ -0,0 +1,261 @@
> +RMC Disro
> +--------------------------------------------------------------------------------
> +Table of Contents
> +
> +Introduction
> +Usage
> +Enable
> +Examples
> +Troubleshooting
> +
> +
> +
> +Introduction:
> +--------------------------------------------------------------------------------
> +RMC Distro feature fills functionality gaps between a build for a generic
> +machine like intel-corei7-64 and boards which require board-specific quirks and
> +configurations. These special customizations cannot be covered by conventional
> +auto-detection features based on probing a hardware module because they happen
> +at a board or a product level. For example:
> + - tty console to be specified for a type of board in kernel cmdline
> + - default audio configuration
> + - network configuration
> + - UI layout
> + - requirement to software driven by a mechanical design
> + - or static configuration bits for a physical bus that doesn't support to
> +   identify devices or their presence at runtime
> +
> +An image with the feature has ability to configure supported boards with data
> +associated only to a type of board to get full functionality of the target at
> +runtime, yet still with a single image.
> +
> +The effect is identical as what a conventional image customized for a type of
> +board (depending on the way to deploy image).
> +
> +This feature is based on RMC project and two modified software components (RMC
> +clients):
> +
> +systemd-boot EFI bootloader:
> +We modified systemd-boot so that it queries board-specific boot entries and show
> +them in boot menu.
> +
> +It also queries a global kernel command line fragment from RMC. If there is such
> +fragment in RMC database for the running board, bootloader appends it to the end
> +of kernel commandline to all of boot entries, regardless if an entry is got from
> +ESP or RMC database.

What is ESP?

> +
> +OE EFI installer:
> +The existing installer is modified to call RMC tool in installation stage. It
> +queries board-specific files from RMC database and deploy data to file systems.
> +
> +The below depicts the usage and configuration files. You can also find several
> +examples in common/recipes-bsp/rmc/boards/. The "Examples" section in this file
> +depicts expected results when running rmc-distro image on these boards.
> +
> +
> +
> +Usage
> +--------------------------------------------------------------------------------
> +Developers are suggested to organize all board-specific files in their own layer
> +following this example, so that RMC recipes can pick up them correctly in build.
> +
> +- my_top_dir/                    Top directory of your board (Note 0)
> +  |- rmc-distro.bbappend         bbappend file to rmc recipe at a lower level
> +  |- rmc/
> +      |- target_board_1/         subdirectory of a board.
> +      |   |- board1.fp           fingerprint file must be provided (NOTE 1)
> +      |   |- BOOTENTRY.CONFIG    optional config file for boot entries. (NOTE 2)
> +      |   |- INSTALLER.CONFIG    optional config file for installer. (NOTE 3)
> +      |   |- board_file_1        A file blob specific to the type of board
> +      |   |- board_file_2        An another file specific to the type of board
> +      |   |- ...more files
> +      |- target_board_2/         subdirectory of another board.
> +          |- board_2_v2.fp       fingerprint file for board 2.
> +          |- BOOTENTRY.CONFIG
> +          |- INSTALLER.CONFIG
> +          |- board_file_1
> +          |- board_file_2
> +          |- ...more files
> +
> +Note 0:
> +To add your boards into rmc distro image, you simply put this line in
> +your rmc-distro.bbappend:
> +
> +RMC_DISTRO_BOARD_DIRS_append := " ${THISDIR}/my_top_dir"
> +
> +RMC distro will take all top directories specified in RMC_DISTRO_BOARD_DIRS when
> +constructing a central RMC database in distro image. The bbclass of the bare RMC
> +project also provide function for other components to construct their own RMC
> +database file. Please refer to rmc-native.bbclass for more information.
> +
> +Subdirectory is not supported in a board's directory.
> +
> +Note 1:
> +Fingerprint files must be provided and with ".fp" at the end of their names.
> +Fingerprint can be obtained by running RMC tool on your board. An easy way is to
> +live-boot USB stick flashed with RMC distro image on your board, then run this
> +command:
> +
> +# rmc -F -o my_board.fp
> +
> +Or you will need to build rmc tool for the architecture of your board, 32 or
> +64 bit x86, from rmc project.
> +
> +You can run rmc tool without any argument to get usage and examples. 
> +
> +DO NOT NAME ANY FILE ENDING WITH '.fp' IF IT IS NOT A RMC FINGERPRINT FILE.
> +
> +If you do need a .fp file deployed onto target, please rename it in source and
> +specify the real name of file on target in INSTALLER.CONFIG.
> +
> +Note 2:
> +At runtime, RMC bootloader tries to fetch this file specific to the board at run
> +time, then tries to fetch each boot entry file specified in BOOTENTRY.CONFIG and
> +show them in boot menu options. The format of this file is very simple. Each
> +line is the name of a boot entry file:
> +
> +boot.entry
> +Install.entry
> +myrmcboot.conf
> +
> +Name of a boot entry file is defined by developer so it can be anything. But the
> +name of config file is what RMC bootloader looks up in RMC database, so it must
> +be named “BOOTENTRY.CONFIG” as a naming convention. If this config file is not
> +provided, only default entries “boot” and “install” from OE are in boot menu.
> +
> +Note 3:
> +At runtime, RMC installer tries to fetch INSTALLER.CONFIG file specific to the
> +board, then tries to fetch each file specified in this config file, and then
> +deploy the file onto target with its permissions, UID, GID and other attributes
> +also specified in this config file if file for the board can be retrieved from
> +RMC database. The format of this file is (# is for comment line)
> +
> +# name:uid:gid:mode:path_on_target
> +# to create a directory, add a “/” at the end of path_on_target:
> +audio_policy:0:0:600:/etc/audio/
> +audio_def_policy:0:0:600:/etc/audio/audio_policy
> +
> +The first line tells RMC installer to create a directory “audio” in /etc. If any
> +parent directory doesn’t exist, installer will create it. The above example
> +creates /etc/audio directory first, then fetch a file named “audio_def_policy”
> +from RMC database for the board, then copy it to /etc/audio/ with a new name
> +“audio_policy”.
> +
> +If this config file is not provided, only default entries “boot” and “install”
> +from OE are in boot menu. The name of this config file is what installer looks
> +up first, so it must be “INSTALLER.CONFIG”.
> +
> +
> +
> +Enable
> +--------------------------------------------------------------------------------
> +To Enable RMC distro feature in build, add the below line in a conf file:
> +EFI_PROVIDER="rmc-distro"
> +

Is rmc-distro actually meant to be an EFI provider?  I know you only
support systemd-boot for now, but assuming there was also support in
grub-efi for rmc, how would the user specify that?

> +
> +
> +Examples
> +--------------------------------------------------------------------------------
> +We checked in configuration data in common/recipes-bsp/rmc/boards/ for several
> +boards, to help users to understand the rmc distro feature. These examples are
> +also used for validation purposes. For any example you find not working as this
> +section depicts, it should be treated as a bug to be fixed.
> +
> +A single rmc distro image shall be used in validation with examples of same
> +architecture, 32 or 64 bit X86. It is invalid to prove rmc distro feature with
> +different images even if they are built for the same architecture.

Please clarify this - I can't understand what you're trying to say here.

> +
> +MinnowBoard MAX and B3 version:
> +common/recipes-bsp/rmc/boards/minnowmax
> +common/recipes-bsp/rmc/boards/minnowmaxB3
> +
> +We have found two identities (type of board) exist for the "same" Minnow Max
> +hardware, so they have to be treated as two different types of hardware. The two
> +examples show you a boot entry specific to a type of board. Titles shown in boot
> +menu have different names according to the type of running board, "Minnow Max
> +boot" or "Minnow Max B3 boot". in /proc/cmdline, "console=ttyS0,115200n8" shall
> +be there. Kernel prints logs from 6-pin FTDI serial port on Minnow Max(s). This
> +console setting is in board-specific entries, so you won't see it effective if
> +you select default "boot" entry to boot the device.
> +
> +NUC Gen 6:
> +common/recipes-bsp/rmc/boards/nucgen6
> +This is a combo example with all supported configuration data for NUC Gen 6
> +product. It shows two boot entries in bootloader menu when you boot image on NUC
> +Gen 6 product, with "NUC Gen6" in entry titles. There shall no any "console=" in
> +/proc/cmdline when you boot with either of two "NUC Gen6"entries. We designed it
> +this way because there is no accessible tty port on NUC Gen 6 with housing.
> +
> +This example also includes a global kernel cmdline fragment KBOOTPARAM. Content
> +of KBOOTPARAM shall be at the end of /proc/cmdline no matter which boot entry
> +you selected to boot NUC Gen6.
> +
> +INSTALLER.CONFIG directs installer to create a directory and deploy a file in it
> +when install the image on NUC Gen6.
> +
> +Choose "NUC Gen6 install" boot entry to boot shall start installation. Once
> +the device reboots after installation, we can verify the configurations.
> +
> +The boot entry "NUC Gen6 boot" shall be shown in boot menu.
> +
> +The content of KBOOTPARAM shall be in /proc/cmdline too.
> +
> +A directory /etc/mylib/ is created and a file "mylib.conf" is there. The content
> +of that file shall be what we put in mylib.conf in
> +common/recipes-bsp/rmc/boards/nucgen6
> +
> +T100 (32bit):
> +common/recipes-bsp/rmc/boards/T100-32bit
> +This example is provided for validation on 32 bit X86 architecture. It doesn't
> +provide any new function not mentioned in above examples.
> +
> +
> +
> +Troubleshooting
> +--------------------------------------------------------------------------------
> +Issue: Cannot obtain RMC fingerprint for a board
> +
> +RMC tool requires UEFI BIOS and SMBIOS support in firmware. It doesn't support
> +other type of firmware, e.g. legacy BIOS. It also requires EFI driver enabled
> +in Linux kernel.
> +
> +Issue: Configuration for a board seems not effective at runtime.
> +
> +Check if board is booted from the storage where RMC image or installation
> +lives when you have multiple boot options in BIOS.
> +
> +Make sure configuration files (BOOTENTRY.CONFIG, INSTALLER.CONFIG and,
> +KBOOTPARAM ...) are properly named in the board directory.
> +
> +Make sure configuration files have correct contents.
> +
> +Some file attributes could not be supported by targeted file system. Installer
> +cannot setup file blobs as you wish. It simply move to the next step if a step
> +fails.
> +
> +Kernel command line can be customized globally with KBOOTPARAM or just in a boot
> +entry for the type of board. They have different effective scopes.
> +
> +If no any board-specific configuration becomes effective on your board but it
> +works on other boards of same product, you can run rmc tool to obtain
> +fingerprint file on your board and compare it with fingerprint of a working
> +board. It is possible they have different firmware versions and unluckily, some
> +information for fingerprint changes between two versions. You can update BIOS
> +on every board to the same BIOS version if it is feasible. Otherwise you have
> +to treat them as two different type of boards. We could extend rmc design to
> +allow multiple fingerprints in a board directory as a workaround.
> +

Would it be possible to do some kind of fingerprint wildcarding in
addition/instead?

> +Issue: RMC reports error because it cannot find fingerprint when building image.
> +
> +Make sure you have a fingerprint file. Its name must be ended with '.fp'. You
> +can put a fingerprint file in a board directory and provide data later.
> +
> +Issue: Any problems the above troubleshooting cannot help
> +
> +Please report it to us. Extra information like the type of your board or a dump
> +file from dmidecode tool is helpful. We will investigate the problem and keep
> +improving this feature.
> +
> +Thanks
> +
> +Jianxun Zhang <jianxun.zhang at linux.intel.com>
> diff --git a/common/recipes-bsp/rmc/boards/T100-32bit/BOOTENTRY.CONFIG b/common/recipes-bsp/rmc/boards/T100-32bit/BOOTENTRY.CONFIG
> new file mode 100644
> index 0000000..b2fabe8
> --- /dev/null
> +++ b/common/recipes-bsp/rmc/boards/T100-32bit/BOOTENTRY.CONFIG
> @@ -0,0 +1,2 @@
> +boot.conf
> +install.conf
> diff --git a/common/recipes-bsp/rmc/boards/T100-32bit/T100-32bit.fp b/common/recipes-bsp/rmc/boards/T100-32bit/T100-32bit.fp
> new file mode 100644
> index 0000000000000000000000000000000000000000..86ecea7344bfc744f7f9d57f3f1810d6f7ba0db1
> GIT binary patch
> literal 116
> zcmZQ%Ehx%QDNQbk&r8frWe71eFbHvEV8SZOB2boERGgWg$KaV)lA5Ctq^aOolAo&)
> p;;X6P91yAyWo&L at px~fjsAp{K?oq{1&rp<FoLW?tn!<p>1_02dA-n(p
> 

It would be nice if you could provide an example layer as you mentioned
in your documentation, which would allow a user to just include that
layer and throw the build result on one of these common machines to see
if it works or not.  These are already meant as examples, so having an
always-working example layer around them would seem to kill three birds
with one stone (examples and testing and user ramp-up).

Tom