[yocto] [PATCH] [yocto-ab-helper] scripts/run-jinja-parser: Add Jinja2 parser extension in autobuilder

[Chan, Aaron Chun Yew]

Hi Richard,

Let me do a quick explanation on what I intended to do with this script.

1. I don't have much lava knowledge but at a quick glance this may be ok, apart from one of the file names which I've commented on below.

[Ans]: Basically this jinja template is mapped to lava.py module where we defines job configuration for several architectures (ARM, MIPS, PPC, x86).
            lava/device/bsp-packages.jinja2 maybe we can name it to "lava-yaml-template.jinja2" since this is YAML template for LAVA? I'm open for all suggestions.

2. I'm not sure we want to call this "lava.py". Would something like "lava-conf-intelqa-minnowboard.py" be a better description (as that is what it appears to be)?

[Ans]: So I've put the LAVA job config into lava.py, the concept is very similar to our yoctoabb (config.py)           
            If we change the name from lava.py to lava-conf-intelqa-minnowboard.py, in the code we need to change to "import lava-conf-intelqa-minnow" instead of "import lava".
            I am good with the rename but it will be just long.

3. Should configuration files be places somewhere outside scripts?

[Ans]: So, basically my script is able to load the module/config outside (in any file structure) and I have it tested like the examples below:

$ run-jinja-parser <Module Path> $2 $3 $4 $5

e.g. It works in this case:
~/yocto-autobuilder/lava.py
$ run-jinja-parser "~/yocto-autobuilder-helper" $2 $3 $4 $5

Or 
~/yocto-autobuilder/scripts/lava.py
$ run-jinja-parser "~/yocto-autobuilder-helper/scripts" $2 $3 $4 $5

Let me know your thoughs about these. Once these changes are added, the community will benefit from using Yocto autobuilder to trigger 
their automated BSP test (using LAVA) on their platform starting with benchmarking own manual BSP test case(s).

Cheers,
Aaron

-----Original Message-----
From: richard.purdie at linuxfoundation.org [mailto:richard.purdie at linuxfoundation.org] 
Sent: Friday, July 6, 2018 9:30 PM
To: Chan, Aaron Chun Yew <aaron.chun.yew.chan at intel.com>; yocto at yoctoproject.org
Subject: Re: [PATCH] [yocto-ab-helper] scripts/run-jinja-parser: Add Jinja2 parser extension in autobuilder

On Fri, 2018-07-06 at 17:15 +0800, Aaron Chan wrote:
> This patch is introduced as a feature in 2.6 M2 to support the 
> extension of autobuilder to LAVA (Linaro Automated Validation 
> Architecture).
> run-jinja2-parser loads lava config module and generates LAVA job 
> config in a YAML format before its triggers LAVA server to execute a 
> task.

I don't have much lava knowledge but at a quick glance this may be ok, apart from one of the file names which I've commented on below.

> Signed-off-by: Aaron Chan <aaron.chun.yew.chan at intel.com>
> ---
>  lava/device/bsp-packages.jinja2 | 43 ++++++++++++++++++
>  scripts/lava.py                 | 76
> ++++++++++++++++++++++++++++++++
>  scripts/run-jinja-parser        | 97
> +++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 216 insertions(+)
>  create mode 100644 lava/device/bsp-packages.jinja2  create mode 
> 100644 scripts/lava.py  create mode 100755 scripts/run-jinja-parser
> 
> diff --git a/lava/device/bsp-packages.jinja2 
> b/lava/device/bsp-packages.jinja2 new file mode 100644 index 
> 0000000..61fbcad
> --- /dev/null
> +++ b/lava/device/bsp-packages.jinja2
> @@ -0,0 +1,43 @@
> +device_type: {{ device_type }}
> +job_name: {{ job_name }}
> +timeouts: 
> +  job:
> +    minutes: {{ timeout.job.minutes }}
> +  action:
> +    minutes: {{ timeout.action.minutes }}
> +  connection:
> +    minutes: {{ timeout.connection.minutes }}
> +priority: {{ priority }}
> +visibility: {{ visibility }}
> +actions:
> +- deploy:
> +    timeout:
> +      minutes: {{ deploy.timeout }}
> +    to: {{ deploy.to }}
> +    kernel:
> +      url: {{ deploy.kernel.url }}
> +      type: {{ deploy.kernel.type }}
> +    modules:
> +      url: {{ deploy.modules.url }}
> +      compression: {{ deploy.modules.compression }}
> +    nfsrootfs:
> +      url: {{ deploy.nfsrootfs.url }}
> +      compression: {{ deploy.nfsrootfs.compression }}
> +    os: {{ deploy.os }}
> +- boot:
> +    timeout:
> +      minutes: {{ boot.timeout }}
> +    method: {{ boot.method }}
> +    commands: {{ boot.commands }}
> +    auto_login: { login_prompt: {{ boot.auto_login.login_prompt }},
> username: {{ boot.auto_login.username }} }
> +    prompts:
> +      - {{ boot.prompts }}
> +- test:
> +    timeout:
> +      minutes: {{ test.timeout }}
> +    name: {{ test.name }}
> +    definitions:
> +    - repository: {{ test.definitions.repository }}
> +      from: {{ test.definitions.from }}
> +      path: {{ test.definitions.path }}
> +      name: {{ test.definitions.name }}
> diff --git a/scripts/lava.py b/scripts/lava.py new file mode 100644 
> index 0000000..be18529
> --- /dev/null
> +++ b/scripts/lava.py

I'm not sure we want to call this "lava.py". Would something like
"lava-conf-intelqa-minnowboard.py" be a better description (as that is
what it appears to be)?

Should configuration files be places somewhere outside scripts?

Cheers,

Richard