[yocto] gitlab-ci helper scripts for OpenEmbedded builds

[Thomas Goodwin]

Hi Yann,

Thanks for sharing! We're working through something similar using a tweak
to the CROPS docker containers and GitLab-CI (we started with autobuilder
2, so we've actually merged quite a bit of that experience with our GitLab
setup).

Using docker runners as our cluster, we setup volume mounts to the host
system for caching the shared state and downloads directories (as well as
other artifacts upon successful build).  As part of the CI setup, we then
insert a build/conf/auto.conf file that enforces the usage of that volume
(DL_DIR and SSTATE_DIR).  At the deploy stage (package feeds, etc. like
you're doing), we did the same thing -- copy out to that path so that it
ends up on the network share rather than the container.

Also like you, we're using the "include" and "extends" instructions
throughout so that we have a top-level "common" set of CI YAML includes
alongside a growing set of repositories for each machine-specific build.
Each of those repositories follows the Yocto release branching style so
that we can trace build success to newer releases by simply creating a
branch and using the branch name for the initial clone of the layers.

One thing that came directly from our autobuilder 2 experience is a way to
inject extra variables into the autoconf.  AB2 called this *EXTRAARGS*,
which was simply a list of variables to add.  For our common CI "setup"
stage, we check for a file of the same name (extraargs.conf) and append it
to the auto.conf if it exists.  In this way our downstream projects can
simply include that file if necessary and version control it as new release
branches are added.

One of the drawbacks we've seen in going this route is that every
repository has its own build timeout limit, which is always laughably
small.  There's a backlog issue on GitLab for a global definition of this
value, but it's slated for 12.7 (I think).

I'm hoping to put a write-up out soon with examples for how this all worked
together, but I want to hold off until we can get the CROPS changes we
needed upstreamed in a way that doesn't break those containers for everyone
else (something about the entrypoint doesn't appreciate the runner's
bash/shell detection script).

Cheers,

Thomas

Geon Technologies, LLC


On Mon, Sep 23, 2019 at 5:50 PM Yann Dirson <yann.dirson at blade-group.com>
wrote:

> Hi all,
>
> We released our scripts to help in setting up continuous integration
> of a yocto-based project
> using Gitlab-CI.  You'll find the repository at
> https://github.com/BladeGroup/gitlab-oe
>
> Although we're using this in production, it still has a couple of
> rough edges and may need
> tuning for different usages.  We'd love to hear how well it fits (or
> doesn't fit ;) with other use
> cases, and will gladly consider evolutions to make it more generally
> usable.
>
> Best regards,
> --
> Yann Dirson <yann at blade-group.com>
> Blade / Shadow -- http://shadow.tech
> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190924/6d47954c/attachment.html>