[yocto] git fetcher - AUTOREV and best practices

[Joshua Watt]

On Wed, Jun 5, 2019 at 8:00 AM Maciej Pijanowski
<maciej.pijanowski at 3mdeb.com> wrote:
>
> Hello,
>
> As explained in the mega manual [1], when using the git:// fetcher,
> setting the
> SRCREV to ${AUTOREV} will result in building the latest commit from
> given git
> branch (master, if not specified otherwise).
>
> Using AUTOREV feature in recipe has following implications as far as I
> can see:
>
> - the same recipe might get built using different git commit, depending
> on when
>   the build was run, which breaks the reproducibility,
> - it imposes some potential security risk - by specifying the exact
> commit in
>   the recipe, we can at least say that this revision of this package is fine
>   and we want to build it; with AUTOREV we might not be aware of the
> code we're
>   fetching
>
> I'm wondering whether there are any best practices or strict rules
> written down
> for recipes getting upstream to follow in this area. When inspecting some of
> the layers from the git.yoctoprojects.org, it appears that the AUTOREV
> feature
> is almost not used, besides a few exceptions.
>
> I'm wondering whether it would make sense to raise a warning when git
> fetcher
> with AUTOREV is used, so it would be easier to build on top OE / Yocto with
> reproducibility / security in mind.
>
> I understand that this feature is mostly meant for development purposes. I'm
> just looking for a tools how one could easily make sure that each
> fetched source code
> is verified prior compilation.

Yes, I think warning about using AUTOREV makes a lot of sense for all
the reasons you have raised. I'm not sure that putting that in the git
fetcher is necessarily the best place for that. OE has an extensive
set of QA checks that run when recipes are built, and I think that
this might be a better/easier place to put this check. IMHO, recipes
that force you to use AUTOREV in the base recipe are broken and should
be fixed, in which case a QA check should be sufficient. The harder
part is making sure that it doesn't trigger when someone is using
AUTOREV legitimately for development purposes. OE already has a class
that encapsulates the logic for reproducible builds
(reproducible_build.bbclass); one approach might be to add the QA
check, and then add it to the list of QA checks that fail the build
(ERROR_QA from insane.bbclass) in reproducible_build.bbclass.

As a bonus, the 3.0 zeus release adds a QA check for reproducible
builds, so such a QA check would get detected on the autobuilder if
someone changed a oe-core recipe to use AUTOREV by default. I suspect
that doesn't help your immeidate use case, since I *really* hope none
of the oe-core recipes actually do this, but the QA test can be
extended in your layers to test your own images if you want.

>
> I've already looked at the https:// fetcher (which is mostly used for
> fetching tarballs).
> It requires the recipe to contain valid md5 and sha256 sums. Even if we
> suppress the
> error in case checksum mismatch in the recipe by setting the
> BB_STRICT_CHECKSUM
> to 0, we are still getting the warning, which is the desired behavior I
> believe.
>
>
> [1]:
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-AUTOREV
> [2]:
> https://www.yoctoproject.org/docs/2.0.1/bitbake-user-manual/bitbake-user-manual.html#var-BB_STRICT_CHECKSUM
>
> --
> Maciej Pijanowski
> Embedded Systems Engineer
> https://3mdeb.com | @3mdeb_com
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto