[yocto] git fetcher - AUTOREV and best practices

[Joshua Watt]

On Wed, Oct 2, 2019 at 10:41 AM Joshua Watt <jpewhacker at gmail.com> wrote:
>
> On Wed, Jun 5, 2019 at 8:00 AM Maciej Pijanowski
> <maciej.pijanowski at 3mdeb.com> wrote:
> >
> > Hello,
> >
> > As explained in the mega manual [1], when using the git:// fetcher,
> > setting the
> > SRCREV to ${AUTOREV} will result in building the latest commit from
> > given git
> > branch (master, if not specified otherwise).
> >
> > Using AUTOREV feature in recipe has following implications as far as I
> > can see:
> >
> > - the same recipe might get built using different git commit, depending
> > on when
> >   the build was run, which breaks the reproducibility,
> > - it imposes some potential security risk - by specifying the exact
> > commit in
> >   the recipe, we can at least say that this revision of this package is fine
> >   and we want to build it; with AUTOREV we might not be aware of the
> > code we're
> >   fetching
> >
> > I'm wondering whether there are any best practices or strict rules
> > written down
> > for recipes getting upstream to follow in this area. When inspecting some of
> > the layers from the git.yoctoprojects.org, it appears that the AUTOREV
> > feature
> > is almost not used, besides a few exceptions.
> >
> > I'm wondering whether it would make sense to raise a warning when git
> > fetcher
> > with AUTOREV is used, so it would be easier to build on top OE / Yocto with
> > reproducibility / security in mind.
> >
> > I understand that this feature is mostly meant for development purposes. I'm
> > just looking for a tools how one could easily make sure that each
> > fetched source code
> > is verified prior compilation.
>
> Yes, I think warning about using AUTOREV makes a lot of sense for all
> the reasons you have raised. I'm not sure that putting that in the git
> fetcher is necessarily the best place for that. OE has an extensive
> set of QA checks that run when recipes are built, and I think that
> this might be a better/easier place to put this check. IMHO, recipes
> that force you to use AUTOREV in the base recipe are broken and should
> be fixed, in which case a QA check should be sufficient. The harder
> part is making sure that it doesn't trigger when someone is using
> AUTOREV legitimately for development purposes. OE already has a class
> that encapsulates the logic for reproducible builds
> (reproducible_build.bbclass); one approach might be to add the QA
> check, and then add it to the list of QA checks that fail the build
> (ERROR_QA from insane.bbclass) in reproducible_build.bbclass.
>
> As a bonus, the 3.0 zeus release adds a QA check for reproducible
> builds, so such a QA check would get detected on the autobuilder if
> someone changed a oe-core recipe to use AUTOREV by default. I suspect
> that doesn't help your immeidate use case, since I *really* hope none
> of the oe-core recipes actually do this, but the QA test can be
> extended in your layers to test your own images if you want.

FYI: I raised a bugzilla to track this:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13567

>
> >
> > I've already looked at the https:// fetcher (which is mostly used for
> > fetching tarballs).
> > It requires the recipe to contain valid md5 and sha256 sums. Even if we
> > suppress the
> > error in case checksum mismatch in the recipe by setting the
> > BB_STRICT_CHECKSUM
> > to 0, we are still getting the warning, which is the desired behavior I
> > believe.
> >
> >
> > [1]:
> > https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-AUTOREV
> > [2]:
> > https://www.yoctoproject.org/docs/2.0.1/bitbake-user-manual/bitbake-user-manual.html#var-BB_STRICT_CHECKSUM
> >
> > --
> > Maciej Pijanowski
> > Embedded Systems Engineer
> > https://3mdeb.com | @3mdeb_com
> >
> >
> > --
> > _______________________________________________
> > yocto mailing list
> > yocto at yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto