[yocto] Image introspection

[Mikko.Rapeli at bmw.de]

On Fri, Mar 29, 2019 at 09:50:20AM +0100, Dimitris Tassopoulos wrote:
> Hi all,
> 
> I was thinking that the mail list shouldn't be only for problems and
> questions and that from time to time, it would be nice to see some guides,
> tutorials or success stories from people that follow the list.
> 
> Anyway, a few days ago someone had an issue with one of the BSPs I'm
> maintaining and I wrote him a small guide on how to -try- to resolve future
> issues like that. Then I thought that I haven't found a small tutorial
> like this. Maybe it already exists, but nevertheless I haven't seen it.
> Of course, those things are in the documentation, but they are documented
> as individual tools (which is the correct thing to do), but it's not very
> clear how to use all those things together to perform more complex tasks.
> 
> So, in my case the issue was that ofono was installed in the image and that
> needed to be removed. Probably a lot of you already know the answer but for
> me that I've never bothered with that I had to track it down how it got in
> there.
> 
> Everything from now on assumes that you've setup up your bitbake environment
> of your build with whatever setup scripts you're using (e.g.
> *oe-init-build-env*)
> 
> There are several ways to do introspection on an image. For example,
> let's say that you found a file in the in rootfs and you want to know which
> recipe added that file. Then you can use this command:
> 
> oe-pkgdata-util find-path /usr/sbin/ofonod
> 
> 
> *oe-pkgdata-util* is a utility in *poky/scripts/oe-pkgdata-util* and
> `find-path` is pretty obvious what it does. This will return:
> 
> ofono: /usr/sbin/ofonod
> 
> 
> So now you know that it's indeed the *ofono* recipe that adds this file.
> Next,
> you need how this did get in the image (probably some other dependency
> because you didn't). Then you can use the `-g` parameter with bitbake like
> this:
> 
> bitbake -g allwinner-image
> 
> 
> This will create a file called `recipe-depends.dot`. This is a dot file that
> has all the dependencies in the image. You can use the same command to get
> the dependencies for a recipe, but now we care about the image.
> 
> Next step is to search in that file, why that key is there. Why is `-w` and
> key is `-k`. You can always remember this as "-w-hy that -k-ey is there?"
> and you
> run this command:
> 
> oe-depends-dot -k ofono -w recipe-depends.dot
> 
> 
> This will return the recipe that has this as dependency.
> 
> 
> > Because: allwinner-image packagegroup-base
> 
> 
> That means that the key is there because of allwinner-image (this is the
> image
> recipe, but it can be any other image) and because the *allwinner-image*
> inherits the
> *packagegroup-base*. So this packagegroup is the guilty.
> 
> Let's find this thing now. Get in the meta layer sources folder and run this
> 
> find . | grep packagegroup-base
> 
> 
> This will return
> 
> > ./poky/meta/recipes-core/packagegroups/packagegroup-base.bb
> 
> 
> Great. Open this file to an editor and find where is *ofono*. Gotcha, is in:
> 
> RDEPENDS_packagegroup-base-3g
> 
> 
> Then it's the *packagegroup-base-3g* that does that. Probably that's a
> recipe
> or package group file, so you can run:
> 
> find . | grep packagegroup-base-3g
> 
> 
> But you get nothing... Then probably this is declared somewhere a file with
> another name, so let's search inside the files in the poky layer:
> 
> grep -nriI ackagegroup-base-3g
> 
> 
> And we get:
> 
> poky/meta/recipes-core/packagegroups/packagegroup-base.bb:35:
> > ${@bb.utils.contains("DISTRO_FEATURES", "3g", "packagegroup-base-3g", "",
> > d)} \
> > poky/meta/recipes-core/packagegroups/packagegroup-base.bb:73:
> > ${@bb.utils.contains('COMBINED_FEATURES', '3g', 'packagegroup-base-3g',
> > '',d)} \
> > poky/meta/recipes-core/packagegroups/packagegroup-base.bb:122:
> > d.setVar("ADD_3G", "packagegroup-base-3g")
> > poky/meta/recipes-core/packagegroups/packagegroup-base.bb:316:SUMMARY_packagegroup-base-3g
> > = "Cellular data support"
> > poky/meta/recipes-core/packagegroups/packagegroup-base.bb:317:RDEPENDS_packagegroup-base-3g
> > = "\
> > poky/meta/recipes-core/packagegroups/packagegroup-base.bb:320:RRECOMMENDS_packagegroup-base-3g
> > = "\
> 
> 
> 
> So it's actually in the same file that we already opened. Here you can
> facepalm,
> but we didn't know that, so this would be the procedure anyways to track it
> down.
> Now, search for *packagegroup-base-3g* inside the
> *poky/meta/recipes-core/packagegroups/packagegroup-base.bb
> <http://packagegroup-base.bb>*
> and you see this line:
> 
> ${@bb.utils.contains("DISTRO_FEATURES", "3g", "packagegroup-base-3g", "",
> > d)} \
> 
> 
> Therefore the *3g* in the *DISTRO_FEATURES* actually added *ofono*. That
> means that,
> we need to remove the *3g* string from our *DISTRO_FEATURES*.
> 
> To do that, add this to your build/conf/local.conf file
> 
> DISTRO_FEATURES_remove = " 3g"
> 
> 
> Now just to be sure, run this to clean *ofono* from cache and everywhere
> else:
> 
> bitbake -c cleanall ofono
> 
> 
> And then rebuild the image:
> 
> bitbake image-name
> 
> 
> Now you'll see that *ofono* won't b e downloaded or get built and it won't
> be in your image.
> 
> I hope the above guide helps a bit.
> 
> I would be glad to get better suggestions or other people experience on
> how they introspect their images and solve related issues.

Thanks for this useful summary of tools.

I use buildhistory a lot. It contains recipe, binary package, image and SDK
metadata. Many of the queries and tools you mentioned can be 'git grep'ed
from buildhistory. I've added some more details to buildhistory like recipe
layer name and SRC_URI which help too. DISTRO_FEATURES and their impact
to packagegroups, image contents etc are currently missing, and they would
be nice to have.

https://www.yoctoproject.org/docs/2.7/dev-manual/dev-manual.html#maintaining-build-output-quality

Hope this helps,

-Mikko

> Regards,
> Dimitris

> -- 
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto