[yocto] Building Out-of-Tree Modules on the BBB Target

Bruce Ashfield bruce.ashfield at gmail.com
Fri May 24 05:48:35 PDT 2019 

On Thu, May 23, 2019 at 9:00 PM Khem Raj <raj.khem at gmail.com> wrote:

>
>
> On 5/23/19 3:32 AM, Zoran Stojsavljevic wrote:
> > After some tests (and I had other problems to take care of, as well),
> > here is the following:
> >
> >> These have all been discussed off an on over the past 5 years.
> >> I can't get at bugzilla right now, but all the details are logged in
> cases.
> >> A survey of all the distros, their kernel package, etc, were all looked
> at.
> >> We had to balance the traditional packaging with some new concepts
> >> and landed with what we have now.
> >
> > I tried several tests. This is my final conclusion (two cases):
> >
> https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/Issues/kernel-development.txt
> >
> > The kernel issue is described here: there is need to have the YOCTO
> > minimum configuration with the kernel setup:
> > [1] The entire kernel source code in:
> > /usr/src/kernel/`uname-r`/<kernel source code>
> > [2] The header files in: /usr/src/kernel/`uname-r`/<header file
> > directory structures>
> >
> > Point [1] is achieved with the following local.config file:
> >
> https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/Issues/local-devsrc.conf
> >
> > Namely, with the following snippets in the local.conf:
> > TOOLCHAIN_TARGET_TASK_append = " packagegroup-core-tools-profile
> > packagegroup-core-buildessential kernel-devsrc"
> > KERNEL_DEV_TOOLS = "packagegroup-core-tools-profile
> > packagegroup-core-buildessential kernel-devsrc"
> > KERNEL_DEV_MODULE = "kernel-modules"
> > CORE_IMAGE_EXTRA_INSTALL += "${KERNEL_DEV_MODULE} \
> > ${KERNEL_DEV_TOOLS} \
> > systemtap \
> > "
> >
> > Problem with this approach is that such a kernel makes the rootfs too
> > big and impractical:
> > -rw-r--r--. 2 user vboxusers 101499952 May 17 14:32
> > core-image-minimal-beaglebone.rootfs.tar.xz
> >
> > The main issue is point [2]: how to achieve it?
> > The suggestion is to introduce the new package in YOCTO kernel,
> > called: kernel-headers
> > The OBVIOUS benefit is that it will serve to the purpose of building
> > modules out of the tree on the target with
> > minimal mpact to rootfs!
>
> I think this is a fair suggestion. Having prebuilt kernel available
> that contains the configuration and header files used in the build is
> all that is required for external modules to build in addition to
> toolchain, so maybe its worth a try to create such a package and then
> have kernel-source separated out which can be installed on top if one
> needs
>

That's exactly what the kernel-devsrc package is. It is about 90 Megs
(unpacked) and is the headers + config required to build out of tree
modules against the kernel.

I'm going to create two new optional packages for the fall release that are
simply captured kernel-source and kernel-headers for those uses cases where
someone really does want the entire source tree, or just the headers. The
details for that work are in bugzilla.

Bruce



>
> >
> > Thank you,
> > Zoran Stojsavljevic
> > _______
> >
> > On Thu, May 16, 2019 at 12:04 AM Bruce Ashfield
> > <bruce.ashfield at gmail.com> wrote:
> >>
> >>
> >>
> >> On Wed, May 15, 2019 at 4:09 PM Zoran Stojsavljevic <
> zoran.stojsavljevic at gmail.com> wrote:
> >>>
> >>>> The core-image-kernel-dev image is how I do all my on target
> >>>> testing when I introduce a new reference kernel for a release.
> >>>
> >>> Maybe you are correct. Maybe I should use/add in my local.conf the
> following:
> >>>
> >>> KERNEL_DEV_TOOLS ?= "packagegroup-core-tools-profile
> >>> packagegroup-core-buildessential kernel-devsrc"
> >>> KERNEL_DEV_MODULE ?= "kernel-modules"
> >>> CORE_IMAGE_EXTRA_INSTALL += "${KERNEL_DEV_MODULE} \
> >>>                               ${KERNEL_DEV_TOOLS} \
> >>>                               systemtap \
> >>>                              "
> >>> I need to try these... Maybe this addendum will solve the $1 mio USD
> problem?!
> >>>
> >>>> And IIRC the autobuilders are using a sato based image (Richard
> >>>> could confirm more easily that I could what image type the
> >>>> autobuilders are using for hello-world on target module tests).
> >>>
> >>> I am just advertising something more simple. To have mandatory
> >>> /lib/modules/`uname -r` directory. And introduce few more packages, as
> >>> Fedora distro, for example, has: kernel-headers (assuming YOCTO
> >>> rootfs, the following will be installed: /usr/src/kernel/`uname
> >>> -r`/<header file directory structures>. This also makes addition of
> >>> /lib/modules/`uname -r`/build file (which is soft link to
> >>> usr/src/kernel/`uname -r`).
> >>
> >>
> >> These have all been discussed off an on over the past 5 years. I can't
> get at bugzilla right now, but all the details are logged in cases. A
> survey of all the distros, their kernel package, etc, were all looked at.
> We had to balance the traditional packaging with some new concepts and
> landed with what we have now.
> >>
> >>
> >>>
> >>> Or kernel-devel package. Then, the whole current kernel source code
> >>> will be introduced, and also support for it.
> >>
> >>
> >> There's a case for this one as well, I'll probably have it done for the
> fall release. But our devsrc used to pretty much be the full source it has
> now been pruned down to something more manageable.  There are definitely
> some cases for having the full source on the target again, and it will be a
> separate package, just not the minimal one to build out of tree modules,
> etc.
> >>
> >>
> >>
> >>>
> >>>
> >>> SDK building with such a support is good/cool. But sometimes, before
> >>> introducing SDK, some tests should be done on target. NO need to
> >>> optionally include built-in layer hello-world driver example. Since I
> >>> (or you name the person) have own test drivers, which will be imported
> >>> out of tree, externally, to the target test bed!
> >>>
> >>
> >> I never use the SDK myself, so you are not alone in not going to it
> first. Hopefully I'll get some new patches out in the coming month before
> summer holidays really kick in.
> >>
> >> Bruce
> >>
> >>
> >>>
> >>> Just thinking loud...
> >>>
> >>> Zoran
> >>> _______
> >>>
> >>> On Wed, May 15, 2019 at 4:25 PM Bruce Ashfield <
> bruce.ashfield at gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 15, 2019 at 3:44 AM Zoran Stojsavljevic <
> zoran.stojsavljevic at gmail.com> wrote:
> >>>>>
> >>>>>> That's correct. That command only adds the kernel source and
> >>>>>> build infrastructure to the SDK, not to your target image. You'd
> still
> >>>>>> need to arrange to have the kernel-devsrc package installed on the
> >>>>>> target image if you want it on the board's rootfs. How you arrange
> >>>>>> to have the package installed to the image varies with the image
> >>>>>> (since they all don't have the same image install variables, etc).
> >>>>>
> >>>>> And here is a $1,000,000 USD question? How to do it on Poky (as
> >>>>> example of what you have stated in RED)? ;-)
> >>>>>
> >>>>> In other words: how to arrange it on Poky (as a Referent example)?
> >>>>
> >>>>
> >>>> The core-image-kernel-dev image is how I do all my on target testing
> when I introduce a new reference kernel for a release. And IIRC the
> autobuilders are using a sato based image (Richard could confirm more
> easily that I could what image type the autobuilders are using for
> hello-world on target module tests).
> >>>>
> >>>> Bruce
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>> Thank you,
> >>>>> Zoran
> >>>>> _______
> >>>>>
> >>>>>
> >>>>> On Wed, May 15, 2019 at 7:41 AM Bruce Ashfield <
> bruce.ashfield at gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 14, 2019 at 1:30 PM Zoran Stojsavljevic <
> zoran.stojsavljevic at gmail.com> wrote:
> >>>>>>>
> >>>>>>> Hello Chris, Bruce,
> >>>>>>>
> >>>>>>> I have some additional data to share with you both, since I have
> tried
> >>>>>>> something. And here is my take on the things!
> >>>>>>>
> >>>>>>>> 1. Build using a bb recipe.
> >>>>>>>> Take a look at meta-skeleton/recipes-kernel/hello-mod for an
> example.
> >>>>>>>> You just need to add meta-skeleton to your bblayers.conf and then
> >>>>>>>>   bitbake hello-mod
> >>>>>>>
> >>>>>>> I looked into this example, and, yes, it is classic kernel module
> >>>>>>> definition out of the tree. With some outdated data, all cool, the
> >>>>>>> YOCTO designer should take care himself to fix these data, if using
> >>>>>>> this stuff.
> >>>>>>>
> >>>>>>> But this is NOT mandatory, since I can add out of the tree module
> NOT
> >>>>>>> actually using built-in module. I just use
> .../tmp/deploy/images/bbb/*
> >>>>>>> generated stuff, since I have automated scripts which are bringing
> all
> >>>>>>> these on my BBB target. Then I tftp my source code module to the
> >>>>>>> target.
> >>>>>>>
> >>>>>>>> 2. Build from the SDK:
> >>>>>>>> First, add the kernel source to the SDK by adding this to
> conf/local.conf
> >>>>>>>>   TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc"
> >>>>>>>
> >>>>>>> YES, this is THE command which should generate
> >>>>>>> /usr/src/kernel(s)/`uname -r` or similar... But adding it to
> >>>>>>> local.conf and after deleting kernel, then regenerating bitbake -k
> >>>>>>> core-image-minimal does not bring this path into the rootfs image!?
> >>>>>>
> >>>>>>
> >>>>>> That's correct. That command only adds the kernel source and build
> infrastructure to the SDK, not to your target image. You'd still need to
> arrange to have the kernel-devsrc package installed on the target image if
> you want it on the board's rootfs. How you arrange to have the package
> installed to the image varies with the image (since they all don't have the
> same image install variables, etc).
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I did it actually using meta-bbb, and using poky referent distro as
> >>>>>>> two additional layers to the more complex bbb image!
> >>>>>>> https://github.com/jumpnow/meta-bbb.git
> >>>>>>>
> >>>>>>> The (KAS - you can figure out out of it local.conf) script I am
> using
> >>>>>>> to build such a BBB image is here:
> >>>>>>>
> https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/bbb-releases/bbb-warrior/kas-bbb-warrior.yml
> >>>>>>>
> >>>>>>> I did not try it with BBB reference poky only! Maybe I should try
> it
> >>>>>>> as only referent poky? What do you think?
> >>>>>>>
> >>>>>>> Does in this case is SDK build really mandatory??? Should NOT be!
> >>>>>>>
> >>>>>>
> >>>>>> You only do the SDK steps if you want to support building out of
> tree modules in an SDK install. So it is not mandatory for on target module
> builds.
> >>>>>>
> >>>>>> Bruce
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>> Once the SDK is installed, generate the kernel headers:
> >>>>>>>>   sudo -i
> >>>>>>>>   .
> /opt/poky/2.6.2/environment-setup-cortexa8hf-neon-poky-linux-gnueabi
> >>>>>>>>   cd /opt/poky/2.6.2/sysroots/cortexa8hf-neon-poky-linux-gnueabi
> >>>>>>>>   cd /usr/src/kernel
> >>>>>>>>   make oldconfig scripts
> >>>>>>>>   exit
> >>>>>>>
> >>>>>>> This is in nutshell the same what I did (a bit different) for
> Embedded
> >>>>>>> Debian. This is already on the target BBB, NOT while building YOCTO
> >>>>>>> BBB image!
> >>>>>>>
> >>>>>>>> Finally, build your module using a Makefile like this
> >>>>>>>>   obj-m := hello-mod.o
> >>>>>>>>   all:
> >>>>>>>>         make -C $(SDKTARGETSYSROOT)/usr/src/kernel M=$(shell pwd)
> >>>>>>>
> >>>>>>> As said before: bringing my own module into the target BBB (I have
> my
> >>>>>>> own examples, and I build them on the target with the almost the
> same
> >>>>>>> Makefiles)
> >>>>>>>
> >>>>>>> Zoran
> >>>>>>> _______
> >>>>>>>
> >>>>>>> On Sun, May 12, 2019 at 3:15 PM Chris Simmonds <chris at 2net.co.uk>
> wrote:
> >>>>>>>>
> >>>>>>>> Hi Zoran,
> >>>>>>>>
> >>>>>>>> There are two ways to do this
> >>>>>>>>
> >>>>>>>> 1. Build using a bb recipe.
> >>>>>>>> Take a look at meta-skeleton/recipes-kernel/hello-mod for an
> example.
> >>>>>>>> You just need to add meta-skeleton to your bblaysers.conf and then
> >>>>>>>>    bitbake hello-mod
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2. Build from the SDK:
> >>>>>>>> First, add the kernel source to the SDK by adding this to
> conf/local/conf
> >>>>>>>>    TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc"
> >>>>>>>>
> >>>>>>>> Then build the SDK
> >>>>>>>>    bitbake -c populate_sdk [your image recipe]
> >>>>>>>>
> >>>>>>>> Once the SDK is installed, generate the kernel headers:
> >>>>>>>>    sudo -i
> >>>>>>>>    .
> /opt/poky/2.6.2/environment-setup-cortexa8hf-neon-poky-linux-gnueabi
> >>>>>>>>    cd /opt/poky/2.6.2/sysroots/cortexa8hf-neon-poky-linux-gnueabi
> >>>>>>>>    cd /usr/src/kernel
> >>>>>>>>    make oldconfig scripts
> >>>>>>>>    exit
> >>>>>>>>
> >>>>>>>> Finally, build your module using a Makefile like this
> >>>>>>>>
> >>>>>>>>    obj-m := hello-mod.o
> >>>>>>>>    all:
> >>>>>>>>          make -C $(SDKTARGETSYSROOT)/usr/src/kernel M=$(shell pwd)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> HTH,
> >>>>>>>> Chris
> >>>>>>>>
> >>>>>>>> On 12/05/2019 11:53, Zoran Stojsavljevic wrote:
> >>>>>>>>> Hello to the YOCTO community,
> >>>>>>>>>
> >>>>>>>>> I am using (to build the target for Beagle Bone Black) the
> following script:
> >>>>>>>>> https://github.com/ZoranStojsavljevic/bbb-yocto
> >>>>>>>>>
> https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/bbb-yocto.sh
> >>>>>>>>>
> >>>>>>>>> The latest kernel I am using from the following repo:
> >>>>>>>>> https://github.com/jumpnow/meta-bbb
> >>>>>>>>>
> >>>>>>>>> Is kernel 5.0.14 .
> >>>>>>>>>
> >>>>>>>>> Here is the snippet of the boot traces:
> >>>>>>>>> Starting kernel ...
> >>>>>>>>>
> >>>>>>>>> [    0.000000] Booting Linux on physical CPU 0x0
> >>>>>>>>> [    0.000000] Linux version 5.0.14-jumpnow (oe-user at oe-host)
> (gcc
> >>>>>>>>> version 8.3.0 (GCC)) #1 Fri May 10 13:12:33 UTC 2019
> >>>>>>>>> [    0.000000] CPU: ARMv7 Processor [413fc082] revision 2
> (ARMv7), cr=10c5387d
> >>>>>>>>> [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT
> aliasing
> >>>>>>>>> instruction cache
> >>>>>>>>> [    0.000000] OF: fdt: Machine model: TI AM335x BeagleBone Black
> >>>>>>>>> [    0.000000] Memory policy: Data cache writeback
> >>>>>>>>> [    0.000000] cma: Reserved 16 MiB at 0x9f000000
> >>>>>>>>> [    0.000000] CPU: All CPU(s) started in SVC mode.
> >>>>>>>>> [    0.000000] AM335X ES2.1 (sgx neon)
> >>>>>>>>> [    0.000000] random: get_random_bytes called from
> >>>>>>>>> start_kernel+0xa4/0x460 with crng_init=0
> >>>>>>>>> [    0.000000] Built 1 zonelists, mobility grouping on.  Total
> pages: 130048
> >>>>>>>>> [    0.000000] Kernel command line: console=ttyO0,115200n8
> >>>>>>>>> root=/dev/ram0 ip=dhcp
> >>>>>>>>>
> >>>>>>>>> According to the documentation, the following:
> >>>>>>>>> 2.10.1. Building Out-of-Tree Modules on the Target
> >>>>>>>>>
> https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html
> >>>>>>>>>
> >>>>>>>>> I tried to find /usr/src/kernels/5.0.14... Directory, since I
> see
> >>>>>>>>> from the build that kernel-dev and kernel-devsrc are included:
> >>>>>>>>> [user at fedora29-ssd bbb-yocto]$ bitbake -s | grep kernel
> >>>>>>>>> core-image-kernel-dev                                 :1.0-r0
> >>>>>>>>> kernel-devsrc                                         :1.0-r0
> >>>>>>>>> kernel-selftest                                       :1.0-r0
> >>>>>>>>>
> >>>>>>>>> THE PROBLEM: But I could not find ob BBB target /usr/src/kernels
> >>>>>>>>> directory at all!?
> >>>>>>>>>
> >>>>>>>>> Two questions here?
> >>>>>>>>> [1] Do you have any advice on this problem (what I am missing
> here)?
> >>>>>>>>> [2] Alternative to [1]: how I can use cross compiler from
> >>>>>>>>> .../build/tmp to build Out-of-Tree Module for the BBB target on
> the
> >>>>>>>>> host?
> >>>>>>>>>
> >>>>>>>>> Thank you,
> >>>>>>>>> Zoran
> >>>>>>>>> _______
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Chris Simmonds, trainer and consultant at 2net
> >>>>>>>> http://www.2net.co.uk
> >>>>>>>> Author of "Mastering Embedded Linux Programming"
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> - Thou shalt not follow the NULL pointer, for chaos and madness
> await thee at its end
> >>>>>> - "Use the force Harry" - Gandalf, Star Trek II
> >>>>>>
> >>>>
> >>>>
> >>>> --
> >>>> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> >>>> - "Use the force Harry" - Gandalf, Star Trek II
> >>>>
> >>
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >>
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190524/a11e5063/attachment-0001.html>

More information about the yocto mailing list