[yocto] New Documentation: Linux Kernel Development with the Yocto Project
Bruce Ashfield
bruce.ashfield at windriver.com
Tue Dec 11 16:03:51 PST 2012
On 12-12-11 06:44 PM, Darren Hart wrote:
> In response to much feedback on the linux-yocto recipes and the associated
> kernel tools, we have made a number of improvements in an attempt to
> make them
> more accessible. In particular, the tools now allow for using your own
> sources
> and configurations in a fairly simple manner, while providing a path to more
> complex meta-data management.
>
> The following document is intended to demystify the Yocto Project Linux
> Kernel
> Tools now that some of this work has been done. Ultimately, the goal of
> this
> document is to provide a "task-based" approach to developing Linux kernel
> recipes and meta-data with the Yocto Project. It should present
> information more
> or less in the order a developer is likely to need it. Start off by
> changing a
> config and adding a patch, then using your own sources, incorporating
> out of tree
> drivers, then diving into more complex configuration and source management.
>
> This document has seen some review prior to this announcement. I wanted
> to thank
> everyone for the feedback provided. I have done my best to incorporate
> it. In a
> couple cases I felt the additions were better left to "How Do I" wiki
> pages or a
> separate manual (such as the Development Manual or the BSP Guide). The
> following
> is what I am submitting to Scott R, our tech writer, for a full review and
> docbookification for release with the 1.4 Yocto Project release.
> Eventually I
> would like to see the following sections added to the document.
>
> 1) Workflow
> * Group environements
> * Local sources
> 2) Advanced Source Management
> * working outside of bitbake
> * generating the git tree from the meta-data
> 3) Examples
>
> For now, I feel this document should go a long way towards clarifying
> the use of
> the kernel tools and the linux-yocto recipes.
>
> I would still greatly appreciate any additional feedback and there is still
> plenty of time to incorporate that feedback prior to release. So if you
> haven't
> already and have the inclination, I'd welcome your review!
Thanks for pulling this together Darren. On a quick scan, I see some
sections that I can update and get you new output and remove some of the
TODOs and questions.
I'll try and get to that ASAP so you'll have them in lots of time for
release.
Cheers,
Bruce
>
>
>
> Linux Kernel Development with the Yocto Project
>
>
> Table of Contents
> =================
> INTRODUCTION
> Yocto Project Linux Kernel Recipes
>
> COMMON TASKS
> Preparing a Layer
> Modifying an Existing Recipe (Mostly covered by the Dev Manual)
> Applying Patches
> Changing the Config
> Iterative Development
> Generating Configuration Files
> Modifying Source Code
> Working with Your Own Sources
> Incorporating Out-of-tree Modules
>
> ADVANCED META-DATA
> Using Meta-Data in a Recipe
> Meta-Data Location
> Recipe-Space Meta-Data
> In-Tree Meta-Data
> Meta-Data Syntax
> Configuration
> Patches
> Features
> Kernel Types
> BSPs
> Machine Branches
> Feature Branches
> SCC Reference
>
> EXAMPLES
>
> FAQ (Maybe put this online as a wiki?)
> How do I use my own Linux kernel .config?
> How do I create configuration fragments?
> How do I use my own Linux kernel sources?
> How do I install/not-install the kernel image on the rootfs?
> How do install a specific kernel module?
> How do I changed the Linux kernel command line?
>
>
> 1 Introduction
> ==============
> Regardless of how you intend to make use of the Yocto Project, chances
> are you
> are going to need to work with the Linux kernel. The Yocto Project
> provides a
> powerful set of tools for managing Linux kernel sources and
> configuration data.
> If you want to make a single configuration change, apply a couple of
> patches, or
> work with your own sources, the Yocto Project has tooling in place to
> help you
> do it.
>
>
> 1.1 Yocto Project Linux Kernel Recipes
> --------------------------------------
> Each Yocto Project release introduces a new set of linux-yocto kernel
> recipes,
> tracking the latest upstream developments and introducing newly supported
> platforms. In addition to the new kernel recipes, the previous recipes are
> refreshed and supported for at least one additional release. As they align,
> these previous releases are updated to include the latest from the LTSI
> project.
> Besides these recipes, a linux-yocto-dev recipe is available for working
> with
> the very latest in upstream Linux kernel development as well as meta-data
> development.
>
> If you do not maintain your own kernel sources and need to make only minimal
> changes to the sources, these recipes provide a vetted base upon which
> to layer
> your changes. Doing so allows you to benefit from the continual kernel
> integration and testing performed during development of the Yocto Project.
>
> If, instead, you have a very specific Linux kernel source tree and are
> unable to
> align with one of the many official linux-yocto releases, an alternative
> exists by which you can use the Yocto Project Linux kernel tools with
> your own
> sources.
>
> The sections that follow provide instructions for completing specific Linux
> kernel development tasks. They assume familiarity with working with bitbake
> recipes and basic open-source development tools. Understanding these
> concepts
> will facilitate the process of working with the kernel recipes. If you
> find you
> need some additional background, please be sure to review and understand the
> following documentation:
>
> o Yocto Project Getting Started Guide
> o Yocto Project Development Manual
> - 4.3 Modifying Temporary Source Code
> - 5.1 Understanding and Creating Layers
> - 4.1.2 Modifying the Kernel
> (This section should be retired, as well as 5.6, 5.7 - with this
> document
> providing the necessary information)
>
> Finally, while this document focuses on the manual creation of recipes,
> patches,
> and configuration files, the Yocto Project BSP tools are available to
> automate
> this process with existing content and work well to create the initial
> framework
> and boilerplate code. For details, refer to the Yocto Project BSP
> Developer's
> Guide.
>
>
> 2 Common Tasks
> ==============
>
>
> 2.1 Preparing a Layer
> ---------------------
> Customizing recipes is best done in a layer with bbappend files. Layers also
> provide a convenient mechanism to create your own recipes. This guide
> assumes
> you will be working from within a layer independent from those released
> with the
> Yocto Project. For details on how to create and work with layers, refer to
> section 5.1 Understanding and Creating Layers in the Yocto Project
> Development
> Manual.
>
> (Kernel specific directions in 5.7.4)
>
>
> 2.2 Modifying an Existing Recipe
> --------------------------------
> In many cases, you can customize an existing linux-yocto recipe to meet the
> needs of your project. Each release of the Yocto Project provides a few
> Linux
> kernel recipes to choose from. To get started, prepare a layer and a
> bbappend
> file corresponding to the recipe you wish to modify.
>
> See [New example in Dev Manual] for instructions to create a minimal layer.
>
> The bbappend will typical be located at the following location relative
> to the
> layer (if modifying the linux-yocto_3.4.bb recipe):
>
> recipes-kernel/linux/linux-yocto_3.4.bbappend
>
> And should contain the following text initially:
>
> FILESEXTRAPATHS := "${THISDIR}/${PN}"
>
> The ${PN} will expand to "linux-yocto" in this example. Any new files
> added to
> modify the recipe should then be added to the following path within the
> layer:
>
> recipes-kernel/linux/linux-yocto/
>
> NOTE: If you are working on a new machine BSP, be sure to refer to the Yocto
> Project Board Support Package Developer's Guide.
>
>
> 2.2.1 Applying Patches
> ----------------------
> If you have a patch, or a small series of patches, to apply to the Linux
> kernel
> source, you can do so just as you would with any other recipe. You first
> copy
> the patches to the path added to FILESEXTRAPATHS in the bbappend file as
> described in 2.2 and then reference them in the SRC_URI.
>
> For example, you can apply a three patch series by adding the following
> lines to
> your linux-yocto bbappend file in your layer:
>
> SRC_URI += "file://0001-first-change.patch"
> SRC_URI += "file://0002-first-change.patch"
> SRC_URI += "file://0003-first-change.patch"
>
> At the next build, bitbake will detect the change in the recipe and
> fetch and
> apply the patches before rebuilding the Linux kernel.
>
>
> 2.2.2 Changing the Config
> -------------------------
> Making wholesale or incremental changes to the Linux kernel config can
> be made
> by including a defconfig or configuration fragments in the SRC_URI.
>
> If you have a complete Linux kernel .config file you want to use, copy it as
> "defconfig" to the ${FILES} directory and add the following line to your
> linux-yocto bbappend file in your layer:
>
> SRC_URI += "file://defconfig"
>
> Generally speaking, the preferred approach is to determine the incremental
> change you want to make and add that as a fragment. For example, if you
> wanted
> to add support for a basic serial console, create a file named
> "8250.cfg" in the
> ${FILES} directory with the following content (without indentation):
>
> CONFIG_SERIAL_8250=y
> CONFIG_SERIAL_8250_CONSOLE=y
> CONFIG_SERIAL_8250_PCI=y
> CONFIG_SERIAL_8250_NR_UARTS=4
> CONFIG_SERIAL_8250_RUNTIME_UARTS=4
> CONFIG_SERIAL_CORE=y
> CONFIG_SERIAL_CORE_CONSOLE=y
>
> Then include this configuration fragment in the SRC_URI:
>
> SRC_URI += "file://8250.cfg"
>
> At the next build, bitbake will detect the change in the recipe and
> fetch and
> apply the new configuration before rebuilding the Linux kernel.
>
>
> 2.3 Iterative Development
> -------------------------
> If you do not have existing patches or configuration files, you can easily
> generate them from within the bitbake build environment, as will be
> described
> below. As you do, running previously completed bitbake tasks will cause
> bitbake
> to invalidate the tasks that follow them in the build sequence, causing
> them to
> rebuild at the next invocation of the build. Throughout this section, be
> sure to
> substitute "linux-yocto" with the name of the Linux kernel recipe you are
> working with.
>
>
> 2.3.1 Generating Configuration Files
> ------------------------------------
> You can manipulate the config used to build a linux-yocto recipe with the
> "menuconfig" command.
>
> $ bitbake linux-yocto -c menuconfig
>
> This will start the Linux kernel configuration tool, allowing you to
> prepare a
> new .config for the build. When you exit, be sure to save the changes when
> prompted. The resulting .config file will be located in the ${WORKDIR} under
> the linux-${MACHINE}-${KTYPE}-build directory. You can use this in its
> entirety
> as the defconfig file described in 2.2.2.
>
> Better still, would be to use the "menuconfig" command and take the
> difference
> of the new .config file with the previous one to generate a configuration
> fragment. To do this, be sure to complete a build at least through the
> kernel
> configuration task:
>
> $ bitbake linux-yocto -c kernel_configme -f
>
> Then make a copy of the .config file, calling it "config.orig", for
> example. Run
> the "menuconfig" command as described above. Finally, prepare a
> configuration
> fragment from the difference between the files. Ultimately you want a
> list of
> Linux kernel CONFIG_ assignments, and not something in diff format.
> Something
> like the following will do that automatically (but plan on reviewing the
> output
> as you can usually remove some of the defaults):
>
> $ diff -Nurp config.orig .config | sed -n "s/^\+//p" > frag.cfg
>
> You can use the output as a configuration fragment described in 2.2.2. This
> method can also be used to define a BSP configuration fragment (See 3.3.5).
>
> The Yocto Project kernel tools provide some configuration validation
> tools which
> will warn when a configuration you requested does not appear in the final
> config, or when you override a policy configuration in a hardware
> configuration
> fragment. You can run these tools with the following command:
>
> $ bitbake linux-yocto -c kernel_configcheck -f
>
> ...
>
> NOTE: validating kernel configuration
> This BSP sets 3 invalid/obsolete kernel options.
> These config options are not offered anywhere within this kernel.
> The full list can be found in your kernel src dir at:
> meta/cfg/standard/mybsp/invalid.cfg
>
> This BSP sets 21 kernel options that are possibly non-hardware related.
> The full list can be found in your kernel src dir at:
> meta/cfg/standard/mybsp/specified_non_hdw.cfg
>
> WARNING: There were 2 hardware options requested that do not
> have a corresponding value present in the final ".config" file.
> This probably means you aren't getting the config you wanted.
> The full list can be found in your kernel src dir at:
> meta/cfg/standard/mybsp/mismatch.cfg
>
> The various problems that you can encounter are described in the output
> along
> with where to find the offending configuration items. You can use these
> logs to
> adjust your configuration files and repeat the "kernel_configme" and
> "kernel_configcheck" commands until no warnings are produced.
>
>
> 2.3.2 Modifying Source Code
> ---------------------------
> You can experiment with source code changes and create a simple patch
> without
> leaving the bitbake environment. To get started, be sure to complete a
> build at
> least through the kernel configuration task:
>
> $ bitbake linux-yocto -c kernel_configme -f
>
> This step will ensure you have the sources prepared and the configuration
> completed. You will find the sources in the ${WORKDIR}/linux directory.
>
> You can edit the sources as you would any other Linux source tree, but
> keep in
> mind that your changes will be lost if you trigger the fetch task for the
> recipe. Avoid this by not issuing the "cleanall" or "cleansstate", or
> forcing
> the "fetch" command. Also be sure not to modify the recipe itself while
> working
> with temporary changes or bitbake may run the fetch command (depending
> on the
> changes to the recipe).
>
> To test your temporary changes, instruct bitbake to run the compile
> again. The
> -f option forces the command to run again even though bitbake may think
> it has
> already done so:
>
> $ bitbake linux-yocto -c compile -f
>
> If the compile fails, you can update the sources and repeat the compile
> command. Once it compiles successfully, you can inspect and test the
> resulting
> build (kernel, modules, etc.) from the build directory at
> ${WORKDIR}/linux-${MACHINE}-${KTYPE}-build. Alternatively, you can run the
> deploy command to place the kernel image in the tmp/deploy/images directory:
>
> $ bitbake linux-yocto -c deploy
>
> And of course, you can run through all the remaining installation and
> packaging
> steps by issuing:
>
> $ bitbake linux-yocto
>
> For rapid iterative development, the edit-compile-repeat loop is
> preferable to
> rebuilding the entire recipe as the installation and packaging tasks are
> very
> time consuming.
>
> Once you are happy with your modifications, you can make these permanent by
> generating patches and applying them to the SRC_URI as described in section
> 2.2.1 Applying Patches. If you are not familiar with generating patches,
> refer
> to the Yocto Project Development Manual, section 5.7.3 Creating the Patch.
>
>
> 2.4 Working with Your Own Sources
> ---------------------------------
> If you find yourself unable to work with one of the Linux kernel versions
> supported by existing linux-yocto recipes, you can still make use of the
> Yocto
> Project Linux kernel tooling while working with your own sources. You
> will not
> be able to leverage the existing meta-data and stabilization work of the
> linux-yocto sources, but you will be able to manage your own meta-data
> in the
> same format as the linux-yocto sources which will facilitate converging with
> linux-yocto on a future mutually-supported kernel version.
>
> The linux-yocto-custom recipe, located in the poky repository at:
>
> meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
>
> is provided as an example recipe which uses kernel.org sources and the Yocto
> Project Linux kernel tools for managing meta-data. To get started, copy this
> recipe to your layer and provide it with a meaningful name, such as
> linux-yocto-myproject_3.5.bb, where 3.5 is the base version of the Linux
> kernel
> you will be working with. In the same directory, create a matching
> directory,
> e.g. linux-yocto-myproject to store your patches and configuration files.
>
> Edit the following variables in the recipe as appropriate for your project:
>
> o SRC_URI
> o LINUX_VERSION
> o LINUX_VERSION_EXTENSION
> o SRCREV
> o PR
> o PV
> o COMPATIBLE_MACHINE
>
> The SRC_URI should be a git repository, using one of the supported git
> fetcher
> protocols (file, git, http, etc.). The skeleton recipe provides an example
> SRC_URI as a syntax reference.
>
> Set LINUX_VERSION to the Linux kernel version you are using, such as
> "3.6.3".
>
> LINUX_VERSION_EXTENSION is used to define the Linux kernel
> CONFIG_LOCALVERSION
> which will be compiled in to the resulting kernel and visible via the uname
> command.
>
> Set SRCREV to the commit ID you wish to build from.
>
> Treat the PR as you would the PR of any other recipe. Increment it to
> indicate
> to the build system that the recipe has changed.
>
> The default PV assignment is typically adequate. It combines the
> LINUX_VERSION
> with the SCM revision (from the SRCPV variable) and results in a string
> something like:
>
> "3.4.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c60d32f0d770931e3cf343e2"
>
> While lengthy, this extra verbosity helps ensure you are building from
> the exact
> sources you intend.
>
> Finally, the default COMPATIBLE_MACHINE assignment for
> linux-yocto-custom is set
> to a regular expression matching only the empty string, "(^$)". This will
> trigger an explicit build failure. You must change it to match a list of the
> machines supported by your new recipe, such as "(qemux86|qemux86-64)"
>
> With that in place, you can continue to customize this recipe as you
> would the
> existing linux-yocto recipes. See Section 2.2 Modifying an Existing
> Recipe for
> details.
>
>
> 2.5 Incorporating Out-of-tree Modules
> -------------------------------------
> While it is always preferable to work with sources integrated into the Linux
> kernel sources, if you have need of an external kernel module, the hello-mod
> recipe is available as a template to create your own out-of-tree Linux
> kernel
> module recipe. It is available in the poky repository at:
>
> meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb
>
> To get started, copy this recipe to your layer and provide it with a
> meaningful name, such as
> mymodule_1.0.bb. In the same directory, create a directory named "files"
> where
> you can store any source files, patches, or other files necessary for
> building
> the module which do not come with the sources. Finally, update the recipe as
> appropriate for the module. Typically you will need to set the following
> variables:
>
> o DESCRIPTION
> o LICENSE*
> o SRC_URI
> o PV
>
> Depending on the build system used by the module sources, you may need
> to make
> adjustments. For example, a typical module Makefile will look much like that
> provided with hello-mod:
>
> obj-m := hello.o
>
> SRC := $(shell pwd)
>
> all:
> $(MAKE) -C $(KERNEL_SRC) M=$(SRC)
>
> modules_install:
> $(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install
> ...
>
> The important point to note here is the KERNEL_SRC variable. The module
> bbclass
> class sets this, as well as KERNEL_PATH, to ${STAGING_KERNEL_DIR} with the
> necessary Linux kernel build information to build modules. If your module
> Makefile uses a different variable, you may want to override the
> do_compile()
> step, or create a patch to the Makefile to work with the more typical
> KERNEL_SRC
> or KERNEL_PATH variables.
>
> After you have prepared the recipe, you will likely want to include the
> module
> in your images. To do this, see the documentation for the following
> variables in
> the Yocto Project Reference Manual and set one of them as appropriate in
> your
> machine config file:
>
> MACHINE_ESSENTIAL_EXTRA_RDEPENDS
> MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
> MACHINE_EXTRA_RDEPENDS
> MACHINE_EXTRA_RRECOMMENDS
>
> As modules are often not required for boot and may be excluded from certain
> build configurations, the following allows for the most flexibility:
>
> MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule"
>
> Where the value is derived by appending the module filename without the .ko
> extension to the string "kernel-module-".
>
> As it is an RRECOMMENDS (and not an RDEPENDS) variable, the build will
> not fail
> if this module is not available to include in the image.
>
>
> 3 Advanced Meta-Data
> ====================
> In addition to configuration fragments and patches, the Yocto Project kernel
> tools support rich meta-data which you can use to define complex
> policies and
> BSP support. The purpose of the meta-data and the tools to manage it,
> known as
> the kern-tools (kern-tools-native_git.bb), is to assist in managing the
> complexity of the configuration and sources in support of multiple BSPs and
> Linux kernel types.
>
> In particular, the kernel tools allow you to specify only what you must, and
> nothing more. Where a complete Linux kernel .config includes all the
> automatically selected CONFIG options, the configuration fragments only
> need to
> contain the highest level visible CONFIG options as presented by the Linux
> kernel menuconfig system. This reduces your maintenance effort and
> allows you
> to further separate your configuration in ways that make sense for your
> project.
> A common split is policy and hardware. For example, all your kernels may
> support
> the proc and sys filesystems, but only specific boards will require
> sound, usb,
> or specific drivers. Specifying these individually allows you to
> aggregate them
> together as needed, but maintain them in only one place. Similar logic
> applies
> to source changes.
>
>
> 3.1 Using Meta-Data in a Recipe
> -------------------------------
> The meta-data provided with any linux-yocto style Linux kernel sources must
> define a BSP that corresponds to the definition laid out in the recipe.
> A BSP
> consists of an aggregation of kernel policy and hardware specific feature
> enablement. This can be influenced from within the recipe.
>
> Every linux-yocto style recipe must define the following variables:
>
> KMACHINE
>
> KMACHINE is typically set to the same value as used within the
> recipe-space BSP
> definition, such as "routerstationpro" or "fri2". However, multiple BSPs can
> reuse the same KMACHINE name if they are built using the same BSP
> description
> (see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples
> of such
> a situation where each specifies KMACHINE as "fri2".
>
> They may optionally define the following variables:
> KBRANCH
> KERNEL_FEATURES
> KBRANCH_DEFAULT
> LINUX_KERNEL_TYPE
>
> KBRANCH_DEFAULT defines the default source branch within the Linux
> kernel source
> repository to be used to build the Linux kernel. It is used as the
> default value
> for KBRANCH which may define an alternate branch, typically with a machine
> override, such as:
>
> KBRANCH_fri2 = "standard/fri2"
>
> Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master".
>
> LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the
> configuration and defaults to "standard" if you do not specify otherwise.
> Together with KMACHINE, this defines the search arguments used by the Yocto
> Project Linux kernel tools to find the appropriate description within the
> meta-data with which to build out the sources and configuration. The
> linux-yocto
> recipes define "standard", "tiny", and "preempt-rt" kernel types. See
> 3.3.4 for
> more inforation on kernel types.
>
> During the build, the kern-tools will search for the BSP description
> file that
> most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the
> recipe. It will use the first BSP description it finds matching both
> variables.
> Failing that it will issue a warning such as the following:
>
> WARNING: Can't find any BSP hardware or required configuration fragments.
> WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
> meta/cfg/broken/fri2-broken/required_frags.txt in directory:
> meta/cfg/broken/fri2-broken
>
> In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE
> was set to "broken".
>
> It will then search first for the KMACHINE and then
> for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will
> use the
> sources from the KBRANCH and any configuration specified in the SRC_URI.
>
> KERNEL_FEATURES can be used to include features (configuration fragments,
> patches, or both) that are not already included by the KMACHINE and
> LINUX_KERNEL_TYPE combination. To include a feature specified as
> "features/netfilter.scc" for example, specify:
>
> KERNEL_FEATURES += "features/netfilter.scc"
>
> To include a feature called "cfg/sound.scc" just for the qemux86 machine,
> specify:
>
> KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"
>
> The value of the entries in KERNEL_FEATURES are dependent on their location
> within the meta-data itself. The examples here are taken from the
> linux-yocto-3.4 repository where "features" and "cfg" are subdirectories
> of the
> meta-data directory. For details, see 3.3.
>
> Note: The processing of the these variables has evolved some between the
> 0.9 and 1.3 releases of the Yocto Project and associated
> kern-tools sources. The above is accurate for 1.3 and later
> releases of the Yocto Project.
>
>
> 3.2 Meta-Data Location
> ----------------------
> This meta-data can be defined along with the Linux kernel recipe
> (recipe-space)
> as partially described in section 2.2 as well as within the Linux kernel
> sources
> themselves (in-tree).
>
> Where you choose to store the meta-data depends on what you want to do
> and how
> you intend to work. If you are unfamiliar with the Linux kernel and only
> wish
> to apply a config and possibly a couple of patches provided to you by
> others,
> you may find the recipe-space mechanism to be easier to work with. This
> is also
> a good approach if you are working with Linux kernel sources you do not
> control
> or if you just don't want to maintain a Linux kernel git repository on
> your own.
>
> If you are doing active kernel development and are already maintaining a
> Linux
> kernel git repository of your own, you may find it more convenient to
> work with
> the meta-data in the same repository as the Linux kernel sources. This
> can make
> iterative development of the Linux kernel more efficient outside of the
> bitbake
> environment.
>
> Regardless of where the meta-data is stored, the syntax as
> described in the following sections applies equally.
>
>
> 3.2.1 Recipe-Space Meta-Data
> ----------------------------
> When stored in recipe-space, the meta-data files reside in a directory
> hierarchy
> below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a
> linux-yocto or linux-yocto-custom derived Linux kernel recipe. See 2.2.
>
> By way of example, a trivial tree of meta-data stored in recipe-space
> within a
> BSP layer might look like the following:
>
> meta/
> `-- recipes-kernel
> `-- linux
> `-- linux-yocto
> |-- bsp-standard.scc
> |-- bsp.cfg
> `-- standard.cfg
>
> When the meta-data is stored in recipe-space, you must take steps to ensure
> bitbake has the necessary information to decide which files to fetch and
> when
> they need to be fetched again.
>
> It is only necessary to specify the .scc files on the SRC_URI; bitbake will
> parse them and fetch any files referenced in the .scc files by the include,
> patch, or kconf commands. Because of this, it is necessary to bump the
> recipe PR
> value when changing the content of files not explicitly listed in the
> SRC_URI.
>
>
> 3.2.2 In-Tree Meta-Data
> -----------------------
> When stored in-tree, the meta-data files reside in the "meta" directory
> of the
> Linux kernel sources. They may be present in the same branch as the sources,
> such as "master", or in their own orphan branch, typically named "meta". An
> orphan branch in git is a branch with unique history and content to the
> other
> branches in the repository. This is useful to track meta-data changes
> independently from the sources of the Linux kernel, while still keeping them
> together in the same repository. For the purposes of this document, we will
> discuss all in-tree meta-data as residing below the "meta/cfg/kernel-cache"
> directory.
>
> By way of example, a trivial tree of meta-data stored in a custom Linux
> kernel
> git repository might look like the following:
>
> meta/
> `-- cfg
> `-- kernel-cache
> |-- bsp-standard.scc
> |-- bsp.cfg
> `-- standard.cfg
>
> To use a specific branch for the meta-data, specify the branch in the KMETA
> variable in your Linux kernel recipe, for example:
>
> KMETA = "meta"
>
> To use the same branch as the sources, set KMETA to the empty string:
>
> KMETA = ""
>
> If you are working with your own sources and want to create an orphan meta
> branch, you can do so using the following commands from within your
> Linux kernel
> git repository:
>
> $ git checkout --orphan meta
> $ git rm -rf .
> $ git commit --allow-empty -m "Create orphan meta branch"
>
>
> 3.3 Meta-Data Syntax
> --------------------
> The Yocto Project Linux kernel tools meta-data consists of three primary
> types
> of files: scc* description files, configuration fragments, and patches.
> The scc
> files define variables and include or otherwise reference any of the
> three file
> types. The description files are used to aggregate all types of
> meta-data into
> what ultimately describes the sources and the configuration required to
> build a
> Linux kernel tailored to a specific machine.
>
> The scc description files are used to define two fundamental types of
> meta-data:
> o Features
> o BSPs
>
> Features aggregate sources in the form of patches and configuration in
> the form
> of configuration fragments into a modular reusable unit. Features are
> used to
> implement conceptually separate meta-data descriptions like pure
> configuration
> fragments, simple patches, complex features, and kernel types (ktypes).
> Kernel
> types define general kernel features and policy to be reused in the BSPs.
>
> BSPs define hardware-specific features and aggregate them with kernel
> types to
> form the final description of what will be assembled and built.
>
> While the meta-data syntax does not enforce any logical separation of
> configuration fragments, patches, features or kernel types, best practices
> dictate a logical separation of these types of meta-data. The following
> meta-data file hierarchy is recommended:
>
> <base>/
> bsp/
> cfg/
> features/
> ktypes/
> patches/
>
> The bsp directory should contain the BSP descriptions, described in
> detail in
> 3.3.5. The remaining directories all contain "features"; the separation
> is meant
> to aid in conceptualizing their intended usage. A simple guide to determine
> where your scc description file should go is as follows. If it contains only
> configuration fragments, it belongs in cfg. If it contains only source-code
> fixes, it belongs in patches. If it encapsulates a major feature, often
> combining sources and configurations, it belongs in features. If it
> aggregates
> non-hardware configuration and patches in order to define a base kernel
> policy
> or major kernel type to be reused across multiple BSPs, it belongs in
> ktypes.
> The line between these can easily become blurred, especially as out-of-tree
> features are slowly merged upstream over time. Also remember that this
> is purely
> logical organization and has no impact on the functionality of the
> meta-data as
> all of cfg, features, patches, and ktypes, contain "features" as far as the
> Yocto Project Linux kernel tools are concerned.
>
> Paths used in meta-data files are relative to <base> which is either
> FILESEXTRAPATHS if you are creating meta-data in recipe-space (see
> 3.2.1), or
> meta/cfg/kernel-cache/ if you are creating meta-data in-tree (see 3.2.2).
>
> * scc stands for Series Configuration Control, but the naming has less
> significance in the current implementation of the tooling than it had
> in the
> past. Consider it to be a description file.
>
>
> 3.3.1 Configuration
> -------------------
> The simplest unit of meta-data is the configuration-only feature. It
> consists of
> one or more Linux kernel configuration parameters in a .cfg file (as
> described
> in section XYZ) and an scc file describing the fragment. The SMP fragment
> included in the linux-yocto-3.4 git repository consists of the following two
> files:
>
> cfg/smp.scc:
> define KFEATURE_DESCRIPTION "Enable SMP"
> kconf hardware smp.cfg
>
> cfg/smp.cfg:
> CONFIG_SMP=y
> CONFIG_SCHED_SMT=y
>
> See 2.3.1 for details on creating configuration fragments.
>
> KFEATURE_DESCRIPTION provides a short description of the fragment, the
> primary use is for higher level tooling, such as the Yocto Project BSP Tools
> (TODO:Citation).
>
> The "kconf" command is used to include the actual configuration fragment
> in an
> scc file, and the "hardware" keyword identifies the fragment as being
> hardware
> enabling, as opposed to general policy (which would use the keyword
> "non-hardware"). The distinction is made for the benefit of the
> configuration
> validation tools which will warn you if a hardware fragment overrides a
> policy
> set by a non-hardware fragment.
>
> As described in 2.3.1, the following bitbake command can be used to
> audit your
> configuration:
>
> $ bitbake linux-yocto -c kernel_configcheck -f
>
> The description file can include multiple kconf statements, one per
> fragment.
>
>
> 3.3.2 Patches
> -------------
> Patches are described in a very similar way to configuration fragments (see
> 3.3.1). Instead of a .cfg file, they work with source patches. A typical
> patch
> includes a description file and the patch itself:
>
> patches/mypatch.scc:
> patch mypatch.patch
>
> patches/mypatch.patch:
> <typical patch created with 'diff -Nurp' or 'git format-patch'>
>
> The description file can include multiple patch statements, one per patch.
>
>
> 3.3.3 Features
> --------------
> Features are a combination of configuration fragments and patches, or, more
> accurately, configuration fragments and patches are simple forms of a
> feature, a
> more complex meta-data type. In addition to the kconf and patch commands,
> features often aggregate description files with the include command.
>
> A hypothetical example of a feature description file might look like the
> following:
>
> features/myfeature.scc
> define KFEATURE_DESCRIPTION "Enable myfeature"
>
> patch 0001-myfeature-core.patch
> patch 0002-myfeature-interface.patch
>
> include cfg/myfeature_dependency.scc
> kconf non-hardware myfeature.cfg
>
> Features are typically less granular than configuration fragments and
> are more
> likely than configurations fragments and patches to be the types of
> things you
> will want to specify in the KERNEL_FEATURES variable of the Linux kernel
> recipe
> (see 3.1).
>
>
> 3.3.4 Kernel Types
> ------------------
> Kernel types, or ktypes, are used to aggregate all non-hardware
> configuration
> fragments together with any patches you want to use for all Linux kernel
> builds
> of the specified ktype. In short, ktypes are where you define a high-level
> kernel policy. Syntactically, however, they are no different than
> features (see
> 3.3.3). preempt-rt, and tiny. The ktype is selected by the
> LINUX_KERNEL_TYPE
> variable in the recipe (see 3.1).
>
> By way of example, the linux-yocto-3.4 tree defines three ktypes: standard,
> tiny, and preempt-rt. The standard kernel type includes the generic
> Linux kernel
> policy of the Yocto Project linux-yocto kernel recipes. This includes things
> like which filesystems, which networking options, which core kernel
> features,
> and which debugging and tracing optoins are supported. The preempt-rt kernel
> type applies the PREEMPT_RT patches and the configuration options
> required to
> build a real-time Linux kernel. It inherits from standard. The tiny
> kernel type
> is independent from the standard configuration and defines a bare minimum
> configuration meant to serve as a base for very small Linux kernels.
> Tiny does
> not currently include any source changes, but it may in the future.
>
> The standard ktype is defined by standard.scc:
> # Include this kernel type fragment to get the standard features and
> # configuration values.
>
> # Include all standard features
> include standard-nocfg.scc
>
> kconf non-hardware standard.cfg
>
> # individual cfg block section
> include cfg/fs/devtmpfs.scc
> include cfg/fs/debugfs.scc
> include cfg/fs/btrfs.scc
> include cfg/fs/ext2.scc
> include cfg/fs/ext3.scc
> include cfg/fs/ext4.scc
>
> include cfg/net/ipv6.scc
> include cfg/net/ip_nf.scc
> include cfg/net/ip6_nf.scc
> include cfg/net/bridge.scc
>
> As with any scc file, a ktype definition can aggregate other scc files
> with the
> include command, or directly pull in configuration fragments and patches
> with
> the kconf and patch commands, respectively.
>
> Note: It is not strictly necessary to create a ktype scc file. The BSP
> file can
> define the ktype implicitly with a "define KTYPE myktype" line.
> See 3.3.5.
>
>
> 3.3.5 BSPs
> ----------
> BSP descriptions combine kernel types (see 3.3.4) with hardware-specific
> features (see 3.3.3). The hardware specific portion is typically defined
> independently, and then aggregated with each supported kernel type.
> Consider a
> simple example:
>
> mybsp.scc:
> define KMACHINE mybsp
> define KTYPE standard
> define KARCH i386
>
> kconf mybsp.cfg
>
> Every BSP description should include the definition of the KMACHINE,
> KTYPE, and
> KARCH variables. These variables allow the build-system to identify this
> description as meeting the criteria set by the recipe being built. This
> particular description can be said to support the "mybsp" machine for the
> "standard" kernel type and the "i386" architecture. Note that there is
> no hard
> link between the KTYPE and a ktype description file. If you do not have
> kernel
> types defined in your meta-data, you only need to ensure that the recipe
> LINUX_KERNEL_TYPE and the KTYPE here match.
>
> NOTE: future versions of the tooling make the specification of KTYPE in
> the BSP
> optional.
>
> If you did want to separate your kernel policy from your hardware
> configuration,
> you could do so by specifying a kernel type, such as "standard" (see
> 3.3.4) and
> including that description in the BSP description. You might also have
> multiple
> hardware configurations that you aggregate into a single hardware
> description
> file which you could include here, rather than referencing a single .cfg
> file.
> Consider the following:
>
> mybsp.scc:
> define KMACHINE mybsp
> define KTYPE standard
> define KARCH i386
>
> include standard.scc
> include mybsp.scc
>
> In the above example standard.scc aggregates all the configuration
> fragments,
> patches, and features that make up your standard kernel policy whereas
> mybsp.scc
> aggregates all those necessary to support the hardware available on the
> mybsp
> machine. For information on how to break a complete .config into the various
> fragments, see 2.3.1.
>
> Many real-world examples are more complex. Like any other scc file, BSP
> descriptions can aggregate features. Consider the Fish River Island II
> (fri2)
> BSP definitions from the linux-yocto-3.4 repository:
>
> fri2.scc:
> kconf hardware fri2.cfg
>
> include cfg/x86.scc
> include features/eg20t/eg20t.scc
> include cfg/dmaengine.scc
> include features/ericsson-3g/f5521gw.scc
> include features/power/intel.scc
> include cfg/efi.scc
> include features/usb/ehci-hcd.scc
> include features/usb/ohci-hcd.scc
> include features/iwlwifi/iwlwifi.scc
>
> The fri2.scc description file includes a hardware configuration fragment
> (fri2.cfg) specific to the fri2 BSP as well as several more general
> configuration fragments and features enabling hardware found on the
> fri2. This
> description is then included in each of the three machine-ktype descriptions
> (standard, preempt-rt, and tiny). Consider the fri2 standard description:
>
> fri2-standard.scc:
> define KMACHINE fri2
> define KTYPE standard
> define KARCH i386
>
> include ktypes/standard/standard.scc
> branch fri2
>
> git merge emgd-1.14
>
> include fri2.scc
>
> # Extra fri2 configs above the minimal defined in fri2.scc
> include cfg/efi-ext.scc
> include features/drm-emgd/drm-emgd.scc
> include cfg/vesafb.scc
>
> # default policy for standard kernels
> include cfg/usb-mass-storage.scc
>
> Note the "include fri2.scc" line about midway through the file. By
> defining all
> hardware enablement common to the BSP for all kernel types, duplication is
> significantly reduced.
>
> This description introduces a few more variables and commands worthy of
> further
> discussion. Note the "branch" command which is used to create a
> machine-specific branch into which source changes can be applied. With this
> branch set up, the "git merge" command uses the git SCM to merge in a
> feature
> branch "emgd-1.14". This could also be handled with the patch command,
> but for
> commonly used features such as this, feature branches can be a convenient
> mechanism (see 3.5).
>
> Next consider the fri2 tiny description:
>
> fri2-tiny.scc:
> define KMACHINE fri2
> define KTYPE tiny
> define KARCH i386
>
> include ktypes/tiny/tiny.scc
> branch fri2
>
> include fri2.scc
>
> As you might expect, the tiny description includes quite a bit less. In
> fact,
> it includes only the minimal policy defined by the tiny ktype and the
> hardware-specific configuration required for boot and the most basic
> functionality of the system as defined in the base fri2 description
> file. Note
> again the three critical variables: KMACHINE, KTYPE, and KARCH. Of
> these, only
> the KTYPE has changed, now set to "tiny".
>
>
> 3.4 Machine Branches
> --------------------
> Section 3.1 introduced the KBRANCH variable which defines the source
> branch to
> use from the Linux kernel git repository you are using. Many
> linux-yocto-custom
> derived recipes will be using Linux kernel sources with only a single
> branch:
> "master". However, when you are working with multiple boards and
> architectures,
> you are likely to run into the situation where a series of patches are
> needed
> for one board to boot. Sometimes these patches are works in progress or
> fundamentally wrong, yet still necessary for specific boards. In these
> situations, you most likely do not want to include these patches in
> every kernel
> you build. You have a couple of options.
>
> First, you could encapsulate these patches in a feature description and only
> include them in the BSP description for the board(s) that require them (see
> 3.3.2 and 3.3.5).
>
> Alternatively, you can create a branch in your Linux kernel sources and
> apply
> the patches there. You can then specify this new branch as the KBRANCH
> to use
> for this board. You can do this in the recipe with the KBRANCH variable:
>
> KBRANCH = "mynewbranch"
>
> or in the BSP description using the "branch" command:
>
> mybsp.scc:
> define KMACHINE mybsp
> define KTYPE standard
> define KARCH i386
> include standard.scc
>
> branch mynewbranch
>
> include mybsp.scc
>
> The decision of which approach to take, feature or branch, is entirely
> up to you
> and depends on what works best for your development model. If you are
> actively
> working on board support, you may find that working within a branch is more
> practical than trying to continually reintegrate your patches into a
> feature. On
> the other hand, if you are simply reusing some patches from an external
> tree and
> are not working on them, you may find the encapsulated feature to be
> appropriate
> as it does not require the additional complexity of branching in your Linux
> kernel sources.
>
> If you are supporting multiple boards and architectures and find
> yourself with
> numerous branches, you might consider using a hierarchical branching system
> similar to what the linux-yocto Linux kernel repositories use:
>
> <common>/<ktype>/<machine>
>
> If you had two ktypes, standard and small for instance, and three
> machines, your
> git tree might look like this:
>
> common/base
> common/standard/base
> common/standard/machine_a
> common/standard/machine_b
> common/standard/machine_c
> common/small/base
> common/small/machine_a
>
> This organization can help clarify the relationship of the branches to
> each other. In this case, "common/standard/machine_a" would include
> everything in
> "common/base" and "common/standard/base". The "standard" and "small"
> branches
> add sources specific to those kernel types that for whatever reason are not
> appropriate for the other branches.
>
> Note: The "base" branches are an artifact of the way git manages its data
> internally on the filesystem: it will not allow you to use
> "common/standard" and "common/standard/machine_a" because it would
> have to
> create a file and a directory named "standard".
>
>
> 3.5 Feature Branches
> --------------------
> During active development a new feature, it can be more efficient to
> work with
> that feature as a branch, rather than as a set of patches which have to be
> regularly updated. The Yocto Project Linux kernel tools provide for this
> with
> the "git merge" command.
>
> To merge a feature branch into a BSP, insert the "git merge" command
> after any
> branch commands:
>
> mybsp.scc:
> define KMACHINE mybsp
> define KTYPE standard
> define KARCH i386
> include standard.scc
>
> branch mynewbranch
> git merge myfeature
>
> include mybsp.scc
>
>
> 3.6 SCC Reference
> -----------------
> * branch [ref]
>
> Create a new branch relative to the current branch (typically
> ${KTYPE}) using
> the currently checked-out branch, or "ref" if specified.
>
> TODO: Bruce, we need to clarify the "relative to the current branch" bit.
>
> * define
>
> Define variables, such as KMACHINE, KTYPE, KARCH, and
> KFEATURE_DESCRIPTION.
>
> * include SCC_FILE
>
> Include an scc file in the current file. It will be parsed as if inserted
> inline.
>
> * kconf [hardware|non-hardware] CFG_FILE
>
> Queue a configuration fragment for merging into the final Linux
> .config file.
>
> * merge (or "git merge") GIT_BRANCH
>
> Merge the feature branch into the current branch.
>
> * patch PATCH_FILE
>
> Apply the patch to the current git branch.
>
>
> 4. FAQ
> ======
> Q. How do I use my own Linux kernel .config?
> A. See 2.2.2.
>
> Q. How do I create configuration fragments?
> A. See 2.3.1.
>
> Q. How do I use my own Linux kernel sources?
> A. See 2.4.
>
> Q. How do I install/not-install the kernel image on the rootfs?
> A. The kernel image (e.g. vmlinuz) is provided by the kernel-image package.
> Image recipes depend on kernel-base. To specify whether or no the kernel
> image is installed in the generated root filesystem, override
> RDEPENDS_kernel-base to include or not include "kernel-image".
> See (TODO: development manual reference for overrides).
>
> Q. How do I install a specific kernel module?
> A. Linux kernel modules are packaged individually. To ensure a specific
> kernel
> module is included in an image, include it in the appropriate
> machine RRECOMMENDS variable. See the Yocto Project Reference Manual
> glossary for details on:
>
> MACHINE_ESSENTIAL_EXTRA_RDEPENDS
> MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
> MACHINE_EXTRA_RDEPENDS
> MACHINE_EXTRA_RRECOMMENDS
>
> For example, set the following in the qemux86.conf file to include
> the ab123
> kernel modules with images built for the qemux86 machine:
>
> MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123"
>
> See 2.5 for details.
>
> Q. How do I changed the Linux kernel command line?
> A. The Linux kernel command line is typically specified in the machine
> config in
> the APPEND variable. For example, you can add some helpful debug
> information:
>
> APPEND += "printk.time=y initcall_debug debug"
>
>
More information about the yocto
mailing list