[yocto] New Documentation: Linux Kernel Development with the Yocto Project

[Stewart, David C]

A good read, thanks!

Dave

On 12/12/12 12:44 AM, "Darren Hart" <dvhart at linux.intel.com> wrote:

>In response to much feedback on the linux-yocto recipes and the associated
>kernel tools, we have made a number of improvements in an attempt to
>make them
>more accessible. In particular, the tools now allow for using your own
>sources
>and configurations in a fairly simple manner, while providing a path to
>more
>complex meta-data management.
>
>The following document is intended to demystify the Yocto Project Linux
>Kernel
>Tools now that some of this work has been done.  Ultimately, the goal of
>this
>document is to provide a "task-based" approach to developing Linux kernel
>recipes and meta-data with the Yocto Project. It should present
>information more
>or less in the order a developer is likely to need it. Start off by
>changing a
>config and adding a patch, then using your own sources, incorporating
>out of tree
>drivers, then diving into more complex configuration and source
>management.
>
>This document has seen some review prior to this announcement. I wanted
>to thank
>everyone for the feedback provided. I have done my best to incorporate
>it. In a
>couple cases I felt the additions were better left to "How Do I" wiki
>pages or a
>separate manual (such as the Development Manual or the BSP Guide). The
>following
>is what I am submitting to Scott R, our tech writer, for a full review and
>docbookification for release with the 1.4 Yocto Project release.
>Eventually I
>would like to see the following sections added to the document.
>
>1) Workflow
>   * Group environements
>   * Local sources
>2) Advanced Source Management
>   * working outside of bitbake
>   * generating the git tree from the meta-data
>3) Examples
>
>For now, I feel this document should go a long way towards clarifying
>the use of
>the kernel tools and the linux-yocto recipes.
>
>I would still greatly appreciate any additional feedback and there is
>still
>plenty of time to incorporate that feedback prior to release. So if you
>haven't
>already and have the inclination, I'd welcome your review!
>
>
>
>Linux Kernel Development with the Yocto Project
>
>
>Table of Contents
>=================
>INTRODUCTION
>  Yocto Project Linux Kernel Recipes
>
>COMMON TASKS
>  Preparing a Layer
>  Modifying an Existing Recipe (Mostly covered by the Dev Manual)
>  Applying Patches
>  Changing the Config
>  Iterative Development
>    Generating Configuration Files
>    Modifying Source Code
>  Working with Your Own Sources
>  Incorporating Out-of-tree Modules
>
>ADVANCED META-DATA
>  Using Meta-Data in a Recipe
>  Meta-Data Location
>    Recipe-Space Meta-Data
>    In-Tree Meta-Data
>  Meta-Data Syntax
>    Configuration
>    Patches
>    Features
>    Kernel Types
>    BSPs
>  Machine Branches
>  Feature Branches
>  SCC Reference
>
>EXAMPLES
>
>FAQ (Maybe put this online as a wiki?)
>  How do I use my own Linux kernel .config?
>  How do I create configuration fragments?
>  How do I use my own Linux kernel sources?
>  How do I install/not-install the kernel image on the rootfs?
>  How do install a specific kernel module?
>  How do I changed the Linux kernel command line?
>
>
>1 Introduction
>==============
>Regardless of how you intend to make use of the Yocto Project, chances
>are you
>are going to need to work with the Linux kernel. The Yocto Project
>provides a
>powerful set of tools for managing Linux kernel sources and
>configuration data.
>If you want to make a single configuration change, apply a couple of
>patches, or
>work with your own sources, the Yocto Project has tooling in place to
>help you
>do it.
>
>
>1.1 Yocto Project Linux Kernel Recipes
>--------------------------------------
>Each Yocto Project release introduces a new set of linux-yocto kernel
>recipes,
>tracking the latest upstream developments and introducing newly supported
>platforms. In addition to the new kernel recipes, the previous recipes are
>refreshed and supported for at least one additional release. As they
>align,
>these previous releases are updated to include the latest from the LTSI
>project.
>Besides these recipes, a linux-yocto-dev recipe is available for working
>with
>the very latest in upstream Linux kernel development as well as meta-data
>development.
>
>If you do not maintain your own kernel sources and need to make only
>minimal
>changes to the sources, these recipes provide a vetted base upon which
>to layer
>your changes. Doing so allows you to benefit from the continual kernel
>integration and testing performed during development of the Yocto Project.
>
>If, instead, you have a very specific Linux kernel source tree and are
>unable to
>align with one of the many official linux-yocto releases, an alternative
>exists by which you can use the Yocto Project Linux kernel tools with
>your own
>sources.
>
>The sections that follow provide instructions for completing specific
>Linux
>kernel development tasks. They assume familiarity with working with
>bitbake
>recipes and basic open-source development tools. Understanding these
>concepts
>will facilitate the process of working with the kernel recipes. If you
>find you
>need some additional background, please be sure to review and understand
>the
>following documentation:
>
>o Yocto Project Getting Started Guide
>o Yocto Project Development Manual
>  - 4.3 Modifying Temporary Source Code
>  - 5.1 Understanding and Creating Layers
>  - 4.1.2 Modifying the Kernel
>    (This section should be retired, as well as 5.6, 5.7 - with this
>document
>     providing the necessary information)
>
>Finally, while this document focuses on the manual creation of recipes,
>patches,
>and configuration files, the Yocto Project BSP tools are available to
>automate
>this process with existing content and work well to create the initial
>framework
>and boilerplate code. For details, refer to the Yocto Project BSP
>Developer's
>Guide.
>
>
>2 Common Tasks
>==============
>
>
>2.1 Preparing a Layer
>---------------------
>Customizing recipes is best done in a layer with bbappend files. Layers
>also
>provide a convenient mechanism to create your own recipes. This guide
>assumes
>you will be working from within a layer independent from those released
>with the
>Yocto Project. For details on how to create and work with layers, refer to
>section 5.1 Understanding and Creating Layers in the Yocto Project
>Development
>Manual.
>
>(Kernel specific directions in 5.7.4)
>
>
>2.2 Modifying an Existing Recipe
>--------------------------------
>In many cases, you can customize an existing linux-yocto recipe to meet
>the
>needs of your project. Each release of the Yocto Project provides a few
>Linux
>kernel recipes to choose from. To get started, prepare a layer and a
>bbappend
>file corresponding to the recipe you wish to modify.
>
>See [New example in Dev Manual] for instructions to create a minimal
>layer.
>
>The bbappend will typical be located at the following location relative
>to the
>layer (if modifying the linux-yocto_3.4.bb recipe):
>
>	recipes-kernel/linux/linux-yocto_3.4.bbappend
>
>And should contain the following text initially:
>
>	FILESEXTRAPATHS := "${THISDIR}/${PN}"
>
>The ${PN} will expand to "linux-yocto" in this example. Any new files
>added to
>modify the recipe should then be added to the following path within the
>layer:
>
>	recipes-kernel/linux/linux-yocto/
>
>NOTE: If you are working on a new machine BSP, be sure to refer to the
>Yocto
>Project Board Support Package Developer's Guide.
>
>
>2.2.1 Applying Patches
>----------------------
>If you have a patch, or a small series of patches, to apply to the Linux
>kernel
>source, you can do so just as you would with any other recipe. You first
>copy
>the patches to the path added to FILESEXTRAPATHS in the bbappend file as
>described in 2.2 and then reference them in the SRC_URI.
>
>For example, you can apply a three patch series by adding the following
>lines to
>your linux-yocto bbappend file in your layer:
>
>	SRC_URI += "file://0001-first-change.patch"
>	SRC_URI += "file://0002-first-change.patch"
>	SRC_URI += "file://0003-first-change.patch"
>
>At the next build, bitbake will detect the change in the recipe and
>fetch and
>apply the patches before rebuilding the Linux kernel.
>
>
>2.2.2 Changing the Config
>-------------------------
>Making wholesale or incremental changes to the Linux kernel config can
>be made
>by including a defconfig or configuration fragments in the SRC_URI.
>
>If you have a complete Linux kernel .config file you want to use, copy it
>as
>"defconfig" to the ${FILES} directory and add the following line to your
>linux-yocto bbappend file in your layer:
>
>	SRC_URI += "file://defconfig"
>
>Generally speaking, the preferred approach is to determine the incremental
>change you want to make and add that as a fragment. For example, if you
>wanted
>to add support for a basic serial console, create a file named
>"8250.cfg" in the
>${FILES} directory with the following content (without indentation):
>
>	CONFIG_SERIAL_8250=y
>	CONFIG_SERIAL_8250_CONSOLE=y
>	CONFIG_SERIAL_8250_PCI=y
>	CONFIG_SERIAL_8250_NR_UARTS=4
>	CONFIG_SERIAL_8250_RUNTIME_UARTS=4
>	CONFIG_SERIAL_CORE=y
>	CONFIG_SERIAL_CORE_CONSOLE=y
>
>Then include this configuration fragment in the SRC_URI:
>
>	SRC_URI += "file://8250.cfg"
>
>At the next build, bitbake will detect the change in the recipe and
>fetch and
>apply the new configuration before rebuilding the Linux kernel.
>
>
>2.3 Iterative Development
>-------------------------
>If you do not have existing patches or configuration files, you can easily
>generate them from within the bitbake build environment, as will be
>described
>below. As you do, running previously completed bitbake tasks will cause
>bitbake
>to invalidate the tasks that follow them in the build sequence, causing
>them to
>rebuild at the next invocation of the build. Throughout this section, be
>sure to
>substitute "linux-yocto" with the name of the Linux kernel recipe you are
>working with.
>
>
>2.3.1 Generating Configuration Files
>------------------------------------
>You can manipulate the config used to build a linux-yocto recipe with the
>"menuconfig" command.
>
>	$ bitbake linux-yocto -c menuconfig
>
>This will start the Linux kernel configuration tool, allowing you to
>prepare a
>new .config for the build. When you exit, be sure to save the changes when
>prompted. The resulting .config file will be located in the ${WORKDIR}
>under
>the linux-${MACHINE}-${KTYPE}-build directory. You can use this in its
>entirety
>as the defconfig file described in 2.2.2.
>
>Better still, would be to use the "menuconfig" command and take the
>difference
>of the new .config file with the previous one to generate a configuration
>fragment. To do this, be sure to complete a build at least through the
>kernel
>configuration task:
>
>	$ bitbake linux-yocto -c kernel_configme -f
>
>Then make a copy of the .config file, calling it "config.orig", for
>example. Run
>the "menuconfig" command as described above. Finally, prepare a
>configuration
>fragment from the difference between the files. Ultimately you want a
>list of
>Linux kernel CONFIG_ assignments, and not something in diff format.
>Something
>like the following will do that automatically (but plan on reviewing the
>output
>as you can usually remove some of the defaults):
>
>	$ diff -Nurp config.orig .config | sed -n "s/^\+//p" > frag.cfg
>
>You can use the output as a configuration fragment described in 2.2.2.
>This
>method can also be used to define a BSP configuration fragment (See
>3.3.5).
>
>The Yocto Project kernel tools provide some configuration validation
>tools which
>will warn when a configuration you requested does not appear in the final
>config, or when you override a policy configuration in a hardware
>configuration
>fragment. You can run these tools with the following command:
>
>	$ bitbake linux-yocto -c kernel_configcheck -f
>
>	...
>
>	NOTE: validating kernel configuration
>	This BSP sets 3 invalid/obsolete kernel options.
>	These config options are not offered anywhere within this kernel.
>	The full list can be found in your kernel src dir at:
>	meta/cfg/standard/mybsp/invalid.cfg
>
>	This BSP sets 21 kernel options that are possibly non-hardware related.
>	The full list can be found in your kernel src dir at:
>	meta/cfg/standard/mybsp/specified_non_hdw.cfg
>
>	WARNING: There were 2 hardware options requested that do not
>	         have a corresponding value present in the final ".config" file.
>	         This probably means you aren't getting the config you wanted.
>	The full list can be found in your kernel src dir at:
>	meta/cfg/standard/mybsp/mismatch.cfg
>
>The various problems that you can encounter are described in the output
>along
>with where to find the offending configuration items. You can use these
>logs to
>adjust your configuration files and repeat the "kernel_configme" and
>"kernel_configcheck" commands until no warnings are produced.
>
>
>2.3.2 Modifying Source Code
>---------------------------
>You can experiment with source code changes and create a simple patch
>without
>leaving the bitbake environment. To get started, be sure to complete a
>build at
>least through the kernel configuration task:
>
>	$ bitbake linux-yocto -c kernel_configme -f
>
>This step will ensure you have the sources prepared and the configuration
>completed. You will find the sources in the ${WORKDIR}/linux directory.
>
>You can edit the sources as you would any other Linux source tree, but
>keep in
>mind that your changes will be lost if you trigger the fetch task for the
>recipe. Avoid this by not issuing the "cleanall" or "cleansstate", or
>forcing
>the "fetch" command. Also be sure not to modify the recipe itself while
>working
>with temporary changes or bitbake may run the fetch command (depending
>on the
>changes to the recipe).
>
>To test your temporary changes, instruct bitbake to run the compile
>again. The
>-f option forces the command to run again even though bitbake may think
>it has
>already done so:
>
>	$ bitbake linux-yocto -c compile -f
>
>If the compile fails, you can update the sources and repeat the compile
>command. Once it compiles successfully, you can inspect and test the
>resulting
>build (kernel, modules, etc.) from the build directory at
>${WORKDIR}/linux-${MACHINE}-${KTYPE}-build. Alternatively, you can run the
>deploy command to place the kernel image in the tmp/deploy/images
>directory:
>
>	$ bitbake linux-yocto -c deploy
>
>And of course, you can run through all the remaining installation and
>packaging
>steps by issuing:
>
>	$ bitbake linux-yocto
>
>For rapid iterative development, the edit-compile-repeat loop is
>preferable to
>rebuilding the entire recipe as the installation and packaging tasks are
>very
>time consuming.
>
>Once you are happy with your modifications, you can make these permanent
>by
>generating patches and applying them to the SRC_URI as described in
>section
>2.2.1 Applying Patches. If you are not familiar with generating patches,
>refer
>to the Yocto Project Development Manual, section 5.7.3 Creating the Patch.
>
>
>2.4 Working with Your Own Sources
>---------------------------------
>If you find yourself unable to work with one of the Linux kernel versions
>supported by existing linux-yocto recipes, you can still make use of the
>Yocto
>Project Linux kernel tooling while working with your own sources. You
>will not
>be able to leverage the existing meta-data and stabilization work of the
>linux-yocto sources, but you will be able to manage your own meta-data
>in the
>same format as the linux-yocto sources which will facilitate converging
>with
>linux-yocto on a future mutually-supported kernel version.
>
>The linux-yocto-custom recipe, located in the poky repository at:
>
>	meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
>
>is provided as an example recipe which uses kernel.org sources and the
>Yocto
>Project Linux kernel tools for managing meta-data. To get started, copy
>this
>recipe to your layer and provide it with a meaningful name, such as
>linux-yocto-myproject_3.5.bb, where 3.5 is the base version of the Linux
>kernel
>you will be working with. In the same directory, create a matching
>directory,
>e.g. linux-yocto-myproject to store your patches and configuration files.
>
>Edit the following variables in the recipe as appropriate for your
>project:
>
>	o SRC_URI
>	o LINUX_VERSION
>	o LINUX_VERSION_EXTENSION
>	o SRCREV
>	o PR
>	o PV
>	o COMPATIBLE_MACHINE
>
>The SRC_URI should be a git repository, using one of the supported git
>fetcher
>protocols (file, git, http, etc.). The skeleton recipe provides an example
>SRC_URI as a syntax reference.
>
>Set LINUX_VERSION to the Linux kernel version you are using, such as
>"3.6.3".
>
>LINUX_VERSION_EXTENSION is used to define the Linux kernel
>CONFIG_LOCALVERSION
>which will be compiled in to the resulting kernel and visible via the
>uname
>command.
>
>Set SRCREV to the commit ID you wish to build from.
>
>Treat the PR as you would the PR of any other recipe. Increment it to
>indicate
>to the build system that the recipe has changed.
>
>The default PV assignment is typically adequate. It combines the
>LINUX_VERSION
>with the SCM revision (from the SRCPV variable) and results in a string
>something like:
>
>"3.4.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c6
>0d32f0d770931e3cf343e2"
>
>While lengthy, this extra verbosity helps ensure you are building from
>the exact
>sources you intend.
>
>Finally, the default COMPATIBLE_MACHINE assignment for
>linux-yocto-custom is set
>to a regular expression matching only the empty string, "(^$)". This will
>trigger an explicit build failure. You must change it to match a list of
>the
>machines supported by your new recipe, such as "(qemux86|qemux86-64)"
>
>With that in place, you can continue to customize this recipe as you
>would the
>existing linux-yocto recipes. See Section 2.2 Modifying an Existing
>Recipe for
>details.
>
>
>2.5 Incorporating Out-of-tree Modules
>-------------------------------------
>While it is always preferable to work with sources integrated into the
>Linux
>kernel sources, if you have need of an external kernel module, the
>hello-mod
>recipe is available as a template to create your own out-of-tree Linux
>kernel
>module recipe. It is available in the poky repository at:
>
>	meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb
>
>To get started, copy this recipe to your layer and provide it with a
>meaningful name, such as
>mymodule_1.0.bb. In the same directory, create a directory named "files"
>where
>you can store any source files, patches, or other files necessary for
>building
>the module which do not come with the sources. Finally, update the recipe
>as
>appropriate for the module. Typically you will need to set the following
>variables:
>
>	o DESCRIPTION
>	o LICENSE*
>	o SRC_URI
>	o PV
>
>Depending on the build system used by the module sources, you may need
>to make
>adjustments. For example, a typical module Makefile will look much like
>that
>provided with hello-mod:
>
>	obj-m := hello.o
>	
>	SRC := $(shell pwd)
>	
>	all:
>		$(MAKE) -C $(KERNEL_SRC) M=$(SRC)
>	
>	modules_install:
>		$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install
>	...
>
>The important point to note here is the KERNEL_SRC variable. The module
>bbclass
>class sets this, as well as KERNEL_PATH, to ${STAGING_KERNEL_DIR} with the
>necessary Linux kernel build information to build modules. If your module
>Makefile uses a different variable, you may want to override the
>do_compile()
>step, or create a patch to the Makefile to work with the more typical
>KERNEL_SRC
>or KERNEL_PATH variables.
>
>After you have prepared the recipe, you will likely want to include the
>module
>in your images. To do this, see the documentation for the following
>variables in
>the Yocto Project Reference Manual and set one of them as appropriate in
>your
>machine config file:
>
>	MACHINE_ESSENTIAL_EXTRA_RDEPENDS
>	MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
>	MACHINE_EXTRA_RDEPENDS
>	MACHINE_EXTRA_RRECOMMENDS
>
>As modules are often not required for boot and may be excluded from
>certain
>build configurations, the following allows for the most flexibility:
>
>	MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule"
>
>Where the value is derived by appending the module filename without the
>.ko
>extension to the string "kernel-module-".
>
>As it is an RRECOMMENDS (and not an RDEPENDS) variable, the build will
>not fail
>if this module is not available to include in the image.
>
>
>3 Advanced Meta-Data
>====================
>In addition to configuration fragments and patches, the Yocto Project
>kernel
>tools support rich meta-data which you can use to define complex
>policies and
>BSP support. The purpose of the meta-data and the tools to manage it,
>known as
>the kern-tools (kern-tools-native_git.bb), is to assist in managing the
>complexity of the configuration and sources in support of multiple BSPs
>and
>Linux kernel types.
>
>In particular, the kernel tools allow you to specify only what you must,
>and
>nothing more.  Where a complete Linux kernel .config includes all the
>automatically selected CONFIG options, the configuration fragments only
>need to
>contain the highest level visible CONFIG options as presented by the Linux
>kernel menuconfig system. This reduces your maintenance effort and
>allows you
>to further separate your configuration in ways that make sense for your
>project.
>A common split is policy and hardware. For example, all your kernels may
>support
>the proc and sys filesystems, but only specific boards will require
>sound, usb,
>or specific drivers. Specifying these individually allows you to
>aggregate them
>together as needed, but maintain them in only one place. Similar logic
>applies
>to source changes.
>
>
>3.1 Using Meta-Data in a Recipe
>-------------------------------
>The meta-data provided with any linux-yocto style Linux kernel sources
>must
>define a BSP that corresponds to the definition laid out in the recipe.
>A BSP
>consists of an aggregation of kernel policy and hardware specific feature
>enablement. This can be influenced from within the recipe.
>
>Every linux-yocto style recipe must define the following variables:
>
>	KMACHINE
>
>KMACHINE is typically set to the same value as used within the
>recipe-space BSP
>definition, such as "routerstationpro" or "fri2". However, multiple BSPs
>can
>reuse the same KMACHINE name if they are built using the same BSP
>description
>(see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples
>of such
>a situation where each specifies KMACHINE as "fri2".
>
>They may optionally define the following variables:
>	KBRANCH
>	KERNEL_FEATURES
>	KBRANCH_DEFAULT
>	LINUX_KERNEL_TYPE
>
>KBRANCH_DEFAULT defines the default source branch within the Linux
>kernel source
>repository to be used to build the Linux kernel. It is used as the
>default value
>for KBRANCH which may define an alternate branch, typically with a machine
>override, such as:
>
>KBRANCH_fri2 = "standard/fri2"
>
>Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master".
>
>LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the
>configuration and defaults to "standard" if you do not specify otherwise.
>Together with KMACHINE, this defines the search arguments used by the
>Yocto
>Project Linux kernel tools to find the appropriate description within the
>meta-data with which to build out the sources and configuration. The
>linux-yocto
>recipes define "standard", "tiny", and "preempt-rt" kernel types. See
>3.3.4 for
>more inforation on kernel types.
>
>During the build, the kern-tools will search for the BSP description
>file that
>most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the
>recipe. It will use the first BSP description it finds matching both
>variables.
>Failing that it will issue a warning such as the following:
>
>	WARNING: Can't find any BSP hardware or required configuration fragments.
>	WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
>	         meta/cfg/broken/fri2-broken/required_frags.txt in directory:
>	         meta/cfg/broken/fri2-broken
>
>	In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE
>	was set to "broken".
>
>It will then search first for the KMACHINE and then
>for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will
>use the
>sources from the KBRANCH and any configuration specified in the SRC_URI.
>
>KERNEL_FEATURES can be used to include features (configuration fragments,
>patches, or both) that are not already included by the KMACHINE and
>LINUX_KERNEL_TYPE combination. To include a feature specified as
>"features/netfilter.scc" for example, specify:
>
>KERNEL_FEATURES += "features/netfilter.scc"
>
>To include a feature called "cfg/sound.scc" just for the qemux86 machine,
>specify:
>
>KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"
>
>The value of the entries in KERNEL_FEATURES are dependent on their
>location
>within the meta-data itself. The examples here are taken from the
>linux-yocto-3.4 repository where "features" and "cfg" are subdirectories
>of the
>meta-data directory. For details, see 3.3.
>
>	Note: The processing of the these variables has evolved some between the
>	      0.9 and 1.3 releases of the Yocto Project and associated
>	      kern-tools sources. The above is accurate for 1.3 and later
>	      releases of the Yocto Project.
>
>
>3.2 Meta-Data Location
>----------------------
>This meta-data can be defined along with the Linux kernel recipe
>(recipe-space)
>as partially described in section 2.2 as well as within the Linux kernel
>sources
>themselves (in-tree).
>
>Where you choose to store the meta-data depends on what you want to do
>and how
>you intend to work. If you are unfamiliar with the Linux kernel and only
>wish
>to apply a config and possibly a couple of patches provided to you by
>others,
>you may find the recipe-space mechanism to be easier to work with. This
>is also
>a good approach if you are working with Linux kernel sources you do not
>control
>or if you just don't want to maintain a Linux kernel git repository on
>your own.
>
>If you are doing active kernel development and are already maintaining a
>Linux
>kernel git repository of your own, you may find it more convenient to
>work with
>the meta-data in the same repository as the Linux kernel sources. This
>can make
>iterative development of the Linux kernel more efficient outside of the
>bitbake
>environment.
>
>Regardless of where the meta-data is stored, the syntax as
>described in the following sections applies equally.
>
>
>3.2.1 Recipe-Space Meta-Data
>----------------------------
>When stored in recipe-space, the meta-data files reside in a directory
>hierarchy
>below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a
>linux-yocto or linux-yocto-custom derived Linux kernel recipe. See 2.2.
>
>By way of example, a trivial tree of meta-data stored in recipe-space
>within a
>BSP layer might look like the following:
>
>meta/
>`-- recipes-kernel
>    `-- linux
>        `-- linux-yocto
>            |-- bsp-standard.scc
>            |-- bsp.cfg
>            `-- standard.cfg
>
>When the meta-data is stored in recipe-space, you must take steps to
>ensure
>bitbake has the necessary information to decide which files to fetch and
>when
>they need to be fetched again.
>
>It is only necessary to specify the .scc files on the SRC_URI; bitbake
>will
>parse them and fetch any files referenced in the .scc files by the
>include,
>patch, or kconf commands. Because of this, it is necessary to bump the
>recipe PR
>value when changing the content of files not explicitly listed in the
>SRC_URI.
>
>
>3.2.2 In-Tree Meta-Data
>-----------------------
>When stored in-tree, the meta-data files reside in the "meta" directory
>of the
>Linux kernel sources. They may be present in the same branch as the
>sources,
>such as "master", or in their own orphan branch, typically named "meta".
>An
>orphan branch in git is a branch with unique history and content to the
>other
>branches in the repository. This is useful to track meta-data changes
>independently from the sources of the Linux kernel, while still keeping
>them
>together in the same repository. For the purposes of this document, we
>will
>discuss all in-tree meta-data as residing below the
>"meta/cfg/kernel-cache"
>directory.
>
>By way of example, a trivial tree of meta-data stored in a custom Linux
>kernel
>git repository might look like the following:
>
>meta/
>`-- cfg
>    `-- kernel-cache
>        |-- bsp-standard.scc
>        |-- bsp.cfg
>        `-- standard.cfg
>
>To use a specific branch for the meta-data, specify the branch in the
>KMETA
>variable in your Linux kernel recipe, for example:
>
>	KMETA = "meta"
>
>To use the same branch as the sources, set KMETA to the empty string:
>
>	KMETA = ""
>
>If you are working with your own sources and want to create an orphan meta
>branch, you can do so using the following commands from within your
>Linux kernel
>git repository:
>
>	$ git checkout --orphan meta
>	$ git rm -rf .
>	$ git commit --allow-empty -m "Create orphan meta branch"
>
>
>3.3 Meta-Data Syntax
>--------------------
>The Yocto Project Linux kernel tools meta-data consists of three primary
>types
>of files: scc* description files, configuration fragments, and patches.
>The scc
>files define variables and include or otherwise reference any of the
>three file
>types. The description files are used to aggregate all types of
>meta-data into
>what ultimately describes the sources and the configuration required to
>build a
>Linux kernel tailored to a specific machine.
>
>The scc description files are used to define two fundamental types of
>meta-data:
>	o Features
>	o BSPs
>
>Features aggregate sources in the form of patches and configuration in
>the form
>of configuration fragments into a modular reusable unit. Features are
>used to
>implement conceptually separate meta-data descriptions like pure
>configuration
>fragments, simple patches, complex features, and kernel types (ktypes).
>Kernel
>types define general kernel features and policy to be reused in the BSPs.
>
>BSPs define hardware-specific features and aggregate them with kernel
>types to
>form the final description of what will be assembled and built.
>
>While the meta-data syntax does not enforce any logical separation of
>configuration fragments, patches, features or kernel types, best practices
>dictate a logical separation of these types of meta-data. The following
>meta-data file hierarchy is recommended:
>
>	<base>/
>		bsp/
>		cfg/
>		features/
>		ktypes/
>		patches/
>
>The bsp directory should contain the BSP descriptions, described in
>detail in
>3.3.5. The remaining directories all contain "features"; the separation
>is meant
>to aid in conceptualizing their intended usage.  A simple guide to
>determine
>where your scc description file should go is as follows. If it contains
>only
>configuration fragments, it belongs in cfg. If it contains only
>source-code
>fixes, it belongs in patches. If it encapsulates a major feature, often
>combining sources and configurations, it belongs in features. If it
>aggregates
>non-hardware configuration and patches in order to define a base kernel
>policy
>or major kernel type to be reused across multiple BSPs, it belongs in
>ktypes.
>The line between these can easily become blurred, especially as
>out-of-tree
>features are slowly merged upstream over time. Also remember that this
>is purely
>logical organization and has no impact on the functionality of the
>meta-data as
>all of cfg, features, patches, and ktypes, contain "features" as far as
>the
>Yocto Project Linux kernel tools are concerned.
>
>Paths used in meta-data files are relative to <base> which is either
>FILESEXTRAPATHS if you are creating meta-data in recipe-space (see
>3.2.1), or
>meta/cfg/kernel-cache/ if you are creating meta-data in-tree (see 3.2.2).
>
>* scc stands for Series Configuration Control, but the naming has less
>  significance in the current implementation of the tooling than it had
>in the
>  past. Consider it to be a description file.
>
>
>3.3.1 Configuration
>-------------------
>The simplest unit of meta-data is the configuration-only feature. It
>consists of
>one or more Linux kernel configuration parameters in a .cfg file (as
>described
>in section XYZ) and an scc file describing the fragment. The SMP fragment
>included in the linux-yocto-3.4 git repository consists of the following
>two
>files:
>
>cfg/smp.scc:
>	define KFEATURE_DESCRIPTION "Enable SMP"
>	kconf hardware smp.cfg
>
>cfg/smp.cfg:
>	CONFIG_SMP=y
>	CONFIG_SCHED_SMT=y
>
>See 2.3.1 for details on creating configuration fragments.
>
>KFEATURE_DESCRIPTION provides a short description of the fragment, the
>primary use is for higher level tooling, such as the Yocto Project BSP 
>Tools
>(TODO:Citation).
>
>The "kconf" command is used to include the actual configuration fragment
>in an
>scc file, and the "hardware" keyword identifies the fragment as being
>hardware
>enabling, as opposed to general policy (which would use the keyword
>"non-hardware"). The distinction is made for the benefit of the
>configuration
>validation tools which will warn you if a hardware fragment overrides a
>policy
>set by a non-hardware fragment.
>
>As described in 2.3.1, the following bitbake command can be used to
>audit your
>configuration:
>
>	$ bitbake linux-yocto -c kernel_configcheck -f
>
>The description file can include multiple kconf statements, one per
>fragment.
>
>
>3.3.2 Patches
>-------------
>Patches are described in a very similar way to configuration fragments 
>(see
>3.3.1). Instead of a .cfg file, they work with source patches. A typical
>patch
>includes a description file and the patch itself:
>
>patches/mypatch.scc:
>	patch mypatch.patch
>
>patches/mypatch.patch:
>	<typical patch created with 'diff -Nurp' or 'git format-patch'>
>
>The description file can include multiple patch statements, one per patch.
>
>
>3.3.3 Features
>--------------
>Features are a combination of configuration fragments and patches, or, 
>more
>accurately, configuration fragments and patches are simple forms of a
>feature, a
>more complex meta-data type. In addition to the kconf and patch commands,
>features often aggregate description files with the include command.
>
>A hypothetical example of a feature description file might look like the
>following:
>
>features/myfeature.scc
>	define KFEATURE_DESCRIPTION "Enable myfeature"
>
>	patch 0001-myfeature-core.patch
>	patch 0002-myfeature-interface.patch
>
>	include cfg/myfeature_dependency.scc
>	kconf non-hardware myfeature.cfg
>
>Features are typically less granular than configuration fragments and
>are more
>likely than configurations fragments and patches to be the types of
>things you
>will want to specify in the KERNEL_FEATURES variable of the Linux kernel
>recipe
>(see 3.1).
>
>
>3.3.4 Kernel Types
>------------------
>Kernel types, or ktypes, are used to aggregate all non-hardware
>configuration
>fragments together with any patches you want to use for all Linux kernel
>builds
>of the specified ktype. In short, ktypes are where you define a high-level
>kernel policy. Syntactically, however, they are no different than
>features (see
>3.3.3).  preempt-rt, and tiny. The ktype is selected by the
>LINUX_KERNEL_TYPE
>variable in the recipe (see 3.1).
>
>By way of example, the linux-yocto-3.4 tree defines three ktypes: 
>standard,
>tiny, and preempt-rt. The standard kernel type includes the generic
>Linux kernel
>policy of the Yocto Project linux-yocto kernel recipes. This includes 
>things
>like which filesystems, which networking options, which core kernel
>features,
>and which debugging and tracing optoins are supported. The preempt-rt 
>kernel
>type applies the PREEMPT_RT patches and the configuration options
>required to
>build a real-time Linux kernel. It inherits from standard. The tiny
>kernel type
>is independent from the standard configuration and defines a bare minimum
>configuration meant to serve as a base for very small Linux kernels.
>Tiny does
>not currently include any source changes, but it may in the future.
>
>The standard ktype is defined by standard.scc:
>	# Include this kernel type fragment to get the standard features and
>	# configuration values.
>	
>	# Include all standard features
>	include standard-nocfg.scc
>	
>	kconf non-hardware standard.cfg
>	
>	# individual cfg block section
>	include cfg/fs/devtmpfs.scc
>	include cfg/fs/debugfs.scc
>	include cfg/fs/btrfs.scc
>	include cfg/fs/ext2.scc
>	include cfg/fs/ext3.scc
>	include cfg/fs/ext4.scc
>	
>	include cfg/net/ipv6.scc
>	include cfg/net/ip_nf.scc
>	include cfg/net/ip6_nf.scc
>	include cfg/net/bridge.scc
>
>As with any scc file, a ktype definition can aggregate other scc files
>with the
>include command, or directly pull in configuration fragments and patches
>with
>the kconf and patch commands, respectively.
>
>Note: It is not strictly necessary to create a ktype scc file. The BSP
>file can
>      define the ktype implicitly with a "define KTYPE myktype" line.
>See 3.3.5.
>
>
>3.3.5 BSPs
>----------
>BSP descriptions combine kernel types (see 3.3.4) with hardware-specific
>features (see 3.3.3). The hardware specific portion is typically defined
>independently, and then aggregated with each supported kernel type.
>Consider a
>simple example:
>
>mybsp.scc:
>	define KMACHINE mybsp
>	define KTYPE standard
>	define KARCH i386
>
>	kconf mybsp.cfg
>
>Every BSP description should include the definition of the KMACHINE,
>KTYPE, and
>KARCH variables. These variables allow the build-system to identify this
>description as meeting the criteria set by the recipe being built. This
>particular description can be said to support the "mybsp" machine for the
>"standard" kernel type and the "i386" architecture. Note that there is
>no hard
>link between the KTYPE and a ktype description file. If you do not have
>kernel
>types defined in your meta-data, you only need to ensure that the recipe
>LINUX_KERNEL_TYPE and the KTYPE here match.
>
>NOTE: future versions of the tooling make the specification of KTYPE in
>the BSP
>      optional.
>
>If you did want to separate your kernel policy from your hardware
>configuration,
>you could do so by specifying a kernel type, such as "standard" (see
>3.3.4) and
>including that description in the BSP description. You might also have
>multiple
>hardware configurations that you aggregate into a single hardware
>description
>file which you could include here, rather than referencing a single .cfg
>file.
>Consider the following:
>
>mybsp.scc:
>	define KMACHINE mybsp
>	define KTYPE standard
>	define KARCH i386
>
>	include standard.scc
>	include mybsp.scc
>
>In the above example standard.scc aggregates all the configuration
>fragments,
>patches, and features that make up your standard kernel policy whereas
>mybsp.scc
>aggregates all those necessary to support the hardware available on the
>mybsp
>machine. For information on how to break a complete .config into the 
>various
>fragments, see 2.3.1.
>
>Many real-world examples are more complex. Like any other scc file, BSP
>descriptions can aggregate features. Consider the Fish River Island II
>(fri2)
>BSP definitions from the linux-yocto-3.4 repository:
>
>fri2.scc:
>	kconf hardware fri2.cfg
>	
>	include cfg/x86.scc
>	include features/eg20t/eg20t.scc
>	include cfg/dmaengine.scc
>	include features/ericsson-3g/f5521gw.scc
>	include features/power/intel.scc
>	include cfg/efi.scc
>	include features/usb/ehci-hcd.scc
>	include features/usb/ohci-hcd.scc
>	include features/iwlwifi/iwlwifi.scc
>
>The fri2.scc description file includes a hardware configuration fragment
>(fri2.cfg) specific to the fri2 BSP as well as several more general
>configuration fragments and features enabling hardware found on the
>fri2. This
>description is then included in each of the three machine-ktype 
>descriptions
>(standard, preempt-rt, and tiny). Consider the fri2 standard description:
>
>fri2-standard.scc:
>	define KMACHINE fri2
>	define KTYPE standard
>	define KARCH i386
>	
>	include ktypes/standard/standard.scc
>	branch fri2
>	
>	git merge emgd-1.14
>	
>	include fri2.scc
>	
>	# Extra fri2 configs above the minimal defined in fri2.scc
>	include cfg/efi-ext.scc
>	include features/drm-emgd/drm-emgd.scc
>	include cfg/vesafb.scc
>	
>	# default policy for standard kernels
>	include cfg/usb-mass-storage.scc
>
>Note the "include fri2.scc" line about midway through the file. By
>defining all
>hardware enablement common to the BSP for all kernel types, duplication is
>significantly reduced.
>
>This description introduces a few more variables and commands worthy of
>further
>discussion.  Note the "branch" command which is used to create a
>machine-specific branch into which source changes can be applied. With 
>this
>branch set up, the "git merge" command uses the git SCM to merge in a
>feature
>branch "emgd-1.14".  This could also be handled with the patch command,
>but for
>commonly used features such as this, feature branches can be a convenient
>mechanism (see 3.5).
>
>Next consider the fri2 tiny description:
>
>fri2-tiny.scc:
>	define KMACHINE fri2
>	define KTYPE tiny
>	define KARCH i386
>	
>	include ktypes/tiny/tiny.scc
>	branch fri2
>	
>	include fri2.scc
>
>As you might expect, the tiny description includes quite a bit less. In
>fact,
>it includes only the minimal policy defined by the tiny ktype and the
>hardware-specific configuration required for boot and the most basic
>functionality of the system as defined in the base fri2 description
>file. Note
>again the three critical variables: KMACHINE, KTYPE, and KARCH. Of
>these, only
>the KTYPE has changed, now set to "tiny".
>
>
>3.4 Machine Branches
>--------------------
>Section 3.1 introduced the KBRANCH variable which defines the source
>branch to
>use from the Linux kernel git repository you are using. Many
>linux-yocto-custom
>derived recipes will be using Linux kernel sources with only a single
>branch:
>"master". However, when you are working with multiple boards and
>architectures,
>you are likely to run into the situation where a series of patches are
>needed
>for one board to boot. Sometimes these patches are works in progress or
>fundamentally wrong, yet still necessary for specific boards. In these
>situations, you most likely do not want to include these patches in
>every kernel
>you build. You have a couple of options.
>
>First, you could encapsulate these patches in a feature description and 
>only
>include them in the BSP description for the board(s) that require them 
>(see
>3.3.2 and 3.3.5).
>
>Alternatively, you can create a branch in your Linux kernel sources and
>apply
>the patches there. You can then specify this new branch as the KBRANCH
>to use
>for this board. You can do this in the recipe with the KBRANCH variable:
>	
>	KBRANCH = "mynewbranch"
>	
>or in the BSP description using the "branch" command:
>
>mybsp.scc:
>	define KMACHINE mybsp
>	define KTYPE standard
>	define KARCH i386
>	include standard.scc
>
>	branch mynewbranch
>
>	include mybsp.scc
>
>The decision of which approach to take, feature or branch, is entirely
>up to you
>and depends on what works best for your development model. If you are
>actively
>working on board support, you may find that working within a branch is 
>more
>practical than trying to continually reintegrate your patches into a
>feature. On
>the other hand, if you are simply reusing some patches from an external
>tree and
>are not working on them, you may find the encapsulated feature to be
>appropriate
>as it does not require the additional complexity of branching in your 
>Linux
>kernel sources.
>
>If you are supporting multiple boards and architectures and find
>yourself with
>numerous branches, you might consider using a hierarchical branching 
>system
>similar to what the linux-yocto Linux kernel repositories use:
>
>	<common>/<ktype>/<machine>
>
>If you had two ktypes, standard and small for instance, and three
>machines, your
>git tree might look like this:
>
>	common/base
>	common/standard/base
>	common/standard/machine_a
>	common/standard/machine_b
>	common/standard/machine_c
>	common/small/base
>	common/small/machine_a
>
>This organization can help clarify the relationship of the branches to
>each other. In this case, "common/standard/machine_a" would include
>everything in
>"common/base" and "common/standard/base". The "standard" and "small"
>branches
>add sources specific to those kernel types that for whatever reason are 
>not
>appropriate for the other branches.
>
>Note: The "base" branches are an artifact of the way git manages its data
>      internally on the filesystem: it will not allow you to use
>      "common/standard" and "common/standard/machine_a" because it would
>have to
>      create a file and a directory named "standard".
>
>
>3.5 Feature Branches
>--------------------
>During active development a new feature, it can be more efficient to
>work with
>that feature as a branch, rather than as a set of patches which have to be
>regularly updated. The Yocto Project Linux kernel tools provide for this
>with
>the "git merge" command.
>
>To merge a feature branch into a BSP, insert the "git merge" command
>after any
>branch commands:
>
>mybsp.scc:
>	define KMACHINE mybsp
>	define KTYPE standard
>	define KARCH i386
>	include standard.scc
>
>	branch mynewbranch
>	git merge myfeature
>
>	include mybsp.scc
>
>
>3.6 SCC Reference
>-----------------
>* branch [ref]
>
>  Create a new branch relative to the current branch (typically
>${KTYPE}) using
>  the currently checked-out branch, or "ref" if specified.
>
>  TODO: Bruce, we need to clarify the "relative to the current branch" 
>bit.
>
>* define
>
>  Define variables, such as KMACHINE, KTYPE, KARCH, and
>KFEATURE_DESCRIPTION.
>
>* include SCC_FILE
>
>  Include an scc file in the current file. It will be parsed as if 
>inserted
>  inline.
>
>* kconf [hardware|non-hardware] CFG_FILE
>
>  Queue a configuration fragment for merging into the final Linux
>.config file.
>
>* merge (or "git merge") GIT_BRANCH
>
>  Merge the feature branch into the current branch.
>
>* patch PATCH_FILE
>
>  Apply the patch to the current git branch.
>
>
>4. FAQ
>======
>Q. How do I use my own Linux kernel .config?
>A. See 2.2.2.
>
>Q. How do I create configuration fragments?
>A. See 2.3.1.
>
>Q. How do I use my own Linux kernel sources?
>A. See 2.4.
>
>Q. How do I install/not-install the kernel image on the rootfs?
>A. The kernel image (e.g. vmlinuz) is provided by the kernel-image 
>package.
>   Image recipes depend on kernel-base. To specify whether or no the 
>kernel
>   image is installed in the generated root filesystem, override
>   RDEPENDS_kernel-base to include or not include "kernel-image".
>   See (TODO: development manual reference for overrides).
>
>Q. How do I install a specific kernel module?
>A. Linux kernel modules are packaged individually. To ensure a specific
>kernel
>   module is included in an image, include it in the appropriate
>   machine RRECOMMENDS variable. See the Yocto Project Reference Manual
>   glossary for details on:
>
>	MACHINE_ESSENTIAL_EXTRA_RDEPENDS
>	MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
>	MACHINE_EXTRA_RDEPENDS
>	MACHINE_EXTRA_RRECOMMENDS
>
>   For example, set the following in the qemux86.conf file to include
>the ab123
>   kernel modules with images built for the qemux86 machine:
>
>	MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123"
>
>   See 2.5 for details.
>
>Q. How do I changed the Linux kernel command line?
>A. The Linux kernel command line is typically specified in the machine
>config in
>   the APPEND variable. For example, you can add some helpful debug
>information:
>
>   APPEND += "printk.time=y initcall_debug debug"
>
>
>-- 
>Darren Hart
>Intel Open Source Technology Center
>Yocto Project - Technical Lead - Linux Kernel