Getting Started: The Yocto Project® Overview

The Yocto Project is an open source collaboration project that helps developers create custom Linux-based systems for embedded products, regardless of the hardware architecture. The project provides a flexible set of tools and a space where embedded developers worldwide can share technologies, software stacks, configurations and best practices which can be used to create tailored Linux images for embedded devices.

The project provides a standard to delivering hardware support and software stacks, allowing the interchange of software configurations and builds. The tools allow users to build and support customizations for multiple hardware platforms and software stacks in a maintainable and scalable way.

Historically the project grew from, and works with the OpenEmbedded Project which is where the build system and some of the meta-data are derived.

The Yocto Project combines, maintains and validates three (3) key development elements.

  1. A set of integrated tools to make working with embedded Linux successful, including tools for automated building and testing, processes for board support and license compliance, and component information for custom Linux-based embedded operating systems
  2. A reference embedded distribution (called Poky)
  3. The OpenEmbedded build system, co-maintained with the OpenEmbedded Project

There are many different open source components and tools within the Yocto Project umbrella.

Poky, the reference embedded OS is actually a working BUILD EXAMPLE which will build a small embedded OS with the included build system (BitBake, the build engine and OpenEmbedded-Core, the core build system metadata).

The build system is downloaded with Poky build instruction "files" called recipes and layers (defined below). You can alter, copy, or use the Poky build specifics in any way you might need to in order to create your custom embedded Linux.

The Layer Model - The Key to Customization

Yocto Project has a development model for embedded and IOT Linux creation which distinguishes it from other simple build systems. It is called the Layer Model.

The Layer Model is designed to support both collaboration and customization at the same time. Layers are repositories containing related sets of instructions which tell the build system what to do. Users can collaborate, share, and reuse layers. Layers can contain changes to previous instructions or settings at any time.

This powerful override capability is what allows you to customize previous collaborative or community supplied layers to suit your product requirements.

Use different layers to logically separate information in your build. As an example, you could have a BSP layer, a GUI layer, a distro configuration, middleware, or an application. Putting your entire build into one layer limits and complicates future customization and reuse. Isolating information into layers, on the other hand, helps keep simplify future customizations and reuse. Use BSP layers from silicon vendors when possible.

Familiarize yourself with the curated (tested) Yocto Project Compatible Layer Index . There is also the OpenEmbedded layer index which contains more layers but the content is less universally validated.

Components and Tools Maintained by the Yocto Project

There is a collection of components and tools that are used by the project to maintain and update the actual project. There are also those used directly by the project itself. Finally, other components and tools are used by developers to create their custom OSs. These components and tools are open source projects themselves, or metadata. Both are separate from the reference distribution and build system and most are downloaded separately.

More About Components and Tools

TERMS for Reference

  • Configuration Files: Files which hold global definitions of variables, user defined variables and hardware configuration information. They tell the build system what to build and put into the image to support a particular platform.
  • Recipe: The most common form of metadata. A recipe will contain a list of settings and tasks (instructions) for building packages which are then used to build the binary image. A recipe describes where you get source code and which patches to apply. Recipes describe dependencies for libraries or for other recipes, as well as configuration and compilation options. They are stored in layers.
  • Layer: A collection of related recipes. Layers allow you to consolidate related metadata to customize your build, and isolate information for multiple architecture builds. Layers are hierarchical in their ability to override previous specifications. You can include any number of available layers from the Yocto Project and customize the build by adding your layers after them. The Layer Index is searchable for layers within Yocto Project.
  • Metadata: A key element of the Yocto Project is the meta-data which is used to construct a Linux distribution, contained in the files that the build system parses when building an image. In general, Metadata includes recipes, configuration files and other information refering to the build instructions themselves, as well as the data used to control what things get built and to affect how they are built. The meta-data also includes commands and data used to indicate what versions of software are used, and where they are obtained from, as well as changes or additions to the software itself (patches or auxiliary files) which are used to fix bugs or customize the software for use in a particular situation. OpenEmbedded Core is an important set of validated metadata.
  • OpenEmbedded-Core: oe-core is meta-data comprised of foundation recipes, classes and associated files that are meant to be common among many different OpenEmbedded-derived systems, including the Yocto Project. It is a curated subset of an original repository developed by the OpenEmbedded community which has been pared down into a smaller, core set of continuously validated recipes resulting in a tightly controlled and an quality-assured core set of recipes.
  • Poky: A reference embedded distribution and a reference test configuration created to 1) provide a base level functional distro which can be used to illustrate how to customize a distribution, 2) to test the Yocto Project components, Poky is used to validate Yocto Project, and 3) as a vehicle for users to download Yocto Project. Poky is not a product level distro, but a good starting point for customization. Poky is an integration layer on top of oe-core.
  • Build System - "Bitbake": a scheduler and execution engine which parses instructions (recipes) and configuration data. It then creates a dependency tree to order the compilation, schedules the compilation of the included code, and finally, executes the building of the specified, custom Linux image (distribution). BitBake is a make-like build tool. BitBake recipes specify how a particular package is built. They include all the package dependencies, source code locations, configuration, compilation, build, install and remove instructions. Recipes also store the metadata for the package in standard variables. Related recipes are consolidated into a layer. During the build process dependencies are tracked and native or cross-compilation of the package is performed. As a first step in a cross-build setup, the framework will attempt to create a cross-compiler toolchain (Extensible SDK) suited for the target platform.
  • Packages: The output of the build system used to create your final image.
  • Extensible Software Development Kit (ESDK): A custom SDK for application developers that allows them to incorporate their library and programming changes back into the image to make their code available to other apps developers.
  • Image: A binary form of a Linux distribution (operating system) intended to be loaded onto a device.

The General "Workflow" - How it All Works

  • To begin, developers specify architecture, policies, patches and configuration details.
  • The build system then fetches and downloads the source code from where ever specified. The project supports standard methods such as tarballs or source code repositories systems such as git.
  • Once downloaded, the sources are extracted into a local work area where patches are applied and common steps for configuring and compiling the software will be run.
  • The software is then installed into a temporary staging area where the binary package format you select (deb, rpm, or ipk) will be used to roll up the software.
  • Different QA and sanity checks are run throughout entire build process.
  • After the binaries are created, a binary package feed is generated which is then used to create the final root file image.
  • The file system image is generated.

When using Yocto Project, this workflow will change depending on what components and tools are actually being used.

Development Environment - Host Setup

1) Most developers use a Linux development host as described in the Quick Start Guide for Systems Developers.

2) For other operating systems, the Yocto Project provides CROPS, an open source, cross-platform development framework that leverages Docker containers. CROPS provides an easily managed, extensible environment allowing developers to build binaries for a variety of architectures on Windows or macOS in addition to native Linux hosts. For more information, see the Development Tasks Manual.

3) A third option is to use toaster, a web interface with useful but limited capability that allows you configure and run builds, and provides information and statistics about the build process. See the Toaster Manual.

Once your development host is set up to use the Yocto Project, the Eclipse IDE is also available for use.

Reference Embedded Distribution (Poky)

"Poky" is the name of the Yocto Project's reference distribution or Reference OS Kit. It contains the build system (BitBake and OpenEmbedded Core) as well as a set of metadata to get you started building your own distro.

To use the Yocto Project tools, you can download Poky and use it to bootstrap your own distribution. Note that Poky does not contain binary files - it is a working example of how to build your own custom Linux distribution from source.

More About the Reference Distro

Yocto Project Features

Apart from inherent capabilities, the Yocto Project has project features which are introduced in one release, and generally expanded on and completed over the course of a few releases. In-depth information about the features can be found in the "readme" files associated with releases and tools. This area provides a high-level overview of feature updates to the 3 latest releases and general project features as well.

High Level Feature Table