This document defines the scope of work requested for the Yocto Project for several different development tasks. The document specifies the deliverables required for each task. The information provided in this document should be used to provide quotations for the required tasks.

The response to this RFQ is expected to provide quotations for the development tasks defined in any of the below sections. Please direct any questions to the Yocto Project Technical Steering Committee at tsc@lists.yoctoproject.org.

Please send quotations to jobs@yoctoproject.org.

 

Background

The Yocto Project faces some challenges. A recent article on linux.com gives some of the background to that. The project has also taken stock of its current situation and where ideally the project would be in the future, this was published here. Given these challenges, the project has been exploring various options to fund work in these areas.

The Yocto Project is both actively recruiting new members and also has a funding application in progress. The project will be announcing several new member companies soon and there are also favorable indications on the funding application although it is not yet guaranteed.

As funds are available, the project would like to engage help to move several of the focus areas of the “future directions” plan that was previously made public. Those topic areas have been distilled into several tasks, containing several specific deliverables which are detailed below. We would like to invite quotations for completing these tasks.

If the funding application was successful, we would be looking to complete several of the tasks here by the year end which would mean executing them in parallel. It is hoped that the project could share these tasks among multiple parties to assist with that timeframe. We appreciate the skill sets needed for some tasks are more specialized than others.

 

Proposed areas of development

We have 8 areas of development. These have been identified as self contained areas of work within specific focus areas where we can write clear deliverables. These are listed below with further details on the following pages.

    • Patchtest – Automated testing of mailing list patches
    • Toaster – A web UI to BitBake
    • Core Workflow – Process Improvements
    • Project Tooling – Feature backlog and bugs
    • Meta-openembedded – embracing the wider OpenEmbedded ecosystem
    • Security – Set up the project and ecosystem for success
    • VSCode IDE Integration – New developer tooling
    • Binary Distro – Enabling faster and easier R&D

 

Patchtest – Automated testing of mailing list patches

Background

The Yocto Project and its components are mailing list based for contributions, similar to the kernel and other older large projects with a diverse set of reviewers. Patchtest refers to the automated testing aspect of the Patchwork tool, which is a tool used to help manage mailing list patch workflows. Other projects which use patchwork include QEMU, Ubuntu kernels, Open vSwitch, OpenBMC, OpenWRT, ALSA and many Linux kernel subsystems.

The basis for Patchtest is to help users with patch submissions with a patch testing service which automatically runs a simple set of tests against patches being submitted. A version was implemented on top of patchwork and ran publicly for a period of time. It was much appreciated by users who found the feedback very helpful in successfully submitting patches against Yocto Project

The patchwork part of the project functioned but was difficult to maintain with recent mailing list changes and it eventually failed. We switched to a new upstream version of patchwork and restored that functionality but lost the patchtest integration

The  tests themselves remain as relevant now as when they were written, we need to integrate patchtest into the new patchwork to re-institute this valuable user service.

Rationale

Projects have changed structure over time and new contributors are much more used to GitHub pull request-style workflows. Our patch/mailing list workflow does have value but we need to do more to help new users adjust to it. By having a way to run simple tests against their potential submissions we encourage them to make better submissions and become active members of the community rather than just consumers.

Secondly, having the basic tests and an automated way of running them removes this burden from the maintainers themselves. It means they don’t need to try and spot that class of issues, or repeat replies about how to fix those issues either.

If common issues are occurring, further tests can be added to reduce maintainer burden and help educate users too.

By addressing these issues, we make it easier to bring on new contributors. The solutions may also be applicable to other mailing list/patch based projects as we share a common struggle on good tooling to support this workflow compared to some of the web developments. There is therefore potential for other projects to benefit from improved automated testing support.

Deliverables

  1. Move tests to OE-Core so submitters can run directly

This means users can run the tests locally even if there was a problem with the mailing list public service. This also means users have the opportunity to ensure there are no obvious mistakes before posting the patches publicly.

  1. Enable a service running tests against mailing list submissions

Automating this means users get a fast response to ‘easy’ issues and the maintainers can focus on other things.

 

Toaster – A web UI to BitBake

Background

Toaster is a web-based graphical interface to BitBake. It was originally developed and contributed by a member organization with significant input from user experience designers based upon user research data. The UI was a good step towards making it easier for less experienced users to work with BitBake. Core work was completed but the second phase involving testing and incorporating user feedback was never completed and the interface has never achieved its potential or widespread usage as a result.

Rationale

Bitbake and OpenEmbedded are extremely complex tools so anything which helps present them to users in an easier to understand way is good for the project and also helps users be more productive. Some users are much more comfortable and able to understand graphically presented information than through text consoles.

Much of the work is already done here, the aim of this work would be to enable automated testing so we can maintain what we have, and then secondly make some smaller changes to allow better workflows in standalone use of the tool. This would allow the UI to be used as a query tool against existing builds.

Deliverables

  1. Ensure toaster works with current code base
  2. Update automated tests to current technology
  3. Ensure automated testing occurs on autobuilder
  4. Enable and document standalone workflows and support existing builds being queried with the UI

 

Core Workflow – Process Improvements

Background

The project builds everything from source by default. This means it has a reputation for being slow and heavy. There are ways the project can accelerate this which means faster workflows and improved developer experience but these are currently cumbersome to use.

Rationale

The project aims to defragment customised embedded Linux. This is important as if we succeed at this, it gives benefits to the wider ecosystem through making it easier to inject security fixes and an ability to share and collaborate without re-inventing the wheel.

To do this, we need to provide best-in-class support and compete against binary distributions for usability and speed. One way we can do this is provide better support for binary artifacts via our sstate mechanism. We do already have some of this functionality in our “extensible SDK” or “eSDK”.

Deliverables

  1. Enable a public sstate mirror via a content delivery network (CDN) and populate using the autobuilder
  2. Ensure CDN sstate is reused under normal use case scenarios, particularly for slow components like rust-native. Identify any common sstate mismatch causes. Ensure test cases are added to cover the use cases and prevent regressions.
  3. Add lock and unlock commands to allow specific components to be locked down to specific sstate checksums or allow them to vary
  4. Allow switching between eSDK and non-eSDK modes
  5. Add tooling so we can understand why something is rebuilding when it isn’t expected to.

 

Project Tooling – Feature backlog and bugs

Background

The project has a number of core tools designed to aid common developer workflows. The open source world continually changes with major additions like new languages  such as Rust, and new user workflows. Our tools have not seen active development in a few years and are falling behind, both technically and compared to new user expectations. There are complex recipes where the tools are currently unable to work correctly.

There are also several components such as pseudo and our user/group management code where there are significant problematic bug backlogs.

Devtool – a companion tool to bitbake which allows developers to perform “operations” against recipes such as modifying them to change source code or upgrading them.

Recipetool – another companion tool which handles generation of recipes for software

Pseudo – an emulation environment/tool which allows our builds to run as a single unprivileged user yet create files with varied ownership and permissions

Rationale

By making the tools fully featured and allowing them to operate on a wider range of recipes, we improve user experience for new developers to match expectations. It also encourages existing developers to use them, which means they’re more likely to fix bugs and keep them up to date.  Where there are complex cases the tools can’t operate in, we should at least identify and report this to the user to also improve the user experience.

Adapting to modern developments, particularly in more recent runtime languages allows the project to match user expectations and be seen as keeping with modern trends and the requirements of new users to the project.

Components like pseudo and our user/group support underpin the entire build process so fixing the backlog of issues in this area ensures the project is working on solid foundations.

Deliverables

  1. Improve devtool support for multiple git repositories and git submodules, complex recipes (such as u-boot, edk2) and better support for appends and overrides in recipes (includes bugs 14070 and 14141). Include automated tests.
  2. Adapt/improve recipetool tool support for common modules in modern runtime languages such as go, perl, python and rust by addressing open bugs in these areas. Include automated tests.
  3. Pseudo tool backlog (bugs 14385, 15023, 15074, proof of concept of filesystem inclusion instead of exclusion)
  4. User/group bug backlog (bugs 12107, 13419, 13904, 14961, 13279)
  5. Add proof of concept to extend runtime image testing to include a QEMU screen dump and comparison to check Sato UI started correctly and basic applications appear correctly (need to mask clock).

(bug numbers are referenced against https://bugzilla.yoctoproject.org/)

 

‘Meta-openembedded’ – embracing the wider OpenEmbedded ecosystem

Background

OpenEmbedded Core is a layer containing build instructions for many pieces of core software that go into most Linux systems. OpenEmbedded-Core has modelled best practices for quality and processes and receives significant development from the Yocto Project.

Meta-OpenEmbedded is a “catch all” layer of various pieces of software often needed for specific use cases. The Yocto Project shares “openembedded-core” with OpenEmbedded but the “meta-openembedded” layer is something which has traditionally been left to the OpenEmbedded community to develop and maintain. This layer contains many different recipes and the quality is highly variable.

In most products produced by users of both OpenEmbedded and Yocto Project, both layers will end up being used. There is a problem with stale recipes in meta-openembedded but no one has taken the time to establish a strong enough deprecation policy.

Rationale

Many developments have been made in OpenEmbedded Core that could help the meta-openembedded layer’s quality. Applying these processes and techniques to this key layer would raise the quality of the whole ecosystem.

The proposal would be to strengthen meta-openembedded by adopting changes, tests and processes made by OE-Core, improving automated testing and establishing the missing policies.

By doing this, we would reduce the variability of user experience with both projects, as well as demonstrate the benefit of these processes and tests to a much wider audience. The quality improvements should benefit the majority of users of both projects.

Deliverables

  1. Add CVE checking as modelled in OE-Core (automated report)
  2. Enable upgrade status checking for meta-oe as modelled in OE-Core
  3. Allow patch-status checks to be enabled per layer and enable for meta-openembedded
  4. Identify a list of meta-openembedded recipes where ptest could be added and implement some key commonly used examples
  5. Mirror sources for LTS releases for meta-oe layer
  6. Propose a deprecation policy for meta-openembedded and how deprecated recipes could move to some form of archive, with provision/policy for re-adding them.
  7. Report the reproducibility status of recipes in meta-openembedded, fixing several key recipes as examples

 

Security – Setup the project and ecosystem for success

Background/Rationale

Security is understandably a focus for companies and governments, and legislation is also increasingly focusing on this. Since many places derive their OS from the Yocto Project, we’re also well placed to inject security fixes at the root. There is also an opportunity to model best practices in areas like Software Bill of Materials (SBoM) and reduce burden on the wider ecosystem by providing standardized tools ‘out of the box’.

Deliverables

  1. Identify security processes the project should follow and document in the project documentation manuals.
  2. Implement the processes identified, providing training and setting up infrastructure as needed.
  3. Establish and train any security team in the processes identified.
  4. Implement proof of concept of upcoming SPDX 3.0 draft standard.
  5. Enable SPDX manifest generation as a project default.

 

VSCode IDE Integration – New developer tooling

Background

VS Code is a popular editor and IDE tool. There is an extension already for Yocto Project usage however the current maintainer  is looking for someone to take over (https://github.com/EugenWiens/vscode-bitbake/pull/41). The plugin features are also limited, there is much more integration work that could be done.

Rationale

The new generation of developers have different expectations and are more used to close editor integration and support. Yocto Project is an older, more traditional project which tends to use console based workflows. By improving the experience within editors like VSCode, we should attract and encourage a new generation of users who would go on to rejuvenate our ecosystem.

Deliverables

The proposal would be to:

  1. Take over maintenance of the existing syntax/completion plugin
  2. Improve syntax parser to:
    1. correctly identify shell/python code blocks 
    2. correctly identify class/configuration/recipe files
  3. Improve accuracy of completion handling (e.g. inherit statement)
  4. Add and enable automated testing through test cases
  5. Assess the feature set and enable any key missing editor functionality
  6. Extend the features to cover SDK usage
  7. Extend the features to cover extensible SDK (eSDK) usage
  8. Document missing functionality for full IDE usage
  9. Extend the features to cover full IDE usage for building simple meson based application
  10. Allow time for fixing issues identified with user testing and world usage

 

Binary Distro – Enabling faster and easier R&D

Background

Yocto Project is a source based solution which means software has to be built before developers can use it. Binary distributions are faster and more convenient to use in some use cases since building from source isn’t required.

The project does have mechanisms such as ‘shared state’ which avoid some level of source building but this still isn’t as convenient as an on target package manager installation or deploying a ready built container.

The project is able to build binary packages, package feeds and a binary distribution as an output but the project has never published its own set of binaries. It hasn’t been clear which configurations and platforms to target or how to handle change requests.

Rationale

We believe that the overhead in maintaining such a reference binary feed has become more feasible over time as our tools and testing infrastructure have advanced. Such a reference feed would be valuable to the community, particularly those using the project for quick R&D work where they don’t need to build everything from source.

This work item would target adding the missing workflow pieces, tools and processes to allow the creation of a binary distro proof of concept. It would then be for the wider community to support and drive it, if there was value seen in it. Regardless, improving the ability to create such distributions would be in keeping with the project objectives and stop others having to duplicate that work, often sub-optimally.

These missing workflow pieces are also fundamental building blocks to other functionality such as binary containers and lowering barriers to keeping devices up to date so they are useful in other ways as well.

Deliverables

  1. Define the scope of a binary distro prototype (which recipes and machines to target)
  2. Add automated testing of upgrading an image from our previous release to the current version using a package feed
  3. Implement PR server functionality to handle “pass through” mode and hierarchy data
  4. Automated testing of ability to assemble images quickly from sstate/PRServ/Hashequiv data
  5. Propose policies/requirements on how a binary reference distro for the project would behave. Need to include proposed policies for including new recipes and extending to cover new platforms/architectures.

 

Quotation

The response to this RFQ is expected to provide quotations for the development tasks defined in any of the above sections. Please direct any questions to the Yocto Project Technical Steering Committee at tsc@lists.yoctoproject.org.

Please send quotations to jobs@yoctoproject.org.

Update 12th July 2023

We’ve had a number of responses and questions to the RFQ, thanks! Based upon those there are a few things we felt we should add:
  • We’re asking for quotations by 24th July.
  • The project and TSC is acutely aware code needs ongoing maintenance. The funding source we’re looking at for this work imposes some constraints on the structure of the tasks and needs specific measurable deliverables. Ongoing maintenance isn’t something we could find a way to work into this funding so it isn’t included.
  • To mitigate against this to some degree, there is a focus on automated testing in the deliverables. This is based on the observation and experience of the areas of the project which sustain easily and have lower regression rates compared to areas of the project where we have significant regressions or in some cases complete loss of functionality over time. Whilst this doesn’t remove the need for ongoing maintenance, the hope is that it, combined with wide appeal and use of the changes will allow the community to sustain the code going forward.
  • We’re ideally looking for quotations for the totality of a specific topic area. The project can only access funding once all the deliverables in an area are completed.
  • The setup of the CDN is out of scope of most people and the project does have plans in progress for this. It is therefore reasonable to assume the project will set this up so the scope of work from the quotation perspective is in testing it.
  • The deliverable are sometimes non-specific and this is partly to allow for issues encountered during development as we can’t know how things might work out.
    It would be fine, advantageous in fact, to quote for allowing us to meet the baseline deliberable but also document stretch goals with some time to be spent against them. An example could be reproducibility for meta-oe. Having automated running of the reproducibilty tests and a report for that layer should be straightforward and a few hours work since we already can do it for OE-Core (ignoring the time actually running the build would take). Analysis and patches fixing say, 5 of the most commonly occurring issues could be a nice to have objective. If patching wasn’t feasible, at last the issues could be documented/summarised.
  • We’re hoping to complete the items in this RFQ in Q3 and Q4 this year with the exception of the binary distro and VSCode/IDE tasks which we anticipate running into Q1 next year.