This document defines the scope of work requested for the Yocto Project for several different development tasks. The document specifies the deliverables required for each task. The information provided in this document should be used to provide quotations for the required tasks.
The response to this RFQ is expected to provide quotations for the development tasks defined in any of the below sections. Please direct any questions to the Yocto Project Technical Steering Committee at tsc@lists.yoctoproject.org.
Please send quotations to jobs@yoctoproject.org.
The Yocto Project faces some challenges. A recent article on linux.com gives some of the background to that. The project has also taken stock of its current situation and where ideally the project would be in the future, this was published here. Given these challenges, the project has been exploring various options to fund work in these areas.
The Yocto Project is both actively recruiting new members and also has a funding application in progress. The project will be announcing several new member companies soon and there are also favorable indications on the funding application although it is not yet guaranteed.
As funds are available, the project would like to engage help to move several of the focus areas of the “future directions” plan that was previously made public. Those topic areas have been distilled into several tasks, containing several specific deliverables which are detailed below. We would like to invite quotations for completing these tasks.
If the funding application was successful, we would be looking to complete several of the tasks here by the year end which would mean executing them in parallel. It is hoped that the project could share these tasks among multiple parties to assist with that timeframe. We appreciate the skill sets needed for some tasks are more specialized than others.
We have 8 areas of development. These have been identified as self contained areas of work within specific focus areas where we can write clear deliverables. These are listed below with further details on the following pages.
Background
The Yocto Project and its components are mailing list based for contributions, similar to the kernel and other older large projects with a diverse set of reviewers. Patchtest refers to the automated testing aspect of the Patchwork tool, which is a tool used to help manage mailing list patch workflows. Other projects which use patchwork include QEMU, Ubuntu kernels, Open vSwitch, OpenBMC, OpenWRT, ALSA and many Linux kernel subsystems.
The basis for Patchtest is to help users with patch submissions with a patch testing service which automatically runs a simple set of tests against patches being submitted. A version was implemented on top of patchwork and ran publicly for a period of time. It was much appreciated by users who found the feedback very helpful in successfully submitting patches against Yocto Project
The patchwork part of the project functioned but was difficult to maintain with recent mailing list changes and it eventually failed. We switched to a new upstream version of patchwork and restored that functionality but lost the patchtest integration
The tests themselves remain as relevant now as when they were written, we need to integrate patchtest into the new patchwork to re-institute this valuable user service.
Rationale
Projects have changed structure over time and new contributors are much more used to GitHub pull request-style workflows. Our patch/mailing list workflow does have value but we need to do more to help new users adjust to it. By having a way to run simple tests against their potential submissions we encourage them to make better submissions and become active members of the community rather than just consumers.
Secondly, having the basic tests and an automated way of running them removes this burden from the maintainers themselves. It means they don’t need to try and spot that class of issues, or repeat replies about how to fix those issues either.
If common issues are occurring, further tests can be added to reduce maintainer burden and help educate users too.
By addressing these issues, we make it easier to bring on new contributors. The solutions may also be applicable to other mailing list/patch based projects as we share a common struggle on good tooling to support this workflow compared to some of the web developments. There is therefore potential for other projects to benefit from improved automated testing support.
Deliverables
This means users can run the tests locally even if there was a problem with the mailing list public service. This also means users have the opportunity to ensure there are no obvious mistakes before posting the patches publicly.
Automating this means users get a fast response to ‘easy’ issues and the maintainers can focus on other things.
Background
Toaster is a web-based graphical interface to BitBake. It was originally developed and contributed by a member organization with significant input from user experience designers based upon user research data. The UI was a good step towards making it easier for less experienced users to work with BitBake. Core work was completed but the second phase involving testing and incorporating user feedback was never completed and the interface has never achieved its potential or widespread usage as a result.
Rationale
Bitbake and OpenEmbedded are extremely complex tools so anything which helps present them to users in an easier to understand way is good for the project and also helps users be more productive. Some users are much more comfortable and able to understand graphically presented information than through text consoles.
Much of the work is already done here, the aim of this work would be to enable automated testing so we can maintain what we have, and then secondly make some smaller changes to allow better workflows in standalone use of the tool. This would allow the UI to be used as a query tool against existing builds.
Deliverables
Background
The project builds everything from source by default. This means it has a reputation for being slow and heavy. There are ways the project can accelerate this which means faster workflows and improved developer experience but these are currently cumbersome to use.
Rationale
The project aims to defragment customised embedded Linux. This is important as if we succeed at this, it gives benefits to the wider ecosystem through making it easier to inject security fixes and an ability to share and collaborate without re-inventing the wheel.
To do this, we need to provide best-in-class support and compete against binary distributions for usability and speed. One way we can do this is provide better support for binary artifacts via our sstate mechanism. We do already have some of this functionality in our “extensible SDK” or “eSDK”.
Deliverables
Background
The project has a number of core tools designed to aid common developer workflows. The open source world continually changes with major additions like new languages such as Rust, and new user workflows. Our tools have not seen active development in a few years and are falling behind, both technically and compared to new user expectations. There are complex recipes where the tools are currently unable to work correctly.
There are also several components such as pseudo and our user/group management code where there are significant problematic bug backlogs.
Devtool – a companion tool to bitbake which allows developers to perform “operations” against recipes such as modifying them to change source code or upgrading them.
Recipetool – another companion tool which handles generation of recipes for software
Pseudo – an emulation environment/tool which allows our builds to run as a single unprivileged user yet create files with varied ownership and permissions
Rationale
By making the tools fully featured and allowing them to operate on a wider range of recipes, we improve user experience for new developers to match expectations. It also encourages existing developers to use them, which means they’re more likely to fix bugs and keep them up to date. Where there are complex cases the tools can’t operate in, we should at least identify and report this to the user to also improve the user experience.
Adapting to modern developments, particularly in more recent runtime languages allows the project to match user expectations and be seen as keeping with modern trends and the requirements of new users to the project.
Components like pseudo and our user/group support underpin the entire build process so fixing the backlog of issues in this area ensures the project is working on solid foundations.
Deliverables
(bug numbers are referenced against https://bugzilla.yoctoproject.org/)
Background
OpenEmbedded Core is a layer containing build instructions for many pieces of core software that go into most Linux systems. OpenEmbedded-Core has modelled best practices for quality and processes and receives significant development from the Yocto Project.
Meta-OpenEmbedded is a “catch all” layer of various pieces of software often needed for specific use cases. The Yocto Project shares “openembedded-core” with OpenEmbedded but the “meta-openembedded” layer is something which has traditionally been left to the OpenEmbedded community to develop and maintain. This layer contains many different recipes and the quality is highly variable.
In most products produced by users of both OpenEmbedded and Yocto Project, both layers will end up being used. There is a problem with stale recipes in meta-openembedded but no one has taken the time to establish a strong enough deprecation policy.
Rationale
Many developments have been made in OpenEmbedded Core that could help the meta-openembedded layer’s quality. Applying these processes and techniques to this key layer would raise the quality of the whole ecosystem.
The proposal would be to strengthen meta-openembedded by adopting changes, tests and processes made by OE-Core, improving automated testing and establishing the missing policies.
By doing this, we would reduce the variability of user experience with both projects, as well as demonstrate the benefit of these processes and tests to a much wider audience. The quality improvements should benefit the majority of users of both projects.
Deliverables
Background/Rationale
Security is understandably a focus for companies and governments, and legislation is also increasingly focusing on this. Since many places derive their OS from the Yocto Project, we’re also well placed to inject security fixes at the root. There is also an opportunity to model best practices in areas like Software Bill of Materials (SBoM) and reduce burden on the wider ecosystem by providing standardized tools ‘out of the box’.
Deliverables
Background
VS Code is a popular editor and IDE tool. There is an extension already for Yocto Project usage however the current maintainer is looking for someone to take over (https://github.com/EugenWiens/vscode-bitbake/pull/41). The plugin features are also limited, there is much more integration work that could be done.
Rationale
The new generation of developers have different expectations and are more used to close editor integration and support. Yocto Project is an older, more traditional project which tends to use console based workflows. By improving the experience within editors like VSCode, we should attract and encourage a new generation of users who would go on to rejuvenate our ecosystem.
Deliverables
The proposal would be to:
Background
Yocto Project is a source based solution which means software has to be built before developers can use it. Binary distributions are faster and more convenient to use in some use cases since building from source isn’t required.
The project does have mechanisms such as ‘shared state’ which avoid some level of source building but this still isn’t as convenient as an on target package manager installation or deploying a ready built container.
The project is able to build binary packages, package feeds and a binary distribution as an output but the project has never published its own set of binaries. It hasn’t been clear which configurations and platforms to target or how to handle change requests.
Rationale
We believe that the overhead in maintaining such a reference binary feed has become more feasible over time as our tools and testing infrastructure have advanced. Such a reference feed would be valuable to the community, particularly those using the project for quick R&D work where they don’t need to build everything from source.
This work item would target adding the missing workflow pieces, tools and processes to allow the creation of a binary distro proof of concept. It would then be for the wider community to support and drive it, if there was value seen in it. Regardless, improving the ability to create such distributions would be in keeping with the project objectives and stop others having to duplicate that work, often sub-optimally.
These missing workflow pieces are also fundamental building blocks to other functionality such as binary containers and lowering barriers to keeping devices up to date so they are useful in other ways as well.
Deliverables
The response to this RFQ is expected to provide quotations for the development tasks defined in any of the above sections. Please direct any questions to the Yocto Project Technical Steering Committee at tsc@lists.yoctoproject.org.
Please send quotations to jobs@yoctoproject.org.