[yocto-ab] Project Status

[Richard Purdie]

Hi All,

I wanted to give everyone an update of where the project is at.

Current Status
==============

2.7 was released on schedule and on the most part with the features as
planned. In particular I want to highlight the improvements in the
automated QA situation. We've removed dependencies on legacy tools like
testopia and now have a bare bones resulttool which gives us customised
results handling. This is very simple code but extremely powerful to
let the project do exactly what we need. An example of what this gives
us is this automated report:

https://autobuilder.yocto.io/pub/non-release/20190512-10/testresults/testresult-report.txt

The top chart gives an index summary of all the tests that are run be
them selftest, runtime, sdk or esdk. This allows easy comparison with
previous builds. During 2.7 we significantly increased the number of
tests which were automated and have started working through automation
of more of the manual tests.

The next section is ptest which gives pass/fail/skip and timing counts
for each recipe. There has been a huge amount of work in having stable
numbers here but I believe we now have consistent results and can work
on reducing the fail counts. These ptests have already caught a kernel
bug which was reported upstream and fixed in the 5.0 stable series.

Post 2.7 we have developments in master where Armin added ltp support
and this report includes those after ptest results. Again, this gives
us a baseline which we can investigate and improve upon.

Challenges
==========

As many know, Stephen who was acting as our program manager in his own
time was injured in an accident and we're having to manage without him.
Several people have been helping out with different meetings but it has
had an impact on 2.8 planning which I'm driving in his absence.

Its become apparent we have some resource gaps in some key areas of the
project for 2.8:

* Patchtest/patchwork offline and no maintainer (bug 13284)
* Reproducible builds automated testing (bug 13323)
* Build artefact reuse (bug 10682)
* License infrastructure improvements (bug 13321, 13322)
* Autobuilder code bugs (bug 13332, 13329)

To give a bit more information on these:

We have no active maintainers for some key pieces of infrastructure.
I'm personally covering the autobuilder, patchwork/patchtest have no
active coverage.

Two key focus areas of the project should be reproducibility and
license handling but we have nobody working on these either.

Finally, the build artefact reuse would be a *huge* feature win for the
project with significant day to day improvement for most users. The
base work on the sstate equivalence server has been done, the remaining
piece is to rewrite runqueue so that it can take advantage of artefact
equivalence mapping. This would benefit sstate cache reuse and improve
multiconfig builds. This one is tricky in that it its highly involved
changes at the core of bitbake and there are only a limited number of
people with the skills. I would love to do it but I can't if I'm
covering everything else that I'm doing.

I've also not mentioned layer setup here as that is another problem
which needs experience to work on it and nobody with the skills has the
bandwidth. I believe the artefact reuse would be the bigger win for the
project. I've also not going into the hundreds of smaller day to day
things we're doing, just the big potential gaps.

The team also decided to try and create a new class of "newcomer" bugs
which would be straight forward enough that someone new to the project
could work on. These can be seen here:
https://wiki.yoctoproject.org/wiki/Bug_Triage#Newcomer_Bugs

We've not seen much take up on newcomer bugs yet, I'm hoping we can
socialise these more and help bring some new people into the project.

I want to give the advisory board an indication of the potential missed
opportunities we have and where we could use help. Regardless, I'm
going to continue to work on things on a priority basis, we have a
strong and stable platform for build off now which will help a lot.

I've tried to keep the email shorter than longer but am happy to
provide more information on anything above, be it via email or
potentially a meeting if there is interest.

Cheers,

Richard