[yocto] Patchwork and incoming patch testing

[Paul Eggleton]

Hi all,

As some of you are aware some of my colleagues and I have been working on 
improving how incoming patches are handled - initially for OE-Core but we hope 
to arrive at something that will be useful for other layers as well. The aim 
was to do so without adversely affecting existing workflows, so that means 
building on top of the things we already have. It's taken a bit longer than 
we'd originally planned - we embarked on this a little over a year ago [1] - 
but now I am happy to be able to show some meaningful progress.

A few months ago we upgraded OE's patchwork instance [2], moving not just to a 
later version but to a fork of patchwork where a bunch of new functionality 
was being developed for freedesktop.org [3], notably support for capturing and 
presenting patch series instead of just individual patches. There were some 
teething problems but we've now resolved most of them. Unfortunately work on 
said freedesktop.org fork appears to have stalled so for now we have forked it 
ourselves [4]; long term we'll have to see if we can merge back with patchwork 
upstream - at least for small fixes we'll try to push those back up 
independent of any wholesale merge. In any event we are now finally in the 
position where our patchwork instance can be relied upon to collect emails, 
and the UI is much improved. This should give us a bit more visibility into 
where patches are at in the process, although we are still working on a few 
places where patch series status needs to be updated (e.g. when a patch goes 
into testing).

On top of patchwork we have built a simple smoke-testing framework called 
"patchtest" [5] along with a suite of corresponding tests for OE [6]. These 
tests are fairly simplistic at this point but check the basics such as whether 
a patch has been properly signed off, etc. We should soon start seeing replies 
sent to the mailing list and to submitters with results if there are any 
failures, saving us from noticing and pointing out some of the more obvious 
classes of mistakes. The tests are easy to run locally without the rest of the 
infrastructure and can be extended without difficulty, and I expect we'll 
continue to work on those as time progresses. Contributions would be very 
welcome.

My sincere thanks to José Lamego, Leonardo Sandoval, Daniela Plascencia, 
Belen Barros Pena, Michael Halstead, Damien Lespiau, Patrick Ohly and others 
that have been part of implementing this, and to everyone else who has put up 
with the delays.

Please let us know if you have issues with any part of this process or 
suggestions on how to improve it. We're tracking improvements in the Yocto 
Project bugzilla [7] so you can see what's being worked on there if you're 
interested.

Cheers,
Paul

[1] Earlier announcement:
    https://www.mail-archive.com/openembedded-core@lists.openembedded.org/msg72952.html

[2] OE's patchwork instance:
    http://patchwork.openembedded.org

[3] Freedesktop.org patchwork fork:
    https://github.com/dlespiau/patchwork

[4] Our patchwork fork:
    http://git.yoctoproject.org/cgit/cgit.cgi/patchwork/

[5] Patchtest main repository:
    http://git.yoctoproject.org/cgit/cgit.cgi/patchtest/

[6] OE test suite for patchtest:
    http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe/

[7] Yocto Project bugzilla area for patchwork/patchtest:
    https://bugzilla.yoctoproject.org/describecomponents.cgi?product=Patchwork%2FPatchtest

-- 

Paul Eggleton
Intel Open Source Technology Centre