[Automated-testing] Structured feeds

[Daniel Axtens]

Dmitry Vyukov <dvyukov at google.com> writes:

> On Wed, Nov 6, 2019 at 4:35 PM Daniel Axtens <dja at axtens.net> wrote:
>>
>> > As soon as we have a bridge from plain-text emails into the structured
>> > form, we can start building everything else in the structured world.
>> > Such bridge needs to parse new incoming emails, try to make sense out
>> > of them (new patch, new patch version, comment, etc) and then push the
>> > information in structured form. Then e.g. CIs can fetch info about
>>
>> This is an non-trivial problem, fwiw. Patchwork's email parser clocks in
>> at almost thirteen hundred lines, and that's with the benefit of the
>> Python standard library. It also regularly gets patched to handle
>> changes to email systems (e.g. DMARC), changes to git (git request-pull
>> format changed subtly in 2.14.3), the bizzare ways people send email,
>> and so on.
>>
>> Patchwork does expose much of this as an API, for example for patches:
>> https://patchwork.ozlabs.org/api/patches/?order=-id so if you want to
>> build on that feel free. We can possibly add data to the API if that
>> would be helpful. (Patches are always welcome too, if you don't want to
>> wait an indeterminate amount of time.)
>
> Hi Daniel,
>
> Thanks!
> Could you provide a link to the code?
> Do you have a test suite for the parser (set of email samples and what
> they should be parsed to)?

Sure:
https://github.com/getpatchwork/patchwork in particular
https://github.com/getpatchwork/patchwork/blob/master/patchwork/parser.py and
https://github.com/getpatchwork/patchwork/tree/master/patchwork/tests

Regards,
Daniel