[Automated-testing] Structured feeds

[Dmitry Vyukov]

On Fri, Nov 8, 2019 at 3:53 PM Don Zickus <dzickus at redhat.com> wrote:
>
> On Fri, Nov 08, 2019 at 09:05:02AM +0100, Dmitry Vyukov wrote:
> > On Thu, Nov 7, 2019 at 9:53 PM Don Zickus <dzickus at redhat.com> wrote:
> > >
> > > On Tue, Nov 05, 2019 at 11:02:21AM +0100, Dmitry Vyukov wrote:
> > > > Hi,
> > > >
> > > > This is another follow up after Lyon meetings. The main discussion was
> > > > mainly around email process (attestation, archival, etc):
> > > > https://lore.kernel.org/workflows/20191030032141.6f06c00e@lwn.net/T/#t
> > > >
> > > > I think providing info in a structured form is the key for allowing
> > > > building more tooling and automation at a reasonable price. So I
> > > > discussed with CI/Gerrit people and Konstantin how the structured
> > > > information can fit into the current "feeds model" and what would be
> > > > the next steps for bringing it to life.
> > > >
> > > > Here is the outline of the idea.
> > > > The current public inbox format is a git repo with refs/heads/master
> > > > that contains a single file "m" in RFC822 format. We add
> > > > refs/heads/json with a single file "j" that contains structured data
> > > > in JSON format. 2 separate branches b/c some clients may want to fetch
> > > > just one of them.
> > > >
> > > > Current clients will only create plain text "m" entry. However, newer
> > > > clients can also create a parallel "j" entry with the same info in
> > > > structured form. "m" and "j" are cross-referenced using the
> > > > Message-ID. It's OK to have only "m", or both, but not only "j" (any
> > > > client needs to generate at least some text representation for every
> > > > message).
> > >
> > > Interesting idea.
> > >
> > > One of the nuisances of email is the client tools have quirks.  In Red Hat,
> > > we have used patchworkV1 for quite a long time.  These email client 'quirks'
> > > broke a lot of expectations in the database leading us to fix the tool and
> > > manually clean up the data.
> > >
> > > In the case of translating to a 'j' file.  What happens if the data is
> > > incorrectly translated due to client 'quirks'?  Is it expected the 'j' data
> > > is manually reviewed before committing (probably not).  Or is it left alone
> > > as-is? Or a follow-on 'j' change is committed?
> >
> > Good point.
> > I would expect that eventually there will be updates to the format and
> > new version. Which is easy to add to json with "version":2 attribute.
> > Code that parses these messages will need to keep quirks for older
> > formats.
> > Realistically nobody will review the data (besides the initial
> > testing). I guess in the end it depends on (1) how bad it's screwed,
> > (2) if correct data is preserved in at least some form or not
> > (consider a client pushes bad structured data, but it's also
> > misrepresented in the plain text form, or simply missing there).
> > Fixing up data later is not possible. Appending corrections is possible.
>
> Ok.  Yeah, in my head I was thinking the data is largely right, just
> occasionally 1 or 2 fields was misrepresented due to bad client tool or
> human error in the text.
>
> In Red Hat was use internal metadata for checking our patches through our
> process (namely Bugzilla id).  It isn't unusual for someone to accidentally
> fat-finger the bugzilla id when posting their patch.
>
> I was thinking if there is a follow-on 'type' that appends corrections as you
> stated, say 'type: correction' that 'corrects the original data.  This would
> have to be linked through message-id or some unique identifier.
>
> Then I assume any tool that parses the feed 'j' would correlate all the data
> based around some unique ids such that picking up corrections would just be
> a natural extension?

Yes, this should be handled naturally in this model. Since it's not
possible to mutate any previously published info, everything is
represented as additions/corrections: adding a comment to a patch,
adding Reviewed-by, adding Nack, adding test results. The final state
of a patch is always reconstructed by "replaying" all messages
published regarding the patch. So naturally if we mis-parsed a message
as "Acked-by: X" and then corrected that to "Nacked-by: X" and
republished, whoever will replay the feed, should replace Acked-by
with Nacked-by.