[yocto] [PATCH][ptest-runner] ptest-runner: Add support timeout failure type to XML result file

[Anibal Limon]

On Sat, 5 Jan 2019 at 12:52, <richard.purdie at linuxfoundation.org> wrote:

> On Sat, 2019-01-05 at 11:30 -0600, Anibal Limon wrote:
> >
> >
> > On Thu, 3 Jan 2019 at 15:45, <richard.purdie at linuxfoundation.org>
> > wrote:
> > > On Thu, 2019-01-03 at 12:02 -0600, Aníbal Limón wrote:
> > > > The ptest-runner support logging results to stdout and to a XML
> > > file
> > > > in stdout the ptest is mark as:
> > > >
> > > > ...
> > > > ERROR: Exit status is 1
> > > > TIMEOUT: ptest-directory
> > > > ...
> > > >
> > > > Add the same support in XML file for example,
> > > >
> > > > ...
> > > > <testcase classname='ptest-directory' name='run-ptest'>
> > > >   <failure type='exit_code' message='run-ptest exited with code:
> > > > 1'></failure>
> > > >   <failure type='timeout'/>
> > > > </testcase>
> > > > ...
> > > >
> > > > [YOCTO #13088]
> > > >
> > > > Signed-off-by: Aníbal Limón <anibal.limon at linaro.org>
> > >
> > > Sounds good, thanks!
> > >
> > > I have a feeling this will show up a few bugs but we should find
> > > and
> > > fix the ones which are timing out...
> > >
> > > I'd still be interested in the time each directory takes too if we
> > > can
> > > get that as it would help us decide which tests to run.
> >
> > The timeout is set to 5 minutes (300 secs) by default and currently
> > there is no way to set a timeout per ptest,
> > I will create other patch to add the timeout value to stdout and XML
> > file.
>
> That is handy but what I mean is the execution time each one takes. If
> we have test which take 5s its easier to run them regularly than tests
> which take 500s. Having an idea of their relative execution time would
> therefore be helpful.
>

Got it, I can add a duration for example:.

In stdout will be (after end),

...
BEGIN: TIME
END: ptest-directory
DURATION: Ns
END: TIME
...

In XML:

...
<testcase classname='%s' name='run-ptest' duration='Ns'>
</testcase>
...

or

...
<testcase classname='%s' name='run-ptest' >
<duration>Ns</duration>
</testcase>
...

What is the better? or any other?


> I assume the timeout takes effect after no output for that period?
>

Right, the timeout is based on stdout data.

Anibal

>
> Cheers,
>
> Richard
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190105/0ac37897/attachment.html>