[yocto] [yocto-autobuilder] [RFC] Add support for generate bitbake error reports

[Aníbal Limón]

On 06/14/2016 10:09 AM, Lock, Joshua G wrote:
> On Thu, 2016-06-09 at 16:23 -0500, Aníbal Limón wrote:
>> Hi folks,
>>
>> Currently we support to send error reports to errors.yoctoproject.org
>> about failing tasks on bitbake using SendErrorReport buildstep but we
>> have a lack of bitbake related errors like exceptions.
>>
>> A bug exist to implement this support into Error report web [1], i'm
>> working on it but for generate bitbake error reports there's a need
>> of
>> some process monitoring the bitbake output in this case the Yocto
>> Autobuilder.
>>
>> This email is for review my current implementation for generate
>> bitbake
>> error reports in the Autobuilder [2] next i'll try to explain how it
>> works.
>>
>> I aadded a BitbakeShellCommand [3] class for use in the buildsteps
>> that
>> executes bitbake the main objective of this class is to have common
>> operations to be made in bitbake commands like create error reports
>> if
>> fails.
>>
>> For create error reports this class add an stdio observer to look at
>> bitbake output and if bitbake fails review the bitbake output for
>> identify if the failure is an non-related task error [4]. If the
>> observer found bitbake error creates an Error report with the
>> information in the master controller.
>>
>> In order to send the bitbake error to Error report web the controller
>> transfer the report to the worker using a new DownloadDirectory
>> implementation that i made [5], the SendErrorReport buildstep works
>> on
>> the worker side so it's easy to transfer the reports from master to
>> worker instead of send it by master.
>>
>> Finally to complete with the job of have the bitbake errors reports
>> the
>> Error report web changes need  (i'm working on it) to be integrated
>> first in order to don't break anything.
>>
>> Please review it and provide me feedback.
> 
> This would've been much easier to review as a series of patches.
> 
> After a quick read via the gitweb the series as a whole looks good.
> 
> A couple of review comments:
> 
> In a04b41d37c29191318386455d8d958ff815a3a10 "lib/buildsteps.py: Add
> BitbakeLogLineObserver for BitbakeShellCommands." you have a comment
> 
> "discard line that are not errors and line that is recipe task errors"
> 
> but the lines are not actually discarded, afaict  from a cursory read
> through they aren't used in the rest of the series
> unless errors['bitbake'] == True? 
> 
> Could we move the error['log'].append() to after the if statement which
> checks whether this is a bitbake error?

Yes and i'll update the commit message to be more consistent.

> 
> Minor nit, in caf472bc696053227825c5a102feef3e17574ba2 "lib/buildsteps:
> BitbakeShellCommand add support for create error reports"
> in get_error_report_bitbake_dir() you use both " and ' for strings.
> 
> Same in 40279597615c49bc4f4f067cbab937584b626671

I'll fix the typos for only use "".

	alimon

> 
> Regards,
> 
> Joshua
> 
>>
>> Cheers,
>> 	alimon
>>
>> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=7583
>> [2]
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/log/?h=co
>> ntrib/alimon/devel
>> [3]
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/lib/
>> python2.7/site-
>> packages/autobuilder/lib/buildsteps.py?h=contrib/alimon/devel#n92
>> [4]
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/tree/lib/
>> python2.7/site-
>> packages/autobuilder/lib/buildsteps.py?h=contrib/alimon/devel#n53
>> [5]
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/commit/?h
>> =contrib/alimon/devel&id=4022920bb0e56d1eef3dfe7c5e9b4699f801cdf1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160614/6a98c250/attachment.pgp>