[yocto] Error Report Tool Purge

[Michael Halstead]

Currently the database is 26GB. I'm going to start purging un-visited
reports older than 180 days today. I see some good suggestions in this
thread. Khem, would you open an enhancement request describing the counts
you are interested in preserving?

On Thu, Jul 19, 2018 at 11:08 PM Martin Jansa <martin.jansa at gmail.com>
wrote:

> OK, so it's only visited links, not linked from ML or wiki or whatever,
> right?
>
> How huge the database is? I've checked our internal instance and on
> relatively slow VM we have:
> 47 472 724 Post_buildstatstask
> 2 715 091 Post_build
> 559 655 Post_buildfailure
>
> we use postgresql, the db currently has 20GB, which is quite big for VM
> with 10GB RAM and 12 E5-2699 cores so it's slow as well, but not terribly
> slow, individual builds or build-failures are really fast, there are some
> bad queries in django which are terrible, but we're not using those usually.
>
> Cheers,
>
>
> On Thu, Jul 19, 2018 at 9:01 PM Brindle, Amanda R <
> amanda.r.brindle at intel.com> wrote:
>
>> Every time a specific report is visited (with the end of the URL being
>> /Errors/Details/<id>), we are tracking if the referrer is another website,
>> the reporting tool itself, or unknown. If the referrer is another website
>> or unknown, then we won’t delete it.
>>
>>
>>
>> The purge script does not look at links to whole builds, so as it is
>> right now, those would get deleted. If it’s common to link to whole builds,
>> though, I can add something to the script to save reports from a visited or
>> linked build.
>>
>>
>>
>> -Amanda Brindle
>>
>>
>>
>> *From:* Martin Jansa [mailto:martin.jansa at gmail.com]
>> *Sent:* Thursday, July 19, 2018 11:38 AM
>> *To:* Brindle, Amanda R <amanda.r.brindle at intel.com>
>> *Cc:* Yocto Project <yocto at yoctoproject.org>
>> *Subject:* Re: [yocto] Error Report Tool Purge
>>
>>
>>
>> I'm just curious, how are you tracking which reports were viewed or
>> linked to (and linked from where)? I often use a link to
>> http://errors.yoctoproject.org in the mailing list or the recipes/commit
>> message instead of copy pasting whole build error, because it already
>> shortens the build paths and shows useful additional information about the
>> error.
>>
>>
>>
>> The links to whole builds on http://errors.yoctoproject.org were also
>> often linked from "bitbake world status" e-mails and wiki like:
>>
>> https://www.openembedded.org/wiki/Bitbake_World_Status_Rocko
>>
>> and on many of them nobody clicked yet - should I expect that these will
>> mostly get broken?
>>
>>
>>
>> Regards
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jul 19, 2018 at 8:30 PM Brindle, Amanda R <
>> amanda.r.brindle at intel.com> wrote:
>>
>> Hello,
>>
>>
>>
>> The Error Reporting Tool’s database (
>> http://errors.yoctoproject.org/Errors/Latest/Autobuilder/)  has grown to
>> a huge size, and this is affecting the performance of the application. We
>> are planning to run a purge to get rid of reports that we don’t need. We
>> will keep reports from the last thirty days, as well as reports that have
>> been viewed or linked to. If you have a specific report that you don’t want
>> purged, please let me know by the end of the month.
>>
>>
>>
>> Amanda Brindle, Software Engineer
>>
>> 503-264-3970
>>
>> amanda.r.brindle at intel.com
>>
>>
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto at yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>>
>> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20180801/5af4b067/attachment.html>