[yocto] cannot re-use shared state cache between build hosts

[Andrea Galbusera]

On Mon, Sep 4, 2017 at 10:56 AM, Paul Eggleton <
paul.eggleton at linux.intel.com> wrote:

> On Monday, 4 September 2017 7:59:55 PM NZST Andrea Galbusera wrote:
> > On Mon, Sep 4, 2017 at 9:33 AM, Patrick Ohly <patrick.ohly at intel.com>
> wrote:
> > > More recent bitbake should not fail like that anymore. It's still
> > > better to use an HTTP server that performs better, though.
> > >
> > > commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f
> > > Author: Patrick Ohly <patrick.ohly at intel.com>
> > > Date:   Mon Jul 17 15:25:10 2017 +0200
> > >
> > >     fetch2/wget.py: improve error handling during sstate check
> > >
> > >     When the sstate is accessed via HTTP, the existence check can fail
> due
> > >     to network issues, in which case bitbake silently continues without
> > >     sstate.
> > >
> > >     One such network issue is an HTTP server like Python's own
> SimpleHTTP
> > >     which closes the TCP connection despite an explicit "Keep-Alive" in
> > >     the HTTP request header. The server does that without a "close" in
> the
> > >     HTTP response header, so the socket remains in the connection
> cache,
> > >     leading to "urlopen failed: <urlopen error [Errno 9] Bad file
> > >     descriptor>" (only visible in "bitbake -D -D" output) when trying
> to
> > >     use the cached connection again.
> > >
> > >     The connection might also get closed for other reasons (proxy,
> > >     timeouts, etc.), so this is something that the client should be
> able
> > >     to handle.
> > >
> > >     This is achieved by checking for the error, removing the bad
> > >     connection, and letting the check_status() method try again with a
> new
> > >     connection. It is necessary to let the second attempt fail
> > >     permanently, because bad proxy setups have been observed to also
> lead
> > >     to such broken connections. In that case, we need to abort for real
> > >     after trying twice, otherwise a build would just hang forever.
> > >
> > >     [YOCTO #11782]
> > >
> >
> > Thank you Patrick for pointing that out. While debugging the issue on
> morty
> > I had some reminiscence of seeing your patch on the list, but I wasn't
> able
> > to find it back in morty's history hence inferring it landed later on.
> > Anyway doing this test on morty was a requirement... Thanks again for
> your
> > clarification. Would such a patch be suitable for eventually being
> > back-ported to morty?
>
> Doesn't look too invasive to me at least, so I'd support it.
>

I did some positive tests with Patrick's patch on top of morty and it
improves a lot the situation even with SimpleHTTPServer. Than I'd like to
propose it is backported to morty and, after further testing, reasonably
should land in pyro too. But I'm unsure how such a request should be
submitted. Since it affects bitbake, what's the proper way and which list
should I send such a request?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170907/3c67c75d/attachment.html>