[yocto] [Yocto] Checking sstate mirror is hanging at 100%

[Vineeth Karumanchi]

> From: Andre McCurdy [mailto:armccurdy at gmail.com]
> Sent: Tuesday, October 31, 2017 1:19 AM
> To: Alexander Kanavin <alexander.kanavin at linux.intel.com>
> Cc: Vineeth Karumanchi <VINEETHC at xilinx.com>; Yocto discussion list
> <yocto at yoctoproject.org>
> Subject: Re: [yocto] [Yocto] Checking sstate mirror is hanging at 100%
> 
> On Mon, Oct 30, 2017 at 2:37 AM, Alexander Kanavin
> <alexander.kanavin at linux.intel.com> wrote:
> > On 10/30/2017 07:54 AM, Vineeth Karumanchi wrote:
> >
> >> In this case i would expect bitbake to "check for sstate mirrors" for
> >> some time and rebuild from scratch. Is there any approach to over come
> this ?
> >
> > The approach is to fix your infrastructure so that fetching from
> > network doesn't fail. We've gotten a similar request recently for
> > enabling 'soft sstate cache failure', and RP is firmly against it:
> >
> > http://lists.openembedded.org/pipermail/openembedded-core/2017-
> August/
> > 141593.html
> 
> It looks like the logic in the wget fetcher retries once, so if it's getting stuck
> then that seems like a genuine bug.
> 
> Vineeth, are you sure it's really continuously retrying? 
Yes, I see continuous retry of wget of same sstate files.
With -DD I see the below messages  for every try:

DEBUG: checkstatus: trying again
DEBUG: checkstatus: trying again
DEBUG: checkstatus: trying again
DEBUG: checkstatus: trying again
DEBUG: checkstatus: trying again

>Or are you just seeing
> timeouts for many fetches (which will eventually complete)? Note that you
> can reduce the wget timeout by setting your own value for
> "FETCHCMD_wget". See bitbake/lib/bb/fetch2/wget.py for the default value
> - use that as a starting point for your modification.

No fetch failure error, Just hanging at 100%.

I did some more analysis:
My network needs proxy to access any site.

To validate a URL, bitbake/lib/bb/fetch2/__init__.py:  urllib.parse.urlparse(uri) and checking for netloc.
This is returning true without proxy being set, but the same uri is not establishing connection without proxies.

bitbake is thinking it as a valid  and keep retrying without any errors.

Thanks
Vineeth