[yocto] Storing Sstate in S3 success stories?

[Brian Walsh]

On Mon, Feb 25, 2019 at 8:46 PM Timothy Froehlich <tfroehlich at archsys.io> wrote:
>
> I've been spending a bit too long this past week trying to build up a reproducable build infrastructure in AWS and I've got very little experience with cloud infrastucture and I'm wondering if I'm going in the wrong direction. I'm attempting to host my sstate_cache as a mirror in a private S3 bucket, and I believe I have everything configured properly, including exposing the bucket to http requests, since I can wget files that I've previously synced up to the bucket. However if I add in the SSTATE_MIRRORS to my build, bitbake slows to a crawl (it's a powerful VM) and barely seems to get anything. The EC2 instance is in the same region as the S3 bucket, roles have been configured properly to allow access, etc.
>
> I'm not looking for help debugging this, I just want to know whether I'm right that hosting my sstate in an S3 bucket should work. I've only been able to find one mention of it being done with no reproduction hints.
>

A lot of the files end up with plus signs in the name. This causes
problems with retrieving files through http access with S3. S3
translates all plus signs to spaces, even those in the file path. So
if my-file_v1.0+g1241876 actually exists as named in S3 an http
request for that file will trigger the server to look for
"my-file_v1.0 g1241876"

I ran into this problem trying to host an opkg repository in S3 for upgrading.

It may mostly work for you but there will be many files that it will
never be able to find in your S3 hosted sstate.

Maybe this has been fixed by AWS. I noticed the problem a year or two ago.

https://stackoverflow.com/questions/36734171/how-to-decide-if-the-filename-has-a-plus-sign-in-it#36758133
https://news.ycombinator.com/item?id=15398804