[yocto] Shallow git clones?

Paul Barker paul at paulbarker.me.uk
Fri Nov 4 00:17:12 PDT 2016


On Thu, 3 Nov 2016 12:19:56 -0700
Khem Raj <raj.khem at gmail.com> wrote:

> 
> > On Nov 3, 2016, at 6:06 AM, Gary Thomas <gary at mlbassoc.com> wrote:
> > 
> > I recall seeing some discussion in the past about using shallow
> > GIT clones when importing repositories?  Is this ever going to
> > happen?
> > 
> > The reason I ask is that I routinely save the GIT tarballs and
> > some of them are obscenely obese :-(  The worst of the bunch
> > is the device firmware for the RaspberryPi (as of today):
> >  -rw-rw-r-- 1 gthomas gthomas 6321877646 Nov  3 12:13 git2_github.com.raspberrypi.firmware.git.tar.gz
> > 
> > This particular tar file increased by more than 300MB since
> > the last time I downloaded it (only 2016-09-13!)  I routinely
> > slosh these files across the oceans (sometimes using tin cans
> > and strings it seems) and this can be very tedious.  Is there
> > anything that can be done to make these files a bit more manageable?
> 
> while shallow clones is a comprehensive solution and we should probably slot it for 2.3 release
> the above recipe should stop using git fetcher and convert to using tarballs
> since this git repo hosts binaries, it will bloat with every time they push
> stuff into it. Someone should teach these RPi folks to not abuse github.
> 

I raised this years ago:
https://github.com/raspberrypi/firmware/issues/175

I also did a bit of investigation and found that the files we actually
use from the firmware repo total under 3MB as a tar.xz file. I have
scripts which can create an appropriate tar.xz file from the git repo
and so I could start publishing these archives again if there's any
interest in using them in the meta-raspberrypi layer.

At 3MB per archive, even if we update the firmware once per week this
would never get as bad as the current firmware repository.

Thanks,
Paul Barker



More information about the yocto mailing list