[yocto] tar ball vs. git development questions

[Bruce Ashfield]

On 12-01-23 08:10 AM, Gary Thomas wrote:
> On 2012-01-23 05:51, jfabernathy wrote:
>> On 01/22/2012 08:12 PM, Gary Thomas wrote:
>>> On 2012-01-22 13:19, James Abernathy wrote:
>>>> I have used both git and the tarball methods of bitbaking projects,
>>>> all of them derivatives of the examples in the Yocto documentation.
>>>> I was having issues using the local clone of
>>>> the Yocto kernel git repository this weekend. I had successfully
>>>> done that before, but I was rebuilding the PC workstation, and
>>>> getting everything setup and tested some of the
>>>> meta-intel BSPs to make sure I had everything right. Cloning the
>>>> linux-yocto-3.0 repository was successful, but the bakes against it
>>>> failed. I made sure I had poky-extras setup
>>>> right, but I still had problems. To isolate the problem, I changed
>>>> to building with the tarballs and everything worked fine.
>>>>
>>>> So that got me thinking what are the differences between the 2 methods:
>>>>
>>>> * I assume that if I use the tarball method, bitbake, using the
>>>> recipes, pulls down files from the online repositories and puts
>>>> those files into the centralized local download
>>>> directory ($DL_DIR), allowing reuse instead of re-downloading each
>>>> time. The content downloaded for linux-yocto-3.0 is exactly what
>>>> would be pulled from the local repository if
>>>> I used a local clone of the git repository for linux-yocto-3.0.
>>>> * If my assumption above is correct, if I'm not modifying the source
>>>> code of the kernel (only changing config parameters), then once
>>>> you've run at least one build with the
>>>> tarball method, the $DL_DIR directory contains all the files you'll
>>>> need to build any image with linux-yocto-3.0. So there is no need to
>>>> have a local clone of the kernel
>>>> repository for speeding up development. Am I right?
>>>> * If I have a successful creation of a bare clone of
>>>> linux-yocto-3.0.git, how could builds of Edison packages be failing?
>>>> That makes me concerned about using git and successfully
>>>> repeating builds of stable branches like Edison.
>>>
>>> If you set BB_GENERATE_MIRROR_TARBALLS = "1" (e.g. in local.conf)
>>> then you'll get tarballs which hold the git repositories after
>>> download. You can then reuse these (by sharing the DL_DIR or
>>> using a local mirror). Does that help with the issue you're seeing?
>>>
>> I'm not sure it does. I don't want poky to do more work. I have my
>> download directory, $DL_DIR, outside my build directory so I can keep
>> it run to run, as mentioned in the comments
>> in local.conf. I was trying to understand 2 basic questions:
>>
>> 1. what could be causing build failures using a freshly created bare
>> clone yesterday vs. using the poky-edison-6.0 tarball. It would be
>> scary if you could clone the linux-yocto-3.0
>> successfully one day and have it be used in a build successfully, but
>> clone it another day and it not work. I figured that bitbake/poky
>> pulled the same information into DL_DIR
>> regardless of whether you pulled from the bare clone locally or
>> straight from the on-line repository.
>
> What kind of errors? Some details might help understand what the problem
> is.
>
>> 2. I was trying to look at items to speed up build runs. I thought
>> that if the downloads in DL_DIR were reused if they existed, it would
>> speed up a run. It seems to. So the
>> question is, after the first build, the files I need for
>> linux-yocto-3.0 are in DL_DIR regardless of whether they came from the
>> online repository or the local bare clone. right?
>
> I think so, but I'm not 100% sure.

 From my experience, this is true. Regardless of the SRCI_URI, the
downloads directory contains everything that poky/bitbake need to
do builds after the initial fetch has been performed. And that's
where subsequent updates are pullled.

Cheers,

Bruce

>
> The way I have my system set up, I never download anything more
> than once :-) I have a shared source repository which I point to
> using MIRRORS and then all my builds are relative to that. If there
> is new code, it gets downloaded and saved (as a tarball) in my sources
> repository to be used by subsequent builds. To do this, I just have
> these lines in <distro>.conf
> # Provide pre-staged sources
> SOURCE_MIRROR_URL ?= "file://${COREBASE}/sources/"
> INHERIT += "own-mirrors"
> BB_GENERATE_MIRROR_TARBALLS = "1"
> I don't define DL_DIR in local.conf so files only come from my mirror.
>
> This process is so successful that I almost always run with
> BB_NO_NETWORK="1"
> which causes the fetcher to die if it can't find the file locally. If I
> find
> that I do need to download something new, I disable this and let the
> fetcher
> download it. I'll then have a saved tarball (either downloaded directly or
> synthesized) in <build_dir>/downloads that I can push to my mirror.
>