[poky] perl binary hard-code path hurts sstate

[Tian, Kevin]

>From: Gary Thomas [mailto:gary at mlbassoc.com]
>Sent: Tuesday, December 21, 2010 11:14 PM
>
>On 12/21/2010 05:30 AM, Gary Thomas wrote:
>> On 12/21/2010 04:51 AM, Tian, Kevin wrote:
>>>> From: Tian, Kevin
>>>> Sent: Tuesday, December 21, 2010 9:10 AM
>>>>
>>>>> From: Gary Thomas
>>>>> Sent: Friday, December 17, 2010 11:09 PM
>>>>>
>>>>> On 12/17/2010 06:58 AM, Richard Purdie wrote:
>>>>>> On Fri, 2010-12-17 at 06:49 -0700, Chris Larson wrote:
>>>>>>> These sorts of issues aren't new, or fun, sadly. We've run into many
>>>>>>> like this using packaged staging. We had to use the variable to
>>>>>>> disable packaged-staging for exactly this reason, so something similar
>>>>>>> for sstate would indeed be useful for this.
>>>>>>
>>>>>> In cases like this I'd suggest adding the encoded path into the checksum
>>>>>> via vardeps. This means the sstate package will be reused when the path
>>>>>> matches but not otherwise.
>>>>>>
>>>>>> We can then work on removing the dependencies which I agree is the goal
>>>>>> but we should get them identified first.
>>>>>
>>>>> Sadly, this problem is not confined to perl. I just tried this experiment:
>>>
>>> Hi, Gary, I can't reproduce in my side, though I use only one machine.
>>>
>>>>> * Build Poky on MachineA to some directory $NEW
>>>>> * Copy Poky ($POKYBASE) and $NEW/sstate-cache to MachineB, in this case
>>>>> the path for $POKYBASE is the same on both machines but $NEW differs.
>>>>> MachineA:$NEW does not exist on MachineB
>>>>> * Try to build on Machine B, pointing SSTATE_MIRRORS to $NEW/sstate-cache
>>>>> This build is in $NEW2
>>>
>>> compare to your steps, mine is:
>>>
>>> * build Poky in $DIR1, with sstate generated in ${SSTATE-CACHE} (a global one)
>>> * then with same $POKYBASE, create a new build $DIR2, with SSTATE_MIRRORS
>>> pointing to ${SSTATE-CACHE}
>>> * move $DIR1 to $DIR1.bak
>>>
>>> This way if there's reference to $DIR1, I should be able to observe failure
>>>
>>>>> The process fails for me building the kernel:
>>>>> | arm-poky-linux-gnueabi-ld: libgcc.a: No such file: No such file or directory
>>>>> Looking carefully, I can see that the compiler is looking for libgcc.a in
>>>>> the directory path $NEW that is only on MachineA.
>>>
>>> However I didn't see above failure and most sstate packages could be reused
>>> successfully except several ones depending on perl-native...
>>>
>>>>>
>>>>> So this is the same problem Paul had, just for the cross compiler
>>>>> instead of perl.
>>>>>
>>>>> n.b. Is there a bug # I should use to be tracking this?
>>>>>
>>>>
>>>> I'll see whether I can reproduce this after RP's new fix about perl-native.
>>>
>>> so Gary, could you try latest poky master or verify whether my single-machine steps
>>> may expose same failure in your side? Or is there any step not equivalent between
>>> our cases?
>>
>> Yes, I'll try it now, thanks
>>
>
>Still fails.  There is one step I wasn't clear on - if you
>just follow the steps above, you'll succeed and get a working
>result, using only the staged (sstate cached) packages.  However,

That's good, which means so far we all reach the same place for the basic sstate usage. :-)

>if you try and build some other package, or in my case I forced
>a build of the kernel, it fails.

OK, this is indeed a different story. But I admit that this scenario is also important
as it's not unusual to have incremental build based on sstate packages.

>
>Amended steps to reproduce this failure:
>   * Build Poky on MachineA, results in MachineA:NEW1
>   * Copy MachineA:NEW1/sstate-cache to MachineB:NEW1_CACHE
>   * Build Poky on MachineB, results in MachineB:NEW2, using
>     MachineB:NEW1_CACHE for SSTATE_MIRRORS
>   * Disable SSTATE_MIRRORS in conf/local.conf on MachineB:NEW2

Any reason to disable SSTATE_MIRRORS? Is it a necessary step to trigger error?

>   * Build a package from scratch.  In my case, I rebuilt the kernel:
>     - bitbake virtual/kernel -c clean
>     - rm sstate-cache/sstate-linux-am*
>     - bitbake virtual/kernel
>     My kernel is based on a local recipe linux-am_X.Y.Z

It's a bit weird why kernel rebuild causes the error you observed. I'd suspect that
there're more recipes rebuilt caused by above action. If there happens to have
gcc bootstrap recipes included, this issue is then known since we know gcc bootstrap
has problem with sstate:

https://lists.yoctoproject.org/pipermail/poky/2010-November/000270.html
https://lists.yoctoproject.org/pipermail/poky/2010-November/000499.html

which requires a rework on gcc/libc bootstrap process which is currently in development.

>
>     Note: I tried to simplify this process by using 'cleanall'
>     but ran into a fetch problem (reported separately)

OK, I see that RP fixed this fetch issue in latest master, but I'll first try to use
same commit as yours to make sure mine is on the same page as you.

>
>Notes:
>   * For my testing, I used a simple package set, e.g. poky-image-minimal
>   * My Poky tree is based on today's master
>4cd40dbaa8aef7b6b9ff9ce892a576b802f4b1ec
>   * I notice that this process now rebuilds some 20 packages from scratch
>     on MachineB.  These include the perl-native, etc, that this thread has
>     been discussing.  Is this expected?

yes, there're 10 recipes rebuilt due to perl-native:
https://lists.yoctoproject.org/pipermail/poky/2010-December/001563.html

Your 20 is a bit higher than my previous case, which is worthy of a look. Could you
post them?

>
>Thanks for working on this; it's getting close and will be
>a big help to my customers.
>

Thanks for you detail info. I'll try it again to understand whether it's a new issue
or something we may expect. :-)

Thanks
Kevin