[yocto] What keeps sstate from being used?

[Gary Thomas]

On 2016-07-06 13:51, Chris Z. wrote:
> Hi,
>
> Check:
> http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shared-state-cache
>
> "... the build system detects changes in the "inputs" to a given task by creating a checksum (or signature) of the
> task's inputs. If the checksum changes, the system assumes the inputs have changed and the task needs to be rerun. For
> the second question, the shared state (sstate) code tracks which tasks add which output to the build process.... "
>

I don't see how anything could have changed between the two runs.  Literally, I ran:
   % bitbake target
   ... failed with error building webkitgtk after some 6000 tasks
   % mv tmp old
   % bitbake target

I traced this down to [at least] these tasks:

gthomas at europa:p0382_2016-01-13$ bitbake-diffsigs 
sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo 
sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo
basehash changed from 366d439ed05d276413ffabf7423ebe44 to acc4c0150665538a4bdd394b6b6bda13
Variable SRC_URI value changed from ' 
git://sourceware.org/git/binutils-gdb.git;branch=binutils-${BINUPV}-branch;protocol=git 
file://0002-configure-widen-the-regexp-for-SH-architectures.patch 
file://0003-Point-scripts-location-to-libdir.patch 
file://0004-Only-generate-an-RPATH-entry-if-LD_RUN_PATH-is-not-e.patch 
file://0005-Explicitly-link-with-libm-on-uclibc.patch      file://0006-Use-libtool-2.4.patch 
file://0007-Add-the-armv5e-architecture-to-binutils.patch 
file://0008-don-t-let-the-distro-compiler-point-to-the-wrong-ins.patch 
file://0009-warn-for-uses-of-system-directories-when-cross-linki.patch 
file://0010-Fix-rpath-in-libtool-when-sysroot-is-enabled.patch 
file://0011-Change-default-emulation-for-mips64-linux.patch      file://0012-Add-support-for-Netlogic-XLP.patch 
file://0013-Fix-GOT-address-computations-in-initial-PLT-entries-.patch 
file://0014-Correct-nios2-_gp-address-computation.patch 
file://0015-fix-the-incorrect-assembling-for-ppc-wait-mnemonic.patch 
file://MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch ' to ' 
git://sourceware.org/git/binutils-gdb.git;branch=binutils-${BINUPV}-branch;protocol=git 
file://0002-configure-widen-the-regexp-for-SH-architectures.patch 
file://0003-Point-scripts-location-to-libdir.patch 
file://0004-Only-generate-an-RPATH-entry-if-LD_RUN_PATH-is-not-e.patch 
file://0005-Explicitly-link-with-libm-on-uclibc.patch      file://0006-Use-libtool-2.4.patch 
file://0007-Add-the-armv5e-architecture-to-binutils.patch 
file://0008-don-t-let-the-distro-compiler-point-to-the-wrong-ins.patch 
file://0009-warn-for-uses-of-system-directories-when-cross-linki.patch 
file://0010-Fix-rpath-in-libtool-when-sysroot-is-enabled.patch 
file://0011-Change-default-emulation-for-mips64-linux.patch      file://0012-Add-support-for-Netlogic-XLP.patch 
file://0013-Fix-GOT-address-computations-in-initial-PLT-entries-.patch 
file://0014-Correct-nios2-_gp-address-computation.patch 
file://0015-fix-the-incorrect-assembling-for-ppc-wait-mnemonic.patch '
Dependency on checksum of file MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch was removed

gthomas at europa:p0382_2016-01-13$ ls -l 
sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo 
sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo
-rw-rw-r-- 1 gthomas gthomas 3966 Jul  6 09:16 
sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo
-rw-rw-r-- 1 gthomas gthomas 3787 Jun 28 05:55 
sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo

Notice that the timestamp from today was 09:16.  I had updated my Poky/Yocto repo
at 08:04 today and then immediately made the first run (seen above).  When I started
the second run it was 11:57, and binutils-cross-arm was created (I think from setscene)
at 12:00.  So why could these hashes have caused all those other ripples (which took
until 14:15 to finish).

I also saw some very weird behaviour in all of this.  I noticed that my virtual/kernel
was totally rebuilt, but the image that's in the deploy directory came from the setscene
task, i.e. the first run :-(

None of this seems quite right to me and I'd really like to understand it and, if needed,
help figure out how to make it correct (and robust).

I'll save the total state of everything in case I can provide more data...

> On Wed, Jul 6, 2016 at 12:10 PM, Gary Thomas <gary at mlbassoc.com <mailto:gary at mlbassoc.com>> wrote:
>
>     I just had a [nearly] complete build failed (building webkitgtk
>     after some 6000 tasks).  I decided to try the 'rebuild from sstate'
>     by removing my 'tmp' directory and rebuild.  Bitbake proceeded to
>     run 2200 setscene tasks and then [it seems] started over on the
>     build, rebuilding the target gcc, the kernel, who knows what else...
>     These are all things that I thought had been completed before
>     during the initial ~6000 steps.
>
>     What could be going on and why couldn't it just pick up using
>     sstate from where it left off?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------