[yocto] gcc LIBRARY_PATH sstate-cache issue

Mon Nov 2 05:51:40 PST 2015

Hi Andre,

Thanks a lot for this big hint that allowed me to find the cause of this problem.

The problem came from the CMakeLists.txt of libB. It was overriding CMAKE_C_FLAGS cmake variable defined in the Yocto generated toolchain.cmake.
CMAKE_C_FLGAS contains the "--sysroot" flag for gcc command.

I don't know how it could work before and what in image/machine configurations' changes triggered this bug to show up.

I don't know also how, when it was bugged, the build system populated the gcc's LIBRARY_PATH variable with sysroots path from the mirrored reference build's sstate-cache (build-machineA's sstate-cache).

The most important is that state sharing cache is working fine.

Thanks everyone for the help,

Cheers
-- 
Jérôme OLIVIER
TEL : +337 52 46 29 25
@ : jerome.olivier at outlook.com

> Date: Wed, 21 Oct 2015 13:58:53 -0700
> Subject: Re: [yocto] gcc LIBRARY_PATH sstate-cache issue
> From: armccurdy at gmail.com
> To: jerome.olivier at outlook.com
> CC: paul.eggleton at linux.intel.com; yocto at yoctoproject.org
> 
> On Wed, Oct 21, 2015 at 5:10 AM, Jérôme OLIVIER
> <jerome.olivier at outlook.com> wrote:
> >
> > My situation is I have two machines:
> > * machineA
> > * machineB: inherits from machine B (only PREFERRED_VERSION for libxml2
> > added)
> >
> > two images:
> > * imageA
> > * imageB: inherits from image A
> >
> > I have one build directory for each machine:
> > * build-machineA
> > * build-machineB (only new packages added), two of the package are shared
> > libraries : libA and libB. libA is linked to libB. libB depends on libA.
> > libA and libB are cmake recipes
> >
> > First I build the imageA in build-machineA for machineA
> >
> > In the build-machineB directory I build imageB for machineB. Moreover I
> > point SSTATE_DIR or SSTATE_MIRRORS of build-machineB/conf/local.conf to
> > build-machineA's sstate-cache directory.
> >
> > libA is built with success but libB's do_compile fails:
> >
> > /.../build-machineB$ bitbake libB
> > [...]
> >
> > /.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/arm-opensdk-linux-gnueabi-gcc
> > -fPIC -g -O2 -W -Werror -Wall -Wextra -Wuninitialized -Wstrict-prototypes
> > -Wcast-align -Wmissing-prototypes -Wchar-subscripts -Wpointer-arith -Winline
> > -Wno-conversion -Wshadow  -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed
> > -shared -Wl,-soname,libsys.so.1 -o libsys.so.1
> > CMakeFiles/sys.dir/src/crc.c.o CMakeFiles/sys.dir/src/fast_32k_utils.c.o
> > CMakeFiles/sys.dir/src/filemap.c.o CMakeFiles/sys.dir/src/hash_tables.c.o
> > CMakeFiles/sys.dir/src/hash_tables_2.c.o
> > CMakeFiles/sys.dir/src/ipcs_keys.c.o CMakeFiles/sys.dir/src/log.c.o
> > CMakeFiles/sys.dir/src/MM.c.o CMakeFiles/sys.dir/src/MM_no_mutex.c.o
> > CMakeFiles/sys.dir/src/random.c.o CMakeFiles/sys.dir/src/registry.c.o
> > CMakeFiles/sys.dir/src/time.c.o CMakeFiles/sys.dir/src/watchdog_timer.c.o
> > CMakeFiles/sys.dir/src/watchdog_client.c.o
> > CMakeFiles/sys.dir/src/watchdog_server.c.o
> > CMakeFiles/sys.dir/src/state_machine.c.o
> > CMakeFiles/sys.dir/src/thread_manager.c.o
> > CMakeFiles/sys.dir/src/sg_log_eina.c.o -lpthread -leina -lpthread -leina
> >
> > /.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/../../libexec/arm-opensdk-linux-gnueabi/gcc/arm-opensdk-linux-gnueabi/4.9.1/ld:
> > cannot find -llibA
> >
> > collect2: error: ld returned 1 exit status
> >
> > ERROR: Function failed: do_compile (log file is located at
> > /.../build-machineB/tmp-glibc/work/cortexa9t2hf-vfp-neon-opensdk-linux-gnueabi/libB/1.0.0/temp/log.do_compile.2284)
> >
> >
> > At this moment I had to find why ld cannot find libA.so in sysroots so I
> > launched a devshell on libB. In the devshell I cd-ed to the B dir of libB
> > and run the cross-gcc command with the verbose "-v" option.
> >
> > /.../build-machineB$ bitbake libB -c devshell
> >
> > [devshell]
> > /.../build-machineB/tmp-glibc/work/cortexa9t2hf-vfp-neon-opensdk-linux-gnueabi/libB/1.0.0/git
> > $ cd ../build
> >
> > [devshell]
> > /.../build-machineB/tmp-glibc/work/cortexa9t2hf-vfp-neon-opensdk-linux-gnueabi/libB/1.0.0/build$
> > /.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/arm-opensdk-linux-gnueabi-gcc
> > -fPIC -g -O2 -W -Werror -Wall -Wextra -Wuninitialized -Wstrict-prototypes
> > -Wcast-align -Wmissing-prototypes -Wchar-subscripts -Wpointer-arith -Winline
> > -Wno-conversion -Wshadow  -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed
> > -shared -Wl,-soname,libsys.so.1 -o libsys.so.1
> > CMakeFiles/sys.dir/src/crc.c.o CMakeFiles/sys.dir/src/fast_32k_utils.c.o
> > CMakeFiles/sys.dir/src/filemap.c.o CMakeFiles/sys.dir/src/hash_tables.c.o
> > CMakeFiles/sys.dir/src/hash_tables_2.c.o
> > CMakeFiles/sys.dir/src/ipcs_keys.c.o CMakeFiles/sys.dir/src/log.c.o
> > CMakeFiles/sys.dir/src/MM.c.o CMakeFiles/sys.dir/src/MM_no_mutex.c.o
> > CMakeFiles/sys.dir/src/random.c.o CMakeFiles/sys.dir/src/registry.c.o
> > CMakeFiles/sys.dir/src/time.c.o CMakeFiles/sys.dir/src/watchdog_timer.c.o
> > CMakeFiles/sys.dir/src/watchdog_client.c.o
> > CMakeFiles/sys.dir/src/watchdog_server.c.o
> > CMakeFiles/sys.dir/src/state_machine.c.o
> > CMakeFiles/sys.dir/src/thread_manager.c.o
> > CMakeFiles/sys.dir/src/sg_log_eina.c.o -lpthread -leina -lpthread -leina
> >
> > /.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/../../libexec/arm-opensdk-linux-gnueabi/gcc/arm-opensdk-linux-gnueabi/4.9.1/ld:
> > cannot find –leina –v
> >
> 
> Try using ${CC} to call the compiler instead of giving an explicit PATH.
> 
> It's normal for the default sysroot compiled into the OE toolchain to
> be wrong or deliberately set to something invalid. You therefore need
> to take great care to always include the correct --sysroot option when
> calling the compiler or linker.
> 
> The CC, LD, etc definitions provided by OE should always include the
> correct --sysroot option, but you will see problems if the package
> you're building over-rides or doesn't use these standard variables (
> e.g. calls the compiler via ${CROSS_COMPILE}gcc ).
> 
> > I will only show you the interesting part of the result.
> >
> > [...]
> >
> >
> > LIBRARY_PATH=/.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/../../lib/arm-opensdk-linux-gnueabi/gcc/arm-opensdk-linux-gnueabi/4.9.1/:/.../build-machineB/tmp-glibc/sysroots/i686-linux/usr/bin/arm-opensdk-linux-gnueabi/../../lib/arm-opensdk-linux-gnueabi/gcc/:/.../build-machineA/tmp-glibc/sysroots/machineA/lib/:/.../build-machineA/tmp-glibc/sysroots/machineA/usr/lib/arm-opensdk-linux-gnueabi/4.9.1//.../build-machineA/tmp-glibc/sysroots/machineA/usr/lib/
> >
> >  [...]
> >
> >
> > The symptom is here. Instead of
> > ‘/.../build-machineA/tmp-glibc/sysroots/machineA/usr/lib/’ it should be
> > ‘/.../build-machineB//tmp-glibc/sysroots/machineB/usr/lib/’
> >
> >
> > I couldn’t find in libB’s recipe (and included bbclasses) where LIBRARY_PATH
> > is set so I guess it comes from elswhere.
> >
> > My guess is that there is a problem in cross-gcc recipe and somewhere the
> > <sysroots-target>/usr/lib path is hardcoded in sstate-cache. When
> > recompilling for a different build bitbake use this cache that is polluted
> > by this hardcoded path.
> >
> >
> > Note: I have the same problem if I do same build type (machine, builddir
> > name) but different locations.
> >
> > The meta layer of machineB/imageB has a override of gcc 4.9 which consists
> > only on:
> > EXTRA_OECONF_GCC_FLOAT = "${@get_gcc_float_setting(bb, d)}"
> >
> > I had some problem two while changing the version the PREFERRED_VERSION of a
> > recipe in local.conf. When recompiling bitbake don't redo do_unpack so
> > configure/compile/install, one of them fails to find file in S directory. I
> > had to run cleanall on this package before doing a full bitbake.
> >
> > I hop I have been clear in my explanation of the situation.
> >
> > Kind Regards,
> > --
> > Jérôme OLIVIER
> > TEL : +337 52 46 29 25
> > @ : jerome.olivier at outlook.com
> >
> >> From: paul.eggleton at linux.intel.com
> >> To: jerome.olivier at outlook.com
> >> CC: yocto at yoctoproject.org
> >> Subject: Re: [yocto] Disable sstate-cache for one recipe
> >> Date: Mon, 19 Oct 2015 12:21:48 +0100
> >>
> >> Hi Jérôme,
> >>
> >> On Monday 19 October 2015 10:18:41 Jérôme OLIVIER wrote:
> >> > I use yocto 1.7.2. I have some problem with sstate-cache.
> >> > I would like to disable totally sstate_cache for only one recipe. Is
> >> > that
> >> > possible?
> >>
> >> Not really I'm afraid - what is the problem you are experiencing?
> >>
> >> Cheers,
> >> Paul
> >>
> >> --
> >>
> >> Paul Eggleton
> >> Intel Open Source Technology Centre
> >
> > --
> > _______________________________________________
> > yocto mailing list
> > yocto at yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20151102/1809c225/attachment.html>