[yocto] Recursions problem with pre-built versioned libraries and oe_soinstall

[John McCabe]

Ross,

Thank you for your reply.

Please see interspersed comments below.

> On 28 June 2019 at 11:53 "Burton, Ross" <ross.burton at intel.com> wrote: 
> 
> On Thu, 27 Jun 2019 at 17:58, John McCabe < JMcCabe at kirintec.com> wrote:
> > Exception: OSError: [Errno 40] too much recursions while resolving '<project-base>/build/tmp/work/cortexa9hf-neon-xilinx-linux-gnueabi/opendds/1.0-r0/packages-split/opendds/usr/lib/libTAO_PI_Server.so.2.2a_p15'; loop in '<project-base>/build/tmp/work/cortexa9hf-neon-xilinx-linux-gnueabi/opendds/1.0-r0/packages-split/opendds/usr/lib/libTAO_PI_Server.so.2.2a_p15'
>   
> > lrwxrwxrwx 1 jmccabe jmccabe 28 Jun 27 17:38 libTAO_PI_Server.so.2.2a_p15 -> libTAO_PI_Server.so.2.2a_p15
>  
> You can't deny that this is a symlink loop.

Yes, it is most definitely a symlink loop/fault.

oe_soinstall does this (from https://git.yoctoproject.org/cgit.cgi/poky/plain/meta/classes/utils.bbclass):

01 oe_soinstall() {
02 	# Purpose: Install shared library file and
03 	#          create the necessary links
04 	# Example: oe_soinstall libfoo.so.1.2.3 ${D}${libdir}
05 	libname=`basename $1`
06 	case "$libname" in
07 	    *.so)
08 	        bbfatal "oe_soinstall: Shared library must haved versioned filename (e.g. libfoo.so.1.2.3)"
09 	        ;;
10 	esac
11 	install -m 755 $1 $2/$libname
12 	sonamelink=`${HOST_PREFIX}readelf -d $1 |grep 'Library soname:' |sed -e 's/.*\[\(.*\)\].*/\1/'`
13 	if [ -z $sonamelink ]; then
14 		bbfatal "oe_soinstall: $libname is missing ELF tag 'SONAME'."
15 	fi
16 	solink=`echo $libname | sed -e 's/\.so\..*/.so/'`
17 	ln -sf $libname $2/$sonamelink
18 	ln -sf $libname $2/$solink
19 }

Which is as I described so, at the end of it, lines 17 & 18 have created symbolic links in the <output>/usr/lib folder to the ${WORKDIR} copy of the versioned shared library.

Later on the "def perform_packagecopy(d)" function (see https://git.yoctoproject.org/cgit.cgi/poky/plain/meta/classes/package.bbclass, probably!) is run. In run.do_package this shows as:

01def perform_packagecopy(d):
02    enabled = oe.data.typed_value('LICENSE_CREATE_PACKAGE', d)
03    if d.getVar('CLASSOVERRIDE', True) == 'class-target' and enabled:
04        lic_files_paths = find_license_files(d)
05
06        # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join D and LICENSE_FILES_DIRECTORY
07        destdir = d.getVar('D', True) + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY', True), d.getVar('PN', True))
08        copy_license_files(lic_files_paths, destdir)
09        add_package_and_files(d)
10    dest = d.getVar('D', True)
11    dvar = d.getVar('PKGD', True)
12
13    # Start by package population by taking a copy of the installed
14    # files to operate on
15    # Preserve sparse files and hard links
16    cmd = 'tar -cf - -C %s -p . | tar -xf - -C %s' % (dest, dvar)
17    (retval, output) = oe.utils.getstatusoutput(cmd)
18    if retval:
19        bb.fatal("file copy failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
20
21    # replace RPATHs for the nativesdk binaries, to make them relocatable
22    if bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('cross-canadian', d):
23        rpath_replace (dvar, d)

It's line 16 that's breaking things; that tar command claims to preserve sparse files and hard links, but I've tried running that command manually and it certainly doesn't preserve symbolic links.

> Note that these are all the users of oe_libinstall in oe-core:
> 
> $ git grep -l oe_soinstall
> meta/classes/utils.bbclass

Yip - I looked through the core files to see if there were other recipes that used it and found the same result.

> So I'm not massively surprised that it's buggy.  In this case, it looks like the library either has an incorrect soname, or the install function isn't handling the soname being the same: it assumes that it is being passed libfoo.so.1.2.3 which has a soname of libfoo.so.1 and will copy libfoo.so.1.2.3 and then create libfoo.so.1 and libfoo.so symlinks. 

Everything that's done as part of do_install is working as it "should" (!?) if you follow through the code. As described, the result of the do_install is that there is a file called libTAO_PI_Server.so.2.2a_p15 in ${WORKDIR}, and in ${WORKDIR}/image/usr/lib there are two symbolic links; libTAO_PI_Server.so.2.2a_p15 and libTAO_PI_Server.so, both of which point to the file in ${WORKDIR}. When the tar commands are run in perform_packagecopy, you end up with this:

lrwxrwxrwx 1 jmccabe jmccabe 28 Jun 27 17:38 libTAO_PI_Server.so -> libTAO_PI_Server.so.2.2a_p15
lrwxrwxrwx 1 jmccabe jmccabe 28 Jun 27 17:38 libTAO_PI_Server.so.2.2a_p15 -> libTAO_PI_Server.so.2.2a_p15

What confuses me are that there are a small number of examples out there on how to use oe_soinstall, and a couple of comments here and there about how it's what you should be using to install versioned shared libraries, particularly here bugzilla.yoctodev.org/wiki/TipsAndTricks/Packaging_Prebuilt_Libraries#Versioned_Libraries, and it looks quite sensible and simple until you come to try to use it!

>   My recommendations:
  
> 1) Write a proper recipe instead of building code outside of OE and trying to copy it in by hand.  Or,

In the long run that would be the plan, however, I'm still in the early stages so thought that it would be nice to avoid having to rebuild the OpenDDS stuff as it takes, as I mentioned, a number of hours to build!

> 2) Determine what the correct structure is regarding sonames, and either fix oe_soinstall

Now there's a question; how to determine whether it's actually oe_soinstall that's at fault (which, clearly, appears to be very rarely used!), or perform_packagecopy (which is vastly more widely used) and then work out how to fix it!

> or just use install/ln directly.

I suspect this will be the path I need to take, unless anyone else has any useful suggestions.

Many thanks
John