[yocto] Cross-compiling python fails with new custom tuning
Mark Hatle
mark.hatle at windriver.com
Mon Nov 30 16:07:37 PST 2015
This is the patch that we ended up using:
http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html
http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html
http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html
On 11/30/15 5:12 PM, Michael Habibi wrote:
> OK I've dug into this further today, and here are some findings:
>
> With my patch (which is similar to yours), it fails do_compile/package_qa
> because of poisoned system directories during the compilation steps. This
> approach seems to be a bust. Originally when I tested it, I hacked up the
> Makefile and only re-ran do_install onward, and didn't re-run do_compile
> (because I had to create the patch). Once I created the patch and re-ran from
> clean, I got some errors.
>
> It appears the only reason it is using that directory in PYTHONPATH at all (the
> one generated from the shell command and pybuilddir.txt) is for a potential
> sysconfig file in there. I tracked it down to a comment in a bug on Python's bug
> tracker:
>
> http://bugs.python.org/issue15484#msg180577
>
> If we change that directory, it will pull in the sysconfig settings from the
> native Python configure (probably defaults to native sysconfig settings because
> the PATH doesn't include the overrides), and then do some really bad things. The
> question then became: in the unmodified build, why does do_install fail but
> do_compile doesn't? They both seem to be using sharedmods recipe which uses
> $PYTHONPATH. The answer is that when we run do_compile the first time around,
> the offending *.so's are not generated yet, and thus are not included by
> python-native. The second time around (in do_install), the *.so's are in the
> path and so python-native will use them and override its own.
>
> My latest experiment is to remove that location altogether from PYTHONPATH and
> see if it works (neither the shell-generated directory in python-cross, nor the
> lib-dynload directory from python-native). The compile works the first time
> through despite there being no sysconfig data, so I'm wondering if the install
> step will also work without finding any data. python-native should continue to
> work with its internal settings to find modules and libraries.
>
> To be continued tomorrow, I'm sure...
>
> On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehabibi at gmail.com
> <mailto:mikehabibi at gmail.com>> wrote:
>
> Mark, I am continuing to look at this. I hope you don't mind if I keep
> updating this thread with my investigation.
>
> First, I figured out my confusion. I was associating the wrong Makefile
> command with the illegal instruction. It was actually the command under the
> recipe for 'sharedmods', but it was silenced in the Makefile so I assumed it
> was another line. You're right that the path with the *.so's is the
> offending path. I've scoured online patches and can't understand how this
> mechanism works outside of Yocto. I figure people trying to compile for ARM
> would have similar issues. There seems to be a fundamental problem with how
> Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
> However, other people seem to have successfully built python for other
> architectures, so I'm still confused.
>
> I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, and
> the python recipe successfully passed. Can you elaborate on the issues you
> were having with your patch? The only difference is that I maintained the
> other two directories in $PYTHONPATH,
> e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
> offending one.
>
> On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehabibi at gmail.com
> <mailto:mikehabibi at gmail.com>> wrote:
>
> Mark,
>
> I ran the same command, including the offending PYTHONPATH, from the
> shell without error. Any reason why that would work, despite it not
> working from within the Makefile? I was thinking it was another
> environment variable that was causing the issue (one that's not set in
> my bash environment by default). My $PATH didn't have python-native so I
> called it using an absolute path, but otherwise it's the same command
> run from the Makefile (parsed, of course):
>
> bash$
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
> -S -m sysconfig --generate-posix-vars
> bash$ ls pybuilddir.txt
> pybuilddir.txt
>
> Maybe I'm missing something.
>
>
> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.hatle at windriver.com
> <mailto:mark.hatle at windriver.com>> wrote:
>
> On 11/25/15 10:11 AM, Michael Habibi wrote:
> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to a
> > native version of Python:
> >
> > *| which python2.7*
> > *|
> >
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> > |
> > _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > _PYTHON_HOST_PLATFORM=linux2-x86_64
> > PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > python2.7 -S -m sysconfig --generate-posix-vars ;\
> > | if test $? -ne 0 ; then \
> > | echo "generate-posix-vars failed" ; \
> > | rm -f ./pybuilddir.txt ; \
> > | exit 1 ; \
> > | fi
> > | Illegal instruction (core dumped)
> > | make: *** [sharedmods] Error 132
> >
> > I guess one of the environmental variables being passed in is screwing up what
> > it's doing. Unfortunately I don't know enough about the inner workings of Python
> > to be of much help moving forward. I gave it my best shot!
>
> The problem I tracked down was:
>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>
> This is full of .so objects when loaded cause the illegal instruction.
>
> I came up with a patch that I thought was going to fix it, but it's
> triggered
> other failures.
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>
> Just to be clear.. this does NOT work properly in many cases. But
> might give
> you or someone else a clue as to how to possibly fix it.
>
> --Mark
>
> >
> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi at gmail.com <mailto:mikehabibi at gmail.com>
> > <mailto:mikehabibi at gmail.com <mailto:mikehabibi at gmail.com>>> wrote:
> >
> > Ah sorry, I misspoke. I walked through the Makefile and configure scripts a
> > bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> > Python. It seems it's not being configured correctly. Either the path is
> > setup wrong and python2.7 is not pointing to the right version, or python2.7
> > needs to be pointing to the absolute path instead of relying on $PATH.
> >
> >
> >
> >
> > On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehabibi at gmail.com <mailto:mikehabibi at gmail.com>
> > <mailto:mikehabibi at gmail.com <mailto:mikehabibi at gmail.com>>>
> wrote:
> >
> > For what it's worth, this is the offending portion of
> Makefile.pre:452:
> >
> > # Create build directory and generate the sysconfig
> build-time data there.
> > # pybuilddir.txt contains the name of the build dir and is
> used for
> > # sys.path fixup -- see Modules/getpath.c.
> > # Since this step runs before shared modules are built,
> try to avoid
> > bootstrap
> > # problems by creating a dummy pybuilddir.txt just to
> allow interpreter
> > # initialization to succeed. It will be overwritten by
> generate-posix-vars
> > # or removed in case of failure.
> > pybuilddir.txt: $(BUILDPYTHON)
> > → @echo "none" > ./pybuilddir.txt
> > → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
> --generate-posix-vars ;\
> > → if test $$? -ne 0 ; then \
> > → → echo "generate-posix-vars failed" ; \
> > → → rm -f ./pybuilddir.txt ; \
> > → → exit 1 ; \
> > → fi
> >
> > I don't know enough about the Python build to understand
> what it's
> > trying to do, but perhaps replacing PYTHON_FOR_BUILD with
> HOSTPYTHON may
> > be acceptable?
> >
> > I'm surprised this seems to work for other people, since
> this should be
> > failing for anyone using Python on a target platform
> different from
> > their host.
> >
> > On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle
> <mark.hatle at windriver.com <mailto:mark.hatle at windriver.com>
> > <mailto:mark.hatle at windriver.com
> <mailto:mark.hatle at windriver.com>>> wrote:
> >
> > On 11/24/15 3:23 PM, Mark Hatle wrote:
> > > My guess is that there is a bug in the python
> integration where
> > it's not
> > > realizing the host and target are different systems,
> so it's
> > trying to run a
> > > target program on your host. Your host isn't
> haswell, so...
> > Illegal Instruction
> > > -- and the system stops.
> >
> > Just an FYI, I hit the same problem today. I suspect
> it is the host
> > trying to
> > run target software and I'm looking into it.
> >
> > --Mark
> >
> > > The alternative would be something is running QEMU
> to execute a
> > target binary
> > > and QEMU doesn't have instruction support -- but
> that doesn't look
> > like the case
> > > here.
> > >
> > > --Mark
> > >
> > > On 11/24/15 3:06 PM, Michael Habibi wrote:
> > >> All,
> > >>
> > >> I added a new machine definition with tuning
> parameters for haswell
> > >> microarchitectures (basically just duplicated
> corei7 but tuned it
> > for haswell).
> > >> This seems to work correctly, but failed when
> running do_install
> > for python
> > >> toward the end of the build process. I am running
> with the Yocto
> > 2.0 framework,
> > >> with very minimal changes to create a new
> distribution and
> > machine for our
> > >> custom embedded device. Note I had this distro
> configuration
> > working before, and
> > >> the only difference is I added a new machine with
> this tuning.
> > >>
> > >> I believe the issue is because, as part of the
> do_install step
> > for Python, it
> > >> attempts to run python on the local host, despite being
> > cross-compiled. However,
> > >> it is tuned for a processor that my host machine
> doesn't run, so
> > I get an
> > >> instruction exception.
> > >>
> > >> Did I do something weird? I figure python would be
> configured for
> > >> cross-compiling and therefore wouldn't try to run
> the target
> > version on the
> > >> host, even for sanity tests. Here is the output of
> the failure:
> > >>
> > >> $ tail -20
> > >>
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> > >>
> > >> x86_64-diags-linux-ar rc libpython2.7.a
> Modules/threadmodule.o
> > >> Modules/signalmodule.o Modules/posixmodule.o
> Modules/errnomodule.o
> > >> Modules/pwdmodule.o Modules/_sre.o
> Modules/_codecsmodule.o
> > >> Modules/_weakref.o Modules/zipimport.o
> Modules/symtablemodule.o
> > >> Modules/md5module.o Modules/md5.o Modules/xxsubtype.o
> > >> x86_64-diags-linux-ranlib libpython2.7.a
> > >> Modules/posixmodule.o: In function `posix_tmpnam':
> > >>
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> > >> warning: the use of `tmpnam_r' is dangerous, better
> use `mkstemp'
> > >> Modules/posixmodule.o: In function `posix_tempnam':
> > >>
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> > >> warning: the use of `tempnam' is dangerous, better
> use `mkstemp'
> > >> x86_64-diags-linux-gcc -m64 -march=haswell
> -mtune=haswell
> > -mfpmath=sse -mavx2
> > >>
> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> > >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker
> -export-dynamic -o
> > python \
> > >> Modules/python.o \
> > >> -L. -lpython2.7 -lpthread
> -ldl -lpthread
> > -lutil -lm
> > >>
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> > >>
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > >> python2.7 -S -m sysconfig --generate-posix-vars ;\
> > >> if test $? -ne 0 ; then \
> > >> echo "generate-posix-vars failed" ; \
> > >> rm -f ./pybuilddir.txt ; \
> > >> exit 1 ; \
> > >> fi
> > >> Illegal instruction (core dumped)
> > >> make: *** [sharedmods] Error 132
> > >> WARNING: exit code 1 from a shell command.
> > >> ERROR: oe_runmake failed
> > >> ERROR: Function failed: do_install (log file is
> located at
> > >>
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
> > >>
> > >> Here is my tune-haswell.inc (ignore some copy/paste
> comment
> > issues right now =):
> > >>
> > >> # Settings for the GCC(1) cpu-type "haswell":
> > >> #
> > >> # Intel Core i7 CPU with 64-bit extensions,
> MOVBE, MMX, SSE,
> > SSE2, SSE3,·
> > >> # SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2,
> AES, PCLMUL,
> > FSGSBASE,
> > >> # RDRND, FMA, BMI, BMI2 and F16C instruction
> set support.
> > >> #
> > >> # This tune is recommended for Intel Nehalem and
> Silvermont (e.g.
> > Bay Trail) CPUs
> > >> # (and beyond).
> > >> #
> > >> DEFAULTTUNE ?= "haswell"
> > >>
> > >> # Pull in the previous tune in to pull in
> PACKAGE_EXTRA_ARCHS
> > >> require conf/machine/include/tune-corei7.inc
> > >>
> > >> # Extra tune features
> > >> TUNEVALID[haswell] = "Enable haswell specific processor
> > optimizations"
> > >> TUNE_CCARGS .=
> "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
> > >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2",
> "", d)}"
> > >>
> > >> # Extra tune selections
> > >> AVAILTUNES += "haswell"
> > >> TUNE_FEATURES_tune-haswell =
> "${TUNE_FEATURES_tune-x86-64} haswell"
> > >> BASE_LIB_tune-haswell = "lib"
> > >> TUNE_PKGARCH_tune-haswell = "haswell"
> > >> PACKAGE_EXTRA_ARCHS_tune-haswell =
> > "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
> > >>
> > >>
> > >>
> > >
> >
> > --
> > _______________________________________________
> > yocto mailing list
> > yocto at yoctoproject.org <mailto:yocto at yoctoproject.org>
> <mailto:yocto at yoctoproject.org <mailto:yocto at yoctoproject.org>>
> > https://lists.yoctoproject.org/listinfo/yocto
> >
> >
> >
> >
>
>
>
>
More information about the yocto
mailing list