[yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/24/15 3:23 PM, Mark Hatle wrote:
> My guess is that there is a bug in the python integration where it's not
> realizing the host and target are different systems, so it's trying to run a
> target program on your host.  Your host isn't haswell, so... Illegal Instruction
> -- and the system stops.

Just an FYI, I hit the same problem today.  I suspect it is the host trying to
run target software and I'm looking into it.

--Mark

> The alternative would be something is running QEMU to execute a target binary
> and QEMU doesn't have instruction support -- but that doesn't look like the case
> here.
> 
> --Mark
> 
> On 11/24/15 3:06 PM, Michael Habibi wrote:
>> All,
>>
>> I added a new machine definition with tuning parameters for haswell
>> microarchitectures (basically just duplicated corei7 but tuned it for haswell).
>> This seems to work correctly, but failed when running do_install for python
>> toward the end of the build process. I am running with the Yocto 2.0 framework,
>> with very minimal changes to create a new distribution and machine for our
>> custom embedded device. Note I had this distro configuration working before, and
>> the only difference is I added a new machine with this tuning.
>>
>> I believe the issue is because, as part of the do_install step for Python, it
>> attempts to run python on the local host, despite being cross-compiled. However,
>> it is tuned for a processor that my host machine doesn't run, so I get an
>> instruction exception.
>>
>> Did I do something weird? I figure python would be configured for
>> cross-compiling and therefore wouldn't try to run the target version on the
>> host, even for sanity tests. Here is the output of the failure:
>>
>> $ tail -20
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>
>> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> x86_64-diags-linux-ranlib libpython2.7.a
>> Modules/posixmodule.o: In function `posix_tmpnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>> Modules/posixmodule.o: In function `posix_tempnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse -mavx2
>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
>>                         Modules/python.o \
>>                         -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil   -lm
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> python2.7 -S -m sysconfig --generate-posix-vars ;\
>>         if test $? -ne 0 ; then \
>>                 echo "generate-posix-vars failed" ; \
>>                 rm -f ./pybuilddir.txt ; \
>>                 exit 1 ; \
>>         fi
>> Illegal instruction (core dumped)
>> make: *** [sharedmods] Error 132
>> WARNING: exit code 1 from a shell command.
>> ERROR: oe_runmake failed
>> ERROR: Function failed: do_install (log file is located at
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>
>> Here is my tune-haswell.inc (ignore some copy/paste comment issues right now =):
>>
>> # Settings for the GCC(1) cpu-type "haswell":
>> #
>> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2, SSE3,·
>> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>> #
>> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay Trail) CPUs
>> # (and beyond).
>> #
>> DEFAULTTUNE ?= "haswell"
>>
>> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>> require conf/machine/include/tune-corei7.inc
>>
>> # Extra tune features
>> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>>
>> # Extra tune selections
>> AVAILTUNES += "haswell"
>> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
>> BASE_LIB_tune-haswell = "lib"
>> TUNE_PKGARCH_tune-haswell = "haswell"
>> PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>>
>>
>>
>