[meta-virtualization] Help (python-jsonschema do_compile Failed)

[Li, Xin]

Hi, Mark

Thank you very much for your kindly reply.

>>>
>>>A download error during do_compile(). My guess is that we have a
>>>missing dependency. With python, Ruby and some other packages we have
>>>to be careful as their builds will actually cause downloads to happen
>>>if a dependency is missing (python will fetch eggs, ruby will fetch
>>>gems...). When I have time I usually do builds with IP tables rules in
>>>place to force errors when these fetches happen so that I can catch
>>>them and fix them. Downloads should only be done by the fetcher and
>>>not bundler, pdb etc. If you see an error about downloading and you are not in
>do_fetch() something is wrong.
>>>
>>>Start by adding python-vcversioner as a DEPENDS in python-jsonschema
>>>and you should be good to go.
>>>
>>
>> The python-vcversioner is already as a DEPENDS in python-jsonschema,
>> and It is still failed.
>>
>> Perhaps it is the problem about proxy set.
>
>No, again this would just be band-aid for a real problem. I did have time to look at
>this today, but not enough to find a suitable solution.
>
>The issue is "setup_requires" in setup.py for python-jsonschema. This is a rarely
>used directive that has some unexpected consequences. See:
>
>https://github.com/pypa/pip/issues/410
>https://pip.pypa.io/en/latest/user_guide/#installation-bundles
>(warning at bottom of section)
>https://pip.pypa.io/en/latest/reference/pip_install/#controlling-setup-requires
>
>I can reproduce your issue by using iptables rules to DROP packets on port 80 and
>443.
>
>The solution is one of 3.
>1) We can cleanup the setup.py to not require vcversioner. This is most likely the
>simplest but may have bad side effects.
>2) We build vcversioner .egg file in an addition to do_compile() and add it to the
>-dev pkg to ensure it is in the sysroot. We then copy the egg from the sysroot to
>our build as a step before python-jsonschema do_compile(). This is most likely the
>best fix.
>3) We build the .egg as in (2) but build an index file in the sysroot, allowing us to
>use find_links
>
>I really believe (2) is the best approach. Building the egg and having it in the
>sysroot is very straightforward, the copy shouldn't be that difficult either. I can
>attempt this but most likely won't have a fix until tomorrow. For now manually
>download the egg for vcversioner and place it in
>tmp/work/core2-64-poky-linux/python-jsonschema/2.5.1-r0/jsonschema-2.5.1/.e
>ggs/vcversioner-2.14.0.0-py2.7.egg
>and you should be able to at least continue to build.

I chose (2) solution. And it is build with no errors, so how about this Modify?

1. Modify python-vcversioner_2.14.0.0.bb, and create vcversioner-2.14.0.0-py2.7.egg by do_compile_append() .
And then install it. 
-------------------------------------
diff --git a/meta-openstack/recipes-devtools/python/python-vcversioner_2.14.0.0.bb b/meta-openstack/recipes-devtools/p
index 2fe96ab..b8131b1 100644
......
-inherit setuptools
+inherit setuptools pythonnative
+
+do_compile_append() {
+    export BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS}
+    ${PYTHON} setup.py -q bdist_egg --dist-dir ./
+}
+
+do_install_append() {
+    install -m 0644 ${S}/vcversioner-2.14.0.0-py2.7.egg ${D}/${PYTHON_SITEPACKAGES_DIR}/
+}
--------------------------------------

2. Modify python-jsonschema_2.5.1.bb, copy vcversioner-2.14.0.0-py2.7.egg from sysroot to ${S}/.eggs/
-------------------------------------- 
$ git diff meta-openstack/recipes-devtools/python/python-jsonschema_2.5.1.bb
diff --git a/meta-openstack/recipes-devtools/python/python-jsonschema_2.5.1.bb b/meta-openstack/recipes-devtools/pytho
index a3a7ec0..a2055c8 100644
--- a/meta-openstack/recipes-devtools/python/python-jsonschema_2.5.1.bb
+++ b/meta-openstack/recipes-devtools/python/python-jsonschema_2.5.1.bb
@@ -16,4 +16,9 @@ S = "${WORKDIR}/${SRCNAME}-${PV}"

 inherit setuptools

-DEPENDS += "python-vcversioner"
+DEPENDS += "python-vcversioner python-pip"
+
+do_compile_prepend() {
+    install -d ${S}/.eggs
+    install -m 0644 ${STAGING_DIR_HOST}/${PYTHON_SITEPACKAGES_DIR}/vcversioner-2.14.0.0-py2.7.egg ${S}/.eggs/
+}
-------------------------------------- 

Thanks a lot for your help.

Regards,
Li

>
>>
>> When I bitbake python-jsonschema, the error occurs in the step of
>> do_compile, But when I into directory
>"python-jsonschema/2.5.1-r0/jsonschema-2.5.1/ ", and run.do_complie, no
>errors occur.
>> -----------------------------
>> $ cd build-x86_64/tmp/work/x86_64
>> -linux/python-jsonschema/2.5.1-r0/jsonschema-2.5.1/
>> $ ../temp/run.do_compile
>> zip_safe flag not set; analyzing archive contents...
>>
>> Installed ***/work/x86_64
>> -linux/python-jsonschema/2.5.1-r0/jsonschema-2.5.1/.eggs/vcversioner-2
>> .14.0.0-py2.7.egg
>> running build
>> running build_py
>> creating build
>> creating build/lib
>> creating build/lib/jsonschema
>> ......
>> $ echo $?
>> 0
>> ---------------------------------
>>
>> And I have proxy set in ~/.wgetrc like this:
>> http_proxy = http://rep.proxy.nic.fujitsu.com:8080/
>> ftp_proxy = http://rep.proxy.nic.fujitsu.com:8080/
>> https_proxy=https://rep.proxy.nic.fujitsu.com:8080/
>>
>> If you know how to fix it, please help me.
>>
>> Regards,
>> Li
>>
>>
>> --
>> _______________________________________________
>> meta-virtualization mailing list
>> meta-virtualization at yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-virtualization
>