[yocto] Error in do_image_wic in Docker container
Frazer, Will
Will.Frazer at eurotech.com
Fri Sep 13 09:11:30 PDT 2019
I thought that I would post a follow up to this as a recent kernel update (in Ubuntu 18.04.3 – kernel 5.0.x) pulled in a patch which breaks the WIC tool in poky when running on overlayfs.
Previously Ubuntu 18.04.1 used 4.15.x which does not exhibit the issue. The patch appears in kernel 4.19.x onwards.
https://lkml.org/lkml/2018/11/19/1430
Following my earlier posts, I was building poky images in docker and generating images with the wic tool.
The python script:
poky/scripts/lib/wic/filemap.py
function
get_block_size()
at :
# Get the block size of the host file-system for the image file by calling
# the FIGETBSZ ioctl (number 2).
binary_data = fcntl.ioctl(file_obj, 2, struct.pack('I', 0))
was getting a return value of zero for this IOCTL while generating the WIC image on overlayfs in docker. Up until the kernel patch, it was succeeding by falling through to use os.stat if we got zero returned.
The kernel patch above causes the return to be EINVAL (-1) so the WIC process exits with an error.
The kernel patch is correct (well that’s one interpretation) so I believe the problem is in the poky script in that it is attempting but failing to handle the IOCTL return value reporting unsupported on overlayfs (which is the recommended FS for Docker).
I suggest not using the IOCTL method and using os.stat instead
HTH
/Will
From: yocto-bounces at yoctoproject.org <yocto-bounces at yoctoproject.org> On Behalf Of Frazer, Will
Sent: 23 July 2018 14:21
To: Tim Orling <ticotimo at gmail.com>; Darcy Watkins <dwatkins at sierrawireless.com>
Cc: yocto at yoctoproject.org
Subject: Re: [yocto] Error in do_image_wic in Docker container
Hi Darcy,
Thanks for your reply.
I wanted to respond so that others know what I believe the cause of my issue is/was:
Basically, in my Docker setup the storage driver is reporting as AUFS. My host is Ubuntu 14.04.x (so not MacOS or Windows, but thanks for the reply Tom). I used the defaults when I configured Docker originally so perhaps this is actually OverlayFS. Even if it is not overlayfs, my concern is that it may suffer similar to aufs.
The yocto ‘do_wic_image’ task calls the poky wic scripts to attempt to get capability information from the filesystem using ‘filemap.py’ etc. For me, this generates a ZeroDivisionError failure when get_block_size() is called.
I attempted to resolve this issue using os.stat as suggested [1] however the error just moved elsewhere. I’m not sure how far I would need to chase this in poky so I’m considering some other options. Currently, these are:
* Adjust the container volume mounts to provide an ext4 filesystem to keep WIC happy. In hindsight, not dissimilar to your arrangement.
* Switch the docker storage driver [2] to use devicemapper with direct-lvm.
I don’t really want to put the build folder outside of the container (I have many containers with similar and dissimilar build objectives) so I’m thinking the latter for me may be a better long term solution.
Anyway, thought I would share.
Thanks again,
Will
Will Frazer | EUROTECH | direct +44 (0) 1223 403431 | mobile: +44 (0) 7736 198115
[1] https://github.com/intel/bmap-tools/issues/15
[2] https://docs.docker.com/storage/storagedriver/select-storage-driver/
From: Darcy Watkins [mailto:dwatkins at sierrawireless.com]
Sent: 18 July 2018 17:10
To: Frazer, Will <Will.Frazer at eurotech.com<mailto:Will.Frazer at eurotech.com>>; yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: RE: Error in do_image_wic in Docker container
Hi Will,
I managed to get a docker Yocto build under Linux working, but using a simplified volume mounting scheme. I ran into an issue building with CentOS 7.5 so I used CentOS 7.4 in a container (since the build used to always work before the CentOS update). I haven’t gotten back to trying it under MacOS.
Regards,
Darcy
Darcy Watkins :: Senior Staff Engineer, Firmware
SIERRA WIRELESS
Direct +1 604 233 7989 :: Fax +1 604 231 1109 :: Main +1 604 231 1100
13811 Wireless Way :: Richmond, BC Canada V6V 3A4
[P2]
dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com> :: www.sierrawireless.com<http://www.sierrawireless.com/>
From: Frazer, Will <Will.Frazer at eurotech.com<mailto:Will.Frazer at eurotech.com>>
Sent: July-17-18 9:32 AM
To: Darcy Watkins <dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com>>; yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: RE: Error in do_image_wic in Docker container
Hi Darcy,
Don’t suppose you solved this?
I have a yocto 2.3 build that works OK in a virtual machine but seems to throw a v.similar error to what you have reported below.
Similarly I have various mounts outside of the container to inject pre-build components & pick up build artifacts.
Thanks,
Will
Will Frazer | EUROTECH | direct +44 (0) 1223 403431 | mobile: +44 (0) 7736 198115
From: yocto-bounces at yoctoproject.org<mailto:yocto-bounces at yoctoproject.org> [mailto:yocto-bounces at yoctoproject.org] On Behalf Of Darcy Watkins
Sent: 25 March 2018 17:41
To: yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: [yocto] Error in do_image_wic in Docker container
Hi,
This is near end of a build based on rocko branch plus a custom BSP layer, etc. I have had it build OK previously under Linux (CentOS 7). This run is inside a Docker container running on my MacBook. From what I gather from google search results, is the underlying OS filesystem doesn’t support an operation.
I use a volume mount scheme as follows…
docker run -it \
--volume=${PWD}:/home/build:cached \
--volume=${HOME}/.ssh:/home/build/.ssh \
--volume=${PWD}/downloads:/home/build/downloads:delegated \
--rm --volume=${WS_NAME}_build_vol:/home/build/build \
--volume=${PWD}/build/conf:/home/build/build/conf:cached \
--volume=${PWD}/build/deploy:/home/build/build/tmp/deploy:delegated \
--volume=${TARPIT}:/home/public/tarballs:cached \
$DIMG bash
The checkout is mounted from MacOS at /home/build to keep source meta-data accessible by MacOS text editors.
The build directory is inside a docker volume so that it doesn’t have case insensitivity issues.
The deploy dir and the conf dir are mounted inside this volume so that when docker exits I still have MacOS access to the conf and the deployed content.
The TARPIT is a premirror implementation and DIMG is the docker container used.
Does WIC do weird filesystem stuff (obviously ‘yes’), but does it do this in the deploy dir rather than in one of the work dirs?
I think I missed something (almost like a “fool’s mate” in Chess, LOL). Any insight on this appreciated.
Does deploy dir have to be entirely inside the volume, or is this a limitation of Docker implementation itself?
Thanks!
---
ERROR: core-image-minimal-1.0-r0 do_image_wic: Function failed: do_image_wic (log file is located at /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406)
ERROR: Logfile of failure stored in: /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/home/build/upstream/yocto/poky/meta/recipes-devtools/python/python-native_2.7.13.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.3.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/opkg/opkg_0.3.5.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/tcltk/tcl_8.6.7.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.43.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/dosfstools/dosfstools_4.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-core/update-rc.d/update-rc.d_0.7.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/bc/bc_1.06.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/binutils/binutils-cross_2.29.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/createrepo-c/createrepo-c_git.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/rpm/rpm_git.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-connectivity/openssl/openssl_1.0.2n.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/meta-freescale/recipes-bsp/change-file-endianess/change-file-endianess.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/subversion/subversion_1.9.6.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.43.5.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-bsp/u-boot/u-boot-mkimage_2017.09.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/libgcc_linaro-7.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-kernel/dtc/dtc_1.4.4.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/gcc-runtime_linaro-7.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/mtools/mtools_4.0.18.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/openembedded/meta-openembedded/meta-python/recipes-devtools/python/python-pyyaml_3.12.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/dnf/dnf_2.6.3.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/pigz/pigz_2.3.4.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/parted/parted_3.2.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-bsp/u-boot/u-boot-qoriq_2017.09.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-bsp/qe-ucode/qe-ucode_git.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/gcc-cross_linaro-7.1.bb:do_populate_sysroot', '/home/build/meta-MG-os-bsp/recipes-bsp/fec/fec_3.0.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-kernel/linux/linux-qoriq_4.9.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.3.5.bb:do_populate_sysroot', '/home/build/meta-MG-os-bsp/recipes-bsp/omg-hardware/omg-hardware_svn.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-support/bmap-tools/bmap-tools_3.4.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: ['python-native', 'gptfdisk-native', 'tcl-native', 'dosfstools-native', 'bc-native', 'binutils-cross-arm', 'change-file-endianess-native', 'subversion-native', 'mtools-native', 'python-pyyaml-native', 'lzop-native', 'parted-native', 'gcc-cross-arm', 'linux-qoriq', 'apr-util-native', 'serf-native', 'python-setuptools-native', 'python-cython-native', 'libyaml-native', 'mpfr-native', 'libmpc-native', 'gmp-native', 'apr-native', 'gdbm-native', 'python-scons-native']
| NOTE: Skipping as already exists in sysroot: ['opkg-native', 'qemuwrapper-cross', 'mklibs-native', 'makedevs-native', 'kmod-native', 'update-rc.d-native', 'createrepo-c-native', 'rpm-native', 'glibc', 'openssl-native', 'e2fsprogs-native', 'u-boot-mkimage-native', 'libgcc', 'depmodwrapper-cross', 'dtc-native', 'gcc-runtime', 'dnf-native', 'ldconfig-native', 'pigz-native', 'quilt-native', 'u-boot-qoriq', 'qe-ucode', 'fec', 'opkg-utils-native', 'omg-hardware', 'bmap-tools-native', 'pseudo-native', 'prelink-native', 'sqlite3-native', 'libtool-native', 'bzip2-native', 'autoconf-native', 'expat-native', 'xz-native', 'gnu-config-native', 'readline-native', 'pkgconfig-native', 'zlib-native', 'automake-native', 'qemu-native', 'util-linux-native', 'popt-native', 'ncurses-native', 'libarchive-native', 'libsolv-native', 'gettext-minimal-native', 'gtk-doc-native', 'texinfo-dummy-native', 'flex-native', 'bison-native', 'glib-2.0-native', 'cmake-native', 'python3-native', 'libxml2-native', 'file-native', 'curl-native', 'dbus-native', 'nss-native', 'elfutils-native', 'db-native', 'linux-libc-headers', 'cryptodev-linux-native', 'makedepend-native', 'attr-native', 'shadow-native', 'librepo-native', 'python3-iniparse-native', 'libdnf-native', 'libcomps-native', 'lzo-native', 'omg-mcu', 'python3-setuptools-native', 'binutils-native', 'm4-native', 'alsa-lib-native', 'libsdl-native', 'pixman-native', 'libffi-native', 'libpcre-native', 'gettext-native', 'nspr-native', 'xproto-native', 'util-macros-native', 'unzip-native', 'libcheck-native', 'gpgme-native', 'python3-six-native', 'gobject-introspection-native', 'libxrender-native', 'libxrandr-native', 'libxext-native', 'libx11-native', 'libpng-native', 'swig-native', 'libassuan-native', 'libgpg-error-native', 'renderproto-native', 'randrproto-native', 'xextproto-native', 'libxcb-native', 'inputproto-native', 'xtrans-native', 'kbproto-native', 'libxau-native', 'libxdmcp-native', 'libpthread-stubs-native', 'xcb-proto-native']
| DEBUG: sed -e 's:^[^/]*/:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native/:g' /home/build/build/tmp/sysroots-components/x86_64/python-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/tcl-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/gcc-cross-arm/fixmepath /home/build/build/tmp/sysroots-components/x86_64/apr-util-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/python-setuptools-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/gmp-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/apr-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native:g' -e 's:FIXME_COMPONENTS_DIR:/home/build/build/tmp/sysroots-components:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/build/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/build/build/tmp/pkgdata/mg90:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/fifo.15406:g'
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_wic
| INFO: Creating image(s)...
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 385, in _invoke_fiemap
| fcntl.ioctl(self._f_image, _FIEMAP_IOCTL, self._buf, 1)
| OSError: [Errno 95] Operation not supported
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 525, in filemap
| return FilemapFiemap(image, log)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 359, in __init__
| self.block_is_mapped(0)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 406, in block_is_mapped
| struct_fiemap = self._invoke_fiemap(block, 1)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 393, in _invoke_fiemap
| raise ErrorNotSupp(errstr)
| wic.filemap.ErrorNotSupp: FilemapFiemap: the FIEMAP ioctl is not supported by the file-system
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/wic", line 525, in <module>
| sys.exit(main(sys.argv[1:]))
| File "/home/build/upstream/yocto/poky/scripts/wic", line 520, in main
| return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/help.py", line 98, in invoke_subcommand
| subcmd[0](args, usage)
| File "/home/build/upstream/yocto/poky/scripts/wic", line 227, in wic_create_subcommand
| native_sysroot, options)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/engine.py", line 202, in wic_create
| plugin.do_create()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 99, in do_create
| self.create()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 194, in create
| self._image.prepare(self)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 352, in prepare
| imager.kernel_dir, imager.native_sysroot)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/partition.py", line 185, in prepare
| kernel_dir, rootfs_dir, native_sysroot)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/source/rawcopy.py", line 78, in do_prepare_partition
| sparse_copy(src, dst)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 543, in sparse_copy
| fmap = api(src_fname)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 527, in filemap
| return FilemapSeek(image, log)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 206, in __init__
| self._probe_seek_hole()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 241, in _probe_seek_hole
| raise ErrorNotSupp("the file-system does not support "
| wic.filemap.ErrorNotSupp: the file-system does not support "SEEK_HOLE" and "SEEK_DATA" but only provides a stub implementation
| WARNING: /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/run.do_image_wic.15406:1 exit 1 from 'BUILDDIR="/home/build/build" wic create "$wks" --vars "/home/build/build/tmp/sysroots/mg90/imgdata/" -e "core-image-minimal" -o "$out/"'
| ERROR: Function failed: do_image_wic (log file is located at /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406)
ERROR: Task (/home/build/upstream/yocto/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3411 tasks of which 2614 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/build/upstream/yocto/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic
Summary: There were 13 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Regards,
Darcy
Darcy Watkins :: Senior Staff Engineer, Firmware
SIERRA WIRELESS
Direct +1 604 233 7989 :: Fax +1 604 231 1109 :: Main +1 604 231 1100
13811 Wireless Way :: Richmond, BC Canada V6V 3A4
[M4]
dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com> :: www.sierrawireless.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sierrawireless.com%2F&data=02%7C01%7Cdwatkins%40sierrawireless.com%7C9c74be6213f14e8d315308d5ec02e29c%7C08059a4c248643dd89e33a747e0dcbe8%7C1%7C0%7C636674419524916074&sdata=UFp9oHJq%2FWno%2FSvypBUhCFkrVPhHbY82BUls%2FnU0L3M%3D&reserved=0>
From: Tim Orling [mailto:ticotimo at gmail.com]
Sent: 18 July 2018 20:19
To: Darcy Watkins <dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com>>
Cc: Frazer, Will <Will.Frazer at eurotech.com<mailto:Will.Frazer at eurotech.com>>; yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: Re: [yocto] Error in do_image_wic in Docker container
On Mac and Windows you need a samba volume.
See:
https://github.com/crops/docker-win-mac-docs/wiki
On Wed, Jul 18, 2018 at 11:44 AM Darcy Watkins <dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com>> wrote:
Hi Will,
I managed to get a docker Yocto build under Linux working, but using a simplified volume mounting scheme. I ran into an issue building with CentOS 7.5 so I used CentOS 7.4 in a container (since the build used to always work before the CentOS update). I haven’t gotten back to trying it under MacOS.
Regards,
Darcy
Darcy Watkins :: Senior Staff Engineer, Firmware
SIERRA WIRELESS
Direct +1 604 233 7989 :: Fax +1 604 231 1109 :: Main +1 604 231 1100
13811 Wireless Way :: Richmond, BC Canada V6V 3A4
[P2]
dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com> :: www.sierrawireless.com<http://www.sierrawireless.com/>
From: Frazer, Will <Will.Frazer at eurotech.com<mailto:Will.Frazer at eurotech.com>>
Sent: July-17-18 9:32 AM
To: Darcy Watkins <dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com>>; yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: RE: Error in do_image_wic in Docker container
Hi Darcy,
Don’t suppose you solved this?
I have a yocto 2.3 build that works OK in a virtual machine but seems to throw a v.similar error to what you have reported below.
Similarly I have various mounts outside of the container to inject pre-build components & pick up build artifacts.
Thanks,
Will
Will Frazer | EUROTECH | direct +44 (0) 1223 403431 | mobile: +44 (0) 7736 198115
From: yocto-bounces at yoctoproject.org<mailto:yocto-bounces at yoctoproject.org> [mailto:yocto-bounces at yoctoproject.org] On Behalf Of Darcy Watkins
Sent: 25 March 2018 17:41
To: yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
Subject: [yocto] Error in do_image_wic in Docker container
Hi,
This is near end of a build based on rocko branch plus a custom BSP layer, etc. I have had it build OK previously under Linux (CentOS 7). This run is inside a Docker container running on my MacBook. From what I gather from google search results, is the underlying OS filesystem doesn’t support an operation.
I use a volume mount scheme as follows…
docker run -it \
--volume=${PWD}:/home/build:cached \
--volume=${HOME}/.ssh:/home/build/.ssh \
--volume=${PWD}/downloads:/home/build/downloads:delegated \
--rm --volume=${WS_NAME}_build_vol:/home/build/build \
--volume=${PWD}/build/conf:/home/build/build/conf:cached \
--volume=${PWD}/build/deploy:/home/build/build/tmp/deploy:delegated \
--volume=${TARPIT}:/home/public/tarballs:cached \
$DIMG bash
The checkout is mounted from MacOS at /home/build to keep source meta-data accessible by MacOS text editors.
The build directory is inside a docker volume so that it doesn’t have case insensitivity issues.
The deploy dir and the conf dir are mounted inside this volume so that when docker exits I still have MacOS access to the conf and the deployed content.
The TARPIT is a premirror implementation and DIMG is the docker container used.
Does WIC do weird filesystem stuff (obviously ‘yes’), but does it do this in the deploy dir rather than in one of the work dirs?
I think I missed something (almost like a “fool’s mate” in Chess, LOL). Any insight on this appreciated.
Does deploy dir have to be entirely inside the volume, or is this a limitation of Docker implementation itself?
Thanks!
---
ERROR: core-image-minimal-1.0-r0 do_image_wic: Function failed: do_image_wic (log file is located at /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406)
ERROR: Logfile of failure stored in: /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/home/build/upstream/yocto/poky/meta/recipes-devtools/python/python-native_2.7.13.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.3.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/opkg/opkg_0.3.5.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/tcltk/tcl_8.6.7.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.43.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/dosfstools/dosfstools_4.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-core/update-rc.d/update-rc.d_0.7.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/bc/bc_1.06.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/binutils/binutils-cross_2.29.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/createrepo-c/createrepo-c_git.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/rpm/rpm_git.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-core/glibc/glibc_2.26.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-connectivity/openssl/openssl_1.0.2n.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/meta-freescale/recipes-bsp/change-file-endianess/change-file-endianess.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/subversion/subversion_1.9.6.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.43.5.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-bsp/u-boot/u-boot-mkimage_2017.09.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/libgcc_linaro-7.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-kernel/dtc/dtc_1.4.4.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/gcc-runtime_linaro-7.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/mtools/mtools_4.0.18.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/openembedded/meta-openembedded/meta-python/recipes-devtools/python/python-pyyaml_3.12.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/dnf/dnf_2.6.3.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/pigz/pigz_2.3.4.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-extended/parted/parted_3.2.bb:do_populate_sysroot', '/home/build/upstream/yocto/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-bsp/u-boot/u-boot-qoriq_2017.09.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-bsp/qe-ucode/qe-ucode_git.bb:do_populate_sysroot', '/home/build/upstream/linaro/meta-linaro/meta-linaro-toolchain/recipes-devtools/gcc/gcc-cross_linaro-7.1.bb:do_populate_sysroot', '/home/build/meta-MG-os-bsp/recipes-bsp/fec/fec_3.0.1.bb:do_populate_sysroot', '/home/build/upstream/yocto/meta-freescale/recipes-kernel/linux/linux-qoriq_4.9.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.3.5.bb:do_populate_sysroot', '/home/build/meta-MG-os-bsp/recipes-bsp/omg-hardware/omg-hardware_svn.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-support/bmap-tools/bmap-tools_3.4.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/pseudo/pseudo_1.8.2.bb:do_populate_sysroot', 'virtual:native:/home/build/upstream/yocto/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: ['python-native', 'gptfdisk-native', 'tcl-native', 'dosfstools-native', 'bc-native', 'binutils-cross-arm', 'change-file-endianess-native', 'subversion-native', 'mtools-native', 'python-pyyaml-native', 'lzop-native', 'parted-native', 'gcc-cross-arm', 'linux-qoriq', 'apr-util-native', 'serf-native', 'python-setuptools-native', 'python-cython-native', 'libyaml-native', 'mpfr-native', 'libmpc-native', 'gmp-native', 'apr-native', 'gdbm-native', 'python-scons-native']
| NOTE: Skipping as already exists in sysroot: ['opkg-native', 'qemuwrapper-cross', 'mklibs-native', 'makedevs-native', 'kmod-native', 'update-rc.d-native', 'createrepo-c-native', 'rpm-native', 'glibc', 'openssl-native', 'e2fsprogs-native', 'u-boot-mkimage-native', 'libgcc', 'depmodwrapper-cross', 'dtc-native', 'gcc-runtime', 'dnf-native', 'ldconfig-native', 'pigz-native', 'quilt-native', 'u-boot-qoriq', 'qe-ucode', 'fec', 'opkg-utils-native', 'omg-hardware', 'bmap-tools-native', 'pseudo-native', 'prelink-native', 'sqlite3-native', 'libtool-native', 'bzip2-native', 'autoconf-native', 'expat-native', 'xz-native', 'gnu-config-native', 'readline-native', 'pkgconfig-native', 'zlib-native', 'automake-native', 'qemu-native', 'util-linux-native', 'popt-native', 'ncurses-native', 'libarchive-native', 'libsolv-native', 'gettext-minimal-native', 'gtk-doc-native', 'texinfo-dummy-native', 'flex-native', 'bison-native', 'glib-2.0-native', 'cmake-native', 'python3-native', 'libxml2-native', 'file-native', 'curl-native', 'dbus-native', 'nss-native', 'elfutils-native', 'db-native', 'linux-libc-headers', 'cryptodev-linux-native', 'makedepend-native', 'attr-native', 'shadow-native', 'librepo-native', 'python3-iniparse-native', 'libdnf-native', 'libcomps-native', 'lzo-native', 'omg-mcu', 'python3-setuptools-native', 'binutils-native', 'm4-native', 'alsa-lib-native', 'libsdl-native', 'pixman-native', 'libffi-native', 'libpcre-native', 'gettext-native', 'nspr-native', 'xproto-native', 'util-macros-native', 'unzip-native', 'libcheck-native', 'gpgme-native', 'python3-six-native', 'gobject-introspection-native', 'libxrender-native', 'libxrandr-native', 'libxext-native', 'libx11-native', 'libpng-native', 'swig-native', 'libassuan-native', 'libgpg-error-native', 'renderproto-native', 'randrproto-native', 'xextproto-native', 'libxcb-native', 'inputproto-native', 'xtrans-native', 'kbproto-native', 'libxau-native', 'libxdmcp-native', 'libpthread-stubs-native', 'xcb-proto-native']
| DEBUG: sed -e 's:^[^/]*/:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native/:g' /home/build/build/tmp/sysroots-components/x86_64/python-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/tcl-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/gcc-cross-arm/fixmepath /home/build/build/tmp/sysroots-components/x86_64/apr-util-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/python-setuptools-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/gmp-native/fixmepath /home/build/build/tmp/sysroots-components/x86_64/apr-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/recipe-sysroot-native:g' -e 's:FIXME_COMPONENTS_DIR:/home/build/build/tmp/sysroots-components:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/build/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/build/build/tmp/pkgdata/mg90:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/fifo.15406:g'
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_wic
| INFO: Creating image(s)...
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 385, in _invoke_fiemap
| fcntl.ioctl(self._f_image, _FIEMAP_IOCTL, self._buf, 1)
| OSError: [Errno 95] Operation not supported
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 525, in filemap
| return FilemapFiemap(image, log)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 359, in __init__
| self.block_is_mapped(0)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 406, in block_is_mapped
| struct_fiemap = self._invoke_fiemap(block, 1)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 393, in _invoke_fiemap
| raise ErrorNotSupp(errstr)
| wic.filemap.ErrorNotSupp: FilemapFiemap: the FIEMAP ioctl is not supported by the file-system
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File "/home/build/upstream/yocto/poky/scripts/wic", line 525, in <module>
| sys.exit(main(sys.argv[1:]))
| File "/home/build/upstream/yocto/poky/scripts/wic", line 520, in main
| return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/help.py", line 98, in invoke_subcommand
| subcmd[0](args, usage)
| File "/home/build/upstream/yocto/poky/scripts/wic", line 227, in wic_create_subcommand
| native_sysroot, options)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/engine.py", line 202, in wic_create
| plugin.do_create()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 99, in do_create
| self.create()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 194, in create
| self._image.prepare(self)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/imager/direct.py", line 352, in prepare
| imager.kernel_dir, imager.native_sysroot)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/partition.py", line 185, in prepare
| kernel_dir, rootfs_dir, native_sysroot)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/plugins/source/rawcopy.py", line 78, in do_prepare_partition
| sparse_copy(src, dst)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 543, in sparse_copy
| fmap = api(src_fname)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 527, in filemap
| return FilemapSeek(image, log)
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 206, in __init__
| self._probe_seek_hole()
| File "/home/build/upstream/yocto/poky/scripts/lib/wic/filemap.py", line 241, in _probe_seek_hole
| raise ErrorNotSupp("the file-system does not support "
| wic.filemap.ErrorNotSupp: the file-system does not support "SEEK_HOLE" and "SEEK_DATA" but only provides a stub implementation
| WARNING: /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/run.do_image_wic.15406:1 exit 1 from 'BUILDDIR="/home/build/build" wic create "$wks" --vars "/home/build/build/tmp/sysroots/mg90/imgdata/" -e "core-image-minimal" -o "$out/"'
| ERROR: Function failed: do_image_wic (log file is located at /home/build/build/tmp/work/mg90-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_image_wic.15406)
ERROR: Task (/home/build/upstream/yocto/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3411 tasks of which 2614 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/build/upstream/yocto/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_wic
Summary: There were 13 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Regards,
Darcy
Darcy Watkins :: Senior Staff Engineer, Firmware
SIERRA WIRELESS
Direct +1 604 233 7989 :: Fax +1 604 231 1109 :: Main +1 604 231 1100
13811 Wireless Way :: Richmond, BC Canada V6V 3A4
[M4]
dwatkins at sierrawireless.com<mailto:dwatkins at sierrawireless.com> :: www.sierrawireless.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sierrawireless.com%2F&data=02%7C01%7Cdwatkins%40sierrawireless.com%7C9c74be6213f14e8d315308d5ec02e29c%7C08059a4c248643dd89e33a747e0dcbe8%7C1%7C0%7C636674419524916074&sdata=UFp9oHJq%2FWno%2FSvypBUhCFkrVPhHbY82BUls%2FnU0L3M%3D&reserved=0>
--
_______________________________________________
yocto mailing list
yocto at yoctoproject.org<mailto:yocto at yoctoproject.org>
https://lists.yoctoproject.org/listinfo/yocto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190913/fcd96468/attachment-0001.html>
More information about the yocto
mailing list