[yocto] [meta-selinux][PATCH] conf/layer.conf: use BBFILES_DYNAMIC for dynamic layers

Wed Sep 11 07:46:06 PDT 2019

[Re: [meta-selinux][PATCH] conf/layer.conf: use BBFILES_DYNAMIC for dynamic layers] On 19.09.11 (Wed 09:22) Yi Zhao wrote:

> 
> On 9/10/19 1:11 AM, Joe MacDonald wrote:
> > Hi Yi,
> > 
> > [[meta-selinux][PATCH] conf/layer.conf: use BBFILES_DYNAMIC for dynamic layers] On 19.09.09 (Mon 14:01) Yi Zhao wrote:
> > 
> > > From: Robert Yang <liezhi.yang at windriver.com>
> > > 
> > > The previous code add all BBFILE_COLLECTIONS/recipes*/*/*.bbappend to BBFILES,
> > > which causes the parsing very slow when there are many layers, e.g., I have 87
> > > layers:
> > > 
> > > * Before:
> > > $ rm -fr tmp-glibc/ cache; time bitbake -p
> > > real    0m45.173s
> > > user    0m0.560s
> > > sys     0m0.060s
> > > 
> > > * After:
> > > $ rm -fr tmp-glibc/ cache; time bitbake -p
> > > real    0m25.542s
> > > user    0m0.572s
> > > sys     0m0.040s
> > > 
> > > It wasted 20s which wasn't worth (The host has 128 threads, it should cost more
> > > time on less power host), use BBFILES_DYNAMIC can fix the problem.
> > 
> > This seems like a big claim, I certainly haven't seen that on my setup:
> > 
> > * Before:
> > $ rm -fr tmp cache
> > real    0m14.751s
> > user    0m0.323s
> > sys     0m0.048s
> > 
> > * After:
> > $ rm -fr tmp cache ; time bitbake -p
> > real    0m14.725s
> > user    0m0.326s
> > sys     0m0.046s
> > 
> > but it's still a sensible change.  When I ran a test before/after
> > configuration for augeas the configuration seemed off, though.  Can you
> > confirm that with this change as is you're getting the correct
> > --with/--without and --enable/--disable and patches applied for your
> > layers?  I just want to confirm since the ~20s difference in parsing
> > seems kind of out of scale for moving essentially three bbappends around
> > and I'm wondering if there's something else siginficant in your tree we
> > want to consider.
> 
> 
> This patch is from Robert Yang. CC to him. Maybe he can give us more
> explanation.
> 
> For the augeas, the current augeas_%.bbapend doesn't work because the augeas
> recipe is in meta-oe layer but not meta-python layer. This patch moves the
> bbappend to the correct layer to fix this issue.
> 
> It works on my local:
> 
> $ cat log.do_configure
> 
> [snip]
> checking for library containing setfilecon... -lselinux
> [snip]
> checking for selinux/selinux.h... (cached) yes
> checking selinux/context.h usability... yes
> checking selinux/context.h presence... yes
> checking for selinux/context.h... yes
> [snip]

Okay, thanks.  Funny that I randomly picked the package that was broken
in multiple ways, but this looks like an improvement overall.

-J.

> 
> 
> //Yi
> 
> 
> > 
> > -J.
> > 
> > > Signed-off-by: Robert Yang <liezhi.yang at windriver.com>
> > > Signed-off-by: Yi Zhao <yi.zhao at windriver.com>
> > > ---
> > >   conf/layer.conf                                               | 11 +++++++----
> > >   .../recipes-daemons/iscsi-initiator-utils/files/initd.debian  |  0
> > >   .../iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend    |  0
> > >   .../iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc   |  0
> > >   .../recipes-support}/augeas/augeas_%.bbappend                 |  0
> > >   .../recipes-containers/lxc/lxc_%.bbappend                     |  0
> > >   6 files changed, 7 insertions(+), 4 deletions(-)
> > >   rename {networking-layer => dynamic-layers/networking-layer}/recipes-daemons/iscsi-initiator-utils/files/initd.debian (100%)
> > >   rename {networking-layer => dynamic-layers/networking-layer}/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend (100%)
> > >   rename {networking-layer => dynamic-layers/networking-layer}/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc (100%)
> > >   rename {meta-python/recipes-extended/augeas => dynamic-layers/openembedded-layer/recipes-support}/augeas/augeas_%.bbappend (100%)
> > >   rename {virtualization-layer => dynamic-layers/virtualization-layer}/recipes-containers/lxc/lxc_%.bbappend (100%)
> > > 
> > > diff --git a/conf/layer.conf b/conf/layer.conf
> > > index 9dd34b1..89b9468 100644
> > > --- a/conf/layer.conf
> > > +++ b/conf/layer.conf
> > > @@ -5,10 +5,13 @@ BBPATH .= ":${LAYERDIR}"
> > >   BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
> > >               ${LAYERDIR}/recipes-*/*/*.bbappend"
> > > -# Let us add layer-specific bbappends which are only applied when that
> > > -# layer is included in our configuration
> > > -BBFILES += "${@' '.join('${LAYERDIR}/%s/recipes*/*/*.bbappend' % layer \
> > > -               for layer in BBFILE_COLLECTIONS.split())}"
> > > +BBFILES_DYNAMIC += "openembedded-layer:${LAYERDIR}/dynamic-layers/openembedded-layer/*/*/*.bb \
> > > +                    openembedded-layer:${LAYERDIR}/dynamic-layers/openembedded-layer/*/*/*.bbappend \
> > > +                    networking-layer:${LAYERDIR}/dynamic-layers/networking-layer/*/*/*.bb \
> > > +                    networking-layer:${LAYERDIR}/dynamic-layers/networking-layer/*/*/*.bbappend \
> > > +                    virtualization-layer:${LAYERDIR}/dynamic-layers/virtualization-layer/recipes*/*/*.bb \
> > > +                    virtualization-layer:${LAYERDIR}/dynamic-layers/virtualization-layer/recipes*/*/*.bbappend \
> > > +                   "
> > >   BBFILE_COLLECTIONS += "selinux"
> > >   BBFILE_PATTERN_selinux = "^${LAYERDIR}/"
> > > diff --git a/networking-layer/recipes-daemons/iscsi-initiator-utils/files/initd.debian b/dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/files/initd.debian
> > > similarity index 100%
> > > rename from networking-layer/recipes-daemons/iscsi-initiator-utils/files/initd.debian
> > > rename to dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/files/initd.debian
> > > diff --git a/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend b/dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend
> > > similarity index 100%
> > > rename from networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend
> > > rename to dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_%.bbappend
> > > diff --git a/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc b/dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc
> > > similarity index 100%
> > > rename from networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc
> > > rename to dynamic-layers/networking-layer/recipes-daemons/iscsi-initiator-utils/iscsi-initiator-utils_selinux.inc
> > > diff --git a/meta-python/recipes-extended/augeas/augeas/augeas_%.bbappend b/dynamic-layers/openembedded-layer/recipes-support/augeas/augeas_%.bbappend
> > > similarity index 100%
> > > rename from meta-python/recipes-extended/augeas/augeas/augeas_%.bbappend
> > > rename to dynamic-layers/openembedded-layer/recipes-support/augeas/augeas_%.bbappend
> > > diff --git a/virtualization-layer/recipes-containers/lxc/lxc_%.bbappend b/dynamic-layers/virtualization-layer/recipes-containers/lxc/lxc_%.bbappend
> > > similarity index 100%
> > > rename from virtualization-layer/recipes-containers/lxc/lxc_%.bbappend
> > > rename to dynamic-layers/virtualization-layer/recipes-containers/lxc/lxc_%.bbappend
> > > -- 
> > > 2.7.4
> > > 

-- 
-Joe MacDonald.
Linux Architect | Mentor® A Siemens Business
:wq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190911/e7168789/attachment-0001.pgp>