[yocto] kernel-devsrc used to be full source - not the case since thud

Andrei Gherzan andrei at gherzan.ro
Thu May 23 08:24:12 PDT 2019


On 23/05/2019 16.10, Bruce Ashfield wrote:
>
>
> On Thu, May 23, 2019 at 11:00 AM Andrei Gherzan <andrei at gherzan.ro
> <mailto:andrei at gherzan.ro>> wrote:
>
>
>     On 23/05/2019 15.39, Bruce Ashfield wrote:
>>
>>
>>     On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield
>>     <bruce.ashfield at gmail.com <mailto:bruce.ashfield at gmail.com>> wrote:
>>
>>
>>
>>         On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan
>>         <andrei at gherzan.ro <mailto:andrei at gherzan.ro>> wrote:
>>
>>             Hello,
>>
>>             This might have been discussed before. I couldn't find a
>>             relevant
>>             thread, but if it is so, just link me to it.
>>
>>             Since thud, more specific since
>>
>>             commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
>>             Author: Bruce Ashfield <bruce.ashfield at windriver.com
>>             <mailto:bruce.ashfield at windriver.com>>
>>             Date:   Sat Aug 18 22:50:44 2018 -0400
>>                 kernel-devsrc: restructure for out of tree (and on
>>             target) module builds
>>
>>             ... we switched from a recipe that was deploying the
>>             entire source code
>>             to one that provides mainly the kernel headers (but not
>>             only). This
>>             change broke people expectations of this recipe while the
>>             description is
>>             also confusing: "Development source linux kernel. When
>>             built, this
>>             recipe packages the \source of the preferred
>>             virtual/kernel provider and
>>             makes it available for full kernel \development or
>>             external module builds".
>>
>>             If size is not a problem (which can be the case when you
>>             compile on a
>>             builder for example and deploy only a OOT kernel module
>>             through other
>>             means), the full kernel source was a painless experience
>>             where things like
>>
>>             commit a9471601fedd1f5087304eaa5fd39b98ae220313
>>             Author: Bruce Ashfield <bruce.ashfield at windriver.com
>>             <mailto:bruce.ashfield at windriver.com>>
>>             Date:   Thu Aug 30 09:45:41 2018 -0400
>>                 kernel-devsrc: fix arm/arm64 target module build
>>
>>             ... would not appear. I understand the size impact on
>>             target and for
>>             those cases, continuously maintaining this recipe with new
>>             files/resources needed from the kernel, makes sense. So
>>             my proposal is
>>             to have two recipes, for example kernel-devsrc and
>>             kernel-fullsrc
>>             (kernel-src etc.) so people can choose what they need/want
>>             deploying/using. Or even have another devsrc provider.
>>             I'm open to any
>>             implementation detail. I'd just want to have an option
>>             for a full kernel
>>             source recipe.
>>
>>
>>         This is already planned, and hidden in bugzilla somewhere.
>>         I'll have some new kernel packaging
>>         options available for the fall release.
>>
>>
>>     It looks like the bugs that I was using for development were
>>     finally moved to resolved (they were a bit old, and contained
>>     collected information on various kernel packaging options .. my
>>     searching of bugzilla isn't turning it up at the moment). So I
>>     just created a new bug to track the development for 2.8.
>>
>>     The issue with the multiple kernel source providers is really
>>     about test cycles. The smaller devsrc is for on-target module
>>     development and builds against the exported uapi headers, and
>>     that is what the nightly / automated tests will use. We had
>>     issues with both the amount of time it took to package the entire
>>     source, and the amount of space that it took up on the images.
>>     Hence the creation of devsrc.
>>
>>     With new kernel-source and kernel-headers packages (the working
>>     names), they are really provided as references to the running
>>     kernel, and will largely be an exercise left up to the developer
>>     to use them as they want.
>
>     Right. So is there anything that holds us from creating two
>     recipes - one for what we currently have and one for what we used
>     to have pre-thud - find some pretty names and start from that? I'm
>     asking just in case I'm missing any internal architecture
>     discussions or so.
>
>
> In some other non-core layer ? Sure :D The pre-thud copying of the
> kernel source was a disaster and needed to be killed with fire, as it
> was. It was all special cases and I commonly needed to fight with it.
> What you see in the current devsrc is not an invention from Yocto, it
> is based on the redhat spec files, and also (a bit) on some of the
> debian packaging of minimal source.

I am aware of all the other distros having the similar approach (and
they do this dance for disk space optimization) but I don't understand
what were all the special cases you had to fight against? The recipe was
barely touched in years. For example there is only one commit on it in
2017.

>
> So no, I'd rather not restore the mess that was the pre-thud devsrc in
> any form.
>  
>
>     And about test cycles, I'm not sure I understand why having a
>     provider vs having two separate recipes impact automated tests. In
>     both cases you will select whatever you want to test - by using a
>     specific recipe or by setting a preferred provider (I don't have a
>     strong opinion here but I was just curious about the time impact
>     on automated tests).
>
>
> If it goes into core, we would need to run a full set of tests against
> it to make sure that it supported development tasks. That would mean
> on target kernel module builds, as well as potential some new cases
> given that the full source is available (i.e. kernel rebuild). We
> already do that for kernel-devrsc, adding multiple providers of
> something similar means that we'd then have to test the multiple
> providers against all the architectures, across the actively supported
> reference kernels in any given release .. something always breaks, so
> then I'm on the hook for sorting through all the issues (devsrc was
> not bug free in its previous form).  Given the amount of test cycles
> available, that's not necessarily a trivial thing to add. In the end,
> it would be up to Richard if thinks something like multiple provides
> are a good thing, and to commit the test resources/cycles to them.
I see. You were comparing something else. That makes sense.
>
>  
>
>     Replying to the other email (this thread was forked). We have
>     maintained a similar kernel headers approach for a couple of years
>     now. And many times we found ourselves in the situation where
>     something had to be added/removed/checked - that was either kernel
>     fork specific or something that was added due to kernel updates.
>     And this loop of fixes here and there was obviously completely
>     resolved by just using the "old" kernel-devsrc which always had
>     "everything" in. A good example is commit
>     a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above.
>     This kind of stuff will always be needed - or at least in our
>     experience was a pretty often loop (when you try to support one
>     kernel headers recipe against tens of BSPs, you'll need cases,
>     quirks and so on).
>
> Those small tweaks really are trivial, I don't consider them to be a
> big load. The advantages of the smaller footprint package have
> outweighed any tuning we've done since things switched over.
Again, I 100% understand the space advantage. I'm trying to see if there
are any other advantages at all. In a world with infinite disk space :D

-- 
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190523/5c8ab286/attachment-0001.html>


More information about the yocto mailing list