[yocto] kernel-devsrc used to be full source - not the case since thud
Andrei Gherzan
andrei at gherzan.ro
Thu May 23 08:24:12 PDT 2019
On 23/05/2019 16.10, Bruce Ashfield wrote:
>
>
> On Thu, May 23, 2019 at 11:00 AM Andrei Gherzan <andrei at gherzan.ro
> <mailto:andrei at gherzan.ro>> wrote:
>
>
> On 23/05/2019 15.39, Bruce Ashfield wrote:
>>
>>
>> On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield
>> <bruce.ashfield at gmail.com <mailto:bruce.ashfield at gmail.com>> wrote:
>>
>>
>>
>> On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan
>> <andrei at gherzan.ro <mailto:andrei at gherzan.ro>> wrote:
>>
>> Hello,
>>
>> This might have been discussed before. I couldn't find a
>> relevant
>> thread, but if it is so, just link me to it.
>>
>> Since thud, more specific since
>>
>> commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
>> Author: Bruce Ashfield <bruce.ashfield at windriver.com
>> <mailto:bruce.ashfield at windriver.com>>
>> Date: Sat Aug 18 22:50:44 2018 -0400
>> kernel-devsrc: restructure for out of tree (and on
>> target) module builds
>>
>> ... we switched from a recipe that was deploying the
>> entire source code
>> to one that provides mainly the kernel headers (but not
>> only). This
>> change broke people expectations of this recipe while the
>> description is
>> also confusing: "Development source linux kernel. When
>> built, this
>> recipe packages the \source of the preferred
>> virtual/kernel provider and
>> makes it available for full kernel \development or
>> external module builds".
>>
>> If size is not a problem (which can be the case when you
>> compile on a
>> builder for example and deploy only a OOT kernel module
>> through other
>> means), the full kernel source was a painless experience
>> where things like
>>
>> commit a9471601fedd1f5087304eaa5fd39b98ae220313
>> Author: Bruce Ashfield <bruce.ashfield at windriver.com
>> <mailto:bruce.ashfield at windriver.com>>
>> Date: Thu Aug 30 09:45:41 2018 -0400
>> kernel-devsrc: fix arm/arm64 target module build
>>
>> ... would not appear. I understand the size impact on
>> target and for
>> those cases, continuously maintaining this recipe with new
>> files/resources needed from the kernel, makes sense. So
>> my proposal is
>> to have two recipes, for example kernel-devsrc and
>> kernel-fullsrc
>> (kernel-src etc.) so people can choose what they need/want
>> deploying/using. Or even have another devsrc provider.
>> I'm open to any
>> implementation detail. I'd just want to have an option
>> for a full kernel
>> source recipe.
>>
>>
>> This is already planned, and hidden in bugzilla somewhere.
>> I'll have some new kernel packaging
>> options available for the fall release.
>>
>>
>> It looks like the bugs that I was using for development were
>> finally moved to resolved (they were a bit old, and contained
>> collected information on various kernel packaging options .. my
>> searching of bugzilla isn't turning it up at the moment). So I
>> just created a new bug to track the development for 2.8.
>>
>> The issue with the multiple kernel source providers is really
>> about test cycles. The smaller devsrc is for on-target module
>> development and builds against the exported uapi headers, and
>> that is what the nightly / automated tests will use. We had
>> issues with both the amount of time it took to package the entire
>> source, and the amount of space that it took up on the images.
>> Hence the creation of devsrc.
>>
>> With new kernel-source and kernel-headers packages (the working
>> names), they are really provided as references to the running
>> kernel, and will largely be an exercise left up to the developer
>> to use them as they want.
>
> Right. So is there anything that holds us from creating two
> recipes - one for what we currently have and one for what we used
> to have pre-thud - find some pretty names and start from that? I'm
> asking just in case I'm missing any internal architecture
> discussions or so.
>
>
> In some other non-core layer ? Sure :D The pre-thud copying of the
> kernel source was a disaster and needed to be killed with fire, as it
> was. It was all special cases and I commonly needed to fight with it.
> What you see in the current devsrc is not an invention from Yocto, it
> is based on the redhat spec files, and also (a bit) on some of the
> debian packaging of minimal source.
I am aware of all the other distros having the similar approach (and
they do this dance for disk space optimization) but I don't understand
what were all the special cases you had to fight against? The recipe was
barely touched in years. For example there is only one commit on it in
2017.
>
> So no, I'd rather not restore the mess that was the pre-thud devsrc in
> any form.
>
>
> And about test cycles, I'm not sure I understand why having a
> provider vs having two separate recipes impact automated tests. In
> both cases you will select whatever you want to test - by using a
> specific recipe or by setting a preferred provider (I don't have a
> strong opinion here but I was just curious about the time impact
> on automated tests).
>
>
> If it goes into core, we would need to run a full set of tests against
> it to make sure that it supported development tasks. That would mean
> on target kernel module builds, as well as potential some new cases
> given that the full source is available (i.e. kernel rebuild). We
> already do that for kernel-devrsc, adding multiple providers of
> something similar means that we'd then have to test the multiple
> providers against all the architectures, across the actively supported
> reference kernels in any given release .. something always breaks, so
> then I'm on the hook for sorting through all the issues (devsrc was
> not bug free in its previous form). Given the amount of test cycles
> available, that's not necessarily a trivial thing to add. In the end,
> it would be up to Richard if thinks something like multiple provides
> are a good thing, and to commit the test resources/cycles to them.
I see. You were comparing something else. That makes sense.
>
>
>
> Replying to the other email (this thread was forked). We have
> maintained a similar kernel headers approach for a couple of years
> now. And many times we found ourselves in the situation where
> something had to be added/removed/checked - that was either kernel
> fork specific or something that was added due to kernel updates.
> And this loop of fixes here and there was obviously completely
> resolved by just using the "old" kernel-devsrc which always had
> "everything" in. A good example is commit
> a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above.
> This kind of stuff will always be needed - or at least in our
> experience was a pretty often loop (when you try to support one
> kernel headers recipe against tens of BSPs, you'll need cases,
> quirks and so on).
>
> Those small tweaks really are trivial, I don't consider them to be a
> big load. The advantages of the smaller footprint package have
> outweighed any tuning we've done since things switched over.
Again, I 100% understand the space advantage. I'm trying to see if there
are any other advantages at all. In a world with infinite disk space :D
--
Andrei Gherzan
gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190523/5c8ab286/attachment-0001.html>
More information about the yocto
mailing list