[yocto] kernel-devsrc used to be full source - not the case since thud

[Bruce Ashfield]

On Thu, May 23, 2019 at 11:00 AM Andrei Gherzan <andrei at gherzan.ro> wrote:

>
> On 23/05/2019 15.39, Bruce Ashfield wrote:
>
>
>
> On Thu, May 23, 2019 at 10:32 AM Bruce Ashfield <bruce.ashfield at gmail.com>
> wrote:
>
>>
>>
>> On Thu, May 23, 2019 at 9:56 AM Andrei Gherzan <andrei at gherzan.ro> wrote:
>>
>>> Hello,
>>>
>>> This might have been discussed before. I couldn't find a relevant
>>> thread, but if it is so, just link me to it.
>>>
>>> Since thud, more specific since
>>>
>>> commit 9af0f1a46bbb6ad9ee8b35957251f4aa826b023f
>>> Author: Bruce Ashfield <bruce.ashfield at windriver.com>
>>> Date:   Sat Aug 18 22:50:44 2018 -0400
>>>     kernel-devsrc: restructure for out of tree (and on target) module
>>> builds
>>>
>>> ... we switched from a recipe that was deploying the entire source code
>>> to one that provides mainly the kernel headers (but not only). This
>>> change broke people expectations of this recipe while the description is
>>> also confusing: "Development source linux kernel. When built, this
>>> recipe packages the \source of the preferred virtual/kernel provider and
>>> makes it available for full kernel \development or external module
>>> builds".
>>>
>>> If size is not a problem (which can be the case when you compile on a
>>> builder for example and deploy only a OOT kernel module through other
>>> means), the full kernel source was a painless experience where things
>>> like
>>>
>>> commit a9471601fedd1f5087304eaa5fd39b98ae220313
>>> Author: Bruce Ashfield <bruce.ashfield at windriver.com>
>>> Date:   Thu Aug 30 09:45:41 2018 -0400
>>>     kernel-devsrc: fix arm/arm64 target module build
>>>
>>> ... would not appear. I understand the size impact on target and for
>>> those cases, continuously maintaining this recipe with new
>>> files/resources needed from the kernel, makes sense. So my proposal is
>>> to have two recipes, for example kernel-devsrc and kernel-fullsrc
>>> (kernel-src etc.) so people can choose what they need/want
>>> deploying/using. Or even have another devsrc provider. I'm open to any
>>> implementation detail. I'd just want to have an option for a full kernel
>>> source recipe.
>>>
>>
>> This is already planned, and hidden in bugzilla somewhere. I'll have some
>> new kernel packaging
>> options available for the fall release.
>>
>
> It looks like the bugs that I was using for development were finally moved
> to resolved (they were a bit old, and contained collected information on
> various kernel packaging options .. my searching of bugzilla isn't turning
> it up at the moment). So I just created a new bug to track the development
> for 2.8.
>
> The issue with the multiple kernel source providers is really about test
> cycles. The smaller devsrc is for on-target module development and builds
> against the exported uapi headers, and that is what the nightly / automated
> tests will use. We had issues with both the amount of time it took to
> package the entire source, and the amount of space that it took up on the
> images. Hence the creation of devsrc.
>
> With new kernel-source and kernel-headers packages (the working names),
> they are really provided as references to the running kernel, and will
> largely be an exercise left up to the developer to use them as they want.
>
> Right. So is there anything that holds us from creating two recipes - one
> for what we currently have and one for what we used to have pre-thud - find
> some pretty names and start from that? I'm asking just in case I'm missing
> any internal architecture discussions or so.
>

In some other non-core layer ? Sure :D The pre-thud copying of the kernel
source was a disaster and needed to be killed with fire, as it was. It was
all special cases and I commonly needed to fight with it. What you see in
the current devsrc is not an invention from Yocto, it is based on the
redhat spec files, and also (a bit) on some of the debian packaging of
minimal source.

So no, I'd rather not restore the mess that was the pre-thud devsrc in any
form.


> And about test cycles, I'm not sure I understand why having a provider vs
> having two separate recipes impact automated tests. In both cases you will
> select whatever you want to test - by using a specific recipe or by setting
> a preferred provider (I don't have a strong opinion here but I was just
> curious about the time impact on automated tests).
>

If it goes into core, we would need to run a full set of tests against it
to make sure that it supported development tasks. That would mean on target
kernel module builds, as well as potential some new cases given that the
full source is available (i.e. kernel rebuild). We already do that for
kernel-devrsc, adding multiple providers of something similar means that
we'd then have to test the multiple providers against all the
architectures, across the actively supported reference kernels in any given
release .. something always breaks, so then I'm on the hook for sorting
through all the issues (devsrc was not bug free in its previous form).
Given the amount of test cycles available, that's not necessarily a trivial
thing to add. In the end, it would be up to Richard if thinks something
like multiple provides are a good thing, and to commit the test
resources/cycles to them.



> Replying to the other email (this thread was forked). We have maintained a
> similar kernel headers approach for a couple of years now. And many times
> we found ourselves in the situation where something had to be
> added/removed/checked - that was either kernel fork specific or something
> that was added due to kernel updates. And this loop of fixes here and there
> was obviously completely resolved by just using the "old" kernel-devsrc
> which always had "everything" in. A good example is commit
> a9471601fedd1f5087304eaa5fd39b98ae220313 I'm mentioning above. This kind of
> stuff will always be needed - or at least in our experience was a pretty
> often loop (when you try to support one kernel headers recipe against tens
> of BSPs, you'll need cases, quirks and so on).
>
Those small tweaks really are trivial, I don't consider them to be a big
load. The advantages of the smaller footprint package have outweighed any
tuning we've done since things switched over.

Bruce

> Regards,
>
> --
> Andrei Gherzan
> gpg: rsa4096/D4D94F67AD0E9640 | t: @agherzan
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20190523/3d4935f2/attachment.html>