[yocto] how to organize my patches for multiple kernels and multiple target boards?

[Robert P. J. Day]

Quoting Bruce Ashfield <bruce.ashfield at windriver.com>:

> On 16-03-01 05:44 AM, Robert P. J. Day wrote:
>> Quoting Andrea Adami <andrea.adami at gmail.com>:
>>
>>> On Tue, Mar 1, 2016 at 12:19 AM, Robert P. J. Day
>>> <rpjday at crashcourse.ca> wrote:
>>>>  (i posted a much lengthier version of this on oe-core recently, but
>>>> i want
>>>> to cut it down and ask specific questions to clarify what i *think*
>>>> is going
>>>> on.)
>>>>
>>>>  i want to pull in an existing layer with recipes for linux-4.0.bb and
>>>> linux-4.1.bb, and extend them with .bbappend files, to support two
>>>> closely-related
>>>> machines i'm defining -- call them "mach1" and "mach2". AFAICT, my
>>>> patches
>>>> will
>>>> fall somewhere in a 3x3 matrix of possibilities:
>>>>
>>>>  * 3 possibilities of applying against mach1, mach2 or both
>>>>  * 3 possibilities of applying against linux-4.0, linux-4.1 or both
>>>>
>>>> so there's my 3x3 matrix.
>>>>
>>>>  the obvious kernel recipe directory structure would be:
>>>>
>>>>  linux/
>>>>    linux-4.0.bbappend
>>>>    linux-4.1.bbappend
>>>>    linux-4.0/            [4.0-specific patches]
>>>>    linux-4.1/            [4.1-specific patches]
>>>>    linux/                [patches that apply to both]
>>>>
>>>> which suggests that both my .bbappend files would have to contain the
>>>> line:
>>>>
>>>>  FILESEXTRAPATHS_prepend := "${THISDIR}/${BP}:${THISDIR}/${BPN}"
>>>>
>>>> so the SRC_URI search path for linux-4.0.bbappend entries would be
>>>> prepended with:
>>>>
>>>>  * linux-4.0/        [4.0-specific patches]
>>>>  * linux/            [patches that apply to both]
>>>>
>>>> and similarly for linux.4.1.bbappend. how am i doing so far? this would
>>>> mean that, for each recipe, the more specific directory would be
>>>> searched
>>>> before the general directory. but wait ... there's more.
>>>>
>>>>  now i want to further categorize patches based on exclusive to mach1,
>>>> exclusive to mach2, or applicable to both, and since the machine name is
>>>> one of the entries in FILESOVERRIDES, i can extend the directory
>>>> structure
>>>> as:
>>>>
>>>>  linux-4.0/
>>>>    mach1/
>>>>    mach2/
>>>>  linux-4.1/
>>>>    mach1/
>>>>    mach2/
>>>>  linux/
>>>>    mach1/
>>>>    mach2/
>>>>
>>>> and there's my 3x3 matrix of patches, correct? and here's where it gets
>>>> unclear.
>>>>
>>>>  i really don't want to have to number all my patches with prefixes
>>>> like
>>>> 0001-, 0002- and so on, so what is the ordering of processing for .scc,
>>>> .cfg and .patch/.diff files? rather than just lump all the patches into
>>>> a single .scc file, i want to refine the patches across multiple .scc
>>>> files. is there an imposed order on SRC_URI entries, .scc files and so
>>>> on? that's probably all i need to finish this off.
>>>>
>>>> rday
>>>
>>> Robert,
>>>
>>> in the past I have done pretty much the same: scc,cfg and patches all
>>> packed in the recipe.
>>> Please see these (outdated) layout examples for linux-yocto* that were
>>> in meta-handheld.
>>>
>>> for 3.10, using .cfg & .scc
>>> http://cgit.openembedded.org/meta-handheld/tree/recipes-kernel/linux?h=dylan
>>>
>>>
>>> Or simplified, for 3.14, using defconfig, with patches listed in SRC_URI
>>> http://cgit.openembedded.org/meta-handheld/tree/recipes-kernel/linux?h=dizzy
>>>
>>
>>   thanks, i'll check that out. first thing i want to be absolutely clear
>> on is, if i have multiple patch directories, i need to add them *all*
>> to the search path, as in:
>>
>>   FILESEXTRAPATHS_prepend := "${THISDIR}/${BP}:${THISDIR}/${BPN}"
>>
>> and certainly in that order, as i want the more specific version
>> patches to be found first. so that bit is correct, yes?
>>
>>   next, i'm still unclear on whether there is any enforced ordering
>> on the processing of .scc files. i know that some folks number their
>> patch files as 0001-*, 0002-* and so on, in order to enforce a
>> patching order (because the order that one specifies patch/diff files
>> in the SRC_URI doesn't mean anything, correct?)
>
> I'm waiting for someone to slap me for saying something so definitive,
> but here it goes anyay ... The order in the SRC_URI definitely matters.
> The order that they are specified is the order the patches are applied.
> Otherwise, a patch stack of any depth wouldn't be possible. Only if you
> are doing wildcard patches and rely on shell globbing would numbering
> be critical.

eggcellent ... i assume that SRC_URI ordering is imposed on *all* SRC_URI
items, that being .scc, .cfg and .patch/.diff files, as in SRC_URI is
processed strictly in order for all of its constituent items? i always
assumed as much, i just don't recall ever seeing that written down. and,
uh, i suspect scott rifenbark can testify that i'm moderately familiar
with the manuals. :-)

>>   however, i would rather not use a numbering scheme like that because,
>> well, it's ugly, and given that i will have patches scattered all over
>> the patch directories and subdirectories on a per-kernel and a
>> per-target board basis, it just wouldn't make much sense.
>
> Agreed.
>
>>   so my plan is to (predictably) group related patch files and .cfg
>> files into a number of .scc files but (again) is there any enforced
>> search order for .scc files? i'm assuming my setting for FILESEXTRAPATHS
>> and use of FILESOVERRIDES will kick in here ... will the order of
>> the .scc files in SRC_URI have any effect as well?
>
> .scc files will be processed in the order they are found on the
> SRC_URI, and then the patches the order they are within the .scc
> files.

getting better and better ...

>>   also, once a .scc file is located, will the location of the listed
>> .cfg and .patch/.diff files inside it start a whole new search process
>> based on FILESEXTRAPATHS and FILESOVERRIDES?
>
> The search path is created relative to the .scc file that is referencing
> a patch, and across any directory that has a .scc file in the SRC_URI.
> The .scc feature descriptions are self contained, hence "reaching outside"
> to a parent, or other directory structure isn't a good thing. Just list
> things on the SRC_URI if that is required.

   ah, so to clarify, if i refer to a .scc file, my FILESEXTRAPATHS and
FILESOVERRIDES will kick in to *find* it, but once found, its internal
references will be treated as local to where the .scc file was found?
good, that makes sense, i like that.

>>   if this is all written down somewhere, just point me to it. thank
>> you kindly.
>
> IIRC it is in the kernel development manual (the .scc file parts), but
> the SRC_URI and patch order is common to any recipe that bitbake/oe-core
> process.

>> p.s. all of this is in aid of trying to avoid ordering mishaps when
>> applying patches, but i'm guessing that if i design my .scc files
>> carefully to be logically self-contained, i can probably avoid
>> accidents like that in the first place.
>
> Correct. The .scc files, kernel-cache and management found within
> the files was created to manage hundreds of patches for overlapping
> boards, much like you are describing.
>
> In the end, when complexity gets really high, a git repo with branches
> is probably easier ... and with that description, I've just covered
> how patch management with .scc files leads to the linux-yocto tree :)
>
> Bruce

   smugness becomes you. :-)

rday