[yocto] how to organize my patches for multiple kernels and multiple target boards?

[Bruce Ashfield]

On 16-03-01 05:44 AM, Robert P. J. Day wrote:
> Quoting Andrea Adami <andrea.adami at gmail.com>:
>
>> On Tue, Mar 1, 2016 at 12:19 AM, Robert P. J. Day
>> <rpjday at crashcourse.ca> wrote:
>>>   (i posted a much lengthier version of this on oe-core recently, but
>>> i want
>>> to cut it down and ask specific questions to clarify what i *think*
>>> is going
>>> on.)
>>>
>>>   i want to pull in an existing layer with recipes for linux-4.0.bb and
>>> linux-4.1.bb, and extend them with .bbappend files, to support two
>>> closely-related
>>> machines i'm defining -- call them "mach1" and "mach2". AFAICT, my
>>> patches
>>> will
>>> fall somewhere in a 3x3 matrix of possibilities:
>>>
>>>   * 3 possibilities of applying against mach1, mach2 or both
>>>   * 3 possibilities of applying against linux-4.0, linux-4.1 or both
>>>
>>> so there's my 3x3 matrix.
>>>
>>>   the obvious kernel recipe directory structure would be:
>>>
>>>   linux/
>>>     linux-4.0.bbappend
>>>     linux-4.1.bbappend
>>>     linux-4.0/            [4.0-specific patches]
>>>     linux-4.1/            [4.1-specific patches]
>>>     linux/                [patches that apply to both]
>>>
>>> which suggests that both my .bbappend files would have to contain the
>>> line:
>>>
>>>   FILESEXTRAPATHS_prepend := "${THISDIR}/${BP}:${THISDIR}/${BPN}"
>>>
>>> so the SRC_URI search path for linux-4.0.bbappend entries would be
>>> prepended with:
>>>
>>>   * linux-4.0/        [4.0-specific patches]
>>>   * linux/            [patches that apply to both]
>>>
>>> and similarly for linux.4.1.bbappend. how am i doing so far? this would
>>> mean that, for each recipe, the more specific directory would be
>>> searched
>>> before the general directory. but wait ... there's more.
>>>
>>>   now i want to further categorize patches based on exclusive to mach1,
>>> exclusive to mach2, or applicable to both, and since the machine name is
>>> one of the entries in FILESOVERRIDES, i can extend the directory
>>> structure
>>> as:
>>>
>>>   linux-4.0/
>>>     mach1/
>>>     mach2/
>>>   linux-4.1/
>>>     mach1/
>>>     mach2/
>>>   linux/
>>>     mach1/
>>>     mach2/
>>>
>>> and there's my 3x3 matrix of patches, correct? and here's where it gets
>>> unclear.
>>>
>>>   i really don't want to have to number all my patches with prefixes
>>> like
>>> 0001-, 0002- and so on, so what is the ordering of processing for .scc,
>>> .cfg and .patch/.diff files? rather than just lump all the patches into
>>> a single .scc file, i want to refine the patches across multiple .scc
>>> files. is there an imposed order on SRC_URI entries, .scc files and so
>>> on? that's probably all i need to finish this off.
>>>
>>> rday
>>
>> Robert,
>>
>> in the past I have done pretty much the same: scc,cfg and patches all
>> packed in the recipe.
>> Please see these (outdated) layout examples for linux-yocto* that were
>> in meta-handheld.
>>
>> for 3.10, using .cfg & .scc
>> http://cgit.openembedded.org/meta-handheld/tree/recipes-kernel/linux?h=dylan
>>
>>
>> Or simplified, for 3.14, using defconfig, with patches listed in SRC_URI
>> http://cgit.openembedded.org/meta-handheld/tree/recipes-kernel/linux?h=dizzy
>>
>
>    thanks, i'll check that out. first thing i want to be absolutely clear
> on is, if i have multiple patch directories, i need to add them *all*
> to the search path, as in:
>
>    FILESEXTRAPATHS_prepend := "${THISDIR}/${BP}:${THISDIR}/${BPN}"
>
> and certainly in that order, as i want the more specific version
> patches to be found first. so that bit is correct, yes?
>
>    next, i'm still unclear on whether there is any enforced ordering
> on the processing of .scc files. i know that some folks number their
> patch files as 0001-*, 0002-* and so on, in order to enforce a
> patching order (because the order that one specifies patch/diff files
> in the SRC_URI doesn't mean anything, correct?)

I'm waiting for someone to slap me for saying something so definitive,
but here it goes anyay ... The order in the SRC_URI definitely matters.
The order that they are specified is the order the patches are applied.
Otherwise, a patch stack of any depth wouldn't be possible. Only if you
are doing wildcard patches and rely on shell globbing would numbering
be critical.

>
>    however, i would rather not use a numbering scheme like that because,
> well, it's ugly, and given that i will have patches scattered all over
> the patch directories and subdirectories on a per-kernel and a
> per-target board basis, it just wouldn't make much sense.

Agreed.

>
>    so my plan is to (predictably) group related patch files and .cfg
> files into a number of .scc files but (again) is there any enforced
> search order for .scc files? i'm assuming my setting for FILESEXTRAPATHS
> and use of FILESOVERRIDES will kick in here ... will the order of
> the .scc files in SRC_URI have any effect as well?

.scc files will be processed in the order they are found on the
SRC_URI, and then the patches the order they are within the .scc
files.

>
>    also, once a .scc file is located, will the location of the listed
> .cfg and .patch/.diff files inside it start a whole new search process
> based on FILESEXTRAPATHS and FILESOVERRIDES?

The search path is created relative to the .scc file that is referencing
a patch, and across any directory that has a .scc file in the SRC_URI.
The .scc feature descriptions are self contained, hence "reaching outside"
to a parent, or other directory structure isn't a good thing. Just list
things on the SRC_URI if that is required.

>
>    if this is all written down somewhere, just point me to it. thank
> you kindly.

IIRC it is in the kernel development manual (the .scc file parts), but
the SRC_URI and patch order is common to any recipe that bitbake/oe-core
process.

>
> rday
>
> p.s. all of this is in aid of trying to avoid ordering mishaps when
> applying patches, but i'm guessing that if i design my .scc files
> carefully to be logically self-contained, i can probably avoid
> accidents like that in the first place.

Correct. The .scc files, kernel-cache and management found within
the files was created to manage hundreds of patches for overlapping
boards, much like you are describing.

In the end, when complexity gets really high, a git repo with branches
is probably easier ... and with that description, I've just covered
how patch management with .scc files leads to the linux-yocto tree :)

Bruce

>
>