[yocto] Idea for bbappend / layer organization and naming convention

[Stephane Desneux]

Hi all,

I've followed this thread with interest. I'll try to bring here some ideas we
had in parallel on Automotive Grade Linux (AGL) side.

What we plan to do with backports on AGL is still WIP (see
https://jira.automotivelinux.org/browse/SPEC-802).

It's still cooking, but we're going to adopt a solution close to the one
proposed by Gunnar in GENIVI: I mean: we agree on the same goal, but what we end
up with differs slightly.

The scheme we're thinking about would be typically something like this:

meta-agl/meta-agl-backports/<meta-origin>/<recipes-origin>/<recipe>/<feature|reason>/<recipe>.bbappend

Where:
* the first meta-agl is our main git repo (not a meta in fact): the real meta is
meta-agl/meta-agl :)
* meta-agl/meta-agl-backports is a layer
* <meta-origin> is the name of the layer on which the backport is applied
* <recipes-origin> is the origin folder name for recipes in the origin layer
* <recipe> is the name of the recipe which is backported/amended
* <feature|reason> is something that defines why we have a backport here (bug
id, quick desc...)

And finally, in the folder where we put the bbappend, we want to add a
README.backport in YAML, similar to this one:

—
Origin: <name>
Origin-Url: <source repo url>
Origin-Ref: <rev/tag>
Destination: <AGL version name: dab, eel,...>
Expiration: <when> (if known)
Bug-AGL: <reference to bug ID>
Upstream-Status: <the same as OE's Upstream-Status tag>
Reason: <comment
   on
   multiple
   lines>
—

With that model, we expect to smooth the workflow and make the alignment on
Yocto/OE/Poky versions easier.

HTH

Best regards,
---
Stephane Desneux - CTO - IoT.bzh
stephane.desneux at iot.bzh - www.iot.bzh

On 18/08/17 21:35, Mark Hatle wrote:
> On 8/18/17 1:47 PM, Bruce Ashfield wrote:
>> On 08/18/2017 10:46 AM, Gunnar Andersson wrote:
>>>
>>> On Mon, 2017-08-14 at 23:02 -0400, Bruce Ashfield wrote:
>>>> On 2017-08-14 3:36 PM, Gunnar Andersson wrote:
>>>>>
>>>>> Yocto community,
>>>>>
>>>>> Triggered by the previous email request to yocto@ about best practices
>>>>> for layer organization I'm finally firing off this email for (hopefully)
>>>>> some feedback on an idea we first kicked around, discussed for rough
>>>>> consensus and then decided to implement in the Yocto based GDP project
>>>>> [1].  You can see it as a trial, anything can be changed, but so far so
>>>>> good...
>>>>>
>>>>> We adopted a somewhat novel (but actually not really unique, see inside)
>>>>> naming convention [2] for all modifications to components that belong to
>>>>> other layers.
>>>>>
>>>>
>>>> I've been using a similar naming/sorting mechanism in layers that
>>>> I maintain for several years now.
>>>
>>> Great.  Is it similar, or /exactly/ like this?  I'm only asking because I
>>> think it would be useful to have some written down rules and if there are
>>> additional tweaks that will improve this proposal, I'd like to hear them.
>>>
> 
> I suspect suggestions for best practices along with examples are what people
> would like to see.  This may be reasonable to do with both the Yocto Project and
> OpenEmbedded communities.
> 
> Obviously there are currently a large number of existing layers.  So showing how
> this works, and examples of how it makes it easier to maintain and manage the
> work over the longer term is a really good idea.
> 
> (Keep in mind, many of the current layers were originally designed/updated
> before there was a large ecosystem of layers.  So a lot of poor examples, of
> this kind of maintenance, may exist.  It is a good idea to steer people away
> from using these as examples, and even a way to potentially help the maintainers
> of those layers evolve into a better and more efficient format.)
> 
> --Mark
> 
>> Exactly. The layer name and recipe-* structure is nested in a layer
>> that is carrying bbappends. That way there's zero confusion about where
>> the main recipe can be found.
>>
>> Bruce
>>
>>> I suspected it might actually be somewhat of a common practice.
>>>
>>>> When you have a single layer that is carrying bbappends to many other
>>>> layers, I find that it really helps keep everything separated and aids
>>>> finding the original recipe.
>>>
>>> Yes, that is the idea.
>>>
>>>>
>>>> (that being said, recent work with layer index, etc, make it
>>>> easier to locate the original recipe and any bbappends .. but that
>>>> doesn't preclude keeping things organized in a layer).
>>>
>>> ... but I didn't get any more feedback than yours, so I'll just leave it
>>> where it is for now.  Maybe this is not something the other Yocto community
>>> cares about, or similar practices are actually done in practice, but not
>>> written down.  Or maybe everyone is OK with the state of mixing .bb and
>>> .bbappend files within the layers...
>>>
>>> So far I think the initial experience on our side is positive.  It's
>>> something that needs to be looked after a bit (we have a few mistakes to
>>> fix).  But since some directories will only have .bb files, and others only
>>> .bbappend files, it's therefore easy to script a warning system for anything
>>> that is out of place.
>>>
>>> Best Regards
>>> - Gunnar
>>>
>>>> Cheers,
>>>>
>>>> Bruce
>>>>
>>> [trimmed]
>>>
>>>>> -- 
>>>>> Gunnar Andersson <gandersson at genivi.org>
>>>>> Development Lead
>>>>> GENIVI Alliance
>>>>>
>>>>> [1] https://github.com/GENIVI/genivi-dev-platform
>>>>> [2] Naming strategy for bitbake extension layers:			
>>>>> https://at.projects.genivi.org/wiki/x/w4Xk
>>>
>>>
>>
>