[yocto] release management

[Russell Peterson]

Thanks very much, Gunnar, for your detailed and helpful response.

Not sure I will go the git submodule path.  My initial try is using a script in my build environment that pulls in whatever layers it needs via git clone and then using a specific commit in a specific branch for that specific build.  The cloned layers are considered “read only”.  Right now we clone directly from yocto or github but we could easily switch to local, non-public mirrors that we could populate to specific versions.

Thanks again!

Regards,

Russell


> On Aug 14, 2017, at 3:10 PM, Gunnar Andersson <gandersson at genivi.org> wrote:
> 
> 
> Hey Russell
> 
> I don't claim to be an authority on best practices but I will share some
> thoughts.
> 
> On Sun, 2017-08-13 at 06:58 -0400, Russell Peterson wrote:
>> Hello.
>> 
>> As I learn more about yocto and more importantly gain practical experience
>> with it I have started to think about my release structure.  Is there a
>> “best practices” document or something like that that speaks to this?
>> For example, how does everyone deal with “external” meta layer
>> dependencies?  My software uses poky and meta-openembedded, of course.  
> 
> We simply use a parent project [1] that includes git submodules, one per
> yocto layer.  I'm of the opinion that if git (only) can be used, why
> introduce other tools?   But it requires you to learn and master that
> feature.
> 
> For the primary layer that is unique to this project, meta-genivi-dev, we
> later on decided to merge it into the parent git repository instead, but
> keep all other layers as submodules.  Since there are frequent changes to
> the primary layer, it reduces the complexity of having to update a submodule
> for them continuously.  But the purpose is also to keep *only* unique things
> there, and thereby such things that are reusable by others will be naturally
> pushed to the other layers (those that are reusable, and therefore should be
> separate repos).
> 
> There's a natural tradeoff between layers being developed independently (and
> widely reused by other projects), vs. the complexity of hacking on your
> project.
> 
> Other projects use Google's "repo" tool but I have yet to understand what
> that adds compared to git-submodules (in a medium sized project at least),
> other than just additional hoops to jump through.
> 
>> It also relies on some recipes in meta-linaro and meta-virtualization.  I
>> suspect there will be more as time goes by.  I have tweaked my layer
>> priorities as well as used BBMASK to avoid unwanted bbappend files etc…
>> works but seems slightly clunky… still better than duplicating recipes in
>> my meta layer I think.
> 
> I think you've got it.  Those are the tools you have.  I haven't found too
> frequent needs to mask away that which is provided by other layers, but it
> depends on your needs...
> 
>> 
>> Also… I quickly came to the conclusion that “thou shall not modify poky or
>> meta-openembedded”.  
>> That is, ALWAYS use bbappend files instead of modifying external layers.  
> 
> Absolutely that's the Yocto way.  But .bbappends are a kind of patches,
> so who's to say what is right or wrong?  Some people might think that git is
> actually a better tool to keep track of local modifications - keeping those
> on branches that can be efficiently diffed and merged when upstream changes.
> 
> But clearly most Yocto proficient developers will be used to a .bbappend
> based workflow, and I would wager that you will be better received with any
> support questions than if you say "I have made my own branch of poky/meta-
> oe).  Strictly speaking the two option ought to be equivalent, but I'm just
> guessing it will still be perceived differently.  Yocto devs might agree or
> disagree.
> 
> 
>> If I think that poky or some other layer has a recipe bug or want to
>> change something in poky, for example, I plan to upstream a fix to the
>> project and when that becomes available I rebase my poky and remove the
>> bbappend from my meta layer. 
> 
>> One thing about modifying poky and not using bbappend files is that I
>> would then need to ship patches for poky instead of just directing users
>> to to use this branch and this commit for release x.y.z.
> 
> You're not really asking a question here - just drawing the standard open-
> source process conclusions.  I for one agree with you.  Use bbappends.  Talk
> to the upstream.  Always second-guess why you need to change anything at
> all.  Then if the appends become very complex, consider if finally
> overriding with a new .bb file fits better.  
> 
> In our policy that I will describe further (below) we also mostly prefer the
> more explicit separate bbappend files that by virtue of having its own path
> location can be selectively included or not included in the local.conf for
> different project variations.  This feels less magical than having various
> hardware-specific overrides and other deeply embedded tweaks.  But there's
> not 100% right or wrong here -- it's clearly a case of use the right tool
> depending on the situation.
> 
>> 
>> Comments and suggestions welcome.
> 
> In the GENIVI community we decided on a quite specific organization for
> layer modifications / bbappends.  I have been meaning to write an email to
> gather community feedback on it, but it's better done in a separate thread I
> think.
> 
> Best Regards
> - Gunnar
> 
> [1] https://github.com//GENIVI/genivi-dev-platform
> 
> -- 
> Gunnar Andersson <gandersson at genivi.org>
> Development Lead
> GENIVI Alliance
> 
> 
>> 
>> Regards,
>> 
>> Russell
>> 
>> 
>> 
>> 
>