[linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...

[Bruce Ashfield]

On 13-11-07 02:40 AM, Darren Hart wrote:
> On Wed, 2013-11-06 at 23:22 -0800, Darren Hart wrote:
>> On Wed, 2013-11-06 at 21:07 -0500, Bruce Ashfield wrote:
>>> On 11/6/2013, 4:40 PM, Darren Hart wrote:
>>>> On Tue, 2013-11-05 at 22:00 -0500, Bruce Ashfield wrote:
>>>>> On 13-11-05 6:36 PM, Darren Hart wrote:
>>>>>> On Tue, 2013-11-05 at 17:15 -0600, Tom Zanussi wrote:
>>>>>>> On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote:
>>>>>>>> I'm working on rewriting the minnow-io feature to just apply patches.
>>>>>>>> It's working.... but something is seriously horked with 3.10 - or my
>>>>>>>> 3.10 tree.... or.... I don't know. HALP!
>>>>>>>>
>>>>>>>> The first problem was it was building PREEMPT_RT from a linux-yocto
>>>>>>>> recipe. Turns out the eg20t feature has a branch (even in the
>>>>>>>> yoctoproject.org git repo) which includes the preempt-rt sources, so
>>>>>>>> those were all getting merged in. I removed the eg20t branch command and
>>>>>>>> now preempt-rt isn't getting merged in. First I don't understand why
>>>>>>>> eg20t even has a branch (did I do that?). Second, why would it contain
>>>>>>>
>>>>>>> I'm not sure how the eg20t branch could be getting merged in - the eg20t
>>>>>>> feature doesn't contain a 'git merge', and is just there for the .cfg.
>>>>>>>
>>>>>>> The eg20t branch originally contained about 70 eg20t patches before
>>>>>>> they'd gone upstream, and is now useless AFAIK, so should probably be
>>>>>>> removed.
>>>>>>>
>>>>>>>> PREEMPT_RT? Fumbled upgrade?
>>>>>>>
>>>>>>> Sounds like it, but based on the above it shouldn't really matter as it
>>>>>>> shouldn't be used at all..
>>>>>>
>>>>>> Right, it didn't have a merge command, just a branch:
>>>>>>
>>>>>> $ cat meta/cfg/kernel-cache/features/eg20t/eg20t.scc
>>>>>> # changes should be staged on "eg20t"
>>>>>> git branch eg20t master
>>>>>>
>>>>>> kconf hardware eg20t.cfg
>>>>>>
>>>>>> Still, including this scc brought in the eg20t branch. Perhaps if the
>>>>>> branch name exists the branch command does a merge? Bug?
>>>>>
>>>>> We picked this up last week during some integration work @ Wind. The
>>>>> git branch was only used when staging the branch, and shouldn't have
>>>>> been in the .scc .. BSPs that are still including it are only working
>>>>> because they aren't adding any patches AFTER the command.
>>>>>
>>>>> In your case, you are .. hence why things are going insane.
>>>>>
>>>>> I have a change queued that drops the branch command, and moves eg20t
>>>>> to staging, and then "to be killed", as Tom points out.
>>>>>
>>>>>>
>>>>>> I've confirmed this reverting to 3.10.10 happens in the do_patch()
>>>>>> task, but I don't know where or why yet.
>>>>>
>>>>> Have you tried the same build with the branch command removed ?
>>>>>
>>>>
>>>> I removed eg20t from minnow.scc completely and I removed all features
>>>> added in the recipe. The do_patch() stage still reverts from 3.10.17 to
>>>> 3.10.10. I'll start instrumenting the tools I guess.
>>>
>>> It's not the tools, they don't mess with SRCREVs like this. My guess is
>>> that you are hitting a corner case in the do_validate_branches. To solve
>>> some issues with dynamic branches, they reset all branches that contain
>>> the machine's SRCREV to that value, and then start processing changes.
>>
>> This happens in patchme do_apply() somewhere. I'm currently
>> instrumenting. I'll have a pile of cleanups and instrumentation
>> patches ... hopefully tomorrow. My instrumentation tells this story so
>> far:
>>
>> NOTE: git HEAD prior to patchme: c03195e kconfig: inhibit unitialized
>> variable warning
>> meta-dir(): meta-branch=meta
>> meta-dir(): using meta_dir=.meta
>> ktgt=minnow
>> meta_series=
>> branch=standard/base
>> meta_series=.meta/cfg/scratch/minnow-standard-meta
>> Git HEAD: c03195e kconfig: inhibit unitialized variable warning
>> do_init()
>> Git HEAD: c03195e kconfig: inhibit unitialized variable warning
>> Git HEAD: c03195e kconfig: inhibit unitialized variable warning
>> do_apply()
>> [INFO] validating against known patches  (minnow-standard-meta)
>>    [##################################################]  (completed in 6
>> seconds)
>> Git HEAD: ebc8428 Merge tag 'v3.10.10' into standard/base
>> NOTE: git HEAD after patchme: ebc8428 Merge tag 'v3.10.10' into
>> standard/base
>>
>>
>>>
>>> So double check two things. What is your machine's SRCREV ? Is it the
>>> 3.10 hash ?
>>
>> It's the HEAD of my standard/base, which is 3.10.17
>>
>>>   And try doing a build with AUTOREV, that disables the branch
>>> reset functionality and will point the smoking gun.
>>
>> OK, let's give that a try.... but it happens patchme, so it should fix
>> it....
>>
>> Same deal.
>>
>>
>> BUT! AHA. Some git branch --contains on the commits listed in the output
>> above reveal that "standard/minnow" HEAD is v3.10.10. Now, it is
>> supposed to be using standard/base, but I think I've seen this before,
>> if the tools try to create a branch that already exists, they just use
>> the existing one.
>>
>> Delete standard/minnow from the publish tree... and vwhalla... no more
>> resetting to an older tree.
>>
>> So.... some kind of 'bbwarn "Hey dummy, this branch already exists"' is
>> in order. Digging in to see if I can find where and include it in my
>> instrumentation patches.
>>
>> 2 days later.... now I can test the new minnow-io and 3.10 BSP.... sigh.
>>
>
> OK, it occurs in wrap_meta_series() at:
>
> source $_series.wrap
>
> Which makes further instrumentation rather complicated.
>
> I had a discussion with Arjan about code that needed to be easily
> debugged... and I think this might qualify. His argument was that in
> such a case, it made sense to code it in explicit steps and not make it
> so generic that you had to unravel it before you could debug it. I think
> we may have a highly generic implementation in a code base that really
> might serve us better if it were written in explicit steps - difficult
> to articulate what I'm trying to say here.... and I'm just a little
> irritated with it ;-)

But that's like saying that how quilt, guilt or even git-am would need
to be unrolled and explicitly called from the top level. I've spent
more than my fair time reverse engineering and debugging the three of
them .. and when they break, they break hard.

It's a balance of making the tools work with bitbake, standalone and
with their other build system bindings. Needing to do some of the work
in the build system side, and the rest in the tools means that there's
a bit of abstraction required by definition :)

The current version of the tools are actually much more straight forward
and actually possible to invoke from the command line.

Also of note .. I'm about to kill off some of the *me scripts, since they
are no longer necessary, and I think that'll get more clarity in the parts
of the build that were causing you pain.

Bruce

>
> More tomorrow.
>