[linux-yocto] kernel-yocto: validate_branches questions

Moore, Tysen Tysen.Moore at xs-embedded.com
Thu Feb 20 18:13:56 PST 2014


I also agree that consistency is a good thing.  Thanks so much for the great support on this.  I look forward to you update.

XS Embedded LLC
29065 Cabot Drive, Suite 200
Novi, MI 48377
USA

Phone +1 (248) 209 - 6613
Fax   +1 (248) 281 - 7020

www.xs-embedded.com
Tysen.Moore at xs-embedded.com

:::::::::: based.on.visions ::::::::::


XS Embedded LLC
Managing Director: Joachim Kobinger

Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

________________________________________
From: Bruce Ashfield [bruce.ashfield at windriver.com]
Sent: Thursday, February 20, 2014 3:55 PM
To: Moore, Tysen; linux-yocto at yoctoproject.org
Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions

On 14-02-20 10:32 AM, Moore, Tysen wrote:
> I see! Thanks for the details.  The reason I got down this rabbit hole [and sorry for dragging you down it as well] is that I need to process

No need to apologize, the conversation has been very interesting, and
I'm happy to explain and discuss .. not to mention, there very well
could have been a bug.
> a file *prior* to the patch step.  From your description it seems that you really can't trust the "WORKDIR/linux" files between validate_branches and patch.  This was the problem for me. What I saw was that on my "do_patch_prepend" the file did not exist because it was still on master and not my branch.  Seems like this is more like a limitation than a bug.  validate_branches does not consistently exit with you on KBRANCH--only does so when KMETA is defined.  Would it be better or a problem to always exit validate_branches on the KBRANCH?  From my point of view it seems to be what you'd expect--or at least a newbie like myself.  What are your thoughts?  Worst case is I have my ugly hack to get me through my use case and satisfy my do_patch_prepend.

It's definitely operating as designed, but yes, there's no reason why
on an early exit it can't do the same sort of checkout that I'm doing
when KMETA is defined .. consistency is a good thing, even if the
next step doesn't care about the branch.

I'm modified the route locally, and will send up a change after soaking
it for a few days.

Bruce

>
> Tysen
>
> XS Embedded LLC
> 29065 Cabot Drive, Suite 200
> Novi, MI 48377
> USA
>
> Phone +1 (248) 209 - 6613
> Fax   +1 (248) 281 - 7020
>
> www.xs-embedded.com
> Tysen.Moore at xs-embedded.com
>
> :::::::::: based.on.visions ::::::::::
>
>
> XS Embedded LLC
> Managing Director: Joachim Kobinger
>
> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>
> ________________________________________
> From: Bruce Ashfield [bruce.ashfield at windriver.com]
> Sent: Thursday, February 20, 2014 10:09 AM
> To: Moore, Tysen; linux-yocto at yoctoproject.org
> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>
> On 14-02-20 08:47 AM, Moore, Tysen wrote:
>> Just to be clear, yes.  Just prior to the KMETA check in validate_branches it is set to master.  Then the KMETA check exits early leaving this on master.  Is there maybe another step that will then correct the problem and move to KBRANCH?  I am not that familiar with the whole process.  Maybe my problem is just my limited understanding of the *whole* process.  I've just been digging mostly into validate_branches and a bit of the patch step.
> No problem, I can explain a bit more. The validate_branches step makes
> sure the tree is sane and contains the branches and commits referenced
> by KBRANCH and the SRCREVs, it also sets the stage for eventual modification
> of the tree in the patch phase, by making sure that the specified SRCREV
> is the HEAD of any branch that contains that same SCREV.
>
> The patching phase, then processes the patches and applied them to the
> tree. Part of that processing will include making sure the right branch
> is checked out before applying the patches.
>
> So if you do a: bitbake -c patch linux-yocto, you should see the proper
> branch checked out after it completes. If it isn't, that's the bug.
>
> Bruce
>
>> Tysen
>>
>> XS Embedded LLC
>> 29065 Cabot Drive, Suite 200
>> Novi, MI 48377
>> USA
>>
>> Phone +1 (248) 209 - 6613
>> Fax   +1 (248) 281 - 7020
>>
>> www.xs-embedded.com
>> Tysen.Moore at xs-embedded.com
>>
>> :::::::::: based.on.visions ::::::::::
>>
>>
>> XS Embedded LLC
>> Managing Director: Joachim Kobinger
>>
>> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>>
>> ________________________________________
>> From: Bruce Ashfield [bruce.ashfield at windriver.com]
>> Sent: Thursday, February 20, 2014 12:57 AM
>> To: Moore, Tysen; linux-yocto at yoctoproject.org
>> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>>
>> On 2/19/2014, 10:20 AM, Moore, Tysen wrote:
>>> The contents are:
>>>
>>> # auto generated BSP file
>>> define KMACHINE xse-j6-axsb
>>> define KTYPE standard
>>> define KARCH arm
>>> branch axsb/ti-glsdk_dra7xx-evm_6_03_00_01.XSE
>> This statement right here .. that's the one that when processing the
>> tree should ensure that the branch "axsb/ti-glsdk_dra7xx-evm_6_03_00_01.XSE"
>> is checked out, and since that is the last statement, it'll be
>> built.
>>
>> So you are saying that after the kernel has finished the patch
>> phase, it is still sitting on master ?
>>
>> I'll go back and try and get myself a tree that can simulate this,
>> since the inputs look fine, something odd must be happening during
>> the processing.
>>
>> Bruce
>>
>>
>>> XS Embedded LLC
>>> 29065 Cabot Drive, Suite 200
>>> Novi, MI 48377
>>> USA
>>>
>>> Phone +1 (248) 209 - 6613
>>> Fax   +1 (248) 281 - 7020
>>>
>>> www.xs-embedded.com
>>> Tysen.Moore at xs-embedded.com
>>>
>>> :::::::::: based.on.visions ::::::::::
>>>
>>>
>>> XS Embedded LLC
>>> Managing Director: Joachim Kobinger
>>>
>>> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>>>
>>> ________________________________________
>>> From: Bruce Ashfield [bruce.ashfield at windriver.com]
>>> Sent: Wednesday, February 19, 2014 10:13 AM
>>> To: Moore, Tysen; linux-yocto at yoctoproject.org
>>> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>>>
>>> On 14-02-19 10:09 AM, Moore, Tysen wrote:
>>>> Since the file is so small I'll copy the contents in text form:
>>>> /home/tmoore/work/stash/alta-axsb/build/tmp-external-linaro-toolchain/work/xse_j6_axsb-oe-linux-gnueabi/linux-ti-glsdk/3.8.13-xse-r6.2+r717bdd79798d9fa41bc8bd2d089b17ebb52819cd/git/meta/cfg/scratch/obj/xse-j6-axsb-standard.scc
>>> And one more quick request, what are the contents of:
>>>
>>> /home/tmoore/work/stash/alta-axsb/build/tmp-external-linaro-toolchain/work/xse_j6_axsb-oe-linux-gnueabi/linux-ti-glsdk/3.8.13-xse-r6.2+r717bdd79798d9fa41bc8bd2d089b17ebb52819cd/git/meta/cfg/scratch/obj/xse-j6-axsb-standard.scc
>>>
>>> Bruce
>>>
>>>
>>>> As for the recipes I am using.  I have copied them below.  The only file slightly neutered is the "xse-kernel.bbclass" which is only the deploy step--not terribly interesting any ways.  If you really need it I could add the code.
>>>>
>>>> I hope this helps.
>>>>
>>>> Tysen
>>>>
>>>> FILE: linux-ti-glsdk_3.8.13-xse.bb
>>>> --------------------------------
>>>> require linux-ti-glsdk.inc
>>>>
>>>> LINUX_VERSION = "3.8.13"
>>>> LINUX_VERSION_EXTENSION = "-00140-gb5b0832"
>>>> KCONFIG_MODE = "--allnoconfig"
>>>>
>>>> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>>>>
>>>> PR = "${INC_PR}.2+r${SRCREV}"
>>>>
>>>> # kernel device tree
>>>> KERNEL_DEVICETREE = "${S}/arch/arm/boot/dts/dra7-axsb.dts"
>>>>
>>>> KBRANCH = "axsb/ti-glsdk_dra7xx-evm_6_03_00_01.XSE"
>>>> SRCREV  = "717bdd79798d9fa41bc8bd2d089b17ebb52819cd"
>>>>
>>>> SRC_URI = "\
>>>>             git://git@git01.xse.de/third-party-ti/ti-linux-kernel.git;protocol=ssh \
>>>>             file://defconfig \
>>>> "
>>>>
>>>> FILE: linux-ti-glsdk.inc
>>>> --------------------------------------
>>>> DESCRIPTION = "Linux kernel for TI devices supported by GLSDK product"
>>>> HOMEPAGE    = "http://www.ti.com"
>>>> SECTION     = "kernel"
>>>> LICENSE = "GPLv2"
>>>> LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"
>>>>
>>>> inherit xse-kernel
>>>>
>>>> COMPATIBLE_MACHINE = "omap-a15"
>>>>
>>>> INC_PR = "r6"
>>>>
>>>> S = "${WORKDIR}/git"
>>>>
>>>> PR = "r0+r${SRCREV}"
>>>>
>>>> # Define outputname for xse-kernel.bbclass
>>>> IBC_KERNEL_NAME ?= "linux"
>>>>
>>>> # dont'keep zImage in deploy folder
>>>> KEEPZIMAGE = "no"
>>>>
>>>> FILE: xse-kernel.bbclass
>>>> -----------------------------------------------
>>>> require recipes-kernel/linux/linux-yocto.inc
>>>> ...
>>>> ... [ Then does some things to package our proprietary boot, just a deploy step ] ...
>>>> ...
>>>> # then my hack
>>>> do_validate_branches_prepend() {
>>>>         KMETA="${KBRANCH}"
>>>>         SRCREV_meta="${SRCREV}"
>>>>         SRCREV_machine="${SRCREV}"
>>>> }
>>>> do_validate_branches_append() {
>>>>         KMETA=""
>>>>         SRCREV_meta=""
>>>>         SRCREV_machine=""
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> XS Embedded LLC
>>>> 29065 Cabot Drive, Suite 200
>>>> Novi, MI 48377
>>>> USA
>>>>
>>>> Phone +1 (248) 209 - 6613
>>>> Fax   +1 (248) 281 - 7020
>>>>
>>>> www.xs-embedded.com
>>>> Tysen.Moore at xs-embedded.com
>>>>
>>>> :::::::::: based.on.visions ::::::::::
>>>>
>>>>
>>>> XS Embedded LLC
>>>> Managing Director: Joachim Kobinger
>>>>
>>>> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>>>>
>>>> ________________________________________
>>>> From: Bruce Ashfield [bruce.ashfield at windriver.com]
>>>> Sent: Wednesday, February 19, 2014 8:31 AM
>>>> To: Moore, Tysen; linux-yocto at yoctoproject.org
>>>> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>>>>
>>>> On 14-02-19 06:32 AM, Moore, Tysen wrote:
>>>>> Bruce,
>>>>>
>>>>> I spoke to a colleague today and got the version info you requested.
>>>> Perfect. In 1.4.x, the file I'm looking for would be in
>>>> WORKDIR/linux/meta/top_tgt
>>>> (i.e. *not* .meta).
>>>>
>>>> Do you see it now ? I'm also just going to try and create a similar
>>>> recipe and build, since you've been patient and waiting long enough
>>>> for this to be resolved. If any of your parts can be made public,
>>>> feel free to send them my way .. and I'll use them to kickstart things.
>>>>
>>>> bottom line is that your use case is supported, has been tested and
>>>> should just work without any mods.
>>>>
>>>> Bruce
>>>>
>>>>> <!ENTITY DISTRO "1.4.1">
>>>>> <!ENTITY DISTRO_COMPRESSED "141">
>>>>> <!ENTITY DISTRO_NAME "dylan">
>>>>> <!ENTITY YOCTO_DOC_VERSION "1.4.1">
>>>>> <!ENTITY POKYVERSION "9.0.1">
>>>>>
>>>>>
>>>>> XS Embedded LLC
>>>>> 29065 Cabot Drive, Suite 200
>>>>> Novi, MI 48377
>>>>> USA
>>>>>
>>>>> Phone +1 (248) 209 - 6613
>>>>> Fax   +1 (248) 281 - 7020
>>>>>
>>>>> www.xs-embedded.com
>>>>> Tysen.Moore at xs-embedded.com
>>>>>
>>>>> :::::::::: based.on.visions ::::::::::
>>>>>
>>>>>
>>>>> XS Embedded LLC
>>>>> Managing Director: Joachim Kobinger
>>>>>
>>>>> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>>>>>
>>>>> ________________________________________
>>>>> From: linux-yocto-bounces at yoctoproject.org [linux-yocto-bounces at yoctoproject.org] on behalf of Moore, Tysen [Tysen.Moore at xs-embedded.com]
>>>>> Sent: Tuesday, February 18, 2014 7:01 PM
>>>>> To: Bruce Ashfield; linux-yocto at yoctoproject.org
>>>>> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>>>>>
>>>>> We are using:
>>>>> BitBake Build Tool Core version 1.18.0, bitbake version 1.18.0
>>>>>
>>>>> Unfortunately I am not 100% sure what version we are using of Yocto/etc.  I do know we are a bit behind.  I did compare validate_branches versus one I found on github and the only difference was some SRCREV_machine checking in the beginning of the routine.  Is there an easy way to tell what version of yocto/poky/etc is loaded?
>>>>>
>>>>> XS Embedded LLC
>>>>> 29065 Cabot Drive, Suite 200
>>>>> Novi, MI 48377
>>>>> USA
>>>>>
>>>>> Phone +1 (248) 209 - 6613
>>>>> Fax   +1 (248) 281 - 7020
>>>>>
>>>>> www.xs-embedded.com
>>>>> Tysen.Moore at xs-embedded.com
>>>>>
>>>>> :::::::::: based.on.visions ::::::::::
>>>>>
>>>>>
>>>>> XS Embedded LLC
>>>>> Managing Director: Joachim Kobinger
>>>>>
>>>>> Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
>>>>>
>>>>> ________________________________________
>>>>> From: Bruce Ashfield [bruce.ashfield at windriver.com]
>>>>> Sent: Tuesday, February 18, 2014 4:01 PM
>>>>> To: Moore, Tysen; linux-yocto at yoctoproject.org
>>>>> Subject: Re: [linux-yocto] kernel-yocto: validate_branches questions
>>>>>
>>>>> On 14-02-18 03:23 PM, Moore, Tysen wrote:
>>>>>> I looked for the file you requested but did not see it.  Where can I
>>>>>> find this?  Is it part of the linux kernel WORKDIR?  If so, I greped the
>>>>>> tree and I don't see it.  Or am I looking in the wrong place?  If I can
>>>>>> find it I will be sure to pass it on.
>>>>> What yocto branch are you using ?
>>>>>
>>>>> It's in the WORKDIR/linux/.meta/ directory.
>>>>>
>>>>>> What I've got are two branches:
>>>>>> - My KBRANCH branch with the last commit being: (717bdd797)
>>>>>> - Another branch with the last commit being (c826758fb), then the commit
>>>>>> (717bdd797)
>>>>>>
>>>>>> So the validate_branches does a checkout of master, the goes over these
>>>>>> two branches and puts them on the commit I wanted--I think this is
>>>>>> fine.  Then the KMETA check takes place and fails because I do not have
>>>>>> KMETA defined, thus exiting early and leaving me at master.  The problem
>>>>>> seems to be the failing of this KMETA that is not defined.  It seems
>>>>>> like KMETA is required for this to work properly.
>>>>> KMETA is definitely not required, I do korg based builds all the time,
>>>>> all without a meta branch, so we can make this work, I just need to know
>>>>> what has broken .. with a bit more information, we'll get there.
>>>>>
>>>>> Bruce
>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>> *From:* Bruce Ashfield [bruce.ashfield at windriver.com]
>>>>>> *Sent:* Tuesday, February 18, 2014 2:29 PM
>>>>>> *To:* Moore, Tysen; linux-yocto at yoctoproject.org
>>>>>> *Subject:* Re: [linux-yocto] kernel-yocto: validate_branches questions
>>>>>>
>>>>>> On 14-02-18 10:51 AM, Moore, Tysen wrote:
>>>>>>
>>>>>> XSe Logo
>>>>>>
>>>>>> *XS Embedded LLC*
>>>>>> 29065 Cabot Drive, Suite 200
>>>>>> Novi, MI 48377
>>>>>> USA
>>>>>>
>>>>>> Phone         +1 (248) 209 - 6613
>>>>>> Fax   +1 (248) 281 - 7020
>>>>>>
>>>>>>
>>>>>> www.xs-embedded.com <http://www.xs-embedded.com>
>>>>>> Tysen.Moore at xs-embedded.com <mailto:Tysen.Moore at xs-embedded.com>
>>>>>>
>>>>>> *:::::::::: based.on.visions ::::::::::*
>>>>>>
>>>>>> XS Embedded LLC
>>>>>> Managing Director: Joachim Kobinger
>>>>>>
>>>>>> Confidentiality Notice: This e-mail message, including any attachments,
>>>>>> is for the sole use of the intended recipient(s) and may contain
>>>>>> confidential and privileged information. Any unauthorized review, use,
>>>>>> disclosure or distribution is prohibited. If you are not the intended
>>>>>> recipient, please contact the sender by return e-mail and destroy all
>>>>>> copies of the original message.
>>>>>>
>>>>>>
>>>>>>> Bruce,
>>>>>>>
>>>>>>> Thanks for the fast response.  I only posted the question yesterday.
>>>>>> I like to reply quickly to problems :)
>>>>>>
>>>>>>> The "KMETA Check" that I am referring to is:
>>>>>>>          ## KMETA branch validation
>>>>>>>           meta_head=`git show-ref -s --heads ${KMETA}`
>>>>>>>           target_meta_head="${SRCREV_meta}"
>>>>>>>          git show-ref --quiet --verify -- "refs/heads/${KMETA}"
>>>>>>>          if [ $? -eq 1 ]; then
>>>>>>>              return                            #<<<<<<<<<<<<<<< EXITS HERE
>>>>>>>          fi
>>>>>>>
>>>>>>> Because with KMETA not defined here it exists early.  The problem is
>>>>>>> that the earlier "SRCREV loop" starts with, "git checkout -q master".
>>>>>>> Since KMETA is not defined and we exit early the code that follows the
>>>>>>> KMETA check to change back to KBRANCH is not executed.
>>>>>> Correct. But that's supposed to be safe, the checkout of the machine
>>>>>> branch at the bottom of the routine is not supposed to be the last
>>>>>> manipulation of the branches. You should still end up on the right
>>>>>> branch when the build happens.
>>>>>>
>>>>>> I'm driving to understand why that is happening.
>>>>>>
>>>>>> Can you share some specific details ?  If this is master, and you
>>>>>> go into the kernel's source directory tmp/work/*/linux/ and cat
>>>>>> .meta/top_tgt, that should point to a generated file, what are the
>>>>>> contents of that generated description.
>>>>>>
>>>>>> One solution would be to allow that final git checkout to run (as
>>>>>> you mentioned), but if other manipulations of the tree happen,
>>>>>> you'll still be broken, and I want to avoid that.
>>>>>>
>>>>>> Bruce
>>>>>>
>>>>>>> The only workaround I've come up with is very hackish--embarrassing to
>>>>>>> post;-)
>>>>>>> do_validate_branches_prepend() {
>>>>>>>          KMETA="${KBRANCH}"
>>>>>>>          SRCREV_meta="${SRCREV}"
>>>>>>>          SRCREV_machine="${SRCREV}"
>>>>>>> }
>>>>>>> do_validate_branches_append() {
>>>>>>>          KMETA=""
>>>>>>>          SRCREV_meta=""
>>>>>>>          SRCREV_machine=""
>>>>>>> }
>>>>>>> [As you can imagine I'd prefer not to use this hack.]
>>>>>>>
>>>>>>> The problem is that if I do not clear KMETA the configme (kgit-meta
>>>>>>> and friends) call later fails.  The kernel_configme code allows KMETA
>>>>>>> to be undefined but it is almost like the validate_branches is coded
>>>>>>> without that same option.  Shouldn't the KMETA "check" code listed
>>>>>>> above be skipped when KMETA is not defined? I think this change to the
>>>>>>> validate_branches code would fix this use case.  I don't think it
>>>>>>> would break other use cases, but I am rather green with Yocto.
>>>>>>>
>>>>>>> Thanks again for the detailed description and help.
>>>>>>>
>>>>>>> Tysen
>>>>>>>
>>>>>>> XSe Logo
>>>>>>>
>>>>>>> *XS Embedded LLC*
>>>>>>> 29065 Cabot Drive, Suite 200
>>>>>>> Novi, MI 48377
>>>>>>> USA
>>>>>>>
>>>>>>> Phone        +1 (248) 209 - 6613
>>>>>>> Fax  +1 (248) 281 - 7020
>>>>>>>
>>>>>>>
>>>>>>> www.xs-embedded.com <http://www.xs-embedded.com>
>>>>>>> Tysen.Moore at xs-embedded.com <mailto:Tysen.Moore at xs-embedded.com>
>>>>>>>
>>>>>>> *:::::::::: based.on.visions ::::::::::*
>>>>>>>
>>>>>>> XS Embedded LLC
>>>>>>> Managing Director: Joachim Kobinger
>>>>>>>
>>>>>>> Confidentiality Notice: This e-mail message, including any
>>>>>>> attachments, is for the sole use of the intended recipient(s) and may
>>>>>>> contain confidential and privileged information. Any unauthorized
>>>>>>> review, use, disclosure or distribution is prohibited. If you are not
>>>>>>> the intended recipient, please contact the sender by return e-mail and
>>>>>>> destroy all copies of the original message.
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> linux-yocto mailing list
>>>>> linux-yocto at yoctoproject.org
>>>>> https://lists.yoctoproject.org/listinfo/linux-yocto
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>






More information about the linux-yocto mailing list