[yocto] swupate.bblcass dependencies not triggering rebuild

[Andreas Fenkart]

Hi

2017-07-28 15:20 GMT+02:00 Andreas Fenkart <afenkart at gmail.com>:
> Hi Stefano,
>
> 2017-06-14 10:32 GMT+02:00 Stefano Babic <sbabic at denx.de>:
>> Hi Andreas,
>>
>> On 13/06/2017 14:18, Andreas Fenkart wrote:
>>> Hi list,
>>>
>>> I want to create a swupdate file, which is a cpio file with rootfs and
>>> some metadata to control software update. My problem is, that it is not
>>> rebuilt everytime the rootfs changes. I checked in the cooker log.
>>>
>>> NOTE: recipe swupdate-image-1.0-r4: task do_populate_sysroot_setscene: Succeeded
>>> NOTE: Running noexec task 5475 of 5478 (ID: 27,
>>> meta-devel/recipes-core/images/swupdate-image.bb, do_build)
>>>
>>> I generated the dot files with bitbake -g swupdate, and while do_swuimage
>>> has to run before do_build, do_build does not depend directly on do_swuimage:
>>>
>>> swupdate-image.do_createlink -> swupdate-image.do_swuimage
>>> swupdate-image.do_install -> swupdate-image.do_createlink
>>> swupdate-image.do_populate_sysroot -> swupdate-image.do_install
>>> swupdate-image.do_build" -> swupdate-image.do_populate_sysroot
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_build
>>> ERROR: No sigdata files found matching swupdate-image do_build
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_populate_sysroot
>>> Unable to find matching sigdata for ....
>>> recipes-core/images/swupdate-image.bb.do_install with hashes
>>> c0bc8039e8dbe8d70939dd3db2ceb922 or e9d7b242613f323d4d6adff3d6d4881b
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_install
>>> ERROR: No sigdata files found matching swupdate-image do_install
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_createlink
>>> Hash for dependent task imagesswupdate-image.bb.do_swuimage changed
>>> from cec64d7e21b0375ff641427d5d631254 to
>>> 45510576a402839f4c4d4ac9b9eff578
>>>   Hash for dependent task imagesrootfs.bb.do_build changed from
>>> 876eb436b444f84efd355c0294b871bf to fabbabdd000427cdf5e721b59784d5c4
>>>   Unable to find matching sigdata for
>>> meta-devel/recipes-core/images/rootfs.bb.do_build with hashes
>>> 876eb436b444f84efd355c0294b871bf or fabbabdd000427cdf5e721b59784d5c4
>>>
>>> The dependency on rootfs is added here:
>>> https://github.com/sbabic/meta-swupdate/blob/master/classes/swupdate.bbclass#L64
>>> with IMAGE_DEPENDS="rootfs"
>>>
>>> Is it correct to 'depend' on :do_build? It seems rather uncommon.
>>
>> I think that the first attempt to set dependency here was with
>> do_rootfs, but this just works for rootfs, not for all other images. In
>> the history. commit b2f47aa9de53a98380b8677eb9e078db608b0ad3 changed
>> from do_populatesysroot to do_build because it was noted that it runs
>> too early.
>>
>> I am not sure if I have this issue, maybe it was sporadic. I wrote a
>> patch, really to fix an issue in the transition morty --> pyro, where I
>> changed the dependency as:
>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>> index 44e4546..511f147 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -61,16 +61,23 @@ def swupdate_getdepends(d):
>>
>>      depstr = ""
>>      for dep in deps:
>> -        depstr += " " + dep + ":do_build"
>> +        if dep.find(":") != -1:
>> +            deptask = ""
>> +        else:
>> +            if (dep == "virtual/kernel") or (dep == "virtual/bootloader"):
>> +                deptask = ":do_deploy"
>> +            else:
>> +                deptask = ":do_image_complete"
>> +
>> +        depstr += " " + dep + deptask
>> +
>>      return depstr
>>
>>
>> So "do_image_complete" instead of "do_build", but I have to check
>> explicitely for kernel and bootloader because they have no
>> do_image_complete. But I do not know if this can be a correct solution.
>>
>>>
>>> I checked the latest swupdate-image.do_populate siginfo in sstate, which is
>>> pretty large, so adding just the task-dependencies:
>>> ..
>>> Tasks this task depends on:
>>> ['..-devel/recipes-core/images/swupdate-image.bb.do_install']
>>> This task depends on the checksums of files: []
>>> Hash for dependent task
>>> ./meta-devel/recipes-core/images/swupdate-image.bb.do_install is
>>> e9d7b242613f323d4d6adff3d6d4881b
>>> Computed Hash is d8b1d389351c25e0c4a519fb8fe9ddda
>>> ..
>>>
>>> Which makes me wonder, since bitbake-diffsig said do_install has no sigdata
>>>
>>
>> We have do_install[noexec] = "1" in the class - is this the reason ?
>
> I removed all the 'noexec' stamps from swupdate.bbclass, but still the
> cooker.log says, that none of the swupdate-image tasks needs
> execution:
>
> NOTE: Setscene stamp current for task
> 5(../recipes-core/images/swupdate-image.bb, do_populate_sysroot)
> NOTE: Setscene stamp current for task
> 6(../recipes-core/images/swupdate-image.bb, do_populate_lic)
> NOTE: Setscene stamp current for task
> 7(../recipes-core/images/swupdate-image.bb, do_packagedata)
> NOTE: Setscene stamp current for task
> 8(../recipes-core/images/swupdate-image.bb, do_package_qa)
> NOTE: Setscene stamp current for task
> 9(../recipes-core/images/swupdate-image.bb, do_package)
>
> The funny thing is, that bitbake swupdate-image actually triggers a built
> of the rootfs. But the task do_swuimage is not even mentioned in the
> cooker.log! Now only do_swuimage depends on rootfs:do_build, so
> how can bitbake built a dependency for task it's not running?
>
> This is yocto-2.0.2, quite old, but I'd expect this to work on that
> release still

I added this workaround, now it builds a new swupdate image
on every bitbake invocation:

--- a/yocto/dS/meta-digitalstrom-devel/classes/swupdate.bbclass
+++ b/yocto/dS/meta-digitalstrom-devel/classes/swupdate.bbclass
@@ -91,6 +91,7 @@ do_createlink () {
     ln -sf ${IMAGE_NAME}.swu ${IMAGE_LINK_NAME}.swu
 }

+do_swuimage[nostamp] = "1"
 python do_swuimage () {
     import shutil

I also experimented with switching the dependency from do_build
to the do_rootfs task, just in case the problem is that do_build has
no proper stamp (0 sized in the stamps folder). To no avail. It does
not seem to matter if swupdate class depends on do_rootfs or
do_build.
Only 'nostamp' forces the do_swuimage to be built reliably. This isn't
an issue in my case, since the rootfs is built on every bitbake
invocation anyway, even if all stamps of the containing packages are
still valid, and do_rootfs is the first task to be executed.

>
>>> I must admit that I feel pretty lost here.
>>
>> Best regards,
>> Stefano

Andi