[yocto] sysroot not being populated

[Patrick Vacek]

On 14.06.2018 16:28, Khem Raj wrote:
> On Thu, Jun 14, 2018 at 5:45 AM Patrick Vacek <patrick.vacek at here.com> wrote:
>> On 12.06.2018 00:48, Andre McCurdy wrote:
>>> On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek at here.com> wrote:
>>>> On 08.06.2018 10:26, Khem Raj wrote:
>>>>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>>>>>> On 07.06.2018 19:06, Khem Raj wrote:
>>>>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>>>>>> <patrick.vacek at here.com> wrote:
>>>>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>>>>>> (aktualizr)
>>>>>>>> to provide an executable and a config file. The former recipe includes
>>>>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>>>>>> aktualizr-hsm-prov has a line something like this:
>>>>>>>>
>>>>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>>>>>
>>>>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>>>>>> can find that. (Although to be honest, I'm not sure which version
>>>>>>>> it is
>>>>>>>> running.) However, it doesn't find the config file, and sure enough,
>>>>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>>>>>> expect. I
>>>>>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>>>>>> but it
>>>>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>>>>>
>>>>>>>> I see this problem in sumo and master, although previously this logic
>>>>>>>> has worked just fine in morty/pyro/rocko.
>>>>>>>>
>>>>>>>> What am I doing wrong? What changed between rocko and sumo?
>>>>> Can you check location of sota.conf in the build tree for
>>>>> aktualizr-native in directory called package/
>>>> Oddly, I do not see a package subdirectory inside the aktualizr-native
>>>> directory in the build tree. I do see it inside the aktualizr directory,
>>>> though, and it contains everything that I would expect. Is there some
>>>> sort of configuration of the packaging system for a native recipe that I
>>>> haven't done correctly?
>>> No, what you see is expected - there's no packaging for -native recipes.
>>>
>>> Back to the original problem, I think you should consider the
>>> aktualizr executable and the sota.conf file as two separate things.
>>> The host executable should always be provided by aktualizr-native.
>>> It's less clear what should provide sota.conf - it depends whether
>>> it's just required for building other recipes (ie like a header file)
>>> or whether it needs to be present in the final rootfs? If it's only
>>> required for building other packages, then it should be in the -dev
>>> package for aktualizr. If it needs to be present in the final rootfs
>>> then it should be in the default package for aktualizr. Either way,
>>> recipes which need to find sota.conf in sysroot would then depend on
>>> "aktualizr". Therefore recipes which need both the host executable and
>>> sota.conf in sysroot should depend on both aktualizr and
>>> aktualizr-native.
>> Actually, after some further research, it appears that my problem is not
>> entirely resolved. The explanation of packaging and native vs. target
>> dependencies all makes sense to me and usually works. However, this is
>> not the case for one specific condition which happens to be part of my
>> test suite.
>>
>> One of my tests is to change the MACHINE in my local.conf, add some
>> additional layers, change some other variables, and then bitbake. Then I
>> undo that and bitbake again. For the most part, that works fine. In
>> rocko and before, it worked consistently. But it seems like on sumo and
>> master, when I do the test by changing the MACHINE and the layers, it
>> somehow clears out everything in my recipe's work directory, despite
>> that that recipe is not used by that test. In fact, it removes every
>> single file in tmp/work/core2-64-poky-linux (but leaves most of the
>> directories). When I switch back to how it was before and bitbake, most
>> of the files get recreated or re-copied, but not all. I have to
>> explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get
>> things to work again. (Perhaps cleanall isn't necessary, but some form
>> of cleaning is, and it has to be both recipes.)
>>
>> Can someone help explain why I am seeing this behavior? I don't
>> understand why the files get erased, why they don't get repopulated, and
>> why this was different back in rocko.
>>
> when machine and other variables are changed then its mostly
> resulting in hash changes which is enforcing it to rebuild and when
> rebuild happens it will clear the recipe specific build area for that
> given recipe and repopulate it as it has to rebuild/repackage it since
> this time it will get most of build artifacts from shared state as it
> has been bullt with same hash once before. so it will most probably be
> re-using that.
That make sense.
> If doing clean build is helping then this might mean that they
> are not staging all the files that you depend on. When you do
> clean build they get staged into build area during compile phase and
> it silently uses it. So ensure that all files you need are accounted
> for in do_install
That makes sense, too, but the first recipe (aktualizr) definitely
installs the config file I need in do_install and lists it in
FILES_${PN}-host-tools. In fact, the entire package subdirectory of
aktualizr is not repopulated, and thus the recipe-sysroot for the second
recipe (aktualizr-hsm-prov, the one that depends on aktualizr) is also
left empty. It seems like something is not triggering or preventing the
repopulation of those directories when I change MACHINE back to the
original value (qemux86-64).