[yocto] Ordering of anonymous Python functions and task signature calculations

[Matt Hoosier]

(Cross-posting to bitbake-devel.)

Quite a while ago I wrote the following message:

On Tue, Jan 24, 2017 at 11:55 AM Matt Hoosier <matt.hoosier at gmail.com>
wrote:

> In order to support a use-case that embeds information about the Git
> revision of Yocto itself that was used to make a build, I would like to run
> some arbitrary Python code and dump the results (Git SHA1's, tag names,
> etc) into a Bitbake variable.
>
> The idea is that the variable would get consumed in a do_install() task to
> populate some cookie-crumb file in the package payload.
>
> Something like:
>
>   DEPENDS = "git-native"
>
>   python () {
>     # This pseudocode isn't strictly functional for invoking Git, but you
> get the idea
>     d.setVar('GIT_INFO', subprocess.Popen(['git', 'rev-list',
> ...]).communicate().stdout)
>   }
>
>   do_install () {
>     install -d ${D}/etc
>     echo "${GIT_INFO}" > ${D}/etc/git-info.txt
>   }
>
> This all works on a fresh run, but Bitbake appears not to be evaluating
> the anonymous Python function on each execution. This leads it to have
> out-of-date information about the Git working copy when global state
> changes but happens not to impact the particular recipe holding this logic.
>
>
The replies given at the time were very helpful in letting me accomplish my
immediate goals, but I never really did get to the bottom of the real
question:


> Is there any trick available to cause the Python function to execute (and
> hence update the value of ${GIT_INFO} each Bitbake execution so that the
> up-to-the-moment contents of it filter into the calculation of the
> signature for do_install()? Or am I just trying a wrong-headed approach and
> doing something illegitimate?
>

Some build systems I know of [1] that are based on hashes rather than
timestamps have a way that you can inform the task execution engine to
always poison the inputs to a certain task's up-to-dateness check. Is there
anything in Bitbake that would allow a declaration to force such-and-such
variable in a recipe always to be re-evaluated:

  EXTERNALLY_INFLUENCED_VARIABLE = "${@int(random.random() * 10)}"
  do_install[vardepsalwaysdirty] = " EXTERNALLY_INFLUENCED_VARIABLE "

  do_install() {
    echo '${ EXTERNALLY_INFLUENCED_VARIABLE }' > ${D}/etc/stuff.txt
  }

Don't read too much into the example calling random() here. That's not
really what I want to do; the point is just that the variable's value
(though deterministic) isn't computed strictly from Bitbake metadata. So
I'm searching for a way to get the value of EXTERNALLY_INFLUENCED_VARIABLE
to be re-computed each time Bitbake starts. If the computed value ends up
matching what was seen on the last execution, then any dependent tasks
wouldn't need re-run.

Any ideas?

[1] I'm thinking particularly of Waf's hash-based task graph, which
supports annotations that can be used to always cause a task to be re-run
even if all the inputs feeding into it appear up-to-date.

Thanks,
Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20180725/df16366e/attachment.html>