[poky] About the operator "??="

[Xu, Dongxiao]

Tian, Kevin wrote:
>> From: Tian, Kevin
>> Sent: Wednesday, December 08, 2010 7:35 PM
>> 
>>> From: Richard Purdie
>>> Sent: Wednesday, December 08, 2010 6:30 PM
>>> 
>>> Hi Dongxiao,
>>> 
>>> On Wed, 2010-12-01 at 09:57 +0800, Xu, Dongxiao wrote:
>>>> Recently when investigating the file parsing speed, I found the
>>>> implementation of "??=" is another hot spot. This operator will not
>>>> apply the default assignment until the end of the parse, which is
>>>> different from "?=". According to current poky, the users for "??="
>>>> is mostly the scm revisions stored in poky-default-revisions.inc.
>>>> 
>>>> Currently the final assignment logic for "??=" is added in
>>>> finalize(), which costs about 20% parsing time.
>>> 
>>> Can I take a look at the graph showing that? I'm a little surprised
>>> it takes 20% of the time, that is a rather scary amount!
>>> 
>> 
>> I don't have the graph, but did do a simple test by printing the lazy
>> value list in finalize(). There're 166 items in total. Regarding
>> finalize() may be invoked multiple times for same recipe and we have
>> over 700 recipes scanned in a fresh run, it may contribute obvious
>> overhead just like what Dongxiao previously optimized for distro
>> tracking fields. 
>> 
>> I think we could remove the lazy key from the list after grabbing it
>> as 
>> the default value, and is now testing it...
>> 
> 
> I have a simple patch:
> 
> diff --git a/bitbake/lib/bb/parse/ast.py
> b/bitbake/lib/bb/parse/ast.py index 870ae65..4b98799 100644 ---
> a/bitbake/lib/bb/parse/ast.py +++ b/bitbake/lib/bb/parse/ast.py
> @@ -301,10 +301,13 @@ def handleInherit(statements, m):
>      statements.append(InheritNode(m.group(1)))
> 
>  def finalize(fn, d, variant = None):
> +    assigned = bb.data.getVar("__lazy_assigned", d) or ()
>      for lazykey in bb.data.getVar("__lazy_assigned", d) or ():
>          if bb.data.getVar(lazykey, d) is None:
>              val = bb.data.getVarFlag(lazykey, "defaultval", d)
>              bb.data.setVar(lazykey, val, d)
> +            assigned.remove(lazykey)
> +    bb.data.setVar("__lazy_assigned", assigned, d)
> 
>      bb.data.expandKeys(d)
>      bb.data.update_data(d)
> 
> However it doesn't work as I expect. Every time the lazy list always
> contains 166 items. It looks that each finalize() is invoked in
> different database context, while I haven't found the global copy
> yet. Or did I make some silly mistake here? :/   

Per my understanding, "d" is an object instance, each time when doing finalize, "d" is a different instance.
Therefore each round "d" will always have 166 items.

Thanks,
Dongxiao

> 
> Thanks
> Kevin