[poky] About the operator "??="

[Tian, Kevin]

>From: Tian, Kevin
>Sent: Wednesday, December 08, 2010 7:35 PM
>
>>From: Richard Purdie
>>Sent: Wednesday, December 08, 2010 6:30 PM
>>
>>Hi Dongxiao,
>>
>>On Wed, 2010-12-01 at 09:57 +0800, Xu, Dongxiao wrote:
>>> Recently when investigating the file parsing speed, I found the
>>> implementation of "??=" is another hot spot. This operator will not
>>> apply the default assignment until the end of the parse, which is
>>> different from "?=". According to current poky, the users for "??=" is
>>> mostly the scm revisions stored in poky-default-revisions.inc.
>>>
>>> Currently the final assignment logic for "??=" is added in finalize(),
>>> which costs about 20% parsing time.
>>
>>Can I take a look at the graph showing that? I'm a little surprised it
>>takes 20% of the time, that is a rather scary amount!
>>
>
>I don't have the graph, but did do a simple test by printing the lazy value
>list in finalize(). There're 166 items in total. Regarding finalize() may be
>invoked multiple times for same recipe and we have over 700 recipes scanned
>in a fresh run, it may contribute obvious overhead just like what Dongxiao
>previously optimized for distro tracking fields.
>
>I think we could remove the lazy key from the list after grabbing it as the
>default value, and is now testing it...
>

I have a simple patch:

diff --git a/bitbake/lib/bb/parse/ast.py b/bitbake/lib/bb/parse/ast.py
index 870ae65..4b98799 100644
--- a/bitbake/lib/bb/parse/ast.py
+++ b/bitbake/lib/bb/parse/ast.py
@@ -301,10 +301,13 @@ def handleInherit(statements, m):
     statements.append(InheritNode(m.group(1)))

 def finalize(fn, d, variant = None):
+    assigned = bb.data.getVar("__lazy_assigned", d) or ()
     for lazykey in bb.data.getVar("__lazy_assigned", d) or ():
         if bb.data.getVar(lazykey, d) is None:
             val = bb.data.getVarFlag(lazykey, "defaultval", d)
             bb.data.setVar(lazykey, val, d)
+            assigned.remove(lazykey)
+    bb.data.setVar("__lazy_assigned", assigned, d)

     bb.data.expandKeys(d)
     bb.data.update_data(d)

However it doesn't work as I expect. Every time the lazy list always contains 166
items. It looks that each finalize() is invoked in different database context, while
I haven't found the global copy yet. Or did I make some silly mistake here? :/

Thanks
Kevin