[yocto] Update on build time performance - not good

[Richard Purdie]

Just for interest, for the first time ever I did a dump out of profile
information on the python side of bitbake's task execution for all tasks
of a core-image-sato build.

I enabled profiling of each forked off task with this patch:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/temp5&id=c221b4174627ac24f4c9cae687b4f015a5b6523a

and then combined the results together in the attached profile report.
Most of it isn't a big surprise but there are some interesting numbers
such as the 278 million calls to _keys() in data_smart. For that
specific issue I have a patch which I'm considering:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/temp5&id=f549077bbd529dfa5ff6ef75ef9432a823b3a3f5

as experience shows that function calls have overhead in python at that
number of iterations and reworking the code likely will reduce overhead.
I've yet to profile the result on this scale though.

Overall, a build makes over 10 billion python function calls. We access
data store variable flags 150 million times.

Not shown here but interestingly, early in the build process, the number
of calls to time.sleep() was low, at the end of this build its rather
significantly higher showing bitbake was hitting its idle handler a lot
more. Time spend there is an illusion as the system should have yielded
to other processes.

There are probably some other interesting things to be noticed somewhere
in that data :)

Cheers,

Richard



-------------- next part --------------
A non-text attachment was scrubbed...
Name: profilelog.bz2
Type: application/x-bzip
Size: 59237 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20110610/0f29939e/attachment.bin>