[yocto] Build time data

[Richard Purdie]

On Thu, 2012-04-12 at 07:34 -0700, Darren Hart wrote:
> 
> On 04/12/2012 07:08 AM, Björn Stenberg wrote:
> > Darren Hart wrote:
> >> /dev/md0        /build          ext4 
> >> noauto,noatime,nodiratime,commit=6000
> > 
> > A minor detail: 'nodiratime' is a subset of 'noatime', so there is no
> > need to specify both.
> 
> Excellent, thanks for the tip.

Note the key here is that for a system with large amounts of memory, you
can effectively keep the build in memory due to the long commit time.

All the tests I've done show we are not IO bound anyway.


> > Yet for all the combined horsepower, I am unable to match your time
> > of 30 minutes for core-image-minimal. I clock in at around 37 minutes
> > for a qemux86-64 build with ipk output:
> > 
> > ------ NOTE: Tasks Summary: Attempted 1363 tasks of which 290 didn't
> > need to be rerun and all succeeded.
> > 
> > real    36m32.118s user    214m39.697s sys     108m49.152s ------
> > 
> > These numbers also show that my build is running less than 9x
> > realtime, indicating that 80% of my cores sit idle most of the time.
> 
> Yup, that sounds about right. The build has a linear component to it,
> and anything above about 12 just doesn't help. In fact the added
> scheduling overhead seems to hurt.
>
> > This confirms what "ps xf" says during the builds: Only rarely is
> > bitbake running more than a handful tasks at once, even with
> > BB_NUMBER_THREADS at 64. And many of these tasks are in turn running
> > sequential loops on a single core.
> > 
> > I'm hoping to find time soon to look deeper into this issue and
> > suggest remedies. It my distinct feeling that we should be able to
> > build significantly faster on powerful machines.
> > 
> 
> Reducing the dependency chains that result in the linear component of
> the build (forcing serialized execution) is one place we've focused, and
> could probably still use some attention. CC'ing RP as he's done a lot there.

The minimal build is about our worst case single threaded build as it is
highly dependency ordered. We've already done a lot of work looking at
the "single thread" of core dependencies and this is for example why we
have gettext-minimal-native which unlocked some of the core path
dependencies. When you look at what we build, there is a reason for most
of it unfortunately. There are emails from me about what I looked and
found on the mailing list, I tried to keep a record of it somewhere at
least. You can get some wins with things like ASSUME_PROVIDED +=
"git-native".

For something like a sato build you should see more parallelism. 

I do also have some small gains in some pending patches:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=2023801e25d81e8cffb643eac259c18b9fecda0b
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=ecf5f5de8368fdcf90c3d38eafc689d6d265514b
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=2190a51ffac71c9d19305601f8a3a46e467b745a

which look at speeding up do_package, do_package_write_rpm and do_rootfs
(with rpm). There were developed too late for 1.2 and are in some cases
only partially complete but they show some ways we can squeeze some
extra performance out the system.

There are undoubtedly ways we can improve performance but I think we've
done the low hanging fruit and we need some fresh ideas.

Cheers,

Richard