[yocto] Shorter build time?

[Daniel.]

Hi Takashi,

I read that python can't execute multiple threads simultaneously, but
that wouldn't be a problem if each python task is executed at its own
interpreter instance (process). This last statement is what I don't
really know, some experts on Yocto's internals may clarify that, maybe
:)

Regards,

2016-06-30 22:29 GMT-03:00 Takashi Matsuzawa <tmatsuzawa at uievolution.com>:
>
> Hello Yocto.
>
> Well, this may not be a very new topic, but I wonder if you have recent suggetions.
> So far I have tried the followings and see some improvement, making the build time to days to hours.
>
> 1) All of the directories, DL_DIR, SSTATE_CACHE, TMPDIR, DEPLOY_DIR are on SSD.
> 2) .repo and working directory (where I synch my recipes) also on SSD.
> 3) Set BB_NUMBER_PARSE_THREADS/PARALLEL_MAKE/BB_NUMBER_THREADS to something like 20, 30, etc.
> 4) Enable use of ICECC.
> 5) Use pigz, pbzip2, pxz instead of gzip, bzip2 and xz.
>
> I can see with 3-5), I am benefited from my multicore PCs and see some scaling effects (if I can add more PCs or CPU core.)
> However I think there is i the build not paralleled or distributed easily.
>
> i) python tasks
> They say python has slimitation of being single thread, and could be a bottle neck since most of yocto build tasks are in python, if not native/cross toolchain jobs.
>
> ii) zip / tar tasks
> e.g. seeing rootfs taking forever, jobs not distributed to the other machines.
> I understand it is writing files to one target tree and doing a bit tar/zip for the files, being one big job though.
>
> I wonder if I can do anything with i) and ii) at least to make them processed in parallel.
> ICECC only can dispatch gcc jobs I believe.
> Any previous attemps to do the similar to non-gcc jobs?
> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto



-- 
"Do or do not. There is no try"
  Yoda Master