[yocto] parallel build of multiple machine/os?

[Mike Looijmans]

On 12/11/2014 08:42 PM, Mark Hatle wrote:
> On 12/11/14, 12:55 AM, Mike Looijmans wrote:
>> On 12/10/2014 08:00 PM, Khem Raj wrote:
>>>
>>>> On Dec 10, 2014, at 10:52 AM, Koehler, Yannick (HP Networking)
>>>> <yannick.koehler at hp.com> wrote:
>>>>
>>>> Can Yocto build for example in a single bitbake call an image for x86,
>>>> ppc, arm, others?  Or is the call to bitbake limited to a single
>>>> architecture?  And if so, how?
>>>>
>>>> I see that lots of *-native package are built, and I guess is that for
>>>> each machine/os those would be reused, as such, it appears it would be a
>>>> benefit to be able to build multiple package version for different
>>>> architecture/distro/machine in a single invocation of bitbake.
>>>
>>> parallel multi-machine builds have been discussed but not thought of worth
>>> doing. you could however share
>>> lot of common packages among machines sequentially.
>>>
>>> [...]
>>
>> A more convincing use-case for multi-machine building is the emerging of
>> "hybrids". The little/big architecture that puts different versions of ARM
>> CPUs into a single machine is one example, the OMAPs with the DSP and PRUs are
>> another. Boards with FPGAs can have several CPU architectures running on a
>> single board. One could imagine that one would want to build the software for
>> all these subsystems in a single call, and that one bitbake process would be
>> managing that the build system gets properly used.
>>
>> Currently, these hybrids have to invoke bitbake for each subsystem, probably
>> needlessly regenerating binaries, and once more for the one image to find them
>> all and in darkness bind them...
>>
>
>  From a userspace perspective, this is more or less solved with the multilib
> configurations.  Perhaps it would be possible to implement some kind of hybrid
> kernel configuration using the same type of multilib setup.  (Note, this would
> be new work, it's not something we have today.)

The OMAP PRU and DSP each run their own bootloader/kernels (the PRU is just 
bare metal code), and do not share code with the ARM that runs the rest. From 
the ARM's perspective, the DSP and PRU just need "firmware".

If I put a Microblaze into the FPGA of a Zynq, I have a system that can run 
multiple kernels, on the built-in ARM CPUs and on the Microblaze(s). The ARM 
is then responsible for bootstrapping the microblaze, but it ends there, as 
the microblaze can run its own kernel and rootfs and applications, and they 
don't need to understand each other's binaries.

One problem here is that the "firmware" for the microblaze would be a "image", 
and images and in particular the rootfs, for reasons yet unknown to me, always 
get rebuilt even if nothing changed and the output would be the same.

I might succeed in spawning another bitbake from within the one that builds 
the ARM system, but on a 8-core system this sub-bitbake would also try to run 
8 parallel processes, each potentially running 8 compiler threads. Some of 
these compilers eat RAM like crazy, so this is not going to be productive. It 
would be much better if the builds for the ARM and mircroblaze would share the 
load. Probably a very challenging thing to do currently.



Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Systems
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: (+31) (0) 499 33 69 79
Telefax:  (+31) (0) 499 33 69 70
E-mail: mike.looijmans at topic.nl
Website: www.topic.nl

Please consider the environment before printing this e-mail

Topic zoekt gedreven (embedded) software specialisten!
http://topic.nl/vacatures/topic-zoekt-software-engineers/