[yocto] Suitable machine for yocto

[Alex Lennon]

On 11/09/2017 00:56, Mark Hatle wrote:
> On 9/10/17 2:31 PM, Alex Lennon wrote:
>>
>> On 10/09/2017 19:17, Mark Hatle wrote:
>>> On 9/10/17 11:14 AM, Alex Lennon wrote:
>>>> On 10/09/2017 17:06, Mark Hatle wrote:
>>>>> On 9/10/17 2:00 AM, Usman Haider wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Can someone please recommend some good machine for yocto environment and
>>>>>> building sdks. I am interested in RAM, hard disk space, processor.
>>>>> You want fast I/O, as much RAM and as many (fast) cores as you can afford.  I
>>>>> don't think there is a single answer as what is 'best'.  It also depends on
>>>>> which Yocto Project versions, and which layers you are using as to which
>>>>> combination is best.
>>>>>
>>>>> I run builds on my laptop, 4-core/8-thread & SSD and 16 GB of ram from a few
>>>>> years ago.  It's fast, but I wouldn't want to do all of my development on it.
>>>>>
>>>>> I've had 8-core/16-thread (32GB ram/standard disk), 16-core/32-thread (72GB
>>>>> ram/SAS-3 RAID), 24-core/48-thread (64GB ram/SATA - software RAID), 72-core/144
>>>>> thread (256 GB ram/hardware raid/SAS-3), and recently upgraded to
>>>>> 96-core/192-thread (256 GB ram/hardware raid/SAS-3).
>>>>>
>>>>> I would not go below quad-core (8-thread) myself.  You can get a quad core, good
>>>>> quality machine for $1000 or less these day.  If you move up to the larger
>>>>> machines, you can even be able to get to a 24-core for less then $5000.  By the
>>>>> time you get to 96-core and all of the googles you are likely talking $50000 or
>>>>> more.
>>>>>
>>>>> By clock raid, the 24-core machine is the fastest..  While the 96-core monster
>>>>> can do the builds the quickest.  But when you figure out cost/performance/etc..
>>>>> the 24-core is probably the best performance per dollar, and with adequate RAM
>>>>> (I'd say at least 64GB if not 128GB), and fast I/O you'll probably get the
>>>>> lowest price for the best performance in that category.
>>>>>
>>>>> If you need sheer speed and price is no option, then the (4 CPU w/ 24 core each)
>>>>> 96-core monster (or even better) is what you want to go with.  256GB ram would
>>>>> be a minimum with that configuration (I'm not sure if more is actually helpful,
>>>>> I rarely end up in swap -- but I go get into situations where more then 50% of
>>>>> ram is used.)  With that many cores, disk I/O starts to become obvious.  So
>>>>> faster the better... SSDs would be the fastest, but of course the most expensive.
>>>>>
>>>>> If your employer is paying for the machine, you may be able to get a better then
>>>>> normal machine by explaining how much time a faster machine will save and how
>>>>> comparing to your salary a machine is inexpensive.  (If you are a contractor or
>>>>> student, that changes of course.)  :)
>>>>>
>>>>> So my point is really, figure out how much money you have to spend.  My rule of
>>>>> thumb is roughly:
>>>>>
>>>>> 1) Buy as many cores as you can.  Try to get a CPU that has Hyperthreading or
>>>>> equivalent to double the effective core count.  Fastest processing speed helps
>>>>> in repetitive cases vs full system builds.
>>>>>
>>>>> So if the choice is a 24 core @ 2.2GHz vs 22 core @ 2.5, I'd probably go with
>>>>> the 22-core.  While if it was 24 core @ 2.2GHz vs 8 core @ 4.2 GHz, I'd go with
>>>>> the 24 core.
>>>>>
>>>>> 2) Try to get at least 1 GB of ram per thread (2 GB per core..)  You can cut
>>>>> back on the ram (if necessary) once you hit 72 threads or so.   (72 threads
>>>>> right now seems to cover most of the parallelization in a full system build.
>>>>> There are points in the system where it can parallelize MUCH more, but they are
>>>>> fairly rare.)
>>>>>
>>>>> 3) You need fast disks.  Software RAID works fine, but you likely need to buy at
>>>>> least a couple of disk to boost performance.  SSDs are fast, but lots of builds
>>>>> take space, so fast SATA or even better SAS drives are the best performance per
>>>>> cost.
>>>> This brings to mind a related question I keep coming back to as to the
>>>> economics of a docker (or similar) image running a fast Yocto build in
>>>> the cloud.
>>>>
>>>> i.e. set config params -> bring up server image on plaform A/B/C ->
>>>> perform build taking time X/Y/Z -> store output images -> bring down
>>>> server  == $ ?
>>>>
>>>> I find myself asking what the optimal cost per-build would be using this
>>>> approach...
>>> I helped someone do some very -preliminary- figured a few years ago.  The
>>> processing was 'cheap', but between storage and network transfer costs.. it was
>>> cheaper to buy a reasonable machine.. payback time was only a few months.
>>>
>>> (cloud 'storage' is often very slow as well, because there are expectations of
>>> migration and things like that.)
>>>
>>> So as of a few years ago at least, the economics didn't factory the cloud -- yet.
>>>
>>>
>> Yeah that is more or less my thinking. Although I do imagine that if a
>> source mirror on an internal network leg could be set up then that might
>> significantly reduce chargeable network b/w...
> premirror for downloads and sstate-cache shared between multiple customers could
> reduce the network b/w far enough to make it economical.   But you would need to
> trust other vendors since the sstate-cache can be manipulated to provide
> malicious software (in some cases).  (And strengthening the sstate-cache
> checking can cause enough extra overhead to make it 'cheaper' to build then use
> sstate-cache in many cases.)
>
>

Interesting... Thanks for your thoughts.

Cheers, Alex