[yocto] [Openembedded-architecture] Proposal: dealing with language-specific build tools/dependency management tools

[Josef Holzmayr]

Hi Alexander,

thanks for kicking off the topic, sounds like its kinda overdue. While I 
have no really good solution (d'oh!), please find below thoughts and 
bits and pieces about some of the points.

*Recipes*

My gut feeling say auto-generation of the recipes is a good way to go. 
Yet I am uncertain which set of functionality such an auto-generated 
recipe would provide. The obvious possibilities:

Option 1: Only dependency/license tracking.
   Pro: provides exactly what it says - known and proven means to keep 
track of the dependencies and licenses.
   Contra: this would mean that we have a recipes that only do 
housekeeping, while the actual workload is taken care of in the main 
recipe. This seperation sounds like a massive headache

Option 2: Full recipes
   Pro: would fit into the OE workflow.
   Contra: requires the sub-package managers to at least "play along". 
Think two applications having the same dependency. It would get 
installed once (globally/system-wide), and both applications have to use 
it. My interpretation is that this just is not how it works (looking at npm)

Coming from there, one can go further, resulting in
Option 1.5: Provide recipes for common base functionality. Those would 
have the all-in-one-locked-down approach, but are meant to be used as 
global dependencies. Example: the MEAN stack. Like its name says, it 
consist of 4 main pieces (-> possible recipes) which are needed by the 
application.
   Pro: reduces recipe number/bloat and makes dependencies readable. The 
mindset fits the classis 'library' thinking.
   Contra: the depending application would have to be packaged with the 
infrastructure in mind. So while the library recipes could rely on the 
locked down sub-package manager, the application would have to intently 
skip it and provide a custom installation. Which is an annyonce if you 
are application dev and packager in union - and a major pain point if 
you want to package some upstream application.

*Lockdown*

To me the approach sounds interesting, yet it comes with a couple of 
points to think about.
- Feature set: having such a lockdown system implies/requires all 
sub-package managers to provide (at least) the functionality to fullfill 
the needs of the lockdown process/recreation. Is that something we can 
take for granted?
- Multilanguage: imagine a package for example having some native go 
code, then nodejs bindings and then a npdejs application on top. How 
would that look like? Multiple lockdown files? What are the implications?

*Sub-Package managers in general*

We've seen the first (perl, php, python...) and second (npm, go, 
rust...) wave of those by now. A third one certainly will come one day. 
Taking a step back for a larger perspective, it sounds like what we 
actually need is some form of nested dependencies. Or scoped 
dependencies. Whatever we want to call it, because to me thats what it 
actually is. The dependencies we have now are always global. But 
especially the second wave things think different. Those sub-package 
managers do not care about the whole system. They care about their small 
part of it. So my interpretation is that we need to take that paradigm 
shift, and decide upon the actual implementation details afterwards.

*Conclusion*

Guess I raised more questions than I offered answers for. Sorry :-(

Greetz (and try to enjoy the weekend)
-- 
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548