[yocto] [Openembedded-architecture] Proposal: dealing with language-specific build tools/dependency management tools

[Derek Straka]

On Fri, Mar 10, 2017 at 10:10 AM, Alexander Kanavin <
alexander.kanavin at linux.intel.com> wrote:

> On 03/10/2017 04:58 PM, Otavio Salvador wrote:
>
>> I'd like to avoid generating entire separate recipes though, because that
>>> implies your custom-written tool would be figuring out where the
>>> dependency
>>> source came from in the first place, and what are its own dependencies,
>>> when
>>> creating the recipe, which can be tricky, breakage-prone guesswork.
>>>
>>
>> In fact not; as you generate the recipes for the dependencies, it goes
>> recursively and is always good.
>>
>
> Would it also be true for npm, Rust, Go, and other languages that will
> come along? In your specific case the metadata may be easily available to
> parse and convert to recipe form, but this many not hold in other
> situations.
>
> npm fetcher for instance was a nightmare to write, from what I've heard:
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/
> lib/bb/fetch2/npm.py
>
> I want to use existing tools (like 'npm install') for getting the stuff
>>> from
>>> the network - we don't really need full recipes, we just want to know the
>>> licenses of the dependencies, and, if possible, lock them down to a
>>> specific
>>> version.
>>>
>>
>> Well we initially thought this would suffice but consider a security
>> flaw. As many apps may be using different versions of same package it
>> becomes a nightmare to figure which ones are affected. If using
>> dependencies it is fine, for free.
>>
>
> The lockdown files would list the versions of the dependencies (if it is
> possible, which is not always true), so you can inspect those to see if
> something is vulnerable. In node.js or Go worlds the libraries are not
> reused between apps anyway, so it really doesn't matter if they're packaged
> as separate recipes or not (I didn't have time to check Rust, but as it's
> also using lockdown files, I believe the libraries are not reused either).
>
> I can chime in on how we do things in meta-rust.  Right now, each
application is statically linked against the crate library versions it
calls out.  At this point, the rust ABI is not stable between versions of
the compiler, so we made the conscious decision to avoid dynamic libraries
for the time being.  We acknowledge this does increase the file system
size, but we didn't want to have to deal with users trying to perform
package updates on individual shared objects.  We have our own cargo module
that we maintain that helps users generate their bitbake recipe for a given
package [1].  There was a small bit of work done on trying to create so
files, but it became an unmanageable version nightmare isn't supported
moving forward[2].  We also maintain our own custom fetcher for crates [3]
and ran into some issues getting it totally supported without integrating
it into the set of bitbake fetchers [4][5].

-Derek

[1] - https://github.com/meta-rust/meta-rust
[2] - https://github.com/cardoe/cargo-bitbake
[3] - https://github.com/meta-rust/meta-rust/tree/master/recipes-core
[4] - https://github.com/meta-rust/meta-rust/blob/master/lib/crate.py
[5] - https://github.com/meta-rust/meta-rust/issues/136
[6] - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10867



> Alex
>
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20170310/b7edd1b5/attachment.html>