[yocto] Building applications and distributions separately, but integrate into final image

[Khem Raj]

> On Aug 24, 2016, at 11:54 AM, Michael Habibi <mikehabibi at gmail.com> wrote:
> 
> All,
> 
> I am looking at Yocto as a replacement for our embedded distribution.
> Currently we build everything as a distribution. This includes
> building the Linux kernel, open source packages, as well as our own IP
> and applications.
> 
> As you can imagine, a vast majority of our build is taken up by
> repeatedly building distribution components and not our actual IP,
> which is a majority of what changes day to day.
> 
> In moving to Yocto, my first plan was to create all the necessary
> layers and separate out what changes from what doesn't (in fact much
> of what doesn't change is already included in Yocto or the many
> freely-available layers online).
> 
> However, this would still require application developers to have to
> learn and use Yocto to build the entire distribution. Ideally, I'd
> like to separate the applications from the main build, so that
> application developers have to learn less about Yocto, and more
> importantly, save on build time when they make their changes.
> 
> I am exploring exactly how I can use the available SDK options as a
> solution. If I create an image with the base distribution and required
> applications, I'm wondering if I can create a separate tree for the
> application developers. They would then build that entire tree with
> the yocto SDK. I haven't quite figured out yet how I can then
> integrate their applications into the Yocto image, and prepare it for
> deployment. I'm also unclear how we can define new dependencies if the
> base distribution is pre-built. For example, if we introduce a new
> dependency (our custom application now requires zlib, for example), it
> doesn't seem for there to be a way to then ensure zlib is built into
> the final image with the SDK option.
> 
> Unfortunately, we can't build the distribution and then install the
> applications directly onto the target. Based on how our codebase is
> currently designed, it requires a fully-complete image to be installed
> on the target as a whole, and not piecemeal. This is why I need to
> integrate on the host/build environment first before we create the
> deployable image.
> 
> Can anyone share any workflows that they use that may fit with what we
> need? In the mean time I am continuing to play with Yocto and the SDK
> to see what else I can come up with.


You can separate out the application development workflow from system
builder workflow. You can generate SDKs from build system e.g.

bitbake -cpopulate_sdk <image>

which should generate an SDK which is suitable for developing the
applications which are part of your image. However, when you add new
packages which has dependencies not as part of your images then you
have to regenerate the SDK and provide it.

In some large app teams, I have seen that they deploy the SDK on a
shared NFS or something drive and then platform/SDK teams can provide
newer versions of SDK and let app developers switch their applications
to new SDK in a staggered approach.


You have two ways to integrate the applications

1. Let application teams generate binary artifacts and then you
write recipes, to package them into final image.

2. second option is to build these applications from source as part
of image builds in your CI or production.

if you follow the API discipline. You might not need to update the SDK
so often. What this means is that the applications need to have there
own component build system. Yocto can deal with a lot of them e.g.
cmake, autotools, qmake, or even simple make.

with this you do not have to immediately expose the app devs to OE
internals.

Key piece is how you unit-test or do CI in such an environment especially
for applications. I have solutions for those too but it really depends
upon what kind of tooling is in place and how devs are used to do
the work.

> 
> Thanks all for your time.
> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 204 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.yoctoproject.org/pipermail/yocto/attachments/20160824/c44aa1eb/attachment.pgp>