[yocto] Building applications and distributions separately, but integrate into final image

[Michael Habibi]

All,

I am looking at Yocto as a replacement for our embedded distribution.
Currently we build everything as a distribution. This includes
building the Linux kernel, open source packages, as well as our own IP
and applications.

As you can imagine, a vast majority of our build is taken up by
repeatedly building distribution components and not our actual IP,
which is a majority of what changes day to day.

In moving to Yocto, my first plan was to create all the necessary
layers and separate out what changes from what doesn't (in fact much
of what doesn't change is already included in Yocto or the many
freely-available layers online).

However, this would still require application developers to have to
learn and use Yocto to build the entire distribution. Ideally, I'd
like to separate the applications from the main build, so that
application developers have to learn less about Yocto, and more
importantly, save on build time when they make their changes.

I am exploring exactly how I can use the available SDK options as a
solution. If I create an image with the base distribution and required
applications, I'm wondering if I can create a separate tree for the
application developers. They would then build that entire tree with
the yocto SDK. I haven't quite figured out yet how I can then
integrate their applications into the Yocto image, and prepare it for
deployment. I'm also unclear how we can define new dependencies if the
base distribution is pre-built. For example, if we introduce a new
dependency (our custom application now requires zlib, for example), it
doesn't seem for there to be a way to then ensure zlib is built into
the final image with the SDK option.

Unfortunately, we can't build the distribution and then install the
applications directly onto the target. Based on how our codebase is
currently designed, it requires a fully-complete image to be installed
on the target as a whole, and not piecemeal. This is why I need to
integrate on the host/build environment first before we create the
deployable image.

Can anyone share any workflows that they use that may fit with what we
need? In the mean time I am continuing to play with Yocto and the SDK
to see what else I can come up with.

Thanks all for your time.