[yocto] Documenting YP Development Environment in more detail

[Trevor Woerner]

Coincidentally, Nicolas and I were recently involved in providing a
whirlwind tour of Yocto/OE to some developers last week.

On 20 June 2013 04:40, Rifenbark, Scott M <scott.m.rifenbark at intel.com> wrote:
> We can get started now on this by starting to define details for some obvious points or large areas of the flow.  What would be nice to get would be some graphical breakdowns of these areas of concern:
>
> * User-defined layers
> * YP provided layers
> * User configuration
> * Machine Configuration
> * Policy Configuration
> * Patches
> * YP provided recipes
> * User provided source code
> * YP provided source code
> * SCMs
> * Generated images
> * Generated SDKs/toolchains
> * Package Output
> * Source fetching process
> * Patch application process
> * Configuration process (fragments, etc.)
> * Key variable use and effects
> * User-initiated commands along the way


In addition to the above, excellent segments, I observed the following
based on my recent experience:

I think it would be fair to assume that most people reading this new
documentation didn't just install Linux for the first time a day or
two ago and are now wondering about Yocto. I think many people have
some sort of build experience. So I think it might be worth
considering having a "how do I ...?" section which attempts to cover
as many common questions/scenarios as possible. Perhaps it might be
interesting to directly compare Yocto/OE to other build systems (not,
of course, to try to get anyone's defenses up, but rather to be able
to answer questions such as "in build system X I do this to get that
result, how do I do the same thing in Yocto?"). Maybe these questions
and answers could be organized based on assuming a user is coming for
a specific other build system? Or perhaps have sections explaining why
certain procedures which are required in other build systems aren't
required in Yocto.

For example: some build systems require you to restart your entire
build process if there is a failure anywhere during this process. So
one of the questions we got was "is bitbake smart enough to restart
the build at the point of failure if there is an error, or does the
build have to be started from scratch every time?".

To some people it might seem bizarre that Yocto generates packages
(e.g. rpms or debs) when (by default) the core-image-minimal target
didn't have any package management tools. It took us a while to
clarify that Yocto is building these (e.g.) rpms by default as part of
its own internal process. The fact package management tools can be
installed with a very simple configuration file change and then these
rpms or debs can be used to update a running image is just an extra
bonus; but you're getting these (e.g.) rpms whether you want them or
not :-) Stranger still: choosing rpms instead of debs doesn't mean
your final system is going to look a lot more like Fedora than Debian.

Tied closely to the above, lots of clarity was required to explain
that Yocto builds each recipe individually, installs it (somewhere),
uses the files in "somewhere" to generate a set of packages, and then
uses the selected packages (i.e. the -doc, -dev, -dbg, ...) to create
the target's root filesystem image. I think the multiple "install"
stages was not intuitive. I think most people by default would simply
install the build output from each component directly into a sysroot
without the indirection of creating a set of packages which are then
selected from to make up the final image. Obviously what Yocto is
doing is brilliant and correct, it's just not obvious as to why it
would do things that way (I think).

As I mentioned in my other comments, sometimes the best way to explain
why Yocto is doing things the way it is can be best understood once a
person has actually tried to do these things by hand. Polluting the
sysroot during the build would be quite obviously wrong if/when a new
version of a component were required, or you wanted to remove a given
thing from the output image.