[yocto] Yocto usability questions

Wed Nov 16 20:03:53 PST 2011

I apologize in advance for the top-post.

Like Mark, I'm a  20+ year veteran of "ordinary" embedded development, and so cross compilation is old hat for me as well. The usual commercial cross compilers, linkers, RTOSes, debuggers, and other libraries are usually fairly well integrated. Those types of environments tolerate their host environments fairly well, because it's pretty easy to keep everything separated.

I am much newer to Linux and embedded Linux development. I haven't used the SDK for Yocto yet, but I have used the Timesys SDK a little. For writing new code/apps, it's fairly straightforward - set up some environment variables, write a makefile and use cross_make, cross_configure, etc. I set up the embedded target to PXE boot, which keeps all executables and RFS on the host workstation. So for development one merely needs to drop the compiled app and any required configuration onto the RFS on the workstation. For writing kernel-level stuff, such as kernel mode drivers, one pretty much has to use the TimeSys factory (an extension to the kernel make environment from what I can tell).  

If one wants to use the TimeSys factory so that proprietary apps are built and loaded/installed into the RFS automagically, or build packages which are not on the TimeSys server, then one writes what is essentially the equivalent of the bitbake recipe, only in makefile format. It has all the same sorts of process steps: fetch, unpack, patch, auto/configure, compile, install.

Of course PXE boot can be set up for Yocto projects - I've already done it for the Kontron Atom board we're going to use in our next product. I hacked out the bzImage and root file system from the cpio image and plopped them into the tftpboot and exported rfs directory, and bingo - PXE booting!

Frankly, as a medical device software developer, I find the "recipe/makefile" paradigm much more familiar and preferable to the "app" developer paradigm. We have to be able to recreate bit-accurate images for our devices. A relatively consistent paradigm for accomplishing this is much better than cobbling together a bunch of ad hoc scripts.

At any rate, it seems to me that writing recipes is writing recipes, just like once you've used 3 or 4 editors or word processors, then you pretty much know what the score is when you go to pick up the 5th. Bitbake recipes aren't all that different from makefiles, except for the syntax,and learning the various meanings of different "build variables" -- oh, and using bitbake/python as glueinstead of jam, or make. I think that this has been the biggest source of frustration for me wrt to Yocto: Trying to map what I already know onto a new paradigm. I know - after 20 years, the mud gets a little thick and hard. ;)

But I understand the need to make things easier for the less experienced developer. And I get that cross compiling an embedded Linux OS is more intertwined with the host workstation OS than other RTOSes, even when they are compiled on a *nix workstation. One way that I'm addressing this for my development team (which has several never-done-Linux-no-way-no-how types) is finding links to relevant tutorials and sprinkling them in places they're likely to need the info. Although there is a ton of documentation out there, it's not always in easy to find places.

As a relative newcomer to Yocto and OE and bitbake, I can't agree with Mark more regarding examples and documentation. I'd suggest that you make a part of your process finding an inexperienced someone to run through your documentation and try to implement the various different features, or demonstrate knowledge of key concepts. The questions such a person might ask are likely pointers to holes in your documentation or process. This could be a good Google Summer of Code type exercise. Not as sexy as some, but definitely a big payoff for Yocto.

Mark mentioned the need for adding packages which are not already in the recipe base, but perhaps already exist in a desktop form. Making it easy to contribute seems like a common FOSS practice. For the difficulty aspect, someone mentioned in the Qt/OpenSSL thread recently that RPMs have a lot of the dependency information built into them, and I suspect also configuration files for install, or recipes for creating them. I also read in one of the Yocto blogs that someone on the Kernel side created a script to keep Kconfig fragments synched across BSPs. Might it be possible to write a script to mine a desktop RPM (or any other usable package format), and their associated makefile(s) to create as much of the bones of a bitbake recipe as can be gleaned from the desktop package? It might be as complex a task as the automake process itself. :)

Hope there was some useful information in there somewhere. :)

Paul Ourada
Sr Principal Software Engineer
Covidien, Surgical Solutions, Energy-based Devices

________________________________________
From: yocto-bounces at yoctoproject.org [yocto-bounces at yoctoproject.org] on behalf of Mark Hatle [mark.hatle at windriver.com]
Sent: Wednesday, November 16, 2011 4:39 PM
To: yocto at yoctoproject.org
Subject: Re: [yocto] Yocto usability questions

On 11/16/11 4:07 PM, Jeff Osier-Mixon wrote:
> Mark Hatle said:
>
>     Yocto is a cross-compiled build environment. This is a departure to a lot of
>     the Moblin/MeeGo work that has occurred in the past. The advantages are you
>     can use any commodity PC to target any (supported) architecture.
>     Disadvantages are that when you introduce new code, you need to ensure that
>     it has a recipe (build instructions for bitbake) and can cross compile. If
>     everyone has to do the same work over and over, this can be time consuming
>     and counter productive. If people work together, the time and support burden
>     are dramatically reduced. This can help negate issues people have had in the
>     past with cross compiling. Note: Yocto -does- have a self hosted compile
>     environment if it is needed, this is usually when cross compiling isn't easy
>     to do for some reason.
>
>
> Mark & everyone else listening:
>
> Would you say that (1) the need for a recipe and (2) the requirement to
> cross-compile are two of the most major usability or learning-curve
> disadvantages of working with the Yocto Project (and oe-core)? What would be a
> third disadvantage from a usability standpoint?

1) Recipe isn't needed, unless you want automatically reproduced builds and to
share the instructions with others... which is one our our goals.

I don't see the recipe as anything different then an SRPM, Debian src.tgz, etc.
  The only obstacle is that it's "different" that what desktop distributions do.

2) I think cross compilation is by far the largest obstacle.  People not
familiar with the GNU auto tools, cross compiling in general, or simply
inexperienced developers seem to have a lot of problems with this.  I think
OE/Yocto does a good job at providing GNU auto tools and make helpers, but it's
far from perfect.  As far as how to improve it...  we need to keep incrementally
improving our support, documentations and examples.  We also need to foster a
community where people share the work they've already done... thus eliminating
this issue.  (The meta-oe layer is a good place for this already.)

I'm sure there are other usability issues, but I've been doing cross compilation
for so long that I'm a bit blind to some of the issues.

To me the biggest thing we need to do is make sure someone who is familiar with
desktop Linux can step in and apply what they know to building a recipe and
fixing cross compilation problems.  If we can do that -- it will go a long way
toward helping resolve the issues that cause people to do self-hosted
compilation on slow target systems.  (Note there are some things that are simply
complicated and difficult to do like Firefox.. for those the only answer is to
have "experts" do the work and make it available to the novices so they can see
and understand how to work around various issues.)

> Another way to put it: if you could change three things about the Yocto Project
> to make it more approachable for someone who has never used it before, what
> would they be?

To re-iterate:

*) We need a resource for contributed packages (meta-oe?) that will eliminate
most of the problem.

*) We need good examples of problems and solutions to cross compilation difficulties

*) We need to continue to identify "common" issues and work to resolve them in
the tooling we already support

--Mark

> --
> Jeff Osier-Mixon http://jefro.net/blog
> Yocto Project Community Manager @Intel http://yoctoproject.org
> <http://yoctoproject.org/>
>
>
>
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

_______________________________________________
yocto mailing list
yocto at yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto