[yocto] RFC - integrating target runtime information into a yocto build

[Tim Bird]

Hi everyone,

I'm doing research on whole-system optimization, and I'd like to ask opinions
on the next stage of my work.

== background ==
After a fair amount of blood, sweat and tears, I have (finally) produced a working
ARM kernel that is optimized with gcc's link-time-optimization (LTO) feature.
Previously, I had completed a system that did syscall detection in object files,
and unused syscall elimination in the Linux kernel.

In the next stage of my research, I'm planning on combining those two
to do kernel optimization and rebuilding, AFTER the distribution image build.

There are a number of other techniques I'll be looking at as well, including
1) profiling applications and re-linking with modified linker scripts to increase
the locality of init-time functions (and reduce flash access during boot)
2) profiling block-layer accesses to convert portions of applications to XIP
using the AXFS filesystem
3) additional automated kernel feature elimination from whole-image static
and runtime analysis.

Finally (by way of background), I already have tools to automate the process
of installing software on the target, rebooting, running programs and
collecting results.

== questions ==
First, has anyone already done anything with OE or YP that takes data
from a running system, and feeds it back into the build system?

Does it make sense to add recipes to YP that wrap my tools, collect
the data and re-start the build process?  Is this even possible?
That is, can bitbake wait for results from a late build stage,
invalidate previous results, and cause rebuilds of key components?
Or is this stretching things too far?

Has anyone built recipes in OE or YP for image deployment,
target control, or test execution?

The alternative is to put this iteration outside of the build process.
That is, to do an initial build, collect results, modify the build inputs,
and do a subsequent build (directed from "above" bitbake, rather than
from recipes inside the build system).

Does anyone have any experience with this?  Any ideas on the best
approach?

I'm not expert enough in YP to know the dangers here.
I would, however, very much like to make this work reproducible
and automated, which is why I have a high interest in doing
this in YP.

Thanks,
 -- Tim


=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================