[yocto] [RFC] possible solutions to bug 2421

[Eliaser, MaX]

At Jeff Osier-Mixon's suggestion, I decided to try to pin down exactly how much space was being used by this unnecessary documentation. I wrote I shell-script to total these numbers up, you can find it at [1]. This script is not exhaustive but it should catch most of it.

After a build of core-image-minimal, I found 126 MiB of documentation specificially for -native packages. [2] After a build of core-image-sato, I found 129 MiB, just 3 more. [3] Seems like the bulk of -native packages with documentation are already being used by the minimal build.

As far as build time is concerned, the easiest way to find out is to actually fix bug 2421 and see how much faster it is, but I'd be surprised if it was more than thirty seconds in either case. Maybe a minute. That's just a wild guess though.

-Max

[1] pastebin.com/raw.php?i=FUW73z3W
[2] pastebin.com/raw.php?i=muTPA4u0
[3] pastebin.com/raw.php?i=DcWg7FpH


-----Original Message-----
From: yocto-bounces at yoctoproject.org [mailto:yocto-bounces at yoctoproject.org] On Behalf Of Eliaser, MaX
Sent: Friday, April 11, 2014 10:21 AM
To: yocto at yoctoproject.org
Subject: [yocto] [RFC] possible solutions to bug 2421

Hi list,

I've been working on bug 2421. [1] The bug report notes that several -native
and -cross packages include man pages, info pages, and other documentation.
This is unnecessary, and build times and disk usage could be improved by
eliminating these files.

However, with the exception of Perl's Module::Build, [2] the build systems we
support have no built-in way to do this, and most (all?) upstream projects
don't implement it manually in their build system configurations. Unless I'm
missing something easy and obvious, which I don't discount at all.

There are actually two different unnecessary things going on. First is the
generation of the man/info pages, HTML/PDF files, etc. from source markup.
This is done using programs like pod2man, asciidoc, groff, nroff, etc. Second
is the copying of the files to their destination, usually in /usr/share
somewhere. There are a few different ways I can think of to attack one or both
of these problems:
 *  Add a --disable-docs feature to each applicable upstream project's build
    setup. Keep in mind that this is at *least* 58 separate autotools
    configurations to add this feature to (specifically just the ones used
    by -native and -cross recipes.) Plus a bunch of CMake ones, plus probably
    a few others. But it would allow you to prevent documentation files from
    being generated *and* copied.
 *  Come up with some patches for each project that will simply nuke the doc
    building altogether instead of making it optional. These patches obviously
    could not be upstreamed, but they'd be easier to write.
 *  Add code to the recipes to re-clear /usr/share/man, /usr/share/info, etc.
    after each install has finished. That saves disk space but actually *adds*
    disk activity and build time.
I'd say that those three options are all legitimate, solid solutions. There 
are also some ways to cheat:
 *  Attack the documentation formatting utilities used by the various build
    systems. There are a finite number of programs that get used for this--
    pod2man, asciidoc, groff, etc. Why not replace these with a simple program
    that does nothing, or just emits a blank file? We could conditionally set
    the PATH environment variable to use or not use these dummy programs. The
    file copy still happens, but at least it's a blank file.
 *  Attack the "install" program, wrapping it in a shell script that checks
    if the install target directory is in some blacklist (i.e. /usr/share/man, 
    /usr/share/info) and if not, invoking the original "install" binary. That
    saves a file copy.
 *  Mark the bug as "won't fix."

There are probably a few other solutions I haven't thought of yet.

Regards,
-Max Eliaser.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2421
[2] http://search.cpan.org/~leont/Module-Build-0.4205/lib/Module/Build.pm

-- 
_______________________________________________
yocto mailing list
yocto at yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto