[yocto] pseudo interaction issue

[Peter Seebach]

On Mon, 26 Mar 2012 22:45:16 +0100
Richard Purdie <richard.purdie at linuxfoundation.org> wrote:

> I'm still a little surprised we don't make any system() calls though.
> I just tried putting a os.system("true") call into the "breakit"
> class and it doesn't trigger the warnings. Could that be down to the
> lack of a popen wrapper?

I think that part could be, but this doesn't explain why adding the
popen() wrapper doesn't fix it.  Oh, wait.  Yes, it does.  I think I am
now happy with this, although I have a loose end or two.

So, here's what I've figured out.

We start bitbake with PSEUDO_PREFIX set.  This then gets stored in the
internal stash.  Thus, on any call we catch, we should be restoring it.
We then unset it, because it's not part of the whitelisted environment.

Now, what this means is that when we spawn child processes, they should
be getting the environment, but the parent bitbake is running with
PSEUDO_DISABLED.  Which, in turn, sets antimagic.  So most calls run
through their non-wrapped form.

The problem:  I wrote my popen() wrapper wrong.  See, I carefully
removed the check for pseudo_disabled from the top of it.  But!  That
code path is not actually the only way in which pseudo_disabled affects
behavior.  That's just an *OPTIMIZATION*.  The pseudo_disabled flag
also means that antimagic is set, and I copied that part of the wrapper
in unmodified:

        if (antimagic > 0) {
                /* call the real syscall */
                rc = (*real_popen)(command, mode);
        } else {
                /* exec*() use this to restore the sig mask */
                pseudo_saved_sigmask = saved;
                rc = wrap_popen(command, mode);
        }

And antimagic is 1, so I call real popen.  Which forks.  And even
though pseudo isn't in the LD_PRELOAD environment variable, it's still
in the process's address space, but PSEUDO_PREFIX isn't set, and for
some reason, the stashed value is missing.  Not sure I can explain that
part yet; maybe we do have a path where we wipe the stashed value.

But the underlying problem is that my popen() wrapper was never
actually doing the setupenv/dropenv, or just a setupenv.

And the other underlying problem is that calling os.popen() directly is
probably something we should discourage, because we really do want to
know, for each subprocess we plan to spawn, whether it is running in
the pseudo environment or not.  So if you call os.popen(), you get
"whatever state bitbake is in", which may not be at all what you wanted
or intended.

If I fix the popen() patch, it may actually start working again,
although I'm still not totally sure why the prefix is getting wiped out.

So... I think...

1.  We should probably whitelist PSEUDO_PREFIX because life is a heck
of a lot easier if we aren't trying to set it and unset it all the time.
2.  I need to fix my popen patch.
3.  Once I've fixed that, I can probably do a much better job of
articulating what's happening to pseudo_prefix that's causing us to end
up with a child process where both the internal stash and the
environment variable are unset.

The big thing I was missing was that PSEUDO_DISABLED implies that
everything will always have antimagic >= 1.

-s
-- 
Listen, get this.  Nobody with a good compiler needs to be justified.