[yocto] about bitbake exec change

[Qing He]

Richard,

I've seen your recent bitbake change that reverts exec to fork, is there
intended to be the preparation for pseudo changes? Do you have some kind
of plan of next steps?

Btw, since pseudo change needs to ship pseudo binary which is kind of
weird, I've been considering some alternatives, i.e. the socket based
data. Can you help to review this idea?

The point is that, since shared memory is not easy if not impossible in
python (including the python internal data structure), can we use
something else like UNIX socket? Because for a single task, the
number of read/write access is limited. I've figured out several
aspects, including the extension to data_smart and how to arrange the
socket, but not sure about things like bb.cooker.

The idea is something like:
1. the extended data_smart:
    a new field  data["_remote_data"] is in place to work like current
    "_data". Then, _findVar is almost the only thing needs to change
    (this is greatly simplified though):
        conn = data["_remote_data"]
	conn.send(key)
	data[key] = pickle.loads(conn.recv(4906))
    Other methods that need change is __init__() and keys()

2. whenever bitbake runs a new task and calls bitbake-runtask, it sets
   a socket and pass the socket file name to bitbake-runtask as a
   command line parameter

3. the bitbake-runtask won't need cooker any more, currently, the
   cooker itself also attributes a lot to the slowness, especially the
   configuration loading phase


And if it's suitable, the socket listenners can even be put into the
bitbake server, then there is no need to ship binaries with poky.

Two things I'm not very sure about:
1. can cooker involvment be cleanly removed in bitbake-runtask, the
only reason of it is to load cache file, isn't it?
2. it would be very difficult for the task's own version of shadow data
be transfer back to main thread.

Do you think this approach is viable?

Thanks,
Qing