[yocto] RFC: Backwards compatibility checking sstate-cache

[Richard Purdie]

On Fri, 2017-06-30 at 09:48 +0000, Michael Ho wrote:
> Hi, at BMW Car IT we are working on an experimental feature to
> improve sstate cache hits and we are looking for comments on the
> approach who might have some insights to the problem and seeing if
> anyone is interested in the feature for mainline.

Sorry I didn't see this before now but it was brought to my attention.

I have given this problem quite some thought off and on. I do worry
that the interface you propose is complex and requires changes
throughout the metadata and those changes impose "policy". One of our
strengths is that we're managed to keep "policy" out of the metadata.

In this context by policy, I mean when a recipe is equivalent and when
it is not.

I do have a counter-proposal of how we could solve this in a less
invasive way. This isn't implemented but wouldn't in theory be hard to
do.

The idea would be to have some kind of equivalence list. The first
built object's sstate checksum would become the definitive checksum and
the object added to the sstate cache.

If a new object is built, it would be compared with the one in sstate.
If deemed equivalent (by whatever policy), a mapping would be added to
the list. If not, there is no mapping and it becomes a new definitive
checksum.

All runqueue would then have to do is present its list of sstate
checksums to the list and convert them (in dependency order) into
canonical checksums.

This list could be a local equivalence file, or an equivalence server
which could resolve list of checksums.

Doing it this way totally isolates the comparison from the metadata
itself and in my view protects us from future changes in definition of
equivalence.

How does that sound?

Cheers,

Richard