[poky] Recipe Updating Status and call to action

[Tian, Kevin]

>From: Saul Wold
>Sent: Thursday, December 16, 2010 8:58 AM
>
>Folks,
>
>As we get ready for M3 opening up shortly, I wanted to get folks aware
>of where we are with M2 and the recipe updating process.  Based on my
>rough reckoning, we had about 266 recipes that needed updating at the
>start of M2, we updated about 110 (possibly more).  There are now about
>160 recipes in the update list that need tackling.
>
>Also, the distro tracking data for Version information is getting stale,
>there are about 135 RECIPE_LATEST_VERSION metadata tags that need to be
>updated.
>
>I have attached a spreadsheet that shows the update status of both the
>upstream (RMatch Column) and tracking data (TMatch Column).
>
>I would like to see the team and community pitch in and complete these
>160 recipe updates for the 1.0 Release. Please be sure the ownership /
>RECIPE_MAINTAINER information is correct.
>

I'm kind of surprised when seeing this data, as I thought we have upgraded more
than 110 recipes. Then distro team gathered just now to understand whether there's
anything mismatch. The major point causing confusion is that the code generating
above sheet is stateless, which doesn't track historical data while there do have
many packages releasing new versions just in M2 window. This then moves some
packages we do upgrade into the '160' group. Basically we think those items with small 
version difference fall into this category. So it's not that bad as I originally thought. :-)

Ke is collecting some data and will reply later to clarify in 160 recieps:
	- what have been upgraded in M2 window already, and
	- what have not been touched at all

With that data we'll see how much work remains in M3 for this task.

On the other hand, along with this I realize that there's one area we need further
discuss. How often should we upgrade packages in a given release cycle? MeeGo 
only does once. For Yocto we want to keep our recipes in cutting-edge which is 
why we schedule two upgrade windows in M2 and M3 this time. The policy for each 
recipe however could be a bit different:

	- M2 is our 1st upgrade window, within which we try to upgrade as many 
recipes as possible to avoid rush push in last release point. The order is done by:
		* first upgrade small version changes in a batch which normally implicates 
those recipes have been touched in previous release cycle and thus small risk
		* then upgrade the rest recipes which may need more debug time 
because they may not been touched for some time or at all

	- M3 (now) is our 2nd upgrade window, within which we try to ensure our 
final release in cutting-edge. Then I suggest a reverse order:
		* first upgrade recipes which haven't been covered in M2, which implicates 
more work required and higher risk
		* then upgrade small version changes at end of the window. This way 
can ensure catching latest version or else we may need to do it multiple times if doing
it too early. This has small risk

So in generally I propose to have a longer window for recipes which have been 
upgraded and got a new version now. For now we can have distro team focusing on
those recipes which haven't been touched yet.

Does it sound OK?

----

BTW, I think our package report system (in experimental phase now) can help
the upgrade decision clearly. A suggested flow will be:

The server checks upstream version every day or every several days. When there's
a new version newly released, a mail will be sent to the recipe owner and CC this
list.

The owner then need react to the notification:
	- verify whether this is a valid message
	- decide whether a upgrade is required. The reason for non-upgrade could be:
		* pure SCM commit (for SCM check we may need more thinking)
		* an unstable release
		* known critical bugs/security holes
		* not worthy of upgrade effort since no new features are introduced
		* ...
	- mark decision on the package report system
		* to-be-upgraded
		* not upgrade (if so, reason). If we still use RECIPE_NO_UPDATE_REASON,
then a patch may be required to update that field
		
The package report system will record all the historical versions and decisions, to
avoid trigger unnecessary messages again.

When a upgrade window starts and ends, we can then use above stateful data to
draw overall picture and track our progress accurately.

Comments? 

Thanks
Kevin