[yocto] [layerindex-web][PATCH 00/29] RRS: rework upgrade history collection [cover letter only]

[Paul Eggleton]

Taking a hard look at the recipe upgrade functionality, there were a
number of shortcomings in how the data was collected which led to
upgrades being missed, history disappearing when recipes were removed or
renamed and scenarios such as multiple recipes with the same name not being
handled correctly. This patchset introduces the concept of a RecipeSymbol
(acting as an umbrella for RRS records for the same named recipe but
independent of actual Recipe records which may be transient) and also
fixes a number of issues in the upgrade data collection for much greater
accuracy.


The following changes since commit 50fc6780e022a95b452fdb6f57b4e1c6b37084f2:

  requirements.txt: bump a couple more versions (2019-10-29 10:22:59 +1300)

are available in the Git repository at:

  git://git.yoctoproject.org/layerindex-web paule/recipesymbol
  http://git.yoctoproject.org/cgit.cgi/layerindex-web/log/?h=paule/recipesymbol

Paul Eggleton (29):
  RRS: collect history independent of current recipes
  RRS: Add deleted recipe handling
  RRS: use more robust RFC2822 date conversion
  RRS: skip problematic OE-Core commits (when a dependency)
  RRS: handle downgrades
  RRS: use more robust method of getting last upgrade record
  rrs_upgrade_history: record start marker in log file
  RRS: handle recipe moves without overwriting data
  RRS: support grouping upgrades by version for multi-version recipes
  RRS: use RecipeUpgradeGroup to determine downgrades
  RRS: detect PN changing without move
  rrs_upgrade_history: implement file path filtering
  RRS: record previous version
  rrs_upgrade_history: add stop commit option
  RRS: fixup handling of upgrades where recipe moved to inc
  RRS: fix some more bad OE-Core commits
  RRS: ensure upgrades recorded at exact same time are correctly ordered
  RRS: avoid historical parsing bug in bitbake
  recipeparse: handle recipes at root of repository
  RRS: handle when recipes get deleted and later re-added
  RRS: exclude lib/ subdirectory of layers to avoid picking up templates
  RRS: ensure default URLs for release/milestone are the latest
  RRS: detect changes in SRCREV as upgrades
  RRS: Add tool to dump upgrades
  RRS: enable grouping recipe upgrades by license
  RRS: Handle two versions added on same day then later one deleted
  Add recipe dependencies tool
  RRS: do not ignore non-numeric characters in versions
  TODO: add some more tasks

 TODO                                          |  16 +
 layerindex/admin.py                           |   1 +
 layerindex/forms.py                           |  13 +
 .../migrations/0044_extendedprovides.py       |  46 ++
 layerindex/models.py                          |   8 +
 layerindex/recipeparse.py                     |  19 +-
 layerindex/update_layer.py                    |   2 +
 layerindex/urls.py                            |   6 +-
 layerindex/views.py                           | 116 ++++-
 rrs/admin.py                                  |  26 +-
 rrs/migrations/0020_recipesymbol_initial.py   |  63 +++
 rrs/migrations/0021_recipesymbol_nonnull.py   |  31 ++
 rrs/migrations/0022_recipesymbol_finish.py    |  27 ++
 rrs/migrations/0023_recipeupgrade_deleted.py  |  25 +
 .../0024_recipeupgrade_downgrade.py           |  20 +
 rrs/migrations/0025_recipeupgrade_move.py     |  25 +
 rrs/migrations/0026_recipeupgrade_grouping.py |  39 ++
 .../0027_recipeupgrade_prev_version.py        |  20 +
 rrs/migrations/0028_recipeupgrade_srcrev.py   |  20 +
 rrs/migrations/0029_rrs_license_group.py      |  44 ++
 rrs/models.py                                 | 145 +++++-
 rrs/tools/common.py                           |  12 +-
 rrs/tools/dump_upgrades.py                    |  89 ++++
 rrs/tools/rrs_maintainer_history.py           |  32 +-
 rrs/tools/rrs_upgrade_history.py              |  68 ++-
 rrs/tools/rrs_upstream_history.py             |  36 +-
 rrs/tools/upgrade_history_internal.py         | 431 +++++++++++++++---
 rrs/views.py                                  | 177 ++++---
 templates/base.html                           |   1 +
 templates/layerindex/recipedeps.html          | 209 +++++++++
 templates/rrs/recipedetail.html               |  27 +-
 31 files changed, 1598 insertions(+), 196 deletions(-)
 create mode 100644 layerindex/migrations/0044_extendedprovides.py
 create mode 100644 rrs/migrations/0020_recipesymbol_initial.py
 create mode 100644 rrs/migrations/0021_recipesymbol_nonnull.py
 create mode 100644 rrs/migrations/0022_recipesymbol_finish.py
 create mode 100644 rrs/migrations/0023_recipeupgrade_deleted.py
 create mode 100644 rrs/migrations/0024_recipeupgrade_downgrade.py
 create mode 100644 rrs/migrations/0025_recipeupgrade_move.py
 create mode 100644 rrs/migrations/0026_recipeupgrade_grouping.py
 create mode 100644 rrs/migrations/0027_recipeupgrade_prev_version.py
 create mode 100644 rrs/migrations/0028_recipeupgrade_srcrev.py
 create mode 100644 rrs/migrations/0029_rrs_license_group.py
 create mode 100755 rrs/tools/dump_upgrades.py
 create mode 100644 templates/layerindex/recipedeps.html

-- 
2.20.1