[yocto] [layerindex-web][PATCH 2/2] Add CSV export for layer recipes

[Mark Hatle]

On 5/6/18 10:35 PM, Paul Eggleton wrote:
> Add the ability to export the recipe listing for a layer to a CSV file
> for importing into external tools. At the moment we include name,
> version and license, but there is a parameter that lets you specify the
> fields to include in the URL if desired.
> 
> Implements [YOCTO #12722].

With the bitbake layer library I'm working on, it should be possible to do this
directly out of bitbake.

This wouldn't limit it to just recipes either, but any data.  Loading (querying)
the json data from the layer index is easy, and the library would know how to
filter to the right branch/layer/etc.

Something to consider as we go forward with this and other work.

--Mark

> Signed-off-by: Paul Eggleton <paul.eggleton at linux.intel.com>
> ---
>  layerindex/urls_branch.py        |  5 ++++-
>  layerindex/views.py              | 22 ++++++++++++++++++++++
>  templates/layerindex/detail.html | 14 +++++++-------
>  3 files changed, 33 insertions(+), 8 deletions(-)
> 
> diff --git a/layerindex/urls_branch.py b/layerindex/urls_branch.py
> index 0e41435e..2809147b 100644
> --- a/layerindex/urls_branch.py
> +++ b/layerindex/urls_branch.py
> @@ -7,7 +7,7 @@
>  from django.conf.urls import *
>  from django.views.defaults import page_not_found
>  from django.core.urlresolvers import reverse_lazy
> -from layerindex.views import LayerListView, RecipeSearchView, MachineSearchView, DistroSearchView, ClassSearchView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, RedirectParamsView, DuplicatesView, LayerUpdateDetailView
> +from layerindex.views import LayerListView, RecipeSearchView, MachineSearchView, DistroSearchView, ClassSearchView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, RedirectParamsView, DuplicatesView, LayerUpdateDetailView, layer_export_recipes_csv_view
>  
>  urlpatterns = [
>      url(r'^$', 
> @@ -20,6 +20,9 @@ urlpatterns = [
>          LayerDetailView.as_view(
>              template_name='layerindex/detail.html'),
>              name='layer_item'),
> +    url(r'^layer/(?P<slug>[-\w]+)/recipes/csv/$',
> +        layer_export_recipes_csv_view,
> +        name='layer_export_recipes_csv'),
>      url(r'^recipes/$',
>          RecipeSearchView.as_view(
>              template_name='layerindex/recipes.html'),
> diff --git a/layerindex/views.py b/layerindex/views.py
> index dbf08497..06d35261 100644
> --- a/layerindex/views.py
> +++ b/layerindex/views.py
> @@ -1022,3 +1022,25 @@ class StatsView(TemplateView):
>                  machine_count=Count('layerbranch__machine', distinct=True),
>                  distro_count=Count('layerbranch__distro', distinct=True))
>          return context
> +
> +
> +def layer_export_recipes_csv_view(request, branch, slug):
> +    import csv
> +    layer = get_object_or_404(LayerItem, name=slug)
> +    layerbranch = layer.get_layerbranch(branch)
> +
> +    response = HttpResponse(content_type='text/csv')
> +    response['Content-Disposition'] = 'attachment; filename="recipes_%s_%s.csv"' % (layer.name, layerbranch.branch.name)
> +
> +    fieldlist = request.GET.get('fields', 'pn,pv,license').split(',')
> +    recipe_fields = [f.name for f in Recipe._meta.get_fields() if not (f.auto_created and f.is_relation)]
> +    for field in fieldlist:
> +        if field not in recipe_fields:
> +            return HttpResponse('Field %s is invalid' % field)
> +
> +    writer = csv.writer(response)
> +    for recipe in layerbranch.sorted_recipes():
> +        values = [getattr(recipe, field) for field in fieldlist]
> +        writer.writerow(values)
> +
> +    return response
> diff --git a/templates/layerindex/detail.html b/templates/layerindex/detail.html
> index 220d475b..67c21126 100644
> --- a/templates/layerindex/detail.html
> +++ b/templates/layerindex/detail.html
> @@ -199,13 +199,13 @@
>                          <div class="navbar-inner">
>                              <a class="brand pull-left">{{ layeritem.name }} recipes <span class="muted">({{ layerbranch.sorted_recipes.count }})</span></a>
>  
> -                            <ul class="nav pull-right">
> -                                <li>
> -                                    <form action="" class="navbar-search pull-right" id="filter-form">
> -                                        <input type="text" placeholder="Search recipes" class="search-query" id="filter">
> -                                    </form>
> -                                </li>
> -                            </ul>
> +                            <div class="pull-right">
> +                                <a href="{% url 'layer_export_recipes_csv' layerbranch.branch.name layerbranch.layer.name %}" class="btn"><i class="icon-file"></i> Export CSV</a>
> +
> +                                <form action="" class="navbar-search nav-spacer pull-right" id="filter-form">
> +                                    <input type="text" placeholder="Search recipes" class="search-query" id="filter">
> +                                </form>
> +                            </div>
>                          </div>
>                      </div>
>  
>