[yocto] [yocto-autobuilder][PATCH V4 1/1] ScrapeTargets.py: improve target search algorithm

[Joshua Lock]

On Wed, 2017-07-05 at 14:27 -0700, Stephano Cetola wrote:
> When scraping the build targets from refkit-ci.inc, grep would only
> return the first line of the search result. By replacing grep with
> awk, we are now searching for multi-line variables, and should now
> scrape build targets even if they have line breaks between them.
> 
> [YOCTO #11720]

Thanks, I've merged and pushed this. Please see my comment below.

> 
> Signed-off-by: Stephano Cetola <stephano.cetola at linux.intel.com>
> ---
>  .../site-packages/autobuilder/buildsteps/ScrapeTargets.py    | 12
> +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/python2.7/site-
> packages/autobuilder/buildsteps/ScrapeTargets.py
> b/lib/python2.7/site-packages/autobuilder/buildsteps/ScrapeTargets.py
> index 80153fe85..07e2fdcf4 100644
> --- a/lib/python2.7/site-
> packages/autobuilder/buildsteps/ScrapeTargets.py
> +++ b/lib/python2.7/site-
> packages/autobuilder/buildsteps/ScrapeTargets.py
> @@ -41,7 +41,9 @@ class ScrapeTargets(ShellCommand):
>          workerdir = os.path.join(os.path.join(YOCTO_ABBASE, "yocto-
> worker"))
>          buildername = self.getProperty("buildername")
>          src = os.path.join(workerdir, buildername, "build",
> self.source)
> -        self.command = "cat " + src + " | grep " + self.targetsvar
> +        # find targetsvar then return lines up to a quote
> +        self.command = ["awk",
> +                '/%s/{flag=1;print;next}/"/{flag=0}flag' %
> self.targetsvar, src]
>          ShellCommand.start(self)
>  
>      def commandComplete(self, cmd):
> @@ -49,10 +51,10 @@ class ScrapeTargets(ShellCommand):
>              return
>  
>          result = cmd.logs['stdio'].getText()
> -        targets = result.replace(self.targetsvar, "")
> -        targets = targets.strip()
> -        targets = targets.replace('=', '')
> -        targets = targets.strip('"')
> +        targets = result.strip()
> +        targets = targets.replace('%s="' % self.targetsvar, '')

This still leaves us a bit brittle. What if someone adds whitespace
around the assignment operator? Could you create a follow-on patch for
that?

Thanks!

> +        targets = targets.replace('\\', '')
> +        targets = targets.replace('\n', '')
>          self.setProperty("scraped_targets",
>                           targets,
>                           'Targets "%s" scraped from %s' % (targets,
> -- 
> 2.13.2
>