[yocto] Multiple conncetion to svn (svn+ssh)

[Paul Eggleton]

On Monday 06 July 2015 15:04:57 Bryan Evenson wrote:
> Paul,
> 
> > -----Original Message-----
> > From: Paul Eggleton [mailto:paul.eggleton at linux.intel.com]
> > Sent: Monday, July 06, 2015 10:08 AM
> > To: Marcin Krzemiński
> > Cc: Bryan Evenson; yocto at yoctoproject.org
> > Subject: Re: [yocto] Multiple conncetion to svn (svn+ssh)
> > 
> > On Monday 06 July 2015 12:57:39 Bryan Evenson wrote:
> > > Marcin,
> > > 
> > > > From: yocto-bounces at yoctoproject.org
> > > > [mailto:yocto-bounces at yoctoproject.org] On Behalf Of Marcin
> > > > Krzeminski
> > > > Sent: Friday, July 03, 2015 7:55 AM
> > > > To: yocto at yoctoproject.org
> > > > Subject: [yocto] Multiple conncetion to svn (svn+ssh)
> > > > 
> > > > Hello again,
> > > > 
> > > > I have 12 recipes that download code from same svn repository (using
> > > > svn+ssh)protocol. Recipes are grouped in packagegroup.
> > > > When I want to bitbake packagegroup fetcher fail, but when I run
> > > > recipe alone all is ok. I think it is because I want to open 8
> > > > connection to one svn repository.How can I fix this, for example by
> > > > allowing only eg. 2 recipes from packagegroup to be executed in
> > > > paralell.
> > > 
> > > There is no fetcher-specific variable that I know of, but I think you
> > > can set PARALLEL_MAKE="2" in your recipes to reduce the number of
> > > fetches on your repository at a time.  The downside is your build will
> > > go slower when it is building your recipes as it won't be doing as
> > > many things in parallel.
> > 
> > That's not going to help. The parallelism we are talking about here is
> > across recipes - PARALLEL_MAKE is just passed through to make within one
> > task in one recipe, and make isn't even involved at the fetch stage.
> 
> Sorry, I grabbed the wrong variable.  I meant BB_NUMBER_THREADS, not
> PARALLEL_MAKE.

Sure, but that's not going to work either from within a recipe. It would work at the configuration level but that'll quite severely impact build performance.

> > Something that might work would be to set a lockfile on the do_fetch task
> > such that only one of the recipes could fetch at once. That could not
> > allow
> > two executing at once, but at least it would solve the problem. e.g. you
> > could add this to all of the recipes:
> > 
> > do_fetch[lockfiles] += "${TMPDIR}/mysvnlock.lock"
> > 
> > (The file name isn't critical, it just needs to be the same for all of the
> > recipes you wish to participate in the exclusive fetching.)
> 
> I had no idea that we could do this.  I don't see any documentation on
> lockfiles anywhere. If you use a lockfile, do the all recipes with the same
> lockfile wait until the lockfile is available before continuing on with
> that step?  Is there a timeout for waiting for the lockfile or does the
> recipe wait indefinitely?

Yes, it's just a lock on the specified file - whoever gets there first can
continue, everyone else blocks, and it's an indefinite wait as far as
I'm aware.

It's a little obscure perhaps, but it is in the BitBake manual:

http://www.yoctoproject.org/docs/current/bitbake-user-manual/bitbake-user-manual.html#variable-flags

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre