[yocto] [layerindex-web][PATCH] layerindex-web : adding asynchronous task execution

[Paul Eggleton]

On Wednesday, 23 August 2017 12:06:09 PM NZST Paul Eggleton wrote:
> Hi Diana
> 
> I did finally manage to get to it today - sorry about the delay. I've
> sorted out the settings issue and dropped the related item from the todo list.

Oops, I neglected to mention I pushed these minor changes to the same
paule/celery branch.

> Assuming that didn't break anything on your end, could you please squash all of
> these into one commit with you as the author and an appropriate shortlog /
> commit message and then send a new patch? We'd probably want to mention that
> the mechanism this establishes will be used later for executing other
> asynchronous tasks.
> 
> Thanks,
> Paul
> 
> On Saturday, 12 August 2017 7:56:15 AM NZST Diana Thayer wrote:
> > Hello! Did you get a chance to look at the PR
> > <https://github.com/garbados/layerindex-web/pull/1>?
> > 
> > On Wed, Jul 26, 2017 at 6:02 PM, Diana Thayer <garbados at gmail.com> wrote:
> > 
> > > Great! Thanks for testing it out and identifying that issue.
> > >
> > > I've put together a PR on GitHub
> > > <https://github.com/garbados/layerindex-web/pull/1> that includes some
> > > changes that appear to fix the issue you're describing. It also updates the
> > > readme to document how to configure the connection to the RabbitMQ server,
> > > and how to start the Celery worker. You can see all the file changes here
> > > <https://github.com/garbados/layerindex-web/pull/1/files>.
> > >
> > > On Mon, Jul 24, 2017 at 2:49 AM, Paul Eggleton <
> > > paul.eggleton at linux.intel.com> wrote:
> > >
> > >> On Friday, 21 July 2017 4:15:26 PM CEST Paul Eggleton wrote:
> > >> > On Tuesday, 27 June 2017 1:59:23 AM CEST Diana Thayer wrote:
> > >> > > This is a work-in-progress solution to issue 11197 within
> > >> layerindex-web:
> > >> > > adding asynchronous task execution.
> > >> > > I used RabbitMQ and Celery. I'm still working with Paul Eggleton on
> > >> adding
> > >> > > install instructions for RabbitMQ to
> > >> > > the site's Dockerfile.
> > >> > >
> > >> > > Previously, when a user submitted a layer, Django would synchronously
> > >> send
> > >> > > emails to users able to publish layers,
> > >> > > such that if an email bounced or otherwise failed, the user
> > >> submitting the
> > >> > > layer would see an error.
> > >> > > This patch sends those emails as asynchronous tasks, so any sending
> > >> errors
> > >> > > don't reach the user submitting a layer.
> > >> >
> > >> > I had a chance to try your patch out today, although I had to apply it
> > >> > partially - I figured a good test would just be the email part. I
> > >> installed
> > >> > rabbitmq and ensured it got started. I started the web interface and
> > >> > submitted  a dummy layer, and the email sent as expected. I then tried
> > >> > killing my local  SMTP server and the layer submission returned an error
> > >> > page, which is what  happened previously - it looks to me as if the
> > >> sending
> > >> > is still being done synchronously. The celery worker isn't reporting
> > >> > anything so I can only assume  the django app is not trying to connect
> > >> to
> > >> > it.
> > >> >
> > >> > I've pushed just the changes I applied to a new branch here:
> > >> >
> > >> > http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/log
> > >> /?h=paule/celery
> > >> >
> > >> > Any idea what's going wrong? Did I mess up in applying the changes, or
> > >> > misconfigure something else?
> > >>
> > >> I figured out the issue - in order to have the task executed
> > >> asynchronously
> > >> you need to call .delay(<args>) on the task object rather than the task
> > >> function itself. That leads to a "django.core.exceptions.Improp
> > >> erlyConfigured"
> > >> error reported by the worker since in my configuration the "import
> > >> settings"
> > >> succeeds but Django isn't set up properly, but at least it's actually
> > >> executing the task via the worker now.
> > >>
> > >> Cheers,
> > >> Paul
> > >>
> 
> 
> 
> 


-- 

Paul Eggleton
Intel Open Source Technology Centre