Query Timeout and Long-running Tasks

Hello, I'm working on some functionality using the HTTP API and frappe.call() to do work that is processor intensive. Is there a way to extend the timeout from 1 minute to something a bit longer? Is that a frappe setting or something in Python?

I'm guessing the recommended way is to put the task in a celery queue. Would there be a way to return information to the browser after completion, or would that be limited to sending the result through email only?

Thanks.



Note:

 

If you are posting an issue,

 

  1. ERPNext is a free and open source software and support is given on this forum by a team (https://frappe.io/webnotes). So please consider donating if you find this forum useful (https://frappe.io/buy). Even a small amount would be helpful.
  2. We should be able to replicate it at our end. So please give us as much information as you can. Please see it from the point of view of the person receiving the communication.
  3. Paste your code at http://pastebin.com or http://gist.github.com and send only the URL via email
  4. For sending images, use http://imgur.com or other similar services. Do not send images as attachments. Links are good. Same goes for any file you are going to send.

     

    End of Note



    You received this message because you are subscribed to the Google Groups “ERPNext Developer Forum” group.

    To unsubscribe from this group and stop receiving emails from it, send an email to erpnext-developer-forum+un…@googlegroups.com.

    To view this discussion on the web visit https://groups.google.com/d/msgid/erpnext-developer-forum/76a488fe-0965-4b55-92e8-0396f35ca6ee%40googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.

On Thu, Sep 11, 2014 at 12:43 PM, Brendan Kiu br...@gmail.com wrote:

Hello, I’m working on some functionality using the HTTP API and
frappe.call() to do work that is processor intensive. Is there a way to
extend the timeout from 1 minute to something a bit longer? Is that a frappe
setting or something in Python?

You can set custom timeout

bench config http-timeout <integer>
bench setup supervisor
bench setup nginx

As root,

supervisctl reload
service nginx reload

I’m guessing the recommended way is to put the task in a celery queue. Would
there be a way to return information to the browser after completion, or
would that be limited to sending the result through email only?

You can define a task in task.py in your app and run it asyncrhonously,

eg,
https://github.com/frappe/erpnext/blob/develop/erpnext/tasks.py#L101

import the function and run func_name.delay(arg1, arg2)

At the moment, we don’t have any implementation for long polling.

Thanks.


Note:

If you are posting an issue,

  1. ERPNext is a free and open source software and support is given on this
    forum by a team (https://frappe.io/webnotes). So please consider donating if
    you find this forum useful (https://frappe.io/buy). Even a small amount
    would be helpful.
  2. We should be able to replicate it at our end. So please give us as much
    information as you can. Please see it from the point of view of the person
    receiving the communication.
  3. Paste your code at http://pastebin.com or http://gist.github.com and send
    only the URL via email
  4. For sending images, use http://imgur.com or other similar services. Do
    not send images as attachments. Links are good. Same goes for any file you
    are going to send.

End of Note

You received this message because you are subscribed to the Google Groups
“ERPNext Developer Forum” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to erpnext-developer-forum+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/erpnext-developer-forum/76a488fe-0965-4b55-92e8-0396f35ca6ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thanks,

Pratik
erpnext

Thanks for the pointers! 

I'm working on getting celery working, but it doesn't seem to be executing with the func_name.delay(args...) command. Do you need to start the dev server in a particular way to enable the schedule queue? I'm using frappe sitename --serve. I'm just trying to get the output to show on the task in the dev server output.

Basic code snippets here:
https://gist.github.com/bkiu/664b688ff6a2e68530a3

Thanks.

On Thursday, September 11, 2014 4:36:15 PM UTC+5:30, Pratik Vyas wrote:
On Thu, Sep 11, 2014 at 12:43 PM, Brendan Kiu <br...@gmail.com> wrote:
> Hello, I'm working on some functionality using the HTTP API and
> frappe.call() to do work that is processor intensive. Is there a way to
> extend the timeout from 1 minute to something a bit longer? Is that a frappe
> setting or something in Python?
>

You can set custom timeout

`bench config http-timeout <integer>`
`bench setup supervisor`
`bench setup nginx`

As root,

```
supervisctl reload
service nginx reload
```

> I'm guessing the recommended way is to put the task in a celery queue. Would
> there be a way to return information to the browser after completion, or
> would that be limited to sending the result through email only?
>

You can define a task in task.py in your app and run it asyncrhonously,

eg,
https://github.com/frappe/erpnext/blob/develop/erpnext/tasks.py#L101

import the function and run `func_name.delay(arg1, arg2)`

At the moment, we don't have any implementation for long polling.

> Thanks.
>
> --
> Note:
>
> If you are posting an issue,
>
> 1. ERPNext is a free and open source software and support is given on this
> forum by a team (https://frappe.io/webnotes). So please consider donating if
> you find this forum useful (https://frappe.io/buy). Even a small amount
> would be helpful.
> 2. We should be able to replicate it at our end. So please give us as much
> information as you can. Please see it from the point of view of the person
> receiving the communication.
> 3. Paste your code at http://pastebin.com or http://gist.github.com and send
> only the URL via email
> 4. For sending images, use http://imgur.com or other similar services. Do
> not send images as attachments. Links are good. Same goes for any file you
> are going to send.
>
> End of Note
> ---
> You received this message because you are subscribed to the Google Groups
> "ERPNext Developer Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to erpnext-developer-forum+un...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/erpnext-developer-forum/76a488fe-0965-4b55-92e8-0396f35ca6ee%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


Thanks,
--
Pratik
erpnext



Note:

 

If you are posting an issue,

 

  1. ERPNext is a free and open source software and support is given on this forum by a team (https://frappe.io/webnotes). So please consider donating if you find this forum useful (https://frappe.io/buy). Even a small amount would be helpful.
  2. We should be able to replicate it at our end. So please give us as much information as you can. Please see it from the point of view of the person receiving the communication.
  3. Paste your code at http://pastebin.com or http://gist.github.com and send only the URL via email
  4. For sending images, use http://imgur.com or other similar services. Do not send images as attachments. Links are good. Same goes for any file you are going to send.

     

    End of Note



    You received this message because you are subscribed to the Google Groups “ERPNext Developer Forum” group.

    To unsubscribe from this group and stop receiving emails from it, send an email to erpnext-developer-forum+un…@googlegroups.com.

    To view this discussion on the web visit https://groups.google.com/d/msgid/erpnext-developer-forum/d7587c50-1c97-44f5-881e-a1c83280dde4%40googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.

On Thu, Sep 11, 2014 at 10:12 PM, Brendan Kiu br...@gmail.com wrote:

Thanks for the pointers!

I’m working on getting celery working, but it doesn’t seem to be executing
with the func_name.delay(args…) command. Do you need to start the dev
server in a particular way to enable the schedule queue? I’m using frappe
sitename --serve. I’m just trying to get the output to show on the task in
the dev server output.

Are you on the new bench?

With the new bench, bench start (instead of frappe --serve), starts
the celery workers and celerybeat process. For production setup, our
supervisor config has you covered for celery too.

Basic code snippets here:
https://gist.github.com/bkiu/664b688ff6a2e68530a3

looks good.

Thanks,

Pratik
erpnext

I just upgraded to the new bench, and following your instructions it's working fine. Thanks.

On Friday, September 12, 2014 12:23:21 AM UTC+5:30, Pratik Vyas wrote:
On Thu, Sep 11, 2014 at 10:12 PM, Brendan Kiu <br...@gmail.com> wrote:
> Thanks for the pointers!
>
> I'm working on getting celery working, but it doesn't seem to be executing
> with the func_name.delay(args...) command. Do you need to start the dev
> server in a particular way to enable the schedule queue? I'm using frappe
> sitename --serve. I'm just trying to get the output to show on the task in
> the dev server output.

Are you on the new bench?

With the new bench, `bench start` (instead of frappe --serve), starts
the celery workers and celerybeat process. For production setup, our
supervisor config has you covered for celery too.

>
> Basic code snippets here:
> https://gist.github.com/bkiu/664b688ff6a2e68530a3
>

looks good.

Thanks,
--
Pratik
erpnext



Note:

 

If you are posting an issue,

 

  1. ERPNext is a free and open source software and support is given on this forum by a team (https://frappe.io/webnotes). So please consider donating if you find this forum useful (https://frappe.io/buy). Even a small amount would be helpful.
  2. We should be able to replicate it at our end. So please give us as much information as you can. Please see it from the point of view of the person receiving the communication.
  3. Paste your code at http://pastebin.com or http://gist.github.com and send only the URL via email
  4. For sending images, use http://imgur.com or other similar services. Do not send images as attachments. Links are good. Same goes for any file you are going to send.

     

    End of Note



    You received this message because you are subscribed to the Google Groups “ERPNext Developer Forum” group.

    To unsubscribe from this group and stop receiving emails from it, send an email to erpnext-developer-forum+un…@googlegroups.com.

    To view this discussion on the web visit https://groups.google.com/d/msgid/erpnext-developer-forum/faeb0f94-6f16-4969-80d0-c6e2f96b045a%40googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.