Scheduler not working in Virtual image

Install / Update Deployment
1

Hi.

I downloaded the virtual image. Updated as instructed. Everything is running fine (great project by the way).

I did no other manual tweaks on the image.

I set up outgoing email. This works fine, I can email things. (Although, seems a bug: in the system settings, it still shows 0 (zero) next to outgoing email).

I set up incoming support email. This saves fine (and it seems to check the credentials), but no emails are retrieved for support. Syncing from mail IS enabled.

Browsing this forum I discovered these fetches are done via the scheduler.

The scheduler is indeed shown as enabled, but no scheduler logs are shown in the UI.
I’ve tried toggling the checkbox.

In workerbeat.error.log, I see “sending due task scheduler…”, but nothing more interesting than that.

Please help, thanks.

Update: installed erpnext, manually, on a fresh virtual machine from scratch, same problem.

Update 2: Noticed this - is it relevant?

erpnext@sv:~$ bench init frappe-bench
Already using interpreter /usr/bin/python
Cloning into ‘frappe’…
remote: Counting objects: 61698, done.
remote: Total 61698 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (61698/61698), 25.63 MiB | 3.06 MiB/s, done.
Resolving deltas: 100% (43712/43712), done.
installing frappe
Compiling /home/erpnext/frappe-bench/env/build/gunicorn/gunicorn/workers/_gaiohttp.py …
SyntaxError: (‘invalid syntax’, (‘/home/erpnext/frappe-bench/env/build/gunicorn/gunicorn/workers/_gaiohttp.py’, 64, 26, ’ yield from self.wsgi.close()\n’))

Wrote ./assets/js/frappe-web.min.js - 90k
Wrote ./assets/js/print_format_v3.min.js - 18k
Wrote ./assets/css/frappe.css - 224k
Wrote ./assets/css/frappe-web.css - 163k
Wrote ./assets/js/editor.min.js - 27k
Wrote ./assets/js/frappe.min.js - 609k
Wrote ./assets/js/slickgrid.min.js - 102k
Wrote ./assets/js/canvasResize.min.js - 22k
no crontab for erpnext

2

Looks good! Ignore the syntax error.

3

To be clear, I had ignored that error. The problem remains…

How can I tell if the scheduler is actually running or not?

4

firstly, make sure that it’s not disabled for you site. Run,
bench frappe --enable_scheduler

Then check if there are no lock files in your site,
ls sites/{sitename}/locks should be empty or no file should be > 10 minutes old

Then check if celery processes are running,

sudo supervisorctl status

1 Like
5

Thank you for response.

  1. responds with is enabled. (and surely it would be after fresh install, and in VM?)

  2. no locks.

  3. 3 processes running, all with uptimes of several hours.

No errors in the /logs/ directory. Nothing to indicate any problems.

says to check table __SchedulerLog.

This table doesn’t exist?

6

The table is renamed to tabScheduler Log You can access this DocType from the desk too, type, Scheduler Log list in the search bar

7

No entries whatsoever. Is that normal?

8

Is this a clue at all? I see 0 next to outgoing email settings, and incoming support settings. Yet both have values.

Which table are these stored in please?

(I’m desperate to get this working before tomorrow, otherwise I can’t pitch it to the company).

9

Not a clue. Turns out this zero thing has been around for at least 6 months: Fetch support emails: Messages disappear · Issue #1642 · frappe/erpnext · GitHub

10

Zero thing seems to be a non issue. Can you post the out the following in bench frappe --ipython?

cint(frappe.db.get_value('Support Email Settings', None, 'sync_support_mails'))
from erpnext.support.doctype.support_ticket.get_support_mails import get_support_mails
get_support_mails()
11

So I switched email provider from gmail, to another provider, and it started working.

So it seems it was running, but encountering errors.

So it seems:

a) the validation when saving the settings might not be adequate
b) when it runs, and fails, in some cases it does not log the failures anywhere, making system administration a little tricky…

I can’t use this other email provider for production, so still need to find these logs so we can proceed with ERPNext.

Output from that command:

In [1]: cint(frappe.db.get_value(‘Support Email Settings’, None, ‘sync_support_mails’))
Out[1]: 1

In [2]: from erpnext.support.doctype.support_ticket.get_support_mails import get_support_mails

In [3]: get_support_mails()

Thanks for help.

12

I agree, and will be fixing the validation part. For GMail, you might have to enable “less secure apps” to use POP3.

The present system is,

  1. If a job fails, (raises exception), it’s logged in Scheduler Log and worker.error.log
  2. Keeps a lock file and would not run anymore if lock file is there
  3. Raises LockTimeoutError in case the lock file is more than 10 mins old

You can configure email notification for scheduler errors. By writing a file, sites/common_site_config.json with content

{
  "celery_error_emails": {
    "ADMINS": [
      [
        "Person 1",
        "person1@example.com"
      ],
      [
        "Person2 ",
        "person2@example.com"
      ]
    ],
    "SERVER_EMAIL": "exceptions@example.com"
  }
}

One limitation is that it’ll use local mailserver on port25 to send the emails.

13

That’s just the thing - in both the virtual image I used, and setting it up on a proper server, there has never been a single entry in that table. Despite it continually failing for gmail, and succeeding for non-gmail, and then failing again on gmail. Not in the db table, or in that worker.error.log file.

As I said earlier as well, also no lock files…

So it very much seems to me like there’s a particular way of failing that is somehow escaping your error handling, and so it’s silently failing.

Thanks for the info on the celete_error_emails.

14

This seems serious, is this on the VM available for download on our website? I will try to replicate this and check.

15

Indeed - downloaded it on 2014/11/08 or 09.
As I say - same thing happened by following the installation instructions for production server.

Thanks.

16

I tried to replicate the issue with putting in a wrong username and password and this got logged in the error logs.

[2014-11-17 14:09:15,132: ERROR/MainProcess] Task frappe.tasks.scheduler_task[c6feb192-fdb5-4c0b-a32e-4ac1408de2f8] raised unexpected: error_proto('-ERR [AUTH] Username and password not accepted.',)
Traceback (most recent call last):
  File "/home/erpnext/frappe-bench/env/local/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/erpnext/frappe-bench/env/local/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/tasks.py", line 77, in scheduler_task
    frappe.get_attr(handler)()
  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/hr/doctype/job_applicant/get_job_applications.py", line 47, in get_job_applications
    JobsMailbox()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/utils/email_lib/receive.py", line 125, in __init__
    self.get_messages()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/utils/email_lib/receive.py", line 153, in get_messages
    self.connect()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/utils/email_lib/receive.py", line 146, in connect
    self.pop.pass_(self.settings.password)
  File "/usr/lib/python2.7/poplib.py", line 189, in pass_
    return self._shortcmd('PASS %s' % pswd)
  File "/usr/lib/python2.7/poplib.py", line 152, in _shortcmd
    return self._getresp()
  File "/usr/lib/python2.7/poplib.py", line 128, in _getresp
    raise error_proto(resp)
error_proto: -ERR [AUTH] Username and password not accepted.

When you ran get_support_mails() from the ipython shell, did it exit cleanly or did it raise an exception?

The system imports a mail only if it’s unread (gmail has edge cases where if if you mark an email as unread, it does not show up in POP).

17

Thanks this worked for me