Can't start scheduler from bench, errors in worker.error.log

Hi,

I installed Frappe and ERPNext using

sudo bash setup_frappe.sh --setup-production --verbose

on a Ubuntu 14.04 32bit DigitalOcean droplet where only one username, swap and SSH keys had been installed and Python upgraded to 2.7.9.

One of the messages I got during the install was “Scheduler is disabled”. I am only guessing what the scheduler should do and I don’t understand what is its’ relation to cron.

I think both cron and something else are recording backups which I don’t know how to control. When I look at Backup Manager I now have

2015-08-30 11:40 /backups/20150830_78004045_files.tar 180.0K
2015-08-30 11:40 /backups/20150830_78004045_database.sql.gz 310.3K
2015-08-30 12:00 /backups/20150830_91229094_database.sql.gz 310.6K

crontab –l gives

0 */6 * * * cd /home/username/frappe-bench && /usr/local/bin/bench --site all backup >> /home/username/frappe-bench/logs/backup.log 2>&1

so I think the third backup file was recorded by cron.

When I try to enable scheduler from the bench I get the following errors:

username@myserver:~/frappe-bench$ bench frappe --enable_scheduler

frappe app is not installed. Run the following command to install frappe
bench get-app frappe https//github.com/frappe/frappe.git
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 9, in
load_entry_point(‘bench==0.92’, ‘console_scripts’, ‘bench’)()
File “/home/username/bench-repo/bench/cli.py”, line 46, in cli
return old_frappe_cli()
File “/home/username/bench-repo/bench/cli.py”, line 98, in old_frappe_cli
os.execv(f, [f] + sys.argv[2:])
OSError: [Errno 2] No such file or directory

username@myserver:~/frappe-bench$ bench get-app frappe

https//github.com/frappe/frappe.git
fatal: destination path ‘frappe’ already exists and is not an empty directory.
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 9, in
load_entry_point(‘bench==0.92’, ‘console_scripts’, ‘bench’)()
File “/home/username/bench-repo/bench/cli.py”, line 60, in cli
bench()
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 700, in call
return self.main(*args, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 680, in main
rv = self.invoke(ctx)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 1027, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 873, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 508, in invoke
return callback(*args, **kwargs)
File “/home/username/bench-repo/bench/cli.py”, line 174, in get_app
_get_app(name, git_url, branch=branch)
File “/home/username/bench-repo/bench/app.py”, line 52, in get_app
cwd=os.path.join(bench, ‘apps’))
File “/home/username/bench-repo/bench/utils.py”, line 104, in exec_cmd
raise CommandFailedError(cmd)
bench.utils.CommandFailedError: git clone https//github.com/frappe/frappe.git --depth 1 --origin upstream frappe
username@myserver:~/frappe-bench$

When I look in worker.error.log I see the following:

worker: Cold shutdown (MainProcess)
[2015-08-30 11:40:19,750: WARNING/MainProcess] celery@myserver ready.
[2015-08-30 11:40:24,213: WARNING/Worker-1] Database backed up
[2015-08-30 11:40:24,218: WARNING/Worker-1] /home/username/frappe-bench/sites/site1.local/private/backups/20150830_78004045_database.sql.gz
[2015-08-30 11:40:24,234: WARNING/Worker-1] Backed up files
[2015-08-30 11:40:24,235: WARNING/Worker-1] /home/username/frappe-bench/sites/site1.local/private/backups/20150830_78004045_files.tar
[2015-08-30 11:40:24,332: WARNING/Worker-1] /home/username/frappe-bench/env/local/lib/python2.7/site-packages/urllib3/util/ssl_.py:97: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see [https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning][1].
InsecurePlatformWarning
[2015-08-30 11:40:25,341: WARNING/Worker-1] Traceback (innermost last):
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 57, in take_backups_dropbox
did_not_upload, error_log = backup_to_dropbox()
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_dropbox.py”, line 92, in backup_to_dropbox
upload_file_to_dropbox(filename, “/database”, dropbox_client)
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_dropbox.py”, line 152, in upload_file_to_dropbox
dropbox_client.put_file(folder + “/” + os.path.basename(filename), f, overwrite=True)
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/dropbox/client.py”, line 385, in put_file
return self.rest_client.PUT(url, file_obj, headers)
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/dropbox/rest.py”, line 327, in PUT
return cls.IMPL.PUT(*n, **kw)
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/dropbox/rest.py”, line 264, in PUT
return self.request(“PUT”, url, body=body, headers=headers, raw_response=raw_response)
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/dropbox/rest.py”, line 235, in request
raise ErrorResponse(r, r.read())
ErrorResponse: [403] u"Invalid app key (consumer key). Check your app’s configuration to make sure everything is correct."
[2015-08-30 11:40:25,586: WARNING/Worker-1] frappe.tasks.scheduler_task[98bc21e7-fd7c-423d-8ad4-c2b890a0db2a]: Method: daily_long, Handler: erpnext.setup.doctype.backup_manager.backup_manager.take_backups_daily
Traceback (innermost last):
File “/home/username/frappe-bench/apps/frappe/frappe/tasks.py”, line 90, in scheduler_task
frappe.get_attr(handler)()
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 39, in take_backups_daily
take_backups_if(“Daily”)
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 47, in take_backups_if
take_backups_dropbox()
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 65, in take_backups_dropbox
send_email(False, “Dropbox”, error_message)
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 101, in send_email
recipients = frappe.db.get_value(“Backup Manager”, None, “send_notifications_to”).split(",")
AttributeError: ‘NoneType’ object has no attribute ‘split’

[2015-08-30 11:40:25,596: ERROR/MainProcess] Task frappe.tasks.scheduler_task[98bc21e7-fd7c-423d-8ad4-c2b890a0db2a] raised unexpected: AttributeError("‘NoneType’ object has no attribute ‘split’",)
Traceback (most recent call last):
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/celery/app/trace.py”, line 240, in trace_task
R = retval = fun(*args, **kwargs)
File “/home/username/frappe-bench/env/local/lib/python2.7/site-packages/celery/app/trace.py”, line 438, in protected_call
return self.run(*args, **kwargs)
File “/home/username/frappe-bench/apps/frappe/frappe/tasks.py”, line 90, in scheduler_task
frappe.get_attr(handler)()
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 39, in take_backups_daily
take_backups_if(“Daily”)
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 47, in take_backups_if
take_backups_dropbox()
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 65, in take_backups_dropbox
send_email(False, “Dropbox”, error_message)
File “/home/username/frappe-bench/apps/erpnext/erpnext/setup/doctype/backup_manager/backup_manager.py”, line 101, in send_email
recipients = frappe.db.get_value(“Backup Manager”, None, “send_notifications_to”).split(",")
AttributeError: ‘NoneType’ object has no attribute ‘split’

My questions are:

  • why can’t I start the scheduler through bench?
  • how do I stop these errors in the log file?
  • where do I control the scheduler?
  • do I need the cron job if the scheduler is running?

Many thanks.

CC @pdvyas

If you are using production setup, the scheduler is managed by supervisord so you do not need to start it again.

Regard the dropbox upload errors, I think was an issue in the code. Just fixed it in develop.

The command to enable scheduler is bench --site all enable-scheduler

Yes, it’s dedicated only to backup. So, you still get backups if the scheduler is not running.

To start and stop the scheduler, use supervisor commands

sudo supervisorctl stop frappe-worker
sudo supervisorctl start frappe-worker

To check any issues, use

bench doctor
1 Like

Thanks, I think I understand better now.

Thanks. Is there any way I can pull the new code so that I stop recording these dropbox upload errors? My worker.error.log is already 1.8 mb, I think all the same error.

We have started to implement already (accounts, customers, contacts…) so it would bring me a peace of mind to see the error logs idle…

Kind regards.