Just trying to answer this query since alot of us face this issue and kind of are out of wits to know the real cause.
First of all thanks to @saurabh6790 for resolving this issue reference: WN-SUP29014.
This is how I have resolved this issue:
Step1:
- First Check if your code is working by `bench --site [site_name] execute [path_to_file]
- If your scheduled task code is fine then check if scheduler is working
bench --site [site-name] trigger-scheduler-event all/hourly/daily/monthly
- Most probably this would not work since schedulers are not working
- Now try
bench --site all clear-cache
(and most probably this won’t work too)
- Now get into site console where the problem is
bench --site [site-name] console
- Enter this command
frappe.cache().get_value('scheduler_events')
, this would show the events in your cache for scheduler and in my case the custom_app tasks were missing as below:
In [2]: frappe.cache().get_value('scheduler_events')
Out[2]:
{u'all': [u'frappe.email.queue.flush',
u'frappe.email.doctype.email_account.email_account.pull',
u'frappe.email.doctype.email_account.email_account.notify_unreplied',
u'frappe.oauth.delete_oauth2_data',
u'frappe.integrations.doctype.razorpay_settings.razorpay_settings.capture_payment',
u'frappe.twofactor.delete_all_barcodes_for_users'],
u'daily': [u'frappe.email.queue.clear_outbox',
u'frappe.desk.notifications.clear_notifications',
u'frappe.core.doctype.error_log.error_log.set_old_logs_as_seen',
u'frappe.desk.doctype.event.event.send_event_digest',
u'frappe.sessions.clear_expired_sessions',
u'frappe.email.doctype.email_alert.email_alert.trigger_daily_alerts',
u'frappe.async.remove_old_task_logs',
u'frappe.utils.scheduler.disable_scheduler_on_expiry',
u'frappe.utils.scheduler.restrict_scheduler_events_if_dormant',
u'frappe.email.doctype.auto_email_report.auto_email_report.send_daily',
u'frappe.core.doctype.feedback_request.feedback_request.delete_feedback_request',
u'frappe.core.doctype.authentication_log.authentication_log.clear_authentication_logs'],
u'daily_long': [u'frappe.integrations.doctype.dropbox_settings.dropbox_settings.take_backups_daily',
u'frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_daily'],
u'hourly': [u'frappe.model.utils.link_count.update_link_count',
u'frappe.model.utils.user_settings.sync_user_settings',
u'frappe.utils.error.collect_error_snapshots',
u'frappe.desk.page.backups.backups.delete_downloadable_backups',
u'frappe.limits.update_space_usage'],
u'monthly': [u'frappe.email.doctype.auto_email_report.auto_email_report.send_monthly'],
u'monthly_long': [u'frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_monthly'],
u'weekly_long': [u'frappe.integrations.doctype.dropbox_settings.dropbox_settings.take_backups_weekly',
u'frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_weekly']}
- Now since the tasks were missing do
sudo supervisorctl restart all
now I got an issue here as some processes exited with error as below:
[frappe@www frappe-bench]$ sudo supervisorctl restart all
frappe-bench-workers:frappe-bench-frappe-schedule: stopped
frappe-bench-workers:frappe-bench-frappe-default-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-long-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-short-worker-0: stopped
frappe-bench-web:frappe-bench-frappe-web: stopped
frappe-bench-web:frappe-bench-node-socketio: stopped
frappe-bench-redis:frappe-bench-redis-queue: stopped
frappe-bench-redis:frappe-bench-redis-cache: stopped
frappe-bench-redis:frappe-bench-redis-socketio: stopped
frappe-bench-workers:frappe-bench-frappe-schedule: started
frappe-bench-workers:frappe-bench-frappe-default-worker-0: ERROR (abnormal termination)
frappe-bench-workers:frappe-bench-frappe-long-worker-0: started
frappe-bench-workers:frappe-bench-frappe-short-worker-0: started
frappe-bench-web:frappe-bench-frappe-web: started
frappe-bench-redis:frappe-bench-redis-cache: started
frappe-bench-redis:frappe-bench-redis-socketio: started
frappe-bench-web:frappe-bench-node-socketio: started
frappe-bench-redis:frappe-bench-redis-queue: started
[frappe@www frappe-bench]$
- Now do run this command to restart the failed process
sudo supervisorctl restart frappe-bench-workers:frappe-bench-frappe-default-worker-0
see how we referenced the process name.
- Now again in site console after restarting the processes check with
frappe.cache().get_value('scheduler_events')
Now if all the frappe-bench workers were started properly then you should be able to see your custom_app processes in the console output.