I have an installation with “System settings > Number of backups” set to 20.
bench backup --with-files
But instead of keeping 20 last files, only 2 files are kept. This seems to be unrelated to the value entered into the “Number of backups”.
ERPNext: v8.5.2 (master)
Frappe Framework: v8.5.5 (master)
Can someone confirm this as a bug or am I missing something?
Same issue on VM instance.
Would you mind creating a github issue for this. Seems to be replicating in test account as well.
Thanks for your cooperation.
If you do a
bench update --upgrade
it reverts all backups to the last two (2) instance.
Please close this issue.
An update on this: this problem is still present in Frappe 8.10.
It cannot be reproduce when manually running
in this scenario it works just fine. But when using the automated backup (nightly), it only keeps 2 nights… To reproduce, set number of backups to any value > 2. Then wait three days and check number of backups. It will be 2. Should be as many as defined.
What is the difference between the nightly backup (crontab, see above) and manual bench backup? Are the --with-files posing an issue?
in a prod scenario you would backup those to a separated backup server anyway (instead ofkeeping them on the same machine). Once you do this the problem is not as severe any longer (even though the automated backup should keep as many copies as you tell it to )
There are different ways to look at this even with external backup storage
- separate server to pickup/receive the files (after bench backup): ok, agreed
- mount different drive into sites/…/private/backup: problem exists (only 2 backups)
- overarching backup (backup separate process over complete frappe structure): problem exists but compensated because the overarching is having the right history
The nice thing about this backup feature is that the end-customer can download them easily. Therefore, I still see this as a bug.
This issue still prevails in Frappe v13, System is not keeping more than 5 backups even if I set the number higher
Hi. Found same issue on V11 and V12. Not more than 5 backups are stored. No idea why it doesn’t go higher as per settings.
Anybody managed to make it work to go higher than 5 backups?
To my understanding, the
System Settings > Number of backups is for auto backup. It will keep daily backups. So for
3 backups (the standard) it will do auto backup every 8 hours (24÷3).
bench backup will do the backup of current state. So only 1 set of backup (4 files if with files: db, files, private-files, site_config.json).
Here is my findings for anyone still dealing with auto backup issues
Auto backup function is triggered for every 6 hours by a cron job present in
[frappe_user]. The backup period is determined by the cron expression of cronjob, For eg:
*/6 * * *
0 */6 * * * cd /home/frappe/frappe-bench && /usr/local/bin/bench --verbose --site all backup >> /home/frappe/frappe-bench/logs/backup.log 2>&1 # bench auto backups set for every 6 hours
The number of backups to be kept is determined by two different variables.
backup_limit field in
System Settings limits the maximum number of backups, this field is only used to delete the extra backups when you are refreshing the
Download Backups page.
site_config.json determines how much time a backup is to be kept and if undefined, it will default to 24. This condition is checked every time a
bench backup is performed so the reason for auto backups maxing out at 5 is the default value of
To solve this inconsistency, You can ignore the
System Settings by assigning it a very large number(Eg: 1000) and use
keep_backups_for_hours for limiting backups.
CC: @aakvatech @Muzzy @rahy
Thank you @manasan for the explanation.
But I found these info a bit confusing:
Which one is the default? 24 hours or 5 hours?
Or 5 is the number of backups acitivty per day (24 hrs) = every 4.8hrs?
keep_backups_for_hours setting in site_config.json should be manually created?
Because I plan to make the field in System Settings to read only and/or hidden so user can’t change it (affecting the storage capacity planning).
If we do this, I think it will be even more unpredictable.
backup_limit = 1000
keep_backups_for_hours = 6 hrs
Now, if in that 6 hrs we do
bench backup 50 times there will be more than 50 backups are available. This can be too much storage.
keep_backups_for_hours default value is 24, which translates to 24 hours. To change the
keep_backups_for_hours, you need to manually define it in the
5 is the number of backups preserved, if default value of 24 hours is unchanged. The default backup interval is 6 hours so 24 hours/6 hours will result in 4 intervals and the number of backups will be 5 because of arithmetic(4 gaps and 5 poles)
If you plan to hide the
System Settings make sure its default value is set to some higher number.
I edit my comment above when your last answer posted.
Can you please comment on my last comment also?
In this case these 50 backups will get deleted on next
bench backup after 6 hours ageing from the file creation time.
backup_limit will not help you here because it deletes the extra backups only when you are refreshing the
Download Backups page. So until you visit the backups page in your desk, you will have the high storage issue.
So in short,
backup_limit is pretty useless
Can these be used in common_sites_config?