[Solved] Would like some help altering how backups work

I know the syntax for running a backup in ERPNext:

bench backup-all-sites

However, I would like to be able to alter where the backup file (.sql.gz) is saved and set my own backup file name.

I like the way the automatic backups work right now. I do not want to alter that function. Instead, I want to schedule a separate set of backups to a different directory that run every hour during regular business hours. These alternate backups are the files I want to send to a different directory and set my own file names.

Are there any ‘undocumented’ command line switches for the bench backup command that would allow setting the file name and/or destination?

If not, then is there an alternate way to do this maybe using cron?

Any pointers would be greatly appreciated. The ultimate goal is to drop hourly backups in a new directory on the half hour and then use git or some other function to move them to a different server for safe keeping. Recent backbone outages in the vicinity of my hosting provider made me painfully aware of the need to have an alternate server in another part of the country ready and waiting “just in case.” :grin:

Thanks,

BKM

1 Like

You can probably do most of this with a rsync backup script …

While ‘rsync’ seems to be quite versatile for moving files around, it doesn’t really have the ability (at least that I am aware of) to run the database backup as that would seem to require the use of MariaDB commands.

So… maybe the correct questions would be:

Is there a way to run the ERPNext database backup using a single command line that would include all the credentials needed, the source database, the target directory and the target sql.gz file name?

-OR-

If there is an rsync approach to this, could someone enlighten me on the correct syntax because the rsync MAN page is fairly sparse and only indicates pretty much how to address source an destination of file copies.

Thanks
BKM

to add, i dont know if there is already an option but a way to keep multiple rolling backups and also complete backups and maybe even something like image backups or similar because shit happens and no one wants to deal with having to get 3 guys near the end of the busy season to get the system back up and running

1 Like

ERPNext already does this in the folder ~/frappe-bench/sites/site1.local/private/backups
It keeps the latest 3 or 4 backups that are run every 6 hours.

Not sure what you mean by this… the database is all you should need, except maybe the files you created but a search here in the forum on backups will take you to multiple waysy to get that.

If you are looking for a complete erpnext backup with your data and files in place, then that would really be an OS level backup which you would have to handle with your VPS provider.

Yeah, that is what your VPS provider would be able to help you get in place.

Anything further automated than this is very expensive and would involve automatic rollovers to different servers in different locations by using a mirroring plan of some sort. It would cost more to setup something like this that it would cost to host 2 separate sites.

It is the 2 separate sites approach that I am working on now.

  • I can already get a full backup of my primary site from my hosting account
  • I can restore that site backup to a second site and have it idling all the time
  • As long as I do not customize the primary site, the 2 sites will be functional identical
  • If I need to move in a hurry, I can just restore the latest backup to the 2nd site and have the users log into the secondary URL until I get the first one fixed.

Hence, the reason for my initial question at the top of this thread. I want to auto script the primary site to run a backup of my own naming convention every hour on the half hour. Then all I have to do is use some other file moving convention to copy the fresh backup file to the secondary site every hour on the 40min mark and overwrite the last file each time.

Using this approach, I can have a secondary server that only takes minutes to start up and would never be more than about an hour from the primary host. It is much easier to get people to re-do the last hour of their work than it is to tell then they lost the entire day or even the last 6 hours (which would be almost the whole day anyway).

So, that is the plan. I am just looking for the right way to generate the hourly backup on the primary server.

Hope this helps…

BKM

you’re right but i am running and planning to run production self hosted in my office so there goes my vps provider. i am also running a few bitnami instances which make it easier to run and manage on virtualbox but the core backup issues remain.

what i mean by complete backups is a disaster proof backups that could be reused in a moments notice without considerations for configs and settings.

Ahh… Well then for you it would be even easier!

Just create a second server box on your network. If I can ever figure out the ability to run a backup on my own naming convention, then you would use the same except you wouln’t have to save the file to one server and copy it across country to the other. In your case you could just save it to the second server directly since it would be on the same internal network.

That way in the event of a failure, you just run the restore command on the second server and keep on running. At worst case your users might have to re-do their last hour of work again.

Full automation in your case would require a special router and a lot of internal addressing to be automatically switched (in addition to the second server). That is pretty sophisticated stuff and usually only attempted by companies with large IT staff in place to keep everything running smoothly.

Alternately you could save yourself some effort by using a RAID disk array in your server to compensate for disk failures.

Trying to host EPRNext yourself on a local server is full of it’s own problems. The more users and tasks you add the harder it is to keep the performance up to a usable standard. In the cloud you can simply add resources in a few seconds and keep on running. The cost of the VPS server is far less than the manpower it takes to keep tweaking a local server. If a local server goes down, it takes more manpower to go fix it in the computer room than it does to spin up a cloud server.

Just my opinion… your mileage may vary :grin:

BKM

BKM

Anyway… Still looking for a way to run backups using my own directory and naming convention.

BKM

you’re that’s fine. i am aware of ‘some’ of the issues. i am currently serving 4 softwares from one windows 7 machine kept as a server, some 3 instances running on Virtualbox. it “just works” and i have daily system image backups on a network drive and hourly file backups on another. i have a working LAN for the bulk of the work and a wireless network that acts as a bridge to the LAN where users can rdp over to the server and work. i have a load of around 10 people working and there isnt much active monitoring that needs to be done besides managing physical or rdp issues that usually crop up.

that said, i think i can get it to work. i wont be switching my production to a vm unless i know it is battle ready so in my case backups are very important and i’m assuming they are too to you.
lets see what comes up

oh… a quick question. how much does a vps hosting outside of the official erpnext cloud costs on a monthly basis? i havent tried that because of internet concerns and stuff.
say 7 users and 100 companies? whats your monthly bill

The closest one I have to your configuration is 24 users and 2 companies with 12 different warehouse locations. It runs on a KVM type VPS host with 5gb RAM and 125gb disk space running 2 CPU cores. It has pretty good performance with those resources and cost about $50 USD per month. The users are spread out over about a 150 miles radius at any given time. (Several are mobile sales reps.) There are many customizations and modifications to my system software. The Bitnami instances I tried last year were pretty bad and had a difficult time trying to do any custom work, so I abandoned them. I tried a local host box and found that it was a real pain in the butt to keep current. I didn’t want to use VM instances on a local box, because it required to many resources just to get the VM OS to run so i did direct installs on a debian box for a while.

Once I got started with the VPS hosting idea, there was no comparison. If I break something I can spin up a fresh host in seconds instead of hours. I ended up doing all my testing and customization work on Google Cloud Platform. Once it was perfetced, I just bought VPS hosting from a good service provider.

The Google (GCP) development and testing work was free with the trial account, and the paid host was about $50. With the cost that low, it didn’t make sense to try to do it in-house any longer. The time savings alone pays me back for NOT doing it in house. BTW… I now keep a paid GCP account just for testing. It gets a restore from the live server every night in case I have to run simulations on reported problems. The GCP test environment cost me bout $20 per month on average.

BKM

Still looking for a way to run a command line ERPNExt backup where I can use my own file naming.

BKM

Ok, I gues I asked too many questions in this version of my query to get any responses. So, after much more refining and reposting my question, I was able to finally solve the problem here:

I hope this helps anyone else trying to solve the backups puzzle. :grin:

BKM