Amazon S3 BackUp App

Hello :smile:

I have made a plugin/app that would backup your database, public and private files to amazon s3. If you have any issues please just post it in https://github.com/ccfiel/s3_backup/issues

How to Install

1) Install boto and filechunkio packages
    a) pip install boto
    b) pip install filechunkio

2) bench get-app s3_backup https://github.com/ccfiel/s3_backup.git
3) bench install-app s3_backup
19 Likes

@ccfiel awesome! More reason for us to start building the app store :smile:

4 Likes

@A Mehta is it possible to put my app to integration menu under setup? and no icon shortcut?

@ccfiel sure. You need to create a file called config/setup.py

See this example: https://github.com/frappe/erpnext_shopify/blob/master/erpnext_shopify/config/setup.py

@anand thanks! works great!

@rmehta yes we really need a app store to encourage more dev to make one :smile:

Next challenged to take “Cash Flow” hope I can do it! so that I can contribute more to ERPNext.

1 Like

@ccfiel did you create the docs for your app? https://frappe.github.io/frappe/user/guides/app-development/generating-docs.html

@A Mehta not yet. I will create one

Good Job!

1 Like

Hi, ccfiel, very good apps.

It is possible to set 3 backups daily and keep more then 1 week of backup?

@synertic

for now its just minimum is 1 backup per day.

I am planning to use your app. Is this Multitenant compatible? I have n number of sites and I was wondering how this will back up for all sites.

Thanks a lot for this kind of app.

Raj

Yes It is.

Thank you for your support.

@ccfiel

Hey! i am trying the steps as stated above but i am faced with the error message below

bench install-app s3_backup

Installing s3_backup…
Updating DocTypes for s3_backup : [========================================]
Traceback (most recent call last):
File “/usr/lib/python2.7/runpy.py”, line 162, in _run_module_as_main
main”, fname, loader, pkg_name)
File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 94, in
main()
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 18, in main
click.Group(commands=commands)(prog_name=‘bench’)
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/home/ubuntu/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/commands/init.py”, line 24, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/commands/site.py”, line 160, in install_app
_install_app(app, verbose=context.verbose)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/installer.py”, line 145, in install_app
add_to_installed_apps(name)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/installer.py”, line 166, in add_to_installed_apps
post_install(rebuild_website)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/installer.py”, line 230, in post_install
init_singles()
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/installer.py”, line 248, in init_singles
doc = frappe.new_doc(single)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/init.py”, line 610, in new_doc
return get_new_doc(doctype, parent_doc, parentfield, as_dict=as_dict)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/model/create_new.py”, line 20, in get_new_doc
frappe.local.new_doc_templates[doctype] = make_new_doc(doctype)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/model/create_new.py”, line 38, in make_new_doc
“docstatus”: 0
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/init.py”, line 634, in get_doc
return frappe.model.document.get_doc(*args, **kwargs)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/model/document.py”, line 66, in get_doc
controller = get_controller(doctype)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/model/base_document.py”, line 35, in get_controller
module = load_doctype_module(doctype, module_name)
File “/home/ubuntu/frappe-bench/apps/frappe/frappe/modules/utils.py”, line 177, in load_doctype_module
raise ImportError(‘Module import failed for {0} ({1})’.format(doctype, module_name + ’ Error: ’ + str(e)))
ImportError: Module import failed for Student Batch Creation Tool (erpnext.schools.doctype.student_batch_creation_tool.student_batch_creation_tool Error: No module named student_batch_creation_tool.student_batch_creation_tool)

its not the app that causes the error is something else look at the error

ImportError: Module import failed for Student Batch Creation Tool

You need first fix the error first before you install the app.

Hey

Please can you help with a guide on setting up a bucket in S3, I have tried setting up one but i am getting error message below

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py”, line 22, in savedocs
doc.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 256, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 290, in _save
self.run_before_save_methods()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 809, in run_before_save_methods
self.run_method(“validate”)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 702, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 964, in composer
return composed(self, method, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 947, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 696, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/integrations/doctype/s3_backup_settings/s3_backup_settings.py”, line 34, in validate
conn.create_bucket(Bucket=bucket_lower)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 312, in _api_call
return self._make_api_call(operation_name, kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 579, in _make_api_call
api_params, operation_model, context=request_context)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 631, in _convert_to_request_dict
params=api_params, model=operation_model, context=context)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/hooks.py”, line 227, in emit
return self._emit(event_name, kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/hooks.py”, line 210, in emit
response = handler(**kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/handlers.py”, line 212, in validate_bucket_name
raise ParamValidationError(report=error_msg)
ParamValidationError: Parameter validation failed:
Invalid bucket name “arn:aws:s3:::xlevel”: Bucket name must match the regex "^[a-zA-Z0-9.-
]{1,255}$"

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/app.py”, line 57, in application
response = frappe.handler.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 22, in handle
data = execute_cmd(cmd)
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 53, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 935, in call
return fn(*args, **newargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py”, line 22, in savedocs
doc.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 256, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 290, in _save
self.run_before_save_methods()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 809, in run_before_save_methods
self.run_method(“validate”)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 702, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 964, in composer
return composed(self, method, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 947, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 696, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/integrations/doctype/s3_backup_settings/s3_backup_settings.py”, line 34, in validate
conn.create_bucket(Bucket=bucket_lower)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 312, in _api_call
return self._make_api_call(operation_name, kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 579, in _make_api_call
api_params, operation_model, context=request_context)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/client.py”, line 631, in _convert_to_request_dict
params=api_params, model=operation_model, context=context)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/hooks.py”, line 227, in emit
return self._emit(event_name, kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/hooks.py”, line 210, in emit
response = handler(**kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/botocore/handlers.py”, line 212, in validate_bucket_name
raise ParamValidationError(report=error_msg)
ParamValidationError: Parameter validation failed:
Invalid bucket name “arn:aws:s3:::xlevel”: Bucket name must match the regex "^[a-zA-Z0-9.-
]{1,255}$"

Bucket name should be existing in s3. Create a bucket first

Thank you @ccfiel its working now
Please is there anyway i can schedule the backup to happen maybe twice daily? or perhaps change the time the backup occurs ?

Thank you for your prompt assist

For now it’s not possible.