S3 Backup integration not working

Have been corresponding here:

but thought I might share here because its now available in integration and looks like someone updated the original functionality?

1 Like

So I may be confused when looking in setup I see this at the bottom under integrations:

selecting leads to this screen ignore the bucket I copied that in before I took screenshot it was not populated:

how is this different from these screens I arrive at by clicking integrations on desk:

There clearly is some duplicity with this installation and what is required to use?

when I click save under setup intergrations s3backup this erro occurs:
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py”, line 22, in savedocs
doc.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 256, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 290, in _save
self.run_before_save_methods()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 809, in run_before_save_methods
self.run_method(“validate”)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 702, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 964, in composer
return composed(self, method, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 947, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 696, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/s3_backup/s3_backup/amazon_s3_backup/doctype/amazon_s3_settings/amazon_s3_settings.py”, line 13, in validate
from boto.s3.connection import S3Connection
ImportError: No module named boto.s3.connection

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/app.py”, line 57, in application
response = frappe.handler.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 22, in handle
data = execute_cmd(cmd)
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 53, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 935, in call
return fn(*args, **newargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py”, line 22, in savedocs
doc.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 256, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 290, in _save
self.run_before_save_methods()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 809, in run_before_save_methods
self.run_method(“validate”)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 702, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 964, in composer
return composed(self, method, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 947, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 696, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/s3_backup/s3_backup/amazon_s3_backup/doctype/amazon_s3_settings/amazon_s3_settings.py”, line 13, in validate
from boto.s3.connection import S3Connection
ImportError: No module named boto.s3.connection

yet boto installed:

python3-boto
Class
P-T
Version
2.40.0-1ubuntu1
Vendor
Ubuntu Developers ubuntu-devel-discuss@lists.ubuntu.com
Architecture
all
Installed

I installed and reinstalled following these updated instructions:

https://erpnext.org/app/amazon-s3-backup

Received these errors I think we need to verify how this is supposed to be installed as there are two different sets of instructions as listed above in this post. clarity allows adoption and not pollution that needs to be cleaned up later.

bench get-app https://github.com/ccfiel/s3_backup
INFO:bench.app:getting app s3_backup
INFO:bench.utils:git clone https://github.com/ccfiel/s3_backup --depth 1 --origin upstream
Cloning into ‘s3_backup’…
remote: Counting objects: 24, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 24 (delta 0), reused 16 (delta 0), pack-reused 0
Unpacking objects: 100% (24/24), done.
Checking connectivity… done.
(‘installing’, u’s3_backup’)
INFO:bench.app:installing s3_backup
INFO:bench.utils:./env/bin/pip install -q -e ./apps/s3_backup --no-cache-dir
/home/frappe/frappe-bench/apps/frappe/frappe/build.py:106: UserWarning: Source /home/frappe/frappe-bench/apps/s3_backup/s3_backup/public does not exists.
warnings.warn(‘Source {source} does not exists.’.format(source = source))
/home/frappe/frappe-bench/apps/frappe/frappe/build.py:106: UserWarning: Source /home/frappe/frappe-bench/apps/s3_backup/s3_backup/docs does not exists.
warnings.warn(‘Source {source} does not exists.’.format(source = source))
Wrote css/frappe-web.css - 65.11 KB
Wrote js/frappe-web.min.js - 132.05 KB
Wrote js/control.min.js - 66.3 KB
Wrote js/dialog.min.js - 111.18 KB
Wrote css/desk.min.css - 293.82 KB
Wrote css/frappe-rtl.css - 32.49 KB
Wrote js/libs.min.js - 861.86 KB
Wrote js/desk.min.js - 506.66 KB
Wrote css/module.min.css - 2.08 KB
Wrote css/form.min.css - 4.47 KB
Wrote js/form.min.js - 195.27 KB
Wrote css/list.min.css - 14.72 KB
Wrote js/list.min.js - 143.46 KB
Wrote css/report.min.css - 7.89 KB
Wrote js/report.min.js - 260.05 KB
Wrote js/web_form.min.js - 247.55 KB
Wrote css/web_form.css - 24.42 KB
Wrote js/print_format_v3.min.js - 23.39 KB
Wrote css/erpnext.css - 8 KB
Wrote js/erpnext-web.min.js - 3.73 KB
Wrote js/erpnext.min.js - 137.32 KB
Wrote js/item-dashboard.min.js - 7.91 KB
INFO:bench.utils:sudo supervisorctl restart frappe-bench-workers: frappe-bench-web:
frappe-bench-workers:frappe-bench-frappe-schedule: stopped
frappe-bench-workers:frappe-bench-frappe-long-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-default-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-short-worker-0: stopped
frappe-bench-web:frappe-bench-node-socketio: stopped
frappe-bench-web:frappe-bench-frappe-web: stopped
frappe-bench-workers:frappe-bench-frappe-schedule: started
frappe-bench-workers:frappe-bench-frappe-default-worker-0: started
frappe-bench-workers:frappe-bench-frappe-long-worker-0: started
frappe-bench-workers:frappe-bench-frappe-short-worker-0: started
frappe-bench-web:frappe-bench-frappe-web: started
frappe-bench-web:frappe-bench-node-socketio: started

So you run these below as instructed here https://github.com/bailabs/s3_backup

bench get-app s3_backup https://github.com/bailabs/s3_backup.git
bench install-app s3_backup

But this repo code runs instead https://github.com/ccfiel/s3_backup ??

Once you sort that out, request that documentation revision https://erpnext.org/app/amazon-s3-backup here: https://github.com/erpnext/foundation/issues/new

@clarkej that is what I am saying the developers instructions on its GitHub https://github.com/bailabs/s3_backup do not match those listed on apps page for ERPNext https://erpnext.org/app/amazon-s3-backup.

looked at this but was unable to resolve despite boto claiming to be installed:

I have tried both and left with nothing working very confusing compared to dropbox which just works out of the box…

since its been added as an app and shows up in ERPNext was hoping from some clarity as I started at the developers site then saw it was added to ERPNext as an app?

It is being added to Core but this is not complete as yet.

I got this working without any issues about a month back.

Have you installed the right package for boto ? I think it’s something like python-boto. The boto package is used by most apps that back up to S3 storage so if you’re problem is specific to here it may be worth looking at other installation notes for similar backup apps such as Duplicity for example that uses boto. I

tired both versions 2 and 3 ofcourse there were not specifics in the readme file:

python-boto 2.40.0-1ubuntu1

python-boto3 1.2.2-2

python-botocore 1.4.70-1~16.10.0

Requirement already up-to-date: boto in /usr/local/lib/python2.7/dist-packages

Well your install logs reports the directories here not found - so manually create them and see what happens?

Julian possibly can confirm those are present and in use in his instance?

Created them and does nothing in addition I still don’t understand why there are two sets of directions/source and why it would be published as an app on ERPNext production if the instructions are invalid inconsistent or don’t work.
This was on dec 15 in the original thread:

Not sure how a year has passed and this ends up as a working app in ERPNext production with little or no documentation.

I have mentioned this before for successful adoption we need quality control.

I have reviewed pip, the python versions including running with version specific switches and I still have a problem with despite trying several recommendations pertaining to other products with the boto library. The developer does not clarify which version of python or boto libraries are required for their app and what versions of Linux this has been tested on, And several days with out response suggest to me it is not ready to be included in the ERPNext as a app and should not have been posted there as it is still in development and is not ready for master.

Installed Apps
ERPNext: v9.2.7 (master)
Frappe Framework: v9.2.9 (master)
Amazon S3 Backup: v0.0.1 (master)

@imllc,

I am the developer and I assure you the backup is working perfectly. Many of our clients and community members have used the app to backup their data. I have already given you a reply regarding your concern https://github.com/ccfiel/s3_backup/issues/6. This is opensource so do not expect an immediate reply nor assume the developer will solve all your problems. A piece of advice, do not rant or generalize coz it doesn’t help your case. Just because you weren’t able to make it work or the developer did not reply as fast as you want, doesn’t mean the app is defective. I’m pretty sure you’ve made missteps in your installation along the way. If you want somebody to give you an immediate answer when you encounter problems, then I suggest you hire experts to help you https://erpnext.org/service-providers.

2 Likes

Agreed, nobody said this stuff is easy. But please skip the drama imllc unless perhaps it is to share a learning…

3 Likes

@ccfiel
I have no doubt that it is working at other locations and its a good app that was not my point and offered you and ERPNext a different perspective.

My point was ( as is I said in the first paragraph) that there were two different sets of instructions and sources. Before it should be included in master those issues should have been cleared up before ERPNext accepted it. Master apps should pretty much work out of the box and have specific documentation all other should stay on dev for the adventurous.

I fully understand open source and expect nothing for free and contribute readily where I can. It would be good that before something be added to master there are sufficient resources and documentation so that community as a whole can be pointed in the right direction and self service that is a goal of open source, working as a team.

When its on master one should not have to reengineer the app or be confused as to which documentation to follow. If they make mistakes on steps that’s on them.

One example of why I had difficulty is why does the app show up in “integration” and “setup/integration” with two separate set of fields. perhap the two instructions and two different sources caused this or maybe its by design. . I have a working drop box backup, if there is some additional information I can post or test that would help let me know.

@ccfiel on another note my response to you here https://github.com/ccfiel/s3_backup/issues/6 states that something has changed in the app since you developed it because I was able to replicate numerous times the creation of the bucket. So if you did not build the functionality as point person of the app you control what gets added so whom did so I can reach out to the person whom modified it? This change was documented with screenshots I added to the post for your reference. Or perhaps someone at ERPNext can comment on the addition to Apps.

@clarkej my post wasn’t meant to be dramatic only point out the inconsistency and document the issues as I did on the submittal at GitHub. As you know there have been several “enhancements” that I have commented on that were not fully developed and should not been released to master much like the inbox which abruptly took away existing functionality…

Hoping the new mantra of the foundation will continue to evolve better quality control which will beneficial to all concerned and increase it use. I personally feel its one of the best erp apps I have every used and want to help it remain that way…