GitHub - zerodha/frappe-attachments-s3: A frappe app to upload file

ERPNext
1

I was installing this app but couldn’t make it work. I tried this on a ERPNext & Frappe Version 13 Instance that already uses S3 for Database and File Backups.

There are a bunch of fields in the configuration DocType. (Fields like: aws key, aws secret, bucket name, aws region, folder name), but I wasn’t too sure the format in which the inputs need to be given for it to work.

Can somebody please help? I had to uninstall and remove the app and missed capturing a screenshot of the configuration file.

But I did have the Error Traceback:

Traceback (most recent call last):
File “env/lib/python3.7/site-packages/boto3/s3/transfer.py”, line 275, in upload_file
future.result()
File “env/lib/python3.7/site-packages/s3transfer/futures.py”, line 73, in result
return self._coordinator.result()
File “env/lib/python3.7/site-packages/s3transfer/futures.py”, line 233, in result
raise self._exception
File “env/lib/python3.7/site-packages/s3transfer/tasks.py”, line 126, in call
return self._execute_main(kwargs)
File “env/lib/python3.7/site-packages/s3transfer/tasks.py”, line 150, in _execute_main
return_value = self._main(**kwargs)
File “env/lib/python3.7/site-packages/s3transfer/upload.py”, line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File “env/lib/python3.7/site-packages/botocore/client.py”, line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File “env/lib/python3.7/site-packages/botocore/client.py”, line 612, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “apps/frappe_s3_attachment/frappe_s3_attachment/controller.py”, line 132, in upload_files_to_s3_with_key
“ContentType”: content_type,
File “env/lib/python3.7/site-packages/boto3/s3/inject.py”, line 110, in upload_file
extra_args=ExtraArgs, callback=Callback)
File “env/lib/python3.7/site-packages/boto3/s3/transfer.py”, line 283, in upload_file
filename, ‘/’.join([bucket, key]), e))
boto3.exceptions.S3UploadFailedError: Failed to upload ./site1.local/public/files/BTPL_Logo_New.png to bindalbackups/erpnext-files//2022/05/08/Website Settings/6U8BC3T6_BTPL_Logo_New.png: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “apps/frappe/frappe/app.py”, line 69, in application
response = frappe.api.handle()
File “apps/frappe/frappe/api.py”, line 55, in handle
return frappe.handler.handle()
File “apps/frappe/frappe/handler.py”, line 37, in handle
data = execute_cmd(cmd)
File “apps/frappe/frappe/handler.py”, line 75, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “apps/frappe/frappe/init.py”, line 1447, in call
return fn(*args, **newargs)
File “apps/frappe_s3_attachment/frappe_s3_attachment/controller.py”, line 322, in migrate_existing_files
upload_existing_files_s3(file[‘name’], file[‘file_name’])
File “apps/frappe_s3_attachment/frappe_s3_attachment/controller.py”, line 278, in upload_existing_files_s3
parent_name
File “apps/frappe_s3_attachment/frappe_s3_attachment/controller.py”, line 139, in upload_files_to_s3_with_key
frappe.throw(frappe._(“File Upload Failed. Please try again.”))
File “apps/frappe/frappe/init.py”, line 510, in throw
as_list=as_list,
File “apps/frappe/frappe/init.py”, line 478, in msgprint
_raise_exception()
File “apps/frappe/frappe/init.py”, line 433, in _raise_exception
raise raise_exception(msg)
frappe.exceptions.ValidationError: File Upload Failed. Please try again.

Thanks

Jay

2

It looks like the Zerodha app requires Access Control Lists. I believe those are off by default. Have you tried enabling them on your bucket?

3

Okay Peter. I will try that and report back here.

Thanks

Jay

4

Hi Jay,

Were you able to install in?

5

No Nasir! I didn’t persist.

Thanks

Jay

1 Like
6

I’ve played with it a bit and it worked fine for me. As discussed in this thread, ACLs are necessary. I like the app, and I’d be tempted to use it on my production installations, but I’m a bit concerned about the longterm support prospects.

7

Another app that allows you to use external S3 compatible buckets is this one:

4 Likes
8

I must say this looks really good ! Well done :+1:t3:

Kind regards,

1 Like
9

Thank you very much @wale :slight_smile:

10

Whether this app saves and fetches documnets directly to s3 or first saves to frappe and keep on sending to s3. I will try this too

11

If you have selected an external storage for the folder you are uploading files, they are directly uploaded to your external storage, so not saved into Frappe files folder.

12

Its great, It means they will be directly written to s3, whether external storage configurable like google drive, one drive or nextcloud other than s3

13

just few observations:

  • minio py client is used, frappe already has boto3
  • put_object() takes the file to backend from browser and then puts it to s3.

wishlist: use aws-sdk https://www.npmjs.com/package/@aws-sdk/client-s3 in frontend/browser. Directly push file from browser to s3 using temp token generated by backend using keys stored in backend