Attachments - internal or external?

Continuing the discussion from Migrate attached files from ERPNext Server to external storage (like s3 i.e.):

is there any rule of thumb about whether attachments are ‘better off’ being stored internally on the ERPNext server or with an external filestorage solution (like s3) and only using links to that files inside ERPNext?

in a current case I am handling all financial transactions should have digital copies (jpg, png, pdf, html, txt) of the related documents (invoices, remittance advices, etc.) attached and I am wondering which road to take.

In case migration of local attachments to en external service was simple and possible in bulk (as per the linked topic) you’d probably start locally, but if such a migration is a manual 1by1 process you’d probably make up your mind on this rather sooner then later

Hello!

Im facing the same situation, what solution ends working best for you?

Hi @vrms

I’m actually interested in this as well… especially if you took the external route. I’d like to know how that’s working for you so far

Cheers

I am using the default way with attachments saved on the internal storage of my ERPNext instance for the time being.

Likewise there is no experience to share thus far. I am reaching limits of storage slowly though, so this matter may come up again sooner or later (incl. migration potentially).

All I can say thus far is that using something like s3 comes with an authentication challenge

S3 buckets would be a game changer. Hard disk space is quite expensive for storage purposes. Also s3 will allow large file attachments like hi-res photos and videos, and delivery through CDN.

S3 has python api - boto3.

An idea is Frappe can have FieldType s3 or more generic bucket where attachments will be uploaded to the assigned s3 bucket. (Attachments on the Doctype level will still be internal). So, you can have fields named Photos, Videos, and Documents (Birth Certificate, Business Registration) in a doctype and these large attachments can be sent to different s3 buckets.

The functionality for s3 buckets is already in ERPNext, I think because you can store backups in s3.

Frappe has the flexibility to be an Enterprise Content Management System like Alfresco. The ability to store to efficient and cheap storage like s3 - allowing for CDN delivery would be a great resource to have.

Another question related to that:
How do use backup?
I am assuming without attachment.

There would be bg bandwidth involve if you do backups with all files every 6 hours or so. So how to deal with backup when lots of storage is involved.

actually I was advised once there is a custom App for this https://github.com/zerodhatech/frappe-attachments-s3. I have not tried this myself yet though. I guess the ideal scenario would be to have add this to the core code by convincing

  1. the author (or anyone else) to deliver a PR and
  2. the maintainers to accept it

(both can be an exhausting exercise).

3 Likes

Wow! Thanks. I will give this a look