When we have about 15 users logged in (mostly POS Users), ERPNext becomes slow. Trying to open anything takes time and the screen Grey’s out for a while as though the system is busy with something… and this happens even during intervals when no sales are getting synced (we use offline POS)
This is happening on an AWS server with 16G RAM and 4 CPUs. There’s enough free RAM and Innodb Buffer Pool size is set to 8GB
What could be the issue and how do we improve this?
Also, do sales coming in from Offline POS transactions get handled as background jobs? If not, shouldn’t they?
How many do you have now? The max recommended is 2*(number of cpu cores) + 1, but since you’re also running the DB and other things on the server, you’ll need to readjust. Test some different values and see where the sweet spot is.
Trust you’re doing great. Seems like there’s a correlation between gunicorn workers and CPU usage. Yesterday evening I bumped the number of workers up to 5 as suggested and there seemed to be a significant improvement but as of this morning it became very slow again. I checked the CPU Utilization % and it was practically hitting 100
I’ve adjusted to 4 workers now but performance is still not okay. I’m beginning to consider moving this account off AWS to a platform where I can get good service and higher capacity at less cost. I still think the current capacity should be more than adequate for our use but not sure what else to tweak
The downside of this on AWS is that they don’t allow custom instance sizes like Google Cloud. The DB needs more memory, while the App server needs more CPU. On AWS, if you split the app and db, you’ll most likely overprovision memory on the App server. Once you reach a certain CPU need, this becomes quite expensive.
Thanks for the input and suggestions. I decided to try a 32GB RAM, 8 CPU Server to see if that would solve the problem. While there’s an improvement, there’s still a significant lag during peak periods. Could it be a fundamental flaw in the design of the POS? (I’m referring to offline POS here as this is what we use)
Why isn’t syncing of transactions from the POS handled as a background job? Is there a technical reason why this isn’t feasible? Other ERPNext users with a decent number of concurrent POS users have also complained of performance issues. I’d like to know if there’s a technical reason for not using background jobs before I open a Github issue for this
Unfortunately, I’m not going to be able to comment further on this because there are too many unknowns and I don’t use the POS. Maybe install an APM solution like NewRelic to see where the slowdowns are. As a test, try disabling the offline mode - do the slowdowns remain?