Missing invoices in Offline PoS

Still on the issues encountered by retailers in trying to use ERPNext for business .

Over the last two years or so, ERPNext has made a lot of progress in PoS, from introducing offline PoS mode (thank you @rohit_w ) , to designing a better user interface as well as the awesome job done by @fkardame and his team on PoS Bahrain.

We have also eliminated some nagging issues like duplicated/triplicated invoices (thank you @tundebabzy) and slow systems.

But we have one issue that seems to be defying a logical solution. This is the challenge of missing invoices from offline PoS transactions.

A sales invoice is generated from offline PoS, the receipt is printed by the cashier, cache is NOT cleared and yet 1 out of 150 invoices do not sync to the server and also are not on the local client. They just disappear.

If you do not run a high traffic retail business you might not witness this anomaly. But if your location does from 300 and above receipts per day, you will experience it.

Can we as a team have a look at this? I am stumped because I do not even have an inkling of what is causing this. For duplicated invoices we had an idea of the root cause and fixed this, but how do you fix a problem whose cause you do not know?

@wale and @noetico you are my partners in this Retail ERPNext journey, any ideas ? Do you also experience this?

I am truly stumped .

This is an unfortunate issue and may well be responsible for what overs some people have complained about.

One question comes to mind; is this the same on say Chrome and Firefox? It will be a good test to try another browser for a good period to see if the occurrence rate is similar to what it is currently. This is important just to eliminate the possibility of some code incompatibility between the POS code and browser implementations.

From the erpnext side of things; hard to tell what, but must be in the javascript code. Will I be correct to say the team uses some of the most cutting edge code to achieve such great feats, so this could be a bug that needs a dedicated look.

Is it a silly idea to think of having the option - option in addition to current design - an sqlite database for those using this offline pos in critical operations. So installers can drop in an sqlite file on each computer and make the necessary configurations and this serves as the ‘cache’ database for offline invoices. Sqlite can always be accessed to pull out and reconcile records that are missing on the server, this will be accessed just locally as recommended by sqlite.

Just a thought. This is a huge trouble for any business.

1 Like

Hi, I can tell you we have seen this happen on Chrome and FireFox. Issue is most probably not related browser. We have seen invoices missing for less than 100 transactions.

Our developer also couldn’t find anything as there is no log of the missing invoice. It was only when customer comes to return, when cash is more than expected or the card balance is higher than daily sales that one knows of the missing invoice.

We feel each POS should have a SQL. One option is PouchDB
If may or may not fix the issue. If it does not work then we would know that it is not the browser at fault. Right now the root cause of this issue in unknown.

3 Likes

Sqlite would obviously make offline mode of pos more reliable.

  • It eliminates danger of clearing local storage cache in browser (after all cache is meant to be a cache)
  • It provides each POS machine its own history of transactions for that particular machine because each machine has its own sqlite database.
2 Likes

Precisely, you have captured it very well. Hopefully this can be considered…

As with all things JavaScript, I always have this fear of ‘brittleness’. What if some very little part of the script somewhere just has some uncaught exceptions and simply fails to update the cache. More reason a local cache db is a good idea, so POS is pushing using reliable SQL statements and on updating code just makes standard queries from sqlite and updates main database. Another advantage as stated above also is that it will finally secure the offline POS as admins can plan to secure the local sqlite file within the os, as it is we can’t avoid the fact that relying on the browser cache is actually a very very expensive risk.

1 Like

@noetico @Joseph_Marie_Alba @Muzzy

What is the next steps with respect to local db? I am willing to contribute to this project if it can be kick started.

Another advantage of a local db is that we will not need to rely on “syncing” to determine the end of day figure for each cashier, we can write a script querying the local db for this.

Anyone with an idea of how to proceed ? We cannot all just sit back and absorb this huge risk!

@Joseph_Marie_Alba

Explanations ? Am not that techy

I’ve never handled any such projects on erpnext, but maybe you can post as a job (if that’s how it’s done).

Maybe we can call it something like ‘POSQlite for ERPNext’ or ‘LoDB for ERPNext’, push the objectives on the job board and see what the experienced developers think.

Truly it will be one of the greatest things to happen to erpnext… It may be under estimated by many not in the retail segment but for those who know, POS issues are the worst, it can outrightly discredit a software, instantly dropping confidence.

It’s a crucial point that relies heavily on the integrity of the system, unlike office style sales orders, invoices and other types where the user can see what is happening to a greater extent, on POS the cashier has minimal access and the business relies on the absolute correctness of transactions posted from POS.

@Muzzy @Joseph_Marie_Alba

Could you help draft a description of the job? I want to post as a Job but am not sure I can effectively write out the technical requirements.

Would you mind helping to write this and I will post the Job and see if we have any takers

Regards

I have been reading around for sometime now about local storage. I am not a developer so cant comment on the code aspect. However, I have few questions in ERPNext deployment

  1. Is the localstoarge Persistent or Temporary Type.
  2. What is the maximum limit of storage. Every browser type has different storage capacity.
  3. Does ERPNext POS data fill the local storage to near of the limit or somewhere close to it.
  4. What usually should be the ideal storage limit of the computer on which the POS is running,

Maybe the issue has nothing to do with POS rather it would be type of browser and its localstorage settings.

@olamide_shodunke I have asked my developer to look finding way of using SQLite or Pouch db instead of localstorage. PM me if you are interested in funding some part of the job. Any financial help is always welcome.

1 Like

I have PMd you

Are you trying to figure out if the missing invoices is because the browser cache storage limit was breached ?

Regards

Really not sure at this stage. It is one of the possibilities. Many factors contribute to the breach like item master data, users using same browser for other work like watching YouTube, available disk space, cache settings etc.

Its one storage for everything user does on the browser which can be using ERPNext and other applications.

Maybe one way is to monitor the size of the local storage using something like the suggestion here but embedding the size in the POS page so that you can start to see when it reaches the danger point of space consumption:

See space consumption for a dormant POS profile
image

One can then stress test by posting a large number of POS entries online/offline and seeing where things start to disappear or what happens when the upper limit of localstorage is hit before a synchronise occurs

I am currently checking this out.

The syncing of Sales Invoice is a single request. (e.g. 150 invoices to one API call)
Inserting 150 invoices could be a huge overload for a server.

My idea is that if we can manage to breakdown 150 invoices into 50 invoices of 3 API calls.
Maybe, it would fix the problem.

Regards,

Ivan

Makes sense

@iRaySpace is this something you want to take a look at?

I have mistake, recently checkout the core code. Current implementation is one API call synced 49 invoices.

Could we reduce this to 20? or 10?

Maybe we test this scenarios and see if this reduces or eliminates the missing data issue

Where do we make this change(s)

@rohit_w you are the acknowledged father of Retail PoS :slightly_smiling_face::slightly_smiling_face:

Can you help with

  • Either pointing us in the direction of how to reduce the maximum number of invoices each API calls at once or

  • Any other approach to resolving this missing invoice issue? It is a real thing and a big deal

If it will take time we are willing to pay for the hours involved

Regards