ERPNext code instrumentation

We want to do performance analysis of some of the transactions (like create sales invoice, cancel old sales invoice etc.) and want to log time taken by each method to know which method is getting called how many times and the time taken for each call.

e.g when an older invoice is cancelled and amended, stock ledger, general ledger entries are re-written.

We wanted to understand the time taken by each method and see if there are any opportunities in tuning high time consuming methods.

Is there any way to instrument the ERPNext code and log time taken by each method? We come across a library called equip GitHub - neuroo/equip: Python bytecode instrumentation library with which we can insert the code in to python bytecode at runtime.

Any help/pointer on where and how to inject equip instrumentation code in ErpNext would really help.

Hey @Sachin_Mane, that’s a cool little python package! For a first pass, you could build a simple counter that uses the standard library time and the methods time.time() or time.clock() they have separate benefits, here’s an article that covers it, I am not an expert.
Just the phrase “bytecode” makes me a little nervous, so a proof of concept on one doctype method with time might make more sense. I’ve played with this a little bit and while it didn’t end up working for what I needed, it worked pretty much the way you’re describing… Also, I would approach this with a decorator syntax in both the time and equip implementations.

@time_this_method
def on_submit(self):
            """ db calls "" 

In another thread, we were discussing the Frappe ORM and the possibility of postgres as a backend for performance reasons. Ultimately the easy-fix is to move away from the ORM when you need a performance boost and use frappe.db.sql("""select... Personally, I’ve made a resolution to try to do this for every database lookup that’s in a loop in my programming moving forward.

One note is you don’t necessarily always need to switch to frappe.db.sql in a loop. You can also use frappe.db.get_list or frappe.db.fetch_as_dict or something similar to pull many records to loop through. Just commenting here because I believe a previous comment of mine may have led to this statement.

https://frappe.io/docs/current/api/frappe.database

@Sachin_Mane An alternative is to use an option like NewRelic - it’s simple to install using their Python Integration.

I hear you @felix. It’s not necessary, it’s not even a best-practices sort of suggestion, it’s a possible route when something is taking too long. Personally, I’m also using it as a way to enforce that I practice my SQL and get better at it. No warranties.

e.g when an older invoice is cancelled and amended, stock ledger, general ledger entries are re-written.

There is a plan to implement ‘Immutable Ledger’ to solve this problem.

We want to do performance analysis of some of the transactions

You can use a profiler like cProfile.

1 Like