Dashboard issue heatmap in v9

[joelios]

Hi All

I’m currently working on a pull request with which the following features will be implemented and the following bugs will be fixed:

Feature:

• Configurable style of heatmap (continuous or discret) in Global Defaults
• The heatmap reflects all transactions against the linked records in the corresponding dashboard

Fixes:

• The heatmap will show all 7 days of a week, not just 6
• The marks on the heatmap will be shown

Today, my feature/fix branch of Frappe and ERPNext (Branch: joelios-heatmap), reflect all features and fixes above. But at the moment only the heatmap of the doctype Customer will reflect all transactions of linked records in the doctype. I will create an pull request of this stand to merge it into master.

Then i will work out another pull request to update all other heatmaps (not just customer), to merge it in a feature step into master.
To know all doctypes, which has a heatmap and should reflect all linked transactions from the corresponding dashboard, i need your help! You have to tell me all affected doctype of what i have planned. For this, i will open e new, clean, topic in this forum.
As soon as i have all inputs from you and coded all this, i will create the second pull request to improve all other doctypes except customer.
@rmehta What do you think about my plans? Do you agree with it?

@lasalesi thanks for your feedback! Nice to know.

Edit:
Frappe Pull request
ERPNext Pull Request

[joelios]

For further discussions and or inputs please see:

[Improvement] Heatmap

Hi All This topic is the continuation of Dashboard issue heatmap in v9. Summary: Several issues have been identified: Heatmap shows only 6 days per week instead of 7 Heatmap shows no marks based on transaction Several feature request have been identified: Style of heatmap is fixed to “Continuous”, but it should be selectable between “Continuous” and “Discrete” It should be possible to set whether the dashboard section is collapsed or not as default In each doctype the marks are set based…

[bghayad]

Hello @joelios

You are doing a great job and thanks a lot for you. This is wonderful.
From my side: at this time, it is 1 PM and date is 5th April, I am able to see the square at the heatmap for day 5th April, but I am not able to see any marks related to yesterday or today activities.

For student attendance (education module): it seems that the marks are not working at the heatmap totally.

What could be the reason for my troubles with the marks (that I am not able to see recently marks, I mean for last few days “recently days”)?

Regards
Bilal

[joelios]

Hi @bghayad

I had a little mistake in my time shift correction.
Please try my lattest pushes on github, now it should work propperly.
Frappe → Branch: joelios-heatmap
ERPNext-> Branch: joelios-heatmap

But please pull both of me, frappe as well as erpnext.
And dont forget to run

$ bench build
$ bench migrate
$ bench restart

after pull

cheers

[bghayad]

Yes @joelios
Wonderful.
Every thing is working fine. BRAVO.
But still Student heatmap (and I know that still you did not do any thing with it) is not working properly. Actually it is showing the marks on the heatmap but in wrong date.

I need to add one more thing for your previous post, it is required to do the following command:

$ bench migrate

Because you need to create the field Heatmap Style in the database.

I have one point out of this post:
Do you have activity regarding grid? Because we would if it possible to resolve the list view problem (as we need to be able to add more fields and to have the horizontal scroll bar to navigate left and right).

Regards
Bilal

[joelios]

I’m glad to hear that it works for you too!

@bghayad

I need to add one more thing for your previous post, it is required to do the following command:

$ bench migrate

Because you need to create the field Heatmap Style in the database.

Yes, sorry, you’re absolut right!

@bghayad

But still Student heatmap (and I know that still you did not do any thing with it) is not working properly. Actually it is showing the marks on the heatmap but in wrong date.

I will look at the heatmap of student in the near future. In the new topic you see my backlog and where I am currently.

@bghayad

I have one point out of this post:
Do you have activity regarding grid? Because we would if it possible to resolve the list view problem (as we need to be able to add more fields and to have the horizontal scroll bar to navigate left and right).

As soon as the heatmap topic is finished, I can look at this.

cheers

[joelios]

Hi @bghayad

@bghayad

But still Student heatmap (and I know that still you did not do any thing with it) is not working properly. Actually it is showing the marks on the heatmap but in wrong date.

I’ve finaly reworked the student heatmap also.
If you pull my latest commit, the student heatmap should work correct

Cheers!

[bghayad]

Hello @joelios
Great full. Amazing.
It came after long long waiting time.

So it needs pull request to be merged.

Regards
Bilal

[joelios]

Pull requests are placed. They waiting to be merged

[bghayad]

Thanks a lot for you @joelios

I was need to ask you about grid and if you can help to resolve the list view problem.
But first of all: what kind of activity you have about grid?
Second: Can we resolve the list view problem: because we would if it possible to resolve the list view problem (as we need to be able to add more fields and to have a horizontal scroll bar to navigate left and right).

Regards
Bilal

[joelios]

Hi @bghayad

With changes on list_render.js, it is possible to add mor fields to listview:

but adding a horizontal scrollbar would involve significant changes to list_render.js!
My question; Do you really need to force more fields in the listview? you can solve the same also over a report …?

Cheers

[bghayad]

Hello @joelios
Wonderful and amazing. Thanks a lot for you on this big effort and great work.
I will try list_render.js as this will resolve maybe 80% of the problem.
Having horizontal scroll bar to navigate left and right for more columns is giving powerful to manage the business because the user will have a look for all the records with very enough information to decide which record is need to work on it. This will prevent from the opposite case where user will need to go through each record and check if this record is the needed or not and coming back to list and repeat this work.

One more feature is also important to have strength while working with the records:
Inside the document, there should be Next and Back button to be able to go for the next record or previous record without coming back to the list (of course this to be within the filtered records).

But let me tell you that you helped us a lot by directing us to list_render.js and as I told you, this resolved 80% of the problem. I will go for the posts that we placed about this problem and direct them for this link.

Regards
Bilal

[joelios]

Youre welcome!

The code youre looking for is in line 204 (if (me.total_colspans < 12)):

	docfields_in_list_view.forEach(function (d) {
		if (overridden.includes(d.fieldname) || d.fieldname === me.title_field) {
			return;
		}
		if (me.total_colspans < 12) {
			me.add_column(d);
		}
	}); 

and line 249:

		// Limit number of columns to 4
		this.columns = this.columns.slice(0, 4);

best regards
Joelios

[bghayad]

Hello @joelios

I would to notify about heatmap for education, certainly for student attendance.
I am getting the following error when I add attendance for the student and coming to student document which contains the heatmap:

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/app.py”, line 62, in application
response = frappe.handler.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 22, in handle
data = execute_cmd(cmd)
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 53, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 939, in call
return fn(*args, **newargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/notifications.py”, line 274, in get_open_count
out[‘timeline_data’] = module.get_timeline_data(doctype, name)
File “/home/frappe/frappe-bench/apps/erpnext/erpnext/education/doctype/student/student.py”, line 63, in get_timeline_data
timestamp = get_timestamp(date)
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 126, in get_timestamp
return time.mktime(getdate(date).timetuple())
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 42, in getdate
return parser.parse(string_date).date()
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 1182, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 556, in parse
res, skipped_tokens = self._parse(timestr, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 675, in _parse
l = _timelex.split(timestr) # Splits the timestr into tokens
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 192, in split
return list(cls(s))
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 61, in init
‘{itype}’.format(itype=instream.class.name))
TypeError: Parser must be a string or character stream, not int

I think that the problem maybe related to the unix_timestamp as the following line is causing the error:

timestamp = get_timestamp(date)

Again, it is happening only with student attendance transactions.

Regards
Bilal

[bghayad]

Hello @joelios

I am sorry, for education: I discovered that I did not do the frappe changes. I will re fix and come back to you.

Regards
Bilal

[bghayad]

Hello @joelios
I confirmed the problem that is related to the education heatmap that you proposed, but it is only related to the attendance and it seems that it is related to the unix_timestamp at the query which cause error that is related to the timestamp = get_timestamp(date) as in the following traceback:

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/app.py”, line 62, in application
response = frappe.handler.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 22, in handle
data = execute_cmd(cmd)
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 53, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 939, in call
return fn(*args, **newargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/notifications.py”, line 274, in get_open_count
out[‘timeline_data’] = module.get_timeline_data(doctype, name)
File “/home/frappe/frappe-bench/apps/erpnext/erpnext/education/doctype/student/student.py”, line 63, in get_timeline_data
timestamp = get_timestamp(date)
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 126, in get_timestamp
return time.mktime(getdate(date).timetuple())
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/data.py”, line 42, in getdate
return parser.parse(string_date).date()
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 1182, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 556, in parse
res, skipped_tokens = self._parse(timestr, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 675, in _parse
l = _timelex.split(timestr) # Splits the timestr into tokens
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 192, in split
return list(cls(s))
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/dateutil/parser.py”, line 61, in init
’{itype}’.format(itype=instream.class.name))
TypeError: Parser must be a string or character stream, not int

Can you please help?
Regards
Bilal

[johnwick]

Hello,

where this list_render.js is?

[clarkej]

Good question it seems nowhere to be found

frappe@ubuntu:~/frappe-bench$ find . -name ‘list_render.js’

[johnwick]

@joelios

list_render.js,

Found it. Now name is list_renderer.js