Event Streaming

[fkardame]

@olamide_shodunke

The hardware the “server” is running on is basically the same configuration as the PCs the staff use for their day to day tasks.

Possible options

• Erpnext running on virtual machine with Windows host.
• Linux desktop environment with erpnext running on it.
• Erpnext running on wsl.

All the solution will need a piece of hardware no matter a simple machine or a server machine and it will need regular maintenance.
The setup time for individual configure of each location will increase automatically.
Its just an overhead to maintain stability, it will require higher support man hours.
My major concern

• Additional expense for hardware and maintenance.
• Extra man hours will be needed to maintain this type of deployment.

This deployment is ideal for large enterprise with high volume of documents but not for an sme with high number of remote locations with low volume of documents.

Will try to test the existing feature in the coming week and share my feedback.
Btw with this feature we will try to compete with Microsoft dynamic, sap and oracle, so it is a plus point.

[olamide_shodunke]

Hello @rucha_mahabal

I keep getting the following error message and it prevents sales invoice from showing up at the consumer site

Traceback (most recent call last):
File “/opt/bench/develop/apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py”, line 205, in sync
set_update(update, producer_site)
File “/opt/bench/develop/apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py”, line 253, in set_update
local_doc.save()
File “/opt/bench/develop/apps/frappe/frappe/model/document.py”, line 281, in save
return self._save(*args, **kwargs)
File “/opt/bench/develop/apps/frappe/frappe/model/document.py”, line 323, in _save
self.validate_update_after_submit()
File “/opt/bench/develop/apps/frappe/frappe/model/document.py”, line 739, in validate_update_after_submit
self._validate_update_after_submit()
File “/opt/bench/develop/apps/frappe/frappe/model/base_document.py”, line 669, in _validate_update_after_submit
frappe.UpdateAfterSubmitError)
File “/opt/bench/develop/apps/frappe/frappe/init.py”, line 385, in throw
msgprint(msg, raise_exception=exc, title=title, indicator=‘red’, is_minimizable=is_minimizable)
File “/opt/bench/develop/apps/frappe/frappe/init.py”, line 364, in msgprint
_raise_exception()
File “/opt/bench/develop/apps/frappe/frappe/init.py”, line 315, in _raise_exception
raise raise_exception(msg)
frappe.exceptions.UpdateAfterSubmitError: Not allowed to change Date after submission

[rucha_mahabal]

Was this sales invoice cancelled and amended on the producer site?

[olamide_shodunke]

No it was not. It was a fresh transaction.

But now that you mention it, In my tests I also found out that if two customers were merged into one. Transactions from the resulting customer always give an error.

But this sales invoice was a fresh transaction on a unique customer

[sjain]

@rucha_mahabal I am trying to sync my POS invoice from producer to consumer. I have configured both the instances with IP address only and it is throwing below error.

frappe.event_streaming.doctype.event_producer.event_producer.pull_from_node

Traceback (most recent call last):
File “/opt/erpnext/erpnext/apps/frappe/frappe/utils/background_jobs.py”, line 100, in execute_job
method(**kwargs)
File “/opt/erpnext/erpnext/apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py”, line 185, in pull_from_node
event_producer = frappe.get_doc(‘Event Producer’, event_producer)
File “/opt/erpnext/erpnext/apps/frappe/frappe/init.py”, line 790, in get_doc
doc = frappe.model.document.get_doc(*args, **kwargs)
File “/opt/erpnext/erpnext/apps/frappe/frappe/model/document.py”, line 75, in get_doc
return controller(*args, **kwargs)
File “/opt/erpnext/erpnext/apps/frappe/frappe/model/document.py”, line 113, in init
self.load_from_db()
File “/opt/erpnext/erpnext/apps/frappe/frappe/model/document.py”, line 156, in load_from_db
frappe.throw((“{0} {1} not found”).format((self.doctype), self.name), frappe.DoesNotExistError)
File “/opt/erpnext/erpnext/apps/frappe/frappe/init.py”, line 409, in throw
msgprint(msg, raise_exception=exc, title=title, indicator=‘red’, is_minimizable=is_minimizable, wide=wide, as_list=as_list)
File “/opt/erpnext/erpnext/apps/frappe/frappe/init.py”, line 388, in msgprint
_raise_exception()
File “/opt/erpnext/erpnext/apps/frappe/frappe/init.py”, line 342, in _raise_exception
raise raise_exception(msg)
frappe.exceptions.DoesNotExistError: Event Producer http://pos-producer not found

[Izat_Afghan]

I have also same issue.

Please anyone help us.

[JayRam]

Do you know if Event Streaming will work on two instances on the same server that’s setup for port based multi-tenancy?

Thanks

Jay

[JayRam]

And what about if I have to make a Sales Invoice with Update Stock Checked? Will that mess up the Stock Quantities, the Warehouse, etc. etc.

I am going to try it out. But just checking.

Thanks

Jay

[iqbalfeb]

Yes, it works. In my case, I need to add “host_name” : “http://localhost:port number” key to the site_config file

[dufani1]

Can you please help me how can I configure event streaming???

[CA_B.C_Chechani]

@mohitchechani try out and share your experiences here

[mohitchechani]

@CA_B.C_Chechani I have created same user in the Event Producer sites and Event Consumer Sites.
Event Prodcuer site was successfully saved with 1 custom doc to be synced.
When I tried to add another document I get the error “TypeError: ‘NoneType’ object is not iterable”
Console Log is as follows
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/app.py”, line 68, in application
response = frappe.api.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/api.py”, line 55, in handle
return frappe.handler.handle()
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 31, in handle
data = execute_cmd(cmd)
File “/home/frappe/frappe-bench/apps/frappe/frappe/handler.py”, line 67, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 1198, in call
return fn(*args, **newargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py”, line 21, in savedocs
doc.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 285, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 337, in _save
self.run_post_save_methods()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 1001, in run_post_save_methods
self.run_method(“on_update”)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 860, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 1158, in composer
return composed(self, method, *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 1141, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 854, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py”, line 50, in on_update
self.update_event_consumer()
File “/home/frappe/frappe-bench/apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py”, line 137, in update_event_consumer
event_consumer = frappe._dict(event_consumer)
TypeError: ‘NoneType’ object is not iterable

Also the event consumer was not created

[iqbalfeb]

Do you use different server for consumer and producer ? Or do you use multitenancy (dns or port based) ?

If you use port based multi tenancy, maybe this thread will help you

Event Consumer Error TypeError:'NoneType'

Hi, while trying to save Event Consumer (changing the Approval Status to Approve) I got error message TypeError : ‘NoneType’ object is not iterable I’m doing this on ERPNext v13.8.0 with Frappe v13.8.1. Producer site is localhost:81 and consumer site is localhost:80, the DocType I’m trying to stream is Stock Entry I tried to find more about the error on worker error log and web error log but can’t find anything. Any help will be much appreciated, thanks in advance

[adam26d]

I just keep getting the error Failed to connect to the Event Producer site. Retry after some time.

[adam26d]

And pymysql.err.InternalError: (1054, "Unknown column 'doctype' in 'where clause'")

Possibly because I’ve got different frappe versions, not by choice

[Mel_George]

Hey Jay,

Were you able to test this setup out. I was trying to set up eventstreaming across sites on the same server (multi-tenant setup) but was facing issues with the same

[iMoshi]

I can’t seem to get it working. I’ve thought of syncing “User” doctype with a no go. I have set up the producer, same users with System Manager role. Then I’ve created 2 same doctypes with a single field (not requiring any mapping) and added it to the Event consumer site and approved it from the producer site. Then I have created documents and waited, seeing I was not getting anything on the consumer site, I’ve also created data in it hoping it might sync to the producer site, no go…

What am I missing? Is there a settings to alter the syncing timing…? Event Sync Log is empty, Event Update Log has the “Created” and “Updated” logs.

Edit: I’ve seen my Error Log list and I get the error. I have set up the producer with https with my error log says http:// is not found…

Is this a known bug or should I create an issue on Github?

[mangroliya]

@iMoshi

Edit: I’ve seen my Error Log list and I get the error. I have set up the producer with https with my error log says http:// is not found…

Hi, did you find any solution for this ? I am also getting same error.

[manal_alsubaei]

welcome to everyone I thank all of you I benefited from these solutions and now I have the following challenge when creating the pos profile after i add its doctype to producer event

Traceback (most recent call last): File "apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py", line 277, in sync set_insert(update, producer_site, event_producer.name) File "apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py", line 310, in set_insert sync_dependencies(doc, producer_site) File "apps/frappe/frappe/event_streaming/doctype/event_producer/event_producer.py", line 492, in sync_dependencies dependency.insert() File "apps/frappe/frappe/model/document.py", line 259, in insert self.run_before_save_methods() File "apps/frappe/frappe/model/document.py", line 1045, in run_before_save_methods self.run_method("validate") File "apps/frappe/frappe/model/document.py", line 914, in run_method out = Document.hook(fn)(self, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1264, in composer return composed(self, method, *args, **kwargs) File "apps/frappe/frappe/model/document.py", line 1246, in runner add_to_return_value(self, fn(self, *args, **kwargs)) File "apps/frappe/frappe/model/document.py", line 911, in fn return method_object(*args, **kwargs) File "apps/erpnext/erpnext/accounts/doctype/account/account.py", line 48, in validate self.validate_root_details() File "apps/erpnext/erpnext/accounts/doctype/account/account.py", line 120, in validate_root_details frappe.throw(_("The root account {0} must be a group").format(frappe.bold(self.name))) File "apps/frappe/frappe/__init__.py", line 526, in throw msgprint( File "apps/frappe/frappe/__init__.py", line 494, in msgprint _raise_exception() File "apps/frappe/frappe/__init__.py", line 443, in _raise_exception raise raise_exception(msg) frappe.exceptions.ValidationError: The root account <strong>Write Off - FC</strong> must be a group

[adam26d]

So event streaming is buggy. To get it to work, you need to alter some core frappe/erpnext code (I need to revisit what I did some time back). Prepare to spend a day debugging.