Try ERPNext Buy Support Partners Foundation Foundation Members

Error in Amazon MWS integration: RuntimeError: dictionary changed size during iteration

I am attempting to integrate Amazon MWS with a new ERPNext installation. Install went fine with no issues… running in developer mode.

When I fill in the Amazon MWS details then hit ‘Synch Products’, the following error pops up:

RuntimeError: dictionary changed size during iteration

This is what I see on the bench terminal (masked IP as 0.0.0.0):

23:10:08 web.1 | 0.0.0.0 - - [07/Aug/2019 23:10:08] “POST /api/method/runserverobj HTTP/1.1” 500 -
23:10:08 web.1 | INFO:werkzeug:0.0.0.0 - - [07/Aug/2019 23:10:08] “POST /api/method/runserverobj HTTP/1.1” 500 -

Any ideas what is going on?

Hi welcome! This means probably an illegal operation.

To know what is going on you need to refer to the logs on either server.

You need to look for a error stack traceback for that RuntimeError.

Thanks - I am posting the log for this error below:

[ERROR] 2019-08-08 21:21:09,887 | /home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py:
Could not take error snapshot: startswith first arg must be str or a tuple of str, not bytes
Traceback (most recent call last):
  File "/home/horballs/frappe-bench/apps/frappe/frappe/app.py", line 60, in application
    response = frappe.api.handle()
  File "/home/horballs/frappe-bench/apps/frappe/frappe/api.py", line 55, in handle
    return frappe.handler.handle()
  File "/home/horballs/frappe-bench/apps/frappe/frappe/handler.py", line 21, in handle
    data = execute_cmd(cmd)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/handler.py", line 56, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/__init__.py", line 1036, in call
    return fn(*args, **newargs)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/handler.py", line 84, in runserverobj
    frappe.desk.form.run_method.runserverobj(method, docs=docs, dt=dt, dn=dn, arg=arg, args=args)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/desk/form/run_method.py", line 36, in runserverobj
    r = doc.run_method(method)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py", line 786, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py", line 1055, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py", line 1038, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py", line 780, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_settings.py", line 22, in get_products_details
    get_products_details()
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_methods.py", line 23, in get_products_details
    report_id = request_and_fetch_report_id("_GET_FLAT_FILE_OPEN_LISTINGS_DATA_", None, None, market_place_list)
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_methods.py", line 90, in request_and_fetch_report_id
    marketplaceids=marketplaceids)
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py", line 373, in request_report
    return self.make_request(data)
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py", line 164, in make_request
    extra_data = remove_empty(extra_data)
  File "/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py", line 74, in remove_empty
    for key in d.keys():
RuntimeError: dictionary changed size during iteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py", line 36, in make_error_snapshot
    snapshot = get_snapshot(exception)
  File "/home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py", line 131, in get_snapshot
    if type(value)==str and value.startswith(b"u'"):
TypeError: startswith first arg must be str or a tuple of str, not bytes

ok the problem is in this loop

Maybe try this: for i in list(d) that apparently works in python 2.7 or 3.x

This explains the problem https://stackoverflow.com/questions/11941817/how-to-avoid-runtimeerror-dictionary-changed-size-during-iteration-error

Great - seemed to fix that issue. Brings us a new one:

AttributeError: module 'urllib' has no attribute 'quote'

Here is the trace:

ERROR] 2019-08-11 18:56:25,779 | /home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py:
Could not take error snapshot: startswith first arg must be str or a tuple of str, not bytes
Traceback (most recent call last):
File “/home/horballs/frappe-bench/apps/frappe/frappe/app.py”, line 60, in application
response = frappe.api.handle()
File “/home/horballs/frappe-bench/apps/frappe/frappe/api.py”, line 55, in handle
return frappe.handler.handle()
File “/home/horballs/frappe-bench/apps/frappe/frappe/handler.py”, line 21, in handle
data = execute_cmd(cmd)
File “/home/horballs/frappe-bench/apps/frappe/frappe/handler.py”, line 56, in execute_cmd
return frappe.call(method, **frappe.form_dict)
File “/home/horballs/frappe-bench/apps/frappe/frappe/init.py”, line 1036, in call
return fn(*args, **newargs)
File “/home/horballs/frappe-bench/apps/frappe/frappe/handler.py”, line 84, in runserverobj
frappe.desk.form.run_method.runserverobj(method, docs=docs, dt=dt, dn=dn, arg=arg, args=args)
File “/home/horballs/frappe-bench/apps/frappe/frappe/desk/form/run_method.py”, line 36, in runserverobj
r = doc.run_method(method)
File “/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py”, line 786, in run_method
out = Document.hook(fn)(self, *args, **kwargs)
File “/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py”, line 1055, in composer
return composed(self, method, *args, **kwargs)
File “/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py”, line 1038, in runner
add_to_return_value(self, fn(self, *args, **kwargs))
File “/home/horballs/frappe-bench/apps/frappe/frappe/model/document.py”, line 780, in
fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_settings.py”, line 22, in get_products_details
get_products_details()
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_methods.py”, line 23, in get_products_details
report_id = request_and_fetch_report_id(“GET_FLAT_FILE_OPEN_LISTINGS_DATA”, None, None, market_place_list)
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_methods.py”, line 90, in request_and_fetch_report_id
marketplaceids=marketplaceids)
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py”, line 373, in request_report
return self.make_request(data)
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py”, line 175, in make_request
request_description = ‘&’.join([’%s=%s’ % (k, urllib.quote(params[k], safe=’-.~’).encode(‘utf-8’)) for k in sorted(params)])
File “/home/horballs/frappe-bench/apps/erpnext/erpnext/erpnext_integrations/doctype/amazon_mws_settings/amazon_mws_api.py”, line 175, in
request_description = ‘&’.join([’%s=%s’ % (k, urllib.quote(params[k], safe=’-
.~’).encode(‘utf-8’)) for k in sorted(params)])
AttributeError: module ‘urllib’ has no attribute ‘quote’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py”, line 36, in make_error_snapshot
snapshot = get_snapshot(exception)
File “/home/horballs/frappe-bench/apps/frappe/frappe/utils/error.py”, line 131, in get_snapshot
if type(value)==str and value.startswith(b"u’"):
TypeError: startswith first arg must be str or a tuple of str, not bytes

Using python 2.7.12 by the way

Glad to help! Just to give you a hint, next look up this for pointers :slight_smile:

I started by trying to figure out the issue - but that error is a python 3.0 error and I have 2.7. I added this as a workaround but it does not fix it:

try:
    from urllib import quote  # Python 2.X
except ImportError:
    from urllib.parse import quote  # Python 3+

Then try this:

from six.moves.urllib.parse import quote

as this suggests https://stackoverflow.com/questions/31827012/python-importing-urllib-quote/43162895

Doesn’t fix it - same issue.

Seems like a really odd error, since I am using python 2.7 and urllib didn’t change until python 3.0.