Try ERPNext Buy Support Partners Foundation

Update fails - TypeError: can't pickle module objects

Hey guys, anyone has any idea where this is coming from ? in a previous post the idea was this is related to redis but it seems redis is running correctly on my server.

Trying to update the system on the latest version-12 developing branch on an up to date ubuntu 18

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/utils/bench_helper.py", line 97, in <module>
    main()
  File "/opt/erpnext/frappe12/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/opt/erpnext/frappe12/env/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/commands/site.py", line 233, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website, skip_failing=skip_failing)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/migrate.py", line 48, in migrate
    frappe.modules.patch_handler.run_all(skip_failing)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/modules/patch_handler.py", line 41, in run_all
    run_patch(patch)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/modules/patch_handler.py", line 30, in run_patch
    if not run_single(patchmodule = patch):
  File "/opt/erpnext/frappe12/apps/frappe/frappe/modules/patch_handler.py", line 71, in run_single
    return execute_patch(patchmodule, method, methodargs)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/modules/patch_handler.py", line 77, in execute_patch
    block_user(True)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/modules/patch_handler.py", line 127, in block_user
    frappe.db.set_global('__session_status', block and 'stop' or None)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/database/database.py", line 698, in set_global
    self.set_default(key, val, user)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/database/database.py", line 712, in set_default
    frappe.defaults.set_default(key, val, parent, parenttype)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/defaults.py", line 134, in set_default
    add_default(key, value, parent)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/defaults.py", line 147, in add_default
    d.insert(ignore_permissions=True)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/model/document.py", line 222, in insert
    self.run_method("before_insert")
  File "/opt/erpnext/frappe12/apps/frappe/frappe/model/document.py", line 787, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/model/document.py", line 1052, in composer
    doc_events = frappe.get_doc_hooks()
  File "/opt/erpnext/frappe12/apps/frappe/frappe/__init__.py", line 899, in get_doc_hooks
    hooks = get_hooks('doc_events', {})
  File "/opt/erpnext/frappe12/apps/frappe/frappe/__init__.py", line 946, in get_hooks
    hooks = _dict(cache().get_value("app_hooks", load_app_hooks))
  File "/opt/erpnext/frappe12/apps/frappe/frappe/utils/redis_wrapper.py", line 80, in get_value
    self.set_value(original_key, val, user=user)
  File "/opt/erpnext/frappe12/apps/frappe/frappe/utils/redis_wrapper.py", line 48, in set_value
    self.set(key, pickle.dumps(val))
TypeError: can't pickle module objects

Did you get any solution on this?

Any solution for this? i am also facing problem with this error

if custom code in custom app’s hooks.py includes imported module, there is such error, change the system conf to activate develop mode, then system will skip pickling hooks.py.

1 Like

Yes, this is the issue that I experienced.

In developer mode, the hooks are not cached which is why it works fine normally.

With developer mode off, hooks are cached. So when you import a module it’s trying to cache the dict() containing a module key which isn’t possible and that causes the pickle error.

Another workaround besides just turning on developer mode, is you can rewrite your imports to be like this: “import xyz as _xyz”

This works because the caching function skips any keys that begin with underscores.

2 Likes

seems more cute workaround.

Update: confirm it is working
:+1:

Hi
Can you please guide where to make this change? Which file and what is its location

@Manan_Shah i think in hooks.py in your custom app

Thanks for the reply
I see these two lines

from __future__ import unicode_literals

from . import __version__ as app_version

Do I need to change anything in these two lines?

What is the

referring to?

Hi
We need to change this line

to

from . import xyz as _xyz and run bench update, this solved the problem

I am getting this same error when trying to run bench migrate.
developer_mode is set as 1 in the site_config.json, yet this issue occurs.
Any help?

The only way I can get it to work is by changing the following in frappe/frappe/__init__.py,

no_cache = conf.developer_mode or False

if app_name:
	hooks = _dict(load_app_hooks(app_name))
else:
	if no_cache:
		hooks = _dict(load_app_hooks())
	else:
		hooks = _dict(cache().get_value("app_hooks", load_app_hooks))

It works if I explicitly set if True instead of if no_cache
Though it should work as no_cache is being set from conf.developer_mode, shouldn’t it? :thinking: