Migrating from V11 to V12 issues with patch

jokermao · September 13, 2019, 8:21pm

Just migrating a database from latest V11 to latest V12

I do have about 3K items on the TabItem_Tax
After about 30min I get always this error message:

Executing erpnext.patches.v12_0.move_item_tax_to_item_tax_template in mvs (_92dd934f94047209)
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/bench/erpnext/apps/frappe/frappe/utils/bench_helper.py", line 97, in <module>
    main()
  File "/opt/bench/erpnext/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/opt/bench/erpnext/env/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/commands/site.py", line 233, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website, skip_failing=skip_failing)
  File "/opt/bench/erpnext/apps/frappe/frappe/migrate.py", line 48, in migrate
    frappe.modules.patch_handler.run_all(skip_failing)
  File "/opt/bench/erpnext/apps/frappe/frappe/modules/patch_handler.py", line 41, in run_all
    run_patch(patch)
  File "/opt/bench/erpnext/apps/frappe/frappe/modules/patch_handler.py", line 30, in run_patch
    if not run_single(patchmodule = patch):
  File "/opt/bench/erpnext/apps/frappe/frappe/modules/patch_handler.py", line 71, in run_single
    return execute_patch(patchmodule, method, methodargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/modules/patch_handler.py", line 91, in execute_patch
    frappe.get_attr(patchmodule.split()[0] + ".execute")()
  File "/opt/bench/erpnext/apps/erpnext/erpnext/patches/v12_0/move_item_tax_to_item_tax_template.py", line 44, in execute
    item.save()
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 271, in save
    return self._save(*args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 324, in _save
    self.run_post_save_methods()
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 917, in run_post_save_methods
    self.run_method("on_update")
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 786, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 1056, in composer
    return composed(self, method, *args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 1039, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/opt/bench/erpnext/apps/frappe/frappe/model/document.py", line 780, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/opt/bench/erpnext/apps/erpnext/erpnext/stock/doctype/item/item.py", line 138, in on_update
    self.update_variants()
  File "/opt/bench/erpnext/apps/erpnext/erpnext/stock/doctype/item/item.py", line 760, in update_variants
    variants=variants, template=self, now=frappe.flags.in_test, timeout=600)
  File "/opt/bench/erpnext/apps/frappe/frappe/__init__.py", line 1466, in enqueue
    return frappe.utils.background_jobs.enqueue(*args, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/utils/background_jobs.py", line 42, in enqueue
    return frappe.call(method, **kwargs)
  File "/opt/bench/erpnext/apps/frappe/frappe/__init__.py", line 1036, in call
    return fn(*args, **newargs)
  File "/opt/bench/erpnext/apps/erpnext/erpnext/stock/doctype/item/item.py", line 1111, in update_variants
    copy_attributes_to_variant(template, variant)
  File "/opt/bench/erpnext/apps/erpnext/erpnext/controllers/item_variant.py", line 274, in copy_attributes_to_variant
    row = copy.deepcopy(d)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/bench/erpnext/env/lib/python3.6/copy.py", line 238, in _deepcopy_dict
    memo[id(x)] = y
MemoryError

smino · September 13, 2019, 10:33pm

The error message will be better understood if you can describe the operating environment, if a VM and what commands you had issued before the one that produced the error… Was it bench update --patch that produced the error?

jokermao · September 14, 2019, 2:36am

Hi smino,
Thanks for answer, I running in a VM and after I restore database on a fresh V12 install in a new site I tried bench update --patch and also bench update , it seems getting stuck on that particular patch all the time.
Thanks

krnkris · September 14, 2019, 6:08am

@jokermao

Memory error

It’s worth to try increasing the VM memory.
V12 + python needs stronger environment, means more memory and more processor to run smoothly.

Hope this helps.

jokermao · September 15, 2019, 12:32am

Hi @krnkris, thanks for tip, I already have over 6GB dedicated to that VM. I doubt is that the issue.

krnkris · September 15, 2019, 12:19pm

@jokermao

I meant the RAM min. 2GB .

What about the VM environment disc space?
It should be around 50GB.

jokermao · September 15, 2019, 11:03pm

@krnkris, I have way over 2GB of ram and VM way over 20GB, is not even using 10GB, I don’t think this is the issue.

Thanks

smino · September 15, 2019, 11:37pm

It might be worthwhile to monitor the memory use while the script is running. Log into a second ssh session and run top or htop. Do bench update (without --patch) and then bench migrate , complete?

jokermao · September 15, 2019, 11:41pm

Bench update still launches migrate and migrate launches patch

jokermao · September 16, 2019, 12:03am

smino · September 16, 2019, 1:13pm

A couple other things to look at,

How does the memory and disk use look on the host?

Can you run: bench mariadb then check tabletabItem Taxmedium;

Also try:

bench console , then

import sys
import os
import platform
platform architecture()
print (sys.version)

trentmu · September 16, 2019, 2:38pm

If it is a particular patch that it gets stuck on, you can temporarily exclude that patch by editing the patches.txt file and commenting out the line with the patch that doesn’t work. Once that has gone through and the system is migrated, you can retry with the patch uncommented

jokermao · September 16, 2019, 11:28pm

@trentmu I have done that, removed patch and I was able to migrate and build site, but on browser I was getting always message “system is updating”. Then on bench update when the trouble patch was launched … same thing again … issue is still there and same as before.

jokermao · September 18, 2019, 9:28pm

Still same issue, anybody having same troubles on migrating from V11 to V12?? It seems very strange.
Thanks

jokermao · September 19, 2019, 9:45am

@smino All good there no issues.

smino · September 19, 2019, 12:22pm

What is the out put from:

bench console , then

import sys
import os
import platform
platform architecture()
print (sys.version)

jokermao · September 19, 2019, 12:31pm

@smino 44%20AM

smino · September 19, 2019, 2:04pm

The output looks ok. Have you tried bench update --reset ?

jokermao · September 19, 2019, 2:04pm

@smino yes I did, keeps getting stuck on that patch, and ubuntu kills the process

smino · September 19, 2019, 2:40pm

I might try backing up the database, start a new VM , updtate Ubuntu ( to Bionic if you know how to deal with the repositories for Mariadb and npm) , python3 and Erpnext 12 , then restoring the database