How to upgrade to python 3.10

[rmeyer]

Recently, Frappe updated its dependencies for version-14 to python > 3.10.

If you’re on version-13, try to upgrade to python3.8 instead.

Here’s what I did to upgrade to python3.10 under Ubuntu 20.04.

Remember to take a backup of your server first!

Install custom PPA, so Ubuntu knows where to find python3.10:

sudo apt install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa

Install required python packages:

sudo apt install python3.10 python3.10-dev python3.10-distutils

Switch to your bench directory:

cd /home/frappe/frappe-bench

Move your old python env folder to env-old. This way you make space for the new env while keeping a copy to restore in case something goes wrong.

mv env env-old

Create a new python environment with python3.8

virtualenv --python python3.10 env

In case the above command doesn’t work (I had problems on one machine) you can try this alternative:

# OPTIONAL: only if the above failed
python3.10 -m venv env

Upgrade pip to the newest version. The later commands will fail in case your pip version is too old.

env/bin/pip install -U pip

Now we can reinstall our apps and their dependencies. Here I only show frappe and erpnext. Please append all your custom apps to the command like this: -e apps/my-app

env/bin/pip install -e apps/frappe -e apps/erpnext

I like to run bench update at the end to see if everything worked:

bench update --reset

If everything went well, you can delete your old env (optional, use with care):

rm -rf env-old

---

Didn’t test this properly yet, but if you get errors from socketio in your browser console, this can possibly be fixed by:

bench setup supervisor
bench restart --supervisor

[Adam.Niescierowicz]

Python wheel demands g++ so you need to install apt-get install g++

[Usman_Ali]

nginx: [emerg] unknown log format “main” in /etc/nginx/conf.d/frappe-bench.conf:101
nginx: configuration file /etc/nginx/nginx.conf test failed
ERROR:
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 8, in
sys.exit(cli())
File “/usr/local/lib/python3.10/dist-packages/bench/cli.py”, line 127, in cli
bench_command()
File “/usr/lib/python3/dist-packages/click/core.py”, line 1128, in call
return self.main(*args, **kwargs)
File “/usr/lib/python3/dist-packages/click/core.py”, line 1053, in main
rv = self.invoke(ctx)
File “/usr/lib/python3/dist-packages/click/core.py”, line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/lib/python3/dist-packages/click/core.py”, line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/lib/python3/dist-packages/click/core.py”, line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/lib/python3/dist-packages/click/core.py”, line 754, in invoke
return __callback(*args, **kwargs)
File “/usr/local/lib/python3.10/dist-packages/bench/commands/setup.py”, line 89, in setup_production
setup_production(user=user, yes=yes)
File “/usr/local/lib/python3.10/dist-packages/bench/config/production_setup.py”, line 87, in setup_production
reload_nginx()
File “/usr/local/lib/python3.10/dist-packages/bench/config/production_setup.py”, line 205, in reload_nginx
exec_cmd(f"sudo {which(‘nginx’)} -t")
File “/usr/local/lib/python3.10/dist-packages/bench/utils/init.py”, line 154, in exec_cmd
raise CommandFailedError
bench.exceptions.CommandFailedError

[Nathaniel_B]

Same error here!

[smino]

Hi,

What does sudo nginx -t return?

[nomi-gee]

It’s better to upgrade to Ubuntu 22.04 as a python upgrade sometime creates more issues with pip and the default python version.

[pradpreeonline]

Hi,

I am getting the below error running this step

env/bin/pip install -e apps/frappe -e apps/erpnext

pxp@erp2-server:~/frappe-bench$ env/bin/pip3.10 install -e apps/frappe/ -e apps/erpnext
Traceback (most recent call last):
File “/home/pxp/frappe-bench/env/bin/pip3.10”, line 8, in
sys.exit(main())
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/main.py”, line 73, in main
command = create_command(cmd_name, isolated=("–isolated" in cmd_args))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/init.py”, line 96, in create_command
module = importlib.import_module(module_path)
File “/usr/lib/python3.10/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1050, in _gcd_import
File “”, line 1027, in _find_and_load
File “”, line 1006, in _find_and_load_unlocked
File “”, line 688, in _load_unlocked
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/install.py”, line 24, in
from pip._internal.cli.req_command import RequirementCommand
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py”, line 15, in
from pip._internal.index.package_finder import PackageFinder
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/package_finder.py”, line 21, in
from pip._internal.index.collector import parse_links
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/collector.py”, line 12, in
from pip._vendor import html5lib, requests
ImportError: cannot import name ‘html5lib’ from ‘pip._vendor’ (/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_vendor/init.py)

pxp@erp2-server:~/frappe-bench$ pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
pxp@erp2-server:~/frappe-bench$ env/bin/
easy_install pip pip3.10 python3.10 wheel-3.10
easy_install3 pip3 python wheel
easy_install-3.10 pip-3.10 python3 wheel3
pxp@erp2-server:~/frappe-bench$ env/bin/

This step was successful though:

virtualenv --python python3.10 env

created virtual environment CPython3.10.6.final.0-64 in 466ms
creator CPython3Posix(dest=/home/pradeep/frappe-bench/env, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/pxp/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

I think it might be to do with the pip version still showing 20.0.2 even after the lastest version was installed. Could that be the problem?

pxp@erp2-server:~/frappe-bench$ sudo -H pip3 install --upgrade pip
Collecting pip
Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 1.3 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Can’t uninstall ‘pip’. No files were found to uninstall.
Successfully installed pip-22.2.2
pxp@erp2-server:~/frappe-bench$ pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

[smino]

Hi ,

I don’t recall encountering that error. I do see that I upgraded pip via: sudo pip3 install pip --upgrade

pip3 -V reports:

pip 22.2.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)

[rmeyer]

Take care which pip you’re using. pip3 is your system’s pip, env/bin/pip is your bench’s pip. They’re entirely different. You can upgrade the latter by running env/bin/pip install -U pip.

[trustedcomputer]

Thanks for this guide!

After installing python3.10 with add-apt-repository and apt install, I was able to migrate the env with this bench command: bench migrate-env python3.10

Then I continued with upgrading pip and re-installing the apps.

[pradpreeonline]

I got this error when I tried to upgrade bench’s pip.

pxp@erp2-server:~/frappe-bench$ env/bin/pip install -U pip
Traceback (most recent call last):
File “/home/pxp/frappe-bench/env/bin/pip”, line 8, in
sys.exit(main())
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/main.py”, line 73, in main
command = create_command(cmd_name, isolated=("–isolated" in cmd_args))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/init.py”, line 96, in create_command
module = importlib.import_module(module_path)
File “/usr/lib/python3.10/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1050, in _gcd_import
File “”, line 1027, in _find_and_load
File “”, line 1006, in _find_and_load_unlocked
File “”, line 688, in _load_unlocked
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/install.py”, line 24, in
from pip._internal.cli.req_command import RequirementCommand
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py”, line 15, in
from pip._internal.index.package_finder import PackageFinder
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/package_finder.py”, line 21, in
from pip._internal.index.collector import parse_links
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/collector.py”, line 12, in
from pip._vendor import html5lib, requests
ImportError: cannot import name ‘html5lib’ from ‘pip._vendor’ (/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_vendor/init.py)
pxp@erp2-server:~/frappe-bench$

I then ran this after doing a little research but the pip version still seems to be not updated:

pxp@erp2-server:~/frappe-bench$ curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2.2
Uninstalling pip-22.2.2:
Successfully uninstalled pip-22.2.2
WARNING: The scripts pip, pip3 and pip3.10 are installed in ‘/home/pxp/.local/bin’ which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.2.2

pxp@erp2-server:~/frappe-bench$ pip -V
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

[pradpreeonline]

I found another way to upgrade version 13 to version 14 by installing version 13 on ubuntu 22.04 and then restoring the data from version 13 (installed originally on ubuntu 20.04). After the data restore was done I then followed the steps from this URL:

How I upgraded from v13 to v14 and Ubuntu 20.04 LTS to 22.04 LTS

First, I made sure my last nightly backup of the VM was successful. Then I backed up the site with bench --site *sitename* backup --with-files and copied the backups off-host. Then I made another snapshot of the VM. Being OCD with backups is helpful with this kind of upgrade! Next, I upgraded frappe-bench to the latest: pip3 install frappe-bench --upgrade At this point, I followed this guide to upgrade to Python 3.10 and pip to the latest version. Next, I ran bench restart and tested the site…

Erpnext upgraded from version 13 to version 14 with a few errors and from the size of the databackup file and the error I see while using the application that I cannot trust the data on the upgraded system.
Has anybody successfully upgraded ERPNext version 13 to 14 with a trustable data? If so, can you please share the instructions? Although version 14 was launched due to the issue in upgrade I am unable to implement and use version 14.