[Solved] Bench Update Permission Error v10

Hello,

I tried running bench update and this is the error i got:

sudo bench update
INFO:bench.utils:updating bench
INFO:bench.utils:git pull
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 8 (delta 7), reused 4 (delta 3), pack-reused 0
error: insufficient permission for adding an object to repository database .git/objects
fatal: failed to write object
fatal: unpack-objects failed
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 11, in
load_entry_point(‘bench’, ‘console_scripts’, ‘bench’)()
File “/home/frappe/.bench/bench/cli.py”, line 40, in cli
bench_command()
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 722, in call
return self.main(*args, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 697, in main
rv = self.invoke(ctx)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 535, in invoke
return callback(*args, **kwargs)
File “/home/frappe/.bench/bench/commands/update.py”, line 34, in update
update_bench()
File “/home/frappe/.bench/bench/utils.py”, line 254, in update_bench
exec_cmd(“git pull”, cwd=cwd)
File “/home/frappe/.bench/bench/utils.py”, line 153, in exec_cmd
raise CommandFailedError(cmd)
bench.utils.CommandFailedError: git pull

i used the sudo command so i’m surprised as to why i still have permission issues. Can someone tell me what to do?

Thanks.

You should have no need for sudo on any bench commands.

The convention is that user account (for eg frappe) has sudo privileges.

Possibly you installed as root, so perhaps you can fix ownership with chown?

The issue seems to have been fixed as i just ran the command and the update worked. I also noticed that the update.py bench command script received some changes so my guess is it was a known issue which has now been fixed.

As for using sudo, my setup runs on Ubuntu 16.04 LTS on google cloud and i’m only allowed to use ubuntu as my default user without sudo privileges and i did not install as root. Hence for most commands i have to use sudo and it’s been fine so far. Thanks for the help though.

I have many test servers running on GCP and they are all restricted to the username you get from your google account. I also run most of my instances on Ubuntu 16.04.

The only way to really have a successful ERPNext system on Google Cloud Platform (GCP) is to install it with the following extra command line switch:

sudo python install.py --production --user [your_username]

When you install with the --user switch, you will never have trouble with bench commands again :grin:

BKM

When you install with the --user switch, you will never have trouble with bench commands again

But don’t you have to create the user first before making the user default?

No, your choice - just be sure to use --user switch in either case!

BKM creates the user account explicitly - search for adduser here [Tutorial] Step by Step for making a ERPNext and Webmin combo VPS server

or alternatively the standard install creates the account - you can dig into the actual install script here https://github.com/frappe/bench/blob/master/playbooks/install.py#L100

or on your local say here /home/frappe/install.py

edit: the one caveat is not to install as root user -

	if args.user == 'root':
	raise Exception('Please run this script as a non-root user with sudo privileges, but without using sudo or pass --user=USER')

On every GCP server I have running right now, the default user is the beginning of your email address linked to the Google cloud account. For example my Google Cloud account is linked to sys.bkm@gmail.com and any server I set up there is going to have a default user account of “sys_bkm” (the underscore is an automatic replacement thing Google does because they don’t like periods).

So when I set up a production server I use the following command on GCP

sudo python install.py --production --user sys_bkm

As for not having sudo available to you, that wouldn’t make much sense because you needed it in order to run the install script like above.

No, because on Google Cloud Platform you already have a default user. All you have to do is include it in the --user swith when you run the install.py script.

As @clarkej accurately points out, I do in fact add the user when I am on a paid production server like Interserver, Hostwinds, or DigitalOcean where I have root access. However on the Google Cloud Platform as you indicated you were using, the user account is generated for you and given sudo access because Google does NOT allow root logins. All you have to do is use the default user account that Google assigned to your server in your --user switch.

Hope this helps to clarify the use of the switch and why it is important on Google Cloud PLatform.

BKM

1 Like

I am self-taught on ERPNext setup and i am not much of a coder so I’ve hit several brick-walls in the past when setting up ERPNext on GCP so don’t mind me taking the easiest route by following the script line for line.

This point is noted and pretty obvious particularly when you login to terminal. However

The point made here suggests that with sudo privilege added to the default user created by GCP (which in this if the first part of your account email as noted above), for commands such as bench update, you don’t have to initiate the command by adding “sudo” at the beginning of the command as the user assumes sudo privileges already. However, i have set up several ERPNext instances on GCP and each time i try to run scripts such as updates, upgrades or even renew ssl certificates, i always hit a permission denied error. For example

felix@erpnext:/home/frappe/frappe-bench$ bench update
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 11, in
load_entry_point(‘bench’, ‘console_scripts’, ‘bench’)()
File “/home/frappe/.bench/bench/cli.py”, line 40, in cli
bench_command()
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 722, in call
return self.main(*args, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 697, in main
rv = self.invoke(ctx)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 1063, in invoke
Command.invoke(self, ctx)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 535, in invoke
return callback(*args, **kwargs)
File “/home/frappe/.bench/bench/commands/init.py”, line 31, in bench_command
setup_logging(bench_path=bench_path)
File “/home/frappe/.bench/bench/utils.py”, line 293, in setup_logging
hdlr = logging.FileHandler(log_file)
File “/usr/lib/python2.7/logging/init.py”, line 913, in init
StreamHandler.init(self, self._open())
File “/usr/lib/python2.7/logging/init.py”, line 943, in _open
stream = open(self.baseFilename, self.mode)
IOError: [Errno 13] Permission denied: ‘/home/frappe/frappe-bench/logs/bench.log’
felix@erpnext:/home/frappe/frappe-bench$

As you can see above, i hit an IOError because i ran bench update without the sudo prefix. Compare that to:

felix@erpnext:/home/frappe/frappe-bench$ sudo bench update
INFO:bench.utils:updating bench
INFO:bench.utils:git pull
Already up-to-date.
INFO:bench.utils:./env/bin/pip install Pillow
Requirement already satisfied: Pillow in ./env/lib/python2.7/site-packages
Requirement already satisfied: olefile in ./env/lib/python2.7/site-packages (from Pillow)
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the ‘pip install --upgrade pip’ command.
INFO:bench.app:pulling frappe
INFO:bench.utils:git pull upstream master
From https://github.com/frappe/frappe

  • branch master -> FETCH_HEAD
    Already up-to-date.
    INFO:bench.utils:find . -name “*.pyc” -delete
    INFO:bench.app:pulling erpnext
    INFO:bench.utils:git pull upstream master
    remote: Counting objects: 83, done.
    remote: Compressing objects: 100% (26/26), done.
    remote: Total 83 (delta 68), reused 72 (delta 57), pack-reused 0
    error: insufficient permission for adding an object to repository database .git/objects
    fatal: failed to write object
    fatal: unpack-objects failed
    Traceback (most recent call last):
    File “/usr/local/bin/bench”, line 11, in
    load_entry_point(‘bench’, ‘console_scripts’, ‘bench’)()
    File “/home/frappe/.bench/bench/cli.py”, line 40, in cli
    bench_command()
    File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 722, in call
    return self.main(*args, **kwargs)
    File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 697, in main
    rv = self.invoke(ctx)
    File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
    File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 535, in invoke
    return callback(*args, **kwargs)
    File “/home/frappe/.bench/bench/commands/update.py”, line 60, in update
    _update(pull, patch, build, bench, auto, restart_supervisor, restart_systemd, requirements, no_backup, force=fo
    rce, reset=reset)
    File “/home/frappe/.bench/bench/commands/update.py”, line 73, in _update
    pull_all_apps(bench_path=bench_path, reset=reset)
    File “/home/frappe/.bench/bench/app.py”, line 240, in pull_all_apps
    remote=remote, branch=get_current_branch(app, bench_path=bench_path)), cwd=app_dir)
    File “/home/frappe/.bench/bench/utils.py”, line 153, in exec_cmd
    raise CommandFailedError(cmd)
    bench.utils.CommandFailedError: git pull upstream master
    felix@erpnext:/home/frappe/frappe-bench$

As you can see, the script runs without an IOError (although it seems there’s an issue with .git again). So unless i am missing something here, there is something uncanny about the way GCP handles ERPNext when it comes to user permissions.

The way bench is designed, it wants the user owning ghe account to be the one executing commands. So you are logged into the server as user “felix” and you are attempting to run bench commands that are owned by “frappe.” GCP only lets you log in as your default user, so you can never really own the “frappe” account. I have been down this road and tried running “chown” to take possession of the frappe account and it never really worked.

The ONLY way to be able to run bench commands on the Google Cloud Platform is to install ERPNext using the following command:

sudo python install.py --production --user felix

At this point all of the frappe framework and erpnext apps will be installed in your “felix” user account. The bench will be installed in /home/felix/frappe-bench and everything you wish to do will work.

Well… almost everything. Unless you can get Google to unlock some port numbers, I doubt you will have very much luck using Lets Encrypt certificates. That is just one of the down sides of using GCP. It also means that you will not be able to get any of the email functions to work either.

As a development and testing platform GCP is a wonderful tool, but as a place to really test and use ERPNext, it has some pretty big limitations. All of those limitations are caused by the GCP policy of blocking all but the very basic ports.

GCP is still a great place to learn to navigate much of ERPNext. You just cannot run a full ERPNext system on it without the SSL certificates and email. It becomes even more complicated if you ever wanted to turn on 2 level authentication that sends text messages to your phone. SMS will not work either due to blocked ports.

A search of the forum will yield many many results of developers telling new users to NOT user sudo with the bench command because it is not designed to work from anything but the owner account.

Hope this helps.

1 Like

Your responses so far have been insightful and it is much appreciated.

Indeed GCP makes users sweat profusely in order to get ERPNext running if you run the installation manually. Even though it seems a tad bit easier compared to AWS.

Although on the instances i have running, i was able to setup email and lets encrypt ssl (caveat: my email service is hosted by Google. I have not attempted to setup 2 level authentication because i don’t think it necessary at this point). Probably this is due to the fact that i did not specify the default user hence the user assigned to me is allowed to communicate through those ports, essentially suggesting you can’t eat your cake and have it. But there are some other basic things i’ve been unable to accomplish which have proved challenging which i don’t know who or what to attribute the challenge to such as my IP address being routinely banned and my inability to restore files during backup restore.

In the end though, GCP offers more for free (for the first year) so i am not looking to jumping ship yet.

Well, a google search about ip banning of GCP accounts will give you some insight. It is because GCP is such an easy to use platform, the scammers would setup quick sites to do some damage and then just as quickly delete them. Because of the quick changing of the ip addresses just by stopping and starting an instance, there are several spam fighting devices and software that will just ban the whole block of IP addresses that GCP might use. However, on the “good news” side of it, the banning seems to stop when you purchase a permanent IP address from Google (probably because it is in a very different range block).

You are correct on the email. Gmail is really the only thing you can use there, but most businesses do not really use Gmail to run their businesses (although that is startgin to change over the past 12 months due to the improvements in the Google Office suite).

I never have trouble restoring database files on GCP for ERPNext. I take regular backups from my paid production server at a different hosting company and restore them to my GCP test servers multiple times per week so I can use my GCP servers to test real life issues my clients experience. But, then again, I have always used the --user switch with my default “sys_bkm” username as the location of the bench.

To be honest, I only suffered your problems when I didn’t know any better and tried to install the system without the --user switch. I always assumed (incorrectly) that the default installation MUST be the best practice. It took me a few months of trial and error to discover the correct way to use GCP. In fact it took me so long to figure this out, that a Step by Step procedure I wrote on this forum forum was too old for me to go back and edit in the corrections for using the --user switch. Yet it is still quite popular for new users. See below:

[Tutorial] Step by Step Primer to setup ERPNext on Google Cloud Platform

If you look at that post, I used the install.py WITHOUT the --user switch in the process and didn’t yet realize the challenges it would create for the new users. I have since been doing my best to help those using GCP to alwyas remember to use the --user switch in order to have the best experiance possible.

Glad you are having success in your quest to learn the system. Even through it’s shortfalls, I still like it better than the other choices.

BKM

1 Like