[Solved] ERROR trying to setup Lets Encrypt w/ bench. Dependency Issue

Install / Update Security
1

Running on Ubuntu 16.04

  • The trace below has been edited to replace my actual site URL with mysitename.com

Ok, here is the problem I encountered and the complete trace of the work after I run the correct command to setup the lets encrypt in bench:

bkm_user@ubuntu16:~/frappe-bench$ bench setup add-domain --site site1.local mysitename.com
bkm_user@ubuntu16:~/frappe-bench$ sudo -H bench setup lets-encrypt site1.local --custom-domain mysitename.com
Running this will stop the nginx service temporarily causing your sites to go offline
Do you want to continue? [y/N]: y
INFO:bench.utils:sudo systemctl stop nginx
INFO:bench.utils:/opt/certbot-auto --config /etc/letsencrypt/configs/mysitename.com.cfg certonly
Bootstrapping dependencies for Debian-based OSes… (you can skip this with --no-bootstrap)
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:4 Index of /ubuntu xenial-security InRelease [102 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [755 kB]
Hit:6 Index of /mariadb/repo/10.2/ubuntu/ xenial InRelease
Hit:7 https://deb.nodesource.com/node_6.x xenial InRelease
Get:8 http://us.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages [698 kB]
Get:9 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [619 kB]
Get:10 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages [573 kB]
Get:11 Index of /ubuntu xenial-security/main amd64 Packages [472 kB]
Get:12 Index of /ubuntu xenial-security/main i386 Packages [424 kB]
Get:13 Index of /ubuntu xenial-security/universe amd64 Packages [340 kB]
Get:14 Index of /ubuntu xenial-security/universe i386 Packages [297 kB]
Fetched 4,484 kB in 1s (2,662 kB/s)
Reading package lists… Done
Reading package lists… Done
Building dependency tree
Reading state information… Done
gcc is already the newest version (4:5.3.1-1ubuntu1).
gcc set to manually installed.
libffi-dev is already the newest version (3.2.1-4).
ca-certificates is already the newest version (20170717~16.04.1).
libssl-dev is already the newest version (1.0.2g-1ubuntu4.11).
openssl is already the newest version (1.0.2g-1ubuntu4.11).
The following packages were automatically installed and are no longer required:
linux-headers-4.4.0-109 linux-headers-4.4.0-109-generic
linux-headers-4.4.0-112 linux-headers-4.4.0-112-generic
linux-headers-4.4.0-97 linux-headers-4.4.0-97-generic
linux-image-4.4.0-109-generic linux-image-4.4.0-112-generic
linux-image-4.4.0-97-generic linux-image-extra-4.4.0-109-generic
linux-image-extra-4.4.0-112-generic linux-image-extra-4.4.0-97-generic
Use ‘sudo apt autoremove’ to remove them.
The following additional packages will be installed:
libpython-all-dev libpython-dev libpython-stdlib python-all python-all-dev
python-minimal python3-virtualenv
Suggested packages:
augeas-doc augeas-tools python-doc python-tk
The following NEW packages will be installed:
augeas-lenses libaugeas0 python-virtualenv python3-virtualenv virtualenv
The following packages will be upgraded:
libpython-all-dev libpython-dev libpython-stdlib python python-all
python-all-dev python-dev python-minimal
8 upgraded, 5 newly installed, 0 to remove and 84 not upgraded.
3 not fully installed or removed.
Need to get 696 kB of archives.
After this operation, 2,309 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-all-dev amd64 2.7.12-1~16.04 [1,016 B]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-dev amd64 2.7.12-1~16.04 [1,186 B]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-all amd64 2.7.12-1~16.04 [996 B]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python-minimal amd64 2.7.12-1~16.04 [28.1 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python amd64 2.7.12-1~16.04 [137 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython-all-dev amd64 2.7.12-1~16.04 [1,006 B]
Get:7 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython-dev amd64 2.7.12-1~16.04 [7,840 B]
Get:8 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython-stdlib amd64 2.7.12-1~16.04 [7,768 B]
Get:9 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 augeas-lenses all 1.4.0-0ubuntu1.1 [263 kB]
Get:10 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libaugeas0 amd64 1.4.0-0ubuntu1.1 [154 kB]
Get:11 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-virtualenv all 15.0.1+ds-3ubuntu1 [46.6 kB]
Get:12 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python3-virtualenv all 15.0.1+ds-3ubuntu1 [43.2 kB]
Get:13 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 virtualenv all 15.0.1+ds-3ubuntu1 [4,342 B]
Fetched 696 kB in 0s (4,377 kB/s)
(Reading database … 209207 files and directories currently installed.)
Preparing to unpack …/python-all-dev_2.7.12-1~16.04_amd64.deb …
Unpacking python-all-dev (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/python-dev_2.7.12-1~16.04_amd64.deb …
Unpacking python-dev (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/python-all_2.7.12-1~16.04_amd64.deb …
Unpacking python-all (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/python-minimal_2.7.12-1~16.04_amd64.deb …
Unpacking python-minimal (2.7.12-1~16.04) over (2.7.11-1) …
Processing triggers for man-db (2.7.5-1) …
Setting up python-minimal (2.7.12-1~16.04) …
(Reading database … 209207 files and directories currently installed.)
Preparing to unpack …/python_2.7.12-1~16.04_amd64.deb …
Unpacking python (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/libpython-all-dev_2.7.12-1~16.04_amd64.deb …
Unpacking libpython-all-dev:amd64 (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/libpython-dev_2.7.12-1~16.04_amd64.deb …
Unpacking libpython-dev:amd64 (2.7.12-1~16.04) over (2.7.11-1) …
Preparing to unpack …/libpython-stdlib_2.7.12-1~16.04_amd64.deb …
Unpacking libpython-stdlib:amd64 (2.7.12-1~16.04) over (2.7.11-1) …
Selecting previously unselected package augeas-lenses.
Preparing to unpack …/augeas-lenses_1.4.0-0ubuntu1.1_all.deb …
Unpacking augeas-lenses (1.4.0-0ubuntu1.1) …
Selecting previously unselected package libaugeas0.
Preparing to unpack …/libaugeas0_1.4.0-0ubuntu1.1_amd64.deb …
Unpacking libaugeas0 (1.4.0-0ubuntu1.1) …
Selecting previously unselected package python-virtualenv.
Preparing to unpack …/python-virtualenv_15.0.1+ds-3ubuntu1_all.deb …
Unpacking python-virtualenv (15.0.1+ds-3ubuntu1) …
Selecting previously unselected package python3-virtualenv.
Preparing to unpack …/python3-virtualenv_15.0.1+ds-3ubuntu1_all.deb …
Unpacking python3-virtualenv (15.0.1+ds-3ubuntu1) …
Selecting previously unselected package virtualenv.
Preparing to unpack …/virtualenv_15.0.1+ds-3ubuntu1_all.deb …
Unpacking virtualenv (15.0.1+ds-3ubuntu1) …
Processing triggers for man-db (2.7.5-1) …
Processing triggers for libc-bin (2.23-0ubuntu10) …
Setting up linux-image-extra-4.4.0-119-generic (4.4.0-119.143) …
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-119-generic /boot/vmlinuz-4.4.0-119-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-119-generic /boot/vmlinuz-4.4.0-119-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-119-generic

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-4.4.0-119-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
dpkg: error processing package linux-image-extra-4.4.0-119-generic (–configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of linux-image-generic:
linux-image-generic depends on linux-image-extra-4.4.0-119-generic; however:
Package linux-image-extra-4.4.0-119-generic is not configured yet.

dpkg: error processing package linux-image-generic (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
dpkg: dependency problems prevent configuration of linux-generic:
linux-generic depends on linux-image-generic (= 4.4.0.119.125); however:
Package linux-image-generic is not configured yet.

dpkg: error processing package linux-generic (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Setting up libpython-stdlib:amd64 (2.7.12-1~16.04) …
Setting up python (2.7.12-1~16.04) …
Setting up python-all (2.7.12-1~16.04) …
Setting up libpython-dev:amd64 (2.7.12-1~16.04) …
Setting up libpython-all-dev:amd64 (2.7.12-1~16.04) …
Setting up python-dev (2.7.12-1~16.04) …
Setting up python-all-dev (2.7.12-1~16.04) …
Setting up augeas-lenses (1.4.0-0ubuntu1.1) …
Setting up libaugeas0 (1.4.0-0ubuntu1.1) …
Setting up python-virtualenv (15.0.1+ds-3ubuntu1) …
Setting up python3-virtualenv (15.0.1+ds-3ubuntu1) …
Setting up virtualenv (15.0.1+ds-3ubuntu1) …
Processing triggers for libc-bin (2.23-0ubuntu10) …
Errors were encountered while processing:
linux-image-extra-4.4.0-119-generic
linux-image-generic
linux-generic
E: Sub-process /usr/bin/dpkg returned an error code (1)
INFO:bench.utils:sudo systemctl start nginx
There was a problem trying to setup SSL for your site
bkm_user@ubuntu16:~/frappe-bench$

.
.
So, this is what I see in the trace…

It indicates that there was an error processing one of the needed packages due to dependency issues.

It also indicates that no report was written because this error had occurred before. I did do “sudo apt-get update” before running everything.

How would I find the previous error report so it could be displayed here?

What might I be able to do to get past this?

Thanks in advance for any guidance you can provide.

BKM

2

Still struggling with this.

Can anyone recognize the error here?

Can anyone point me to where to find the error report file so that maybe I can find more information?

Thanks.

BKM

3

ok you must address this if you haven’t - this is a showstopper :slight_smile:

4

Hmm… I looked at that, but it is not even possible. I have a 125gb SSD hard drive and only about 10gb used.

Does this lets encrypt script setup its own temporary virtual device?

Really stuck on this issue. I have 2 servers both configured the same way

  • 125gb HD
  • 5gb RAM memory
  • Ubuntu 16.04
  • On the same Service Provider.

The first server was put up Feb 1st and running ERPNext 10.0.14.

The second server was put up April 2nd and running ERPNext 10.1.18

Both servers have been configured for DNS Multitenant.
First server only has site1.local and a custom domain - no additional tenants
Second server has site1.local with a custom domain plus 2 additional tenants

First server fails with the error at the top of this thread.
Second server has successful Lets Encrypt certificates for all 3 of the tenants.

Unfortunately the first server is also locked to it’s version by contract. It has many customizations and should not be migrated. Even if I wanted to, the contract with the client prohibits it.

So, I need to find the issue with the dependency and I guess how in the world it thinks it has no space to unpack its packages.

BKM

5

My guess is /boot is in its own partition and that is out of space?

6

Your guess was right. :exploding_head: - :no_entry: -:weary:

I ran the df command and looked more closely at the boot partitions. I never expected to have to manually maintain the boot partitions of a modern OS, but that is what happens with Ubuntu 16.04 and older. It may even still happen in the newer versions. Don’t know, and don’t care right now. I have to deal with this one.

It makes sense the encrypted certificates need to go in the /boot partition and of course it is full of old kernel files. I will have to wait until late tonight when the system is not in use to try to autoremove some of them and purge them out of the system.

I think the more bothersome aspect of this will be having to do that kind of maintenance every month.

@clarkej Thank you again for pointing me in the right direction. Sometimes I feel like the See-No-Evil monkey with his hands over his eyes. :eyes:

I will work on this tonight and report back. This will probably be one of those moments that should be documented for others that go into this maybe a little blind.

BKM

7

Here is the fix!! ( a big Thank You to @clarkej for putting me on the right path).

My boot partition in Ubuntu was almost completely full and there was no room for the Lets Encrypt packages to unpack and get setup. I think it currently requires about 28mb to setup everything and about 12mb of that is to go in the boot partition. My boot partition only had 8mb of remaining space.

It turns out that every time Ubuntu updates itself and new kernels are added, the old kernels are just left there “unused” in case you ever have to go backwards due to bugs or other disasters. In the mean time, this action of leaving all the old kernel files laying around is in itself a slow train wreck of a disaster in the making. Since the boot partition is only about 235mb in size, it doesn’t take much to start choking off the operating system with un-digested old kernels. There is no Tums or Rolaids for your Ubuntu server, just a lot of careful work to be done on some regular schedule.

So, after a bunch of reading I found a very concise set of instructions for cleaning up the mess left behind by the updates. These instructions were fist published by a user on GitHub called “ipbastola” about 2 years ago. His list is here:

Then around January of this year his original instructions were picked up by someone at Northwestern College (without proper acknowledgment of the original author) and republished with some expanded and very useful additions at the bottom> The Northwestern user goes by “rtn361” and his remake is listed here:

https://discourse.apps.northwestern.edu/t/safest-way-to-clean-up-boot-partition-ubuntu-14-04lts-x64/112

Now, I wanted to acknowledge everyone that came before me in relaying this information to you. I also want to make sure the instructions do not get lost like so many other links on this forum that get broken. So to that end I will copy post them here for posterity and the needs of our own community :grin:

Thank you to both previous authors!

Here are the instructions:

Case I: if /boot is not 100% full and apt is working

**Check the current kernel version**

$ uname -r

It will shows the list like below:

3.19.0-64-generic

2. Remove the OLD kernels

2.a. List the old kernel

$ sudo dpkg --list 'linux-image*'|awk '{ if ($1=="ii") print $2}'|grep -v `uname -r`

You will get the list of images something like below:

linux-image-3.19.0-25-generic
linux-image-3.19.0-56-generic
linux-image-3.19.0-58-generic
linux-image-3.19.0-59-generic
linux-image-3.19.0-61-generic
linux-image-3.19.0-65-generic
linux-image-extra-3.19.0-25-generic
linux-image-extra-3.19.0-56-generic
linux-image-extra-3.19.0-58-generic
linux-image-extra-3.19.0-59-generic
linux-image-extra-3.19.0-61-generic

2.b. Now its time to remove old kernel one by one as

$ sudo apt-get purge linux-image-3.19.0-25-generic
$ sudo apt-get purge linux-image-3.19.0-56-generic
$ sudo apt-get purge linux-image-3.19.0-58-generic
$ sudo apt-get purge linux-image-3.19.0-59-generic
$ sudo apt-get purge linux-image-3.19.0-61-generic
$ sudo apt-get purge linux-image-3.19.0-65-generic

When you’re done removing the older kernels, you can run this to remove ever packages you won’t need anymore:

$ sudo apt-get autoremove

And finally you can run this to update grub kernel list:

$ sudo update-grub

Case II: Can’t Use apt i.e. /boot is 100% full

NOTE: this is only if you can’t use apt to clean up due to a 100% full /boot

_Get the list of kernel images_

Get the list of kernel images and determine what you can do without. This command will show installed kernels except the currently running one

$ sudo dpkg --list ‘linux-image*’|awk ‘{ if ($1==“ii”) print $2}’|grep -v uname -r

You will get the list of images somethign like below:

linux-image-3.19.0-25-generic
linux-image-3.19.0-56-generic
linux-image-3.19.0-58-generic
linux-image-3.19.0-59-generic
linux-image-3.19.0-61-generic
linux-image-3.19.0-65-generic
linux-image-extra-3.19.0-25-generic
linux-image-extra-3.19.0-56-generic
linux-image-extra-3.19.0-58-generic
linux-image-extra-3.19.0-59-generic
linux-image-extra-3.19.0-61-generic

Prepare Delete

Craft a command to delete all files in /boot for kernels that don’t matter to you using brace expansion to keep you sane. Remember to exclude the current and two newest kernel images. From above Example, it’s

sudo rm -rf /boot/-3.19.0-{25,56,58,59,61,65}-

Clean up what’s making apt grumpy about a partial install.

sudo apt-get -f install

4. Autoremove

Finally, autoremove to clear out the old kernel image packages that have been orphaned by the manual boot clean.

sudo apt-get autoremove

5. Update Grub

sudo update-grub

6. Now you can update, install packages

sudo apt-get update

.
.
.
Alright folks. I hope that at least one other person find relief in this thread. I know it is really scary when you try to install Lets Encrypt and it fails. You begin to think there might be a serious problem with your server. The reality here is that it could just as easily be any other package you need to install. If some of it needs to go to the /boot partition and there is no room, then you will still trip over the same problem.

So, I’m done for the night. :sleeping:

BKM

1 Like
8

Certainly thanks BKM for this excellent learning doc it’s all about teamwork :slight_smile:

cheers

2 Likes