How to install with Docker with existing nginx

Hi!

I already have a docker setup running with nginx with mutliple services. I wanted to set this up on the same server for the purpose of testing EPRNext out. But the documentation has starts many containers and it is very unclear which are needed for a barebone install and what port they serve the web app to.

Do anyone have any tips of how to set this up if I do not want the reverse proxy and TLS setup via ERPnext. I want ERPNext on other ports than 80, 433, 8000, 9000 which seems to be the ones it uses by default.

It tried to explain what containers are used for

I gave up on making it easy, I need help here. Help wanted for making docker easy to run · Issue #353 · frappe/frappe_docker · GitHub

1 Like

Thank you for respondning.

If I remove the traefik and nginx container from the docker compose file. What port will the ERPNext web app be served to? Do the traefik and nginx container server any other purpose except reverse proxy and lets encrypt setup?
What does the site-creator container do?
Can I run erpnext-python stand alone?

You cannot remove nginx container. If no ports are published for erpnext-nginx container then ERPNext won’t be served on any port. publish port 80 of erpnext-nginx container to get access to ERPNext. e.g. 8080:80, refer docker docs.

traefik container does Lets encrypt and additional config if required.

Nginx container is part of ERPNext production setup it cannot be removed.
Nginx container does reverse proxy to socketio and gunicorn as well as serves the static html/css/js assets.

Container is run once, it creates the first site. If you run it again and site already exists this container will fail to start.

No.

Frappe/ERPNext is combination of:

  • 3 workers
  • 1 scheduler
  • 1 socketio
  • 1 gunicorn
  • 1 nginx assets + proxy

Other services needed by ERPNext are:

  • MariaDB
  • Redis for Cache
  • Redis for Queue
  • Redis for Socketio

I understand tls and for most cases nginx is needed if you want to expose the service to the outside world. But I do not at the moment and already have a container running nginx that I would use in that case. I only want to access it by :.

After reading more, I looked into the multi bench setup. The *publish.yml compose files both use port 80. Doesn’t they block each other or should I only run one of them?

Is port 80 the only port that setup expose to the host and I can just map 80 to something else in the publish file or will nginx complain with reverse proxy on top of a reverse proxy.

Don’t publish erpnext ports. Attach erpnext-nginx container to the docker network in which your current nginx container is running and make changes to the config to serve erpnext-nginx:80

use one of the files not both. In your case, don’t publish ports, use docker network.

The current docker-compose files setup things with default settings.

You can map it to anything as per your setup / configuration need.

Think of the erpnext-nginx as the application itself. It’s not just nginx. It’s mainly the glue that serves all erpnext services + static assets.

It need not be your main ingress nginx. Your main nginx can serve many other things along with ERPNext.

Use this to reverse proxy the internal erpnext-nginx and pass the Host header to the service in nginx configuration.

Example:

Incoming Request → main nginx → config to pass host header and reverse proxy to epnext-nginx:80
assuming erpnext-nginx container is in same docker network as your existing nginx container.

Also make sure all containers are setup to auto restart. Use it is as substitute of supervisord used in normal bench installation.

As you suggested I stopped trying the versions with a published port. Used the commands below.

I added a network to my nginx and to docker-compose-networks.yml, setup nginx to forward traffic to erpnext-nginx.

docker-compose \
        --project-name <project-name> \
        -f installation/docker-compose-common.yml \
        -f installation/docker-compose-erpnext.yml \
        -f installation/docker-compose-networks.yml \
        up -d

But I get this error in the erpnext-nginx container logs

Waiting for frappe-python to be available on erpnext-python port 8000

Frappe-python available on erpnext-python port 8000

Waiting for frappe-socketio to be available on frappe-socketio port 9000

Frappe-socketio available on frappe-socketio port 9000

172.23.0.2 - - [17/Nov/2020:21:29:34 +0000] "GET / HTTP/1.1" 500 141 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Safari/605.1.15" "192.168.10.1"

The request apprently reach erpnext-nginx but returns internal server error.

Just added the network? Use the network in all containers

Changed the default of erpnext. Added like this to the network file.

networks:
  default:
    external:
      name: my-pre-existing-network

I am just getting internal server error and this error in the erpnext-python log

[2020-11-18 22:24:08 +0000] [23] [INFO] Booting worker with pid: 23

[2020-11-18 22:24:08 +0000] [24] [INFO] Booting worker with pid: 24

[2020-11-18 22:26:14 +0000] [24] [ERROR] Error handling request /

Traceback (most recent call last):

  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 54, in application

    init_request(request)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 137, in init_request

    frappe.local.http_request = frappe.auth.HTTPRequest()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/auth.py", line 43, in __init__

    self.set_lang()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/auth.py", line 87, in set_lang

    frappe.local.lang = guess_language()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 31, in guess_language

    lang_list = get_all_languages() or []

  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 87, in get_all_languages

    return frappe.cache().get_value('languages', _get)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 79, in get_value

    val = generator()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 86, in _get

    return frappe.db.sql_list('select name from tabLanguage')

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 244, in sql_list

    return [r[0] for r in self.sql(query, values, debug=debug)]

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 122, in sql

    self.connect()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 75, in connect

    self._conn = self.get_connection()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/database.py", line 91, in get_connection

    local_infile = frappe.conf.local_infile)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/__init__.py", line 94, in Connect

    return Connection(*args, **kwargs)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 325, in __init__

    self.connect()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 599, in connect

    self._request_authentication()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 861, in _request_authentication

    auth_packet = self._read_packet()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 684, in _read_packet

    packet.check_error()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/protocol.py", line 220, in check_error

    err.raise_mysql_exception(self._data)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception

    raise errorclass(errno, errval)

pymysql.err.OperationalError: (1045, "Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)")


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 280, in handle

    keepalive = self.handle_request(req, conn)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 329, in handle_request

    respiter = self.wsgi(environ, resp.start_response)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/werkzeug/local.py", line 231, in application

    return ClosingIterator(app(environ, start_response), self.cleanup)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/werkzeug/wrappers/base_request.py", line 237, in application

    resp = f(*args[:-2] + (request,))

  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 85, in application

    response = handle_exception(e)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 218, in handle_exception

    http_status_code=http_status_code)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/render.py", line 35, in render

    raise_if_disabled(path)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/render.py", line 359, in raise_if_disabled

    'route': ['like', '%{0}'.format(path)]

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 534, in get_all

    return frappe.get_all(*args, **kwargs)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1376, in get_all

    return get_list(doctype, *args, **kwargs)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1349, in get_list

    return frappe.model.db_query.DatabaseQuery(doctype).execute(None, *args, **kwargs)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/db_query.py", line 98, in execute

    self.columns = self.get_table_columns()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/db_query.py", line 335, in get_table_columns

    return get_table_columns(self.doctype)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/meta.py", line 49, in get_table_columns

    return frappe.db.get_table_columns(doctype)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 887, in get_table_columns

    columns = self.get_db_table_columns('tab' + doctype)

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 878, in get_db_table_columns

    where table_name = %s ''', table)]

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 122, in sql

    self.connect()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 75, in connect

    self._conn = self.get_connection()

  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/database.py", line 91, in get_connection

    local_infile = frappe.conf.local_infile)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/__init__.py", line 94, in Connect

    return Connection(*args, **kwargs)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 325, in __init__

    self.connect()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 599, in connect

    self._request_authentication()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 861, in _request_authentication

    auth_packet = self._read_packet()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 684, in _read_packet

    packet.check_error()

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/protocol.py", line 220, in check_error

    err.raise_mysql_exception(self._data)

  File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception

    raise errorclass(errno, errval)

pymysql.err.OperationalError: (1045, "Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)")

My nginx conf is:

server {
        listen 443 ssl;

        root /config/www;
        index index.html index.htm index.php;

        server_name erp.mydomain.se;

        include /config/nginx/ssl.conf;

        client_max_body_size 0;

        location / {
#               auth_basic "Restricted";
#               auth_basic_user_file /config/nginx/.htpasswd;
                proxy_set_header Host $host;
                proxy_redirect http:// https://;
                proxy_http_version 1.1;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_buffering               off;
                proxy_ssl_verify              off;
#               include /config/nginx/proxy.conf;
                proxy_pass http://erpnext-nginx:80;
        }
}

I also get these errors:

2020-11-19 16:11:55+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.3.27+maria~focal started.

2020-11-19 16:11:55+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'

2020-11-19 16:11:55+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.3.27+maria~focal started.

2020-11-19 16:11:55 0 [Note] mysqld (mysqld 10.3.27-MariaDB-1:10.3.27+maria~focal) starting as process 1 ...

2020-11-19 16:11:55 0 [Note] InnoDB: Using Linux native AIO

2020-11-19 16:11:55 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

2020-11-19 16:11:55 0 [Note] InnoDB: Uses event mutexes

2020-11-19 16:11:55 0 [Note] InnoDB: Compressed tables use zlib 1.2.11

2020-11-19 16:11:55 0 [Note] InnoDB: Number of pools: 1

2020-11-19 16:11:55 0 [Note] InnoDB: Using SSE2 crc32 instructions

2020-11-19 16:11:55 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M

2020-11-19 16:11:55 0 [Note] InnoDB: Completed initialization of buffer pool

2020-11-19 16:11:55 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

2020-11-19 16:11:55 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=34502546

2020-11-19 16:11:55 0 [Note] InnoDB: 128 out of 128 rollback segments are active.

2020-11-19 16:11:55 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"

2020-11-19 16:11:55 0 [Note] InnoDB: Creating shared tablespace for temporary tables

2020-11-19 16:11:55 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...

2020-11-19 16:11:55 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.

2020-11-19 16:11:55 0 [Note] InnoDB: Waiting for purge to start

2020-11-19 16:11:56 0 [Note] InnoDB: 10.3.27 started; log sequence number 34502555; transaction id 10516

2020-11-19 16:11:56 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool

2020-11-19 16:11:56 0 [Note] Plugin 'FEEDBACK' is disabled.

2020-11-19 16:11:56 0 [Note] Recovering after a crash using tc.log

2020-11-19 16:11:56 0 [Note] Starting crash recovery...

2020-11-19 16:11:56 0 [Note] Crash recovery finished.

2020-11-19 16:11:56 0 [Note] Server socket created on IP: '::'.

2020-11-19 16:11:56 0 [Warning] 'proxies_priv' entry '@% root@24f22b00c781' ignored in --skip-name-resolve mode.

2020-11-19 16:11:56 0 [Note] Reading of all Master_info entries succeeded

2020-11-19 16:11:56 0 [Note] Added new Master_info '' to hash table

2020-11-19 16:11:56 0 [Note] mysqld: ready for connections.

Version: '10.3.27-MariaDB-1:10.3.27+maria~focal' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

2020-11-19 16:11:58 0 [Note] InnoDB: Buffer pool(s) load completed at 201119 16:11:58

2020-11-19 16:13:09 13 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:13:09 14 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:14:09 15 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:14:09 16 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:15:09 17 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:15:09 18 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:16:09 19 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:16:09 20 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:16:37 21 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)

2020-11-19 16:16:38 22 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)

2020-11-19 16:17:09 23 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:17:09 24 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:18:10 25 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:18:10 26 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:19:10 27 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:19:10 28 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:20:10 29 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:20:10 30 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:21:10 31 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:21:10 32 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:21:22 33 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)

2020-11-19 16:21:22 34 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.7' (using password: YES)

2020-11-19 16:22:10 35 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:22:10 36 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:23:10 37 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:23:10 38 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:24:10 39 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:24:10 40 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:25:10 41 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:25:10 42 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:26:10 43 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:26:10 44 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:27:10 45 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:27:10 46 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:28:10 47 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:28:10 48 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:29:11 49 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:29:11 50 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:30:11 51 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:30:11 52 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:31:11 53 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:31:11 54 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:32:11 55 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:32:11 56 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:33:11 57 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:33:11 58 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:34:11 59 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:34:11 60 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:35:11 61 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:35:11 62 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

2020-11-19 16:36:11 63 [Warning] Access denied for user '_983de9e0903cd073'@'172.23.0.10' (using password: YES)

2020-11-19 16:36:11 64 [Warning] Access denied for user '_aa86fed04bbeade2'@'172.23.0.10' (using password: YES)

There seems to be some issue with MariaDB or the Python App.

I don’t know why it is not allowing Mariadb connection.

Can you manually create/update user with host as ‘%’ and same password which is in site_config.json.

This should be automatic ideally.

What container is this site_config in?

You mean add a user to mariadb?

I do not seem to be able to access the mariadb by using the password from the .env file

The site-creator image that exist in the single bench install does not exist in the multi bench version. Is there some command that needs to be run stop getting 404 errors? I have managed to get the rest to not give errors in the logs at least

For the purpose of playing I found this image instead: Trial Setup · pipech/erpnext-docker-debian Wiki · GitHub

One container solution, I can just point my reverse proxy to the correct port.

1 Like

I am getting similar error and i have updated the mariadb to with the requirements you mentioned here still it does not solve the problem.

my installatin:
-production

  • using docker

Any help or guidance to solve this problem.

you could find this file in : /var/lib/docker/volumes/frappe_docker_sites-vol/_data/<name_of_you_site>/site_config.json with Linux and docker installation of corse.