How I installed ERPNext on a Kubernetes cluster on GKE

I probably spent 30 hours to figure this out, playing with k3s on EC2, EKS and finally GKE. I’m not an expert with K8s, beginner level I would consider myself.

A great deal of help from @revant_one helped me to get here, so special thanks to him.

Here are the exact steps I did.

  1. Create a K8s cluster. In this case I used GKE, but other K8s clusters should work too. This step is irrelevant to ERPNext installation.

  2. Install an in-cluster NFS service

The special thing here is to make the storage larger than 8GiB, otherwise ERPNext will complain when claiming PV.

# prepare NFS, this NFS has to be slightly larger than 8GiB, or ERPNext will fail to bind the storage
kubectl create namespace nfs
helm repo add nfs-ganesha-server-and-external-provisioner https://kubernetes-sigs.github.io/nfs-ganesha-server-and-external-provisioner
helm upgrade --install -n nfs in-cluster nfs-ganesha-server-and-external-provisioner/nfs-server-provisioner --set 'storageClass.mountOptions={vers=4.1}' --set persistence.enabled=true --set persistence.size=9Gi
  1. Install ERPNext

Nothing special here, these are just copy and paste from the helm chart.

# install ERPNext, don't use custom values
kubectl create namespace erpnext
helm repo add frappe https://helm.erpnext.com
helm upgrade --install frappe-bench --namespace erpnext frappe/erpnext --set persistence.worker.storageClass=nfs
  1. Create a site, modify the values.yaml file which you can find in the helm chart. This is just following up the guide.
jobs:
  createSite:
    enabled: true
    siteName: "erp.example.com"
    adminPassword: "secret"

Run the following commands:

helm template frappe-bench -n erpnext frappe/erpnext -f custom-values.yaml -s templates/job-create-site.yaml > create-new-site-job.yaml
kubectl apply -f create-new-site-job.yaml -n erpnext
  1. Verify the site is working. You can certainly check the pods in the K8s cluster too.
kubectl port-forward -n erpnext svc/frappe-bench-erpnext 8080:8080
curl -H "Host: erp.example.com" http://0.0.0.0:8080/api/method/ping

It should return:

{"message":"pong"}
  1. Install ingress-nginx
helm install nginx-ingress ingress-nginx/ingress-nginx
  1. Create a new ingress to route the traffic from internet

Create a yaml file called ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: erp-resource
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - host: "erp.example.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: frappe-bench-erpnext
            port:
              number: 8080
kubectl apply -f ingress.yaml -n erpnext

Then you should be able to visit it from erp.example.com.

This is only http, so you will get insecure access warning from the browser.

2 Likes

if budget is not the limit then go for Service tiers  |  Filestore  |  Google Cloud
instead of in cluster nfs service

CSI installation: Using the Filestore CSI driver  |  Kubernetes Engine Documentation  |  Google Cloud

also instead of In cluster MariaDB, use Google SkySQL

Hi Revant,

Thanks for your inputs. These are great suggestions for taking the basic k8s setup to another level with fully managed persistent storage both for volumes and database.

GKE setup is my test bench for now. I’m also considering AWS options, e.g., EKS + EFS + RDS.

I’m learning managing the letsencrypt cert manager within the cluster for now.

Here are steps for it. castlecraft / aws-eks-erpnext · GitLab

1 Like

Hi Revant,

This is most helpful, I’m spoiled.

thank you for posting this

I don’t know if this needs separate posts.

here’s kubernetes operator that can be used to automate ingress creation on site creation.

it is multiple things:

  • python app
  • fastapi app serving rest api
  • frappe app (bench get-app)
  • kopf k8s operator that watches jobs and manages ingress

source: castlecraft / k8s_bench · GitLab
docs and helm chart: https://k8s-bench.castlecraft.in/

here’s another app that creates benches and site, it is obsolete and uses old Fluxcd API. It was used to host feature board that created sites based on GitHub pull request. I’m sharing it for code reference.