Kubernetes, step by step - apowerb by thaink2

This page prepares the cluster: everything here is kubectl and plain manifests, no Helm. The application itself is installed afterwards, either by the Helm chart or with kubectl alone (Install without Helm), once the cluster passes the checks below. The files cited below live in the hosting repository. Run the commands from a clone of it:

git clone https://github.com/apowerb/apowerb-hosting.git
cd apowerb-hosting

Every step was run on a real cluster (k0s v1.36.3, three nodes) before being written down. The commands are the ones that worked; the Traps section at the end lists what went wrong on the way, with the symptom that gives each one away.

What you need before starting


A Kubernetes cluster	1.28 or later. Three nodes of 4 vCPU / 8 GB is comfortable for the full stack
`kubectl`	within one minor version of the server
A storage class	the cluster does not necessarily have one — step 2
An ingress controller	for a public address — step 3
A DNS name	pointed at the ingress load balancer

1. Reach the cluster

export KUBECONFIG=~/.kube/config
kubectl get nodes

Every node must be Ready. kubectl version should show a client at most one minor version away from the server; further apart, some commands fail in puzzling ways.

2. Storage

Check first — this is the single most common reason nothing starts.

kubectl get storageclass

No resources found means no volume can ever be created: PostgreSQL and the data volume would stay Pending forever, and the pods with them. Managed clusters differ wildly here; some ship a class, some only offer object storage (which does not suit PostgreSQL). If the list is empty, local-path-provisioner gives you node-local volumes in one manifest:

curl -sL -o local-path-storage.yaml \
  https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.37/deploy/local-path-storage.yaml
kubectl apply -f local-path-storage.yaml
kubectl -n local-path-storage rollout status deploy/local-path-provisioner --timeout=120s

Make it the default, otherwise claims that name no class — the chart’s — stay unbound:

kubectl patch storageclass local-path \
  -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl get storageclass

You want to read local-path (default).

A node-local volume lives on one node’s disk. It is enough for a single PostgreSQL replica and for ReadWriteOnce data, and it is not highly available: losing the node loses the volume. Back the database up, or use your provider’s block CSI driver when it has one.

Prove it works before going further — and mind that the check needs a pod, not just a claim (Trap 1):

kubectl apply -f - <<'YAML'
apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: probe-pvc}
spec:
  accessModes: ["ReadWriteOnce"]
  resources: {requests: {storage: 1Gi}}
---
apiVersion: v1
kind: Pod
metadata: {name: probe-pod}
spec:
  restartPolicy: Never
  containers:
    - name: writer
      image: busybox:1.37
      command: ["sh","-c","echo ok > /data/marker && cat /data/marker && sleep 20"]
      volumeMounts: [{name: data, mountPath: /data}]
  volumes:
    - {name: data, persistentVolumeClaim: {claimName: probe-pvc}}
YAML
sleep 25 && kubectl get pvc probe-pvc && kubectl logs probe-pod
kubectl delete pvc probe-pvc pod probe-pod

Bound, and ok in the logs. Anything else, stop here: the rest cannot work.

3. Ingress controller

Needed only for a public address; skip it if you will use port-forward. Install Traefik or ingress-nginx — most managed Kubernetes offerings have both as a one-click add-on, which is the simplest path since the provider then also wires the load balancer. Then:

kubectl get ingressclass
kubectl get svc -A | grep LoadBalancer

Note the class name (traefik, nginx…) and the EXTERNAL-IP of the controller’s service: that address is what your DNS record will point to.

4. cert-manager and an issuer

Installing cert-manager is not enough — it issues nothing until an issuer exists.

kubectl get pods -n cert-manager

Then create the issuers. Two of them: production, and staging — which you use first, because Let’s Encrypt blocks an account for a week after five failures in an hour, and the first attempt rarely succeeds.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: you@example.com          # Let's Encrypt mails expiry notices here
    privateKeySecretRef:
      name: letsencrypt-staging-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik   # your class from step 3

The production issuer is the same with https://acme-v02.api.letsencrypt.org/directory and its own key secret name. Both are ready-made in the hosting repository under k8s/cert-manager/cluster-issuer-letsencrypt.yaml. Replace the email: of both issuers with a mailbox you read before applying it: the file ships with a maintainer’s address.

kubectl apply -f k8s/cert-manager/cluster-issuer-letsencrypt.yaml
kubectl get clusterissuer

READY: True on both means cert-manager registered with the ACME servers.

Use ingressClassName, not class. The latter goes through the legacy kubernetes.io/ingress.class annotation and is only recommended for ingress-gce. With it, the challenge Ingress is created without a recognised class, the controller ignores it, and the certificate sits at Ready: False while nothing reports an error.

5. HTTPS redirect (Traefik)

A certificate does not close the clear-text entrance: once the site is public, http:// keeps serving it unless something redirects. With Traefik, that something is a Middleware, cluster-side, ready-made in the hosting repository:

kubectl create namespace apowerb      # the Middleware lives in the application's namespace
kubectl apply -f k8s/traefik/redirect-https.yaml
kubectl -n apowerb get middleware redirect-https

The application’s Ingress opts into it with one annotation — see Going public. With ingress-nginx there is nothing to apply: ssl-redirect acts as soon as the Ingress has a TLS section.

The manifests in `k8s/`

The hosting repository’s k8s/ folder holds four sets of manifests. Those under k8s/apowerb/ are rendered from the chart, so both install paths deploy the same stack.

Path	What it is
`k8s/apowerb/`	the whole application: PostgreSQL, backend, interface, th2etl and its seed Job, th2pulse, the collector, the data volume
`k8s/apowerb-public/`	the Ingress and a backend that knows its public URL, for `apowerb.example.com`
`k8s/cert-manager/cluster-issuer-letsencrypt.yaml`	the staging and production issuers — step 4
`k8s/traefik/redirect-https.yaml`	the HTTP → HTTPS Middleware — step 5

Install without Helm

./scripts/generate-k8s-secret.sh        # writes k8s/secret.yaml, mode 600
kubectl apply -f k8s/apowerb/00-namespace.yaml -f k8s/secret.yaml
kubectl apply -f k8s/apowerb/
kubectl -n apowerb get pods

Expected: six pods Running and apowerb-th2etl-seed Completed. Back up k8s/secret.yaml and never regenerate it: a new ENCRYPT_KEY makes every connected integration unreadable. Reach the interface with kubectl -n apowerb port-forward svc/apowerb-frontend 13000:3000. Public address. Copy apowerb-public/, put your domain in, apply the copy:

cp -r k8s/apowerb-public my-public
sed -i.bak 's/apowerb\.example\.com/apowerb.your-domain.tld/' my-public/*.yaml
kubectl apply -f my-public/
kubectl -n apowerb get certificate

It uses the staging issuer. Once the certificate is READY, switch the cert-manager.io/cluster-issuer annotation to letsencrypt-prod and delete the apowerb-tls secret. Upgrade. Keep the same secret.yaml. The seed Job is immutable: delete it first.

git pull
kubectl -n apowerb delete job apowerb-th2etl-seed --ignore-not-found
kubectl apply -f k8s/apowerb/

With a public address, skip k8s/apowerb/backend-deployment.yaml and apply my-public/ instead: the plain backend has no public URL, and CORS and OAuth callbacks break until the public copy is back. Remove. kubectl delete -f k8s/apowerb/ deletes the namespace and both volumes, the database included. Nothing protects them, unlike the chart.

Next: install with Helm

Helm chart

Secrets, install, checks, public address, first administrator, upgrades.

Traps

Trap 1: a PVC that stays Pending on purpose

Symptom. You create a claim to test the storage; it stays Pending. It looks like the storage is broken. Cause. Storage classes with volumeBindingMode: WaitForFirstConsumer — local-path among them — only create the volume when a pod needs it. This is by design: the volume is placed on the node the pod is scheduled to. Remedy. Test with a claim and a pod, as in step 2. kubectl get storageclass shows the mode.

Trap 2: no storage class at all

Symptom. Same Pending, but with a pod waiting too, and kubectl describe pvc says: no persistent volumes available for this claim and no storage class is set. Cause. The cluster has no default storage class — common on lean managed clusters. Remedy. Step 2. Check kubectl get storageclass before installing, not after wondering why nothing starts.

Trap 3: a certificate stuck at Ready: False

Symptom. kubectl get certificate shows READY: False and stays there, without an obvious error. Causes, in the order worth checking.

The DNS name does not resolve to the ingress load balancer yet, or port 80 is closed — the HTTP-01 challenge needs both.
The issuer uses class instead of ingressClassName (see step 4).
The Let’s Encrypt rate limit has been hit — five failures in an hour blocks the account for a week. This is what the staging issuer is for.

kubectl -n apowerb describe certificate apowerb-tls
kubectl -n apowerb get challenge

The Challenge object names the real reason.

​What you need before starting

​1. Reach the cluster

​2. Storage

​3. Ingress controller

​4. cert-manager and an issuer

​5. HTTPS redirect (Traefik)

​The manifests in k8s/

​Install without Helm

​Next: install with Helm

Helm chart

​Traps

​Trap 1: a PVC that stays Pending on purpose

​Trap 2: no storage class at all

​Trap 3: a certificate stuck at Ready: False

What you need before starting

1. Reach the cluster

2. Storage

3. Ingress controller

4. cert-manager and an issuer

5. HTTPS redirect (Traefik)

The manifests in `k8s/`

Install without Helm

Next: install with Helm

Traps

Trap 1: a PVC that stays Pending on purpose

Trap 2: no storage class at all

Trap 3: a certificate stuck at Ready: False