dominik/k3s
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

refinement

Browse Source
This commit is contained in:
Dominik Chilla 2021-11-27 20:56:14 +01:00
parent 4bc0d45de5
commit 2fc6180697
1 changed files with 68 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

169
README.md
View File

@ -1,6 +1,6 @@
* [kubectl - BASH autocompletion](#kubectl-bash-autocompletion) * [kubectl - BASH autocompletion](#kubectl-bash-autocompletion)
* [Install k3s](#install-k3s) * [Install k3s](#install-k3s)
* [On on-premises](#install-k3s-on-premises) * [On premises/IaaS](#install-k3s-on-premises)
* [Configure upstream DNS-resolver](#upstream-dns-resolver) * [Configure upstream DNS-resolver](#upstream-dns-resolver)
* [Change NodePort range](#nodeport-range) * [Change NodePort range](#nodeport-range)
* [Clustering](#clustering) * [Clustering](#clustering)
@ -8,9 +8,8 @@
* [Namespaces and resource limits](#namespaces-limits) * [Namespaces and resource limits](#namespaces-limits)
* [Persistent volumes (StorageClass - dynamic provisioning)](#pv) * [Persistent volumes (StorageClass - dynamic provisioning)](#pv)
* [Rancher Local](#pv-local) * [Rancher Local](#pv-local)
* [Rancher Longhorn - distributed in local cluster](#pv-longhorn) * [Rancher Longhorn (distributed in local cluster) - MY FAVOURITE :-)](#pv-longhorn)
* [NFS](#pv-nfs) * [NFS](#pv-nfs)
* [Seaweedfs](#pv-seaweedfs)
* [Ingress controller](#ingress-controller) * [Ingress controller](#ingress-controller)
* [Disable Traefik-ingress](#disable-traefik-ingress) * [Disable Traefik-ingress](#disable-traefik-ingress)
* [Enable NGINX-ingress with OCSP stapling](#enable-nginx-ingress) * [Enable NGINX-ingress with OCSP stapling](#enable-nginx-ingress)
@ -31,12 +30,12 @@
* [Get deployment history](#helm-history) * [Get deployment history](#helm-history)
* [Rollback](#helm-rollback) * [Rollback](#helm-rollback)
* [Kubernetes in action](#kubernetes-in-action) * [Kubernetes in action](#kubernetes-in-action)
* [Running DaemonSets on `hostPort`](#running-daemonsets) * [Running DaemonSets with `hostNetwork: true`](#running-daemonsets)
* [Running StatefulSet with NFS storage](#running-statefulset-nfs) * [Running StatefulSet with NFS storage](#running-statefulset-nfs)
* [Services](#services) * [Services](#services)
* [Client-IP transparency and loadbalancing](#services-client-ip-transparency) * [Client-IP transparency and loadbalancing](#services-client-ip-transparency)
* [Session affinity/persistence](#services-session-persistence) * [Session affinity/persistence](#services-session-persistence)
* [Keep your cluster balanced](#keep-cluster-balanced) * [Keeping the cluster balanced](#keep-cluster-balanced)
* [Node maintenance](#node-maintenance) * [Node maintenance](#node-maintenance)
* [What happens if a node goes down?](#what-happens-node-down) * [What happens if a node goes down?](#what-happens-node-down)
* [Dealing with disruptions](#disruptions) * [Dealing with disruptions](#disruptions)
@ -55,36 +54,12 @@ echo "source <(kubectl completion bash)" >> ~/.bashrc
``` ```
# Install k3s <a name="user-content-install-k3s"></a> # Install k3s <a name="user-content-install-k3s"></a>
## On premises <a name="user-content-install-k3s-on-premises"></a> ## On premises/IaaS <a name="user-content-install-k3s-on-premises"></a>
https://k3s.io/: https://k3s.io/:
``` ```
curl -sfL https://get.k3s.io | sh - curl -sfL https://get.k3s.io | sh -
``` ```
If disired, set a memory consumption limit of the systemd-unit like so:
```
root#> mkdir /etc/systemd/system/k3s.service.d
root#> vi /etc/systemd/system/k3s.service.d/limits.conf
[Service]
MemoryMax=1024M
root#> systemctl daemon-reload
root#> systemctl restart k3s
root#> systemctl status k3s
k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/k3s.service.d
└─limits.conf
Active: active (running) since Thu 2020-11-26 10:46:26 CET; 13min ago
Docs: https://k3s.io
Process: 9618 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 9619 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 9620 (k3s-server)
Tasks: 229
Memory: 510.6M (max: 1.0G)
CGroup: /system.slice/k3s.service
```
### Upstream DNS-resolver <a name="user-content-upstream-dns-resolver"></a> ### Upstream DNS-resolver <a name="user-content-upstream-dns-resolver"></a>
Docs: https://rancher.com/docs/rancher/v2.x/en/troubleshooting/dns/ Docs: https://rancher.com/docs/rancher/v2.x/en/troubleshooting/dns/
@ -114,7 +89,7 @@ ExecStart=/usr/local/bin/k3s \
If you want to build a K3s-cluster the default networking model is *overlay@VXLAN*. In this case make sure that If you want to build a K3s-cluster the default networking model is *overlay@VXLAN*. In this case make sure that
* all of your nodes can reach (ping) each other over the underlying network (local, routed/vpn). This is required for the overlay network to work properly. VXLAN spans a mashed network over all K3s-nodes. * all of your nodes can reach (ping) each other over the underlying network (local, routed/vpn). This is required for the overlay network to work properly. VXLAN spans a mashed network over all K3s-nodes.
* if your nodes are spread over public networks (like the internet) use a VPN (like IPSec or OpenVPN) to secure the traffic between the nodes. **VXLAN uses plain UDP for transport!** * if your nodes are spread over public networks (like the internet) use a VPN (like IPSec or OpenVPN) to secure the traffic between the nodes. **VXLAN uses plain UDP for transport!**
* if your nodes are connected through VPN, `flannel` (overlay network daemon) should explicitly communicate over the vpn network interface instead of the public network interface. Following settings should be made on the nodes: * if your nodes are connected through VPN, `flannel` (overlay network daemon) should explicitly communicate via the vpn network interface instead of the public network interface. Following settings should be made on the nodes:
``` ```
/etc/systemd/system/k3s-agent.service: /etc/systemd/system/k3s-agent.service:
@ -164,7 +139,7 @@ and change the content to
``` ```
forward . ipaddr.of.your.dns-resolver forward . ipaddr.of.your.dns-resolver
``` ```
Finally redeploy the CoreDNS deployment with: Finally re-deploy the CoreDNS deployment with:
`kubectl -n kube-system rollout restart deployment coredns` `kubectl -n kube-system rollout restart deployment coredns`
**Note:** If you restart the cluster (`k3d cluster stop your-cluster` and `k3d cluster start your-cluster`), the changes will be gone! **Note:** If you restart the cluster (`k3d cluster stop your-cluster` and `k3d cluster start your-cluster`), the changes will be gone!
@ -180,7 +155,7 @@ Read more about [AccessModes](https://kubernetes.io/docs/concepts/storage/persis
https://rancher.com/docs/k3s/latest/en/storage/ https://rancher.com/docs/k3s/latest/en/storage/
Only supports *AccessMode*: ReadWriteOnce (RWO) Only supports *AccessMode*: ReadWriteOnce (RWO)
## Longhorn (distributed in local cluster) <a name="user-content-pv-longhorn"></a> ## Rancher Longhorn (distributed in local cluster) - MY FAVOURITE :-) <a name="user-content-pv-longhorn"></a>
* Requirements: https://longhorn.io/docs/0.8.0/install/requirements/ * Requirements: https://longhorn.io/docs/0.8.0/install/requirements/
* Debian: `apt install open-iscsi` * Debian: `apt install open-iscsi`
* Install: https://rancher.com/docs/k3s/latest/en/storage/ * Install: https://rancher.com/docs/k3s/latest/en/storage/
@ -239,11 +214,6 @@ spec:
requests: requests:
storage: 32Mi storage: 32Mi
``` ```
## Seaweedfs <a name="user-content-pv-seaweedfs"></a>
Docs: https://github.com/seaweedfs
Docs: https://github.com/seaweedfs/seaweedfs-csi-driver
In order to use the CSI driver you need to have a working seaweedfs-cluster. As seaweedfs is really lightweight it can be deployed on a bunch (at least three) of raspberries (min. version 3) as well as on the K3s cluster too.
# Ingress controller <a name="user-content-ingress-controller"></a> # Ingress controller <a name="user-content-ingress-controller"></a>
## Disable Traefik-ingress <a name="user-content-disable-traefik-ingress"></a> ## Disable Traefik-ingress <a name="user-content-disable-traefik-ingress"></a>
@ -251,7 +221,7 @@ edit /etc/systemd/system/k3s.service:
``` ```
[...] [...]
ExecStart=/usr/local/bin/k3s \ ExecStart=/usr/local/bin/k3s \
server --disable traefik --resolv-conf /etc/resolv.conf \ server [...] --disable traefik \
[...] [...]
``` ```
Finally `systemctl daemon-reload` and `systemctl restart k3s` Finally `systemctl daemon-reload` and `systemctl restart k3s`
@ -264,36 +234,37 @@ https://kubernetes.github.io/ingress-nginx/deploy/#using-helm
``` ```
kubectl create ns ingress-nginx kubectl create ns ingress-nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install my-release ingress-nginx/ingress-nginx -n ingress-nginx helm install nginx-ingress ingress-nginx/ingress-nginx -n ingress-nginx
``` ```
`kubectl -n ingress-nginx get all`: `kubectl -n ingress-nginx get all`:
``` ```
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
pod/svclb-my-release-ingress-nginx-controller-m6gxl 2/2 Running 0 110s pod/svclb-nginx-ingress-controller-m6gxl 2/2 Running 0 110s
pod/my-release-ingress-nginx-controller-695774d99c-t794f 1/1 Running 0 110s pod/nginx-ingress-controller-695774d99c-t794f 1/1 Running 0 110s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/my-release-ingress-nginx-controller-admission ClusterIP 10.43.116.191 <none> 443/TCP 110s service/nginx-ingress-controller-admission ClusterIP 10.43.116.191 <none> 443/TCP 110s
service/my-release-ingress-nginx-controller LoadBalancer 10.43.55.41 192.168.178.116 80:31110/TCP,443:31476/TCP 110s service/nginx-ingress-controller LoadBalancer 10.43.55.41 192.168.178.116 80:31110/TCP,443:31476/TCP 110s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/svclb-my-release-ingress-nginx-controller 1 1 1 1 1 <none> 110s daemonset.apps/svclb-nginx-ingress-controller 1 1 1 1 1 <none> 110s
NAME READY UP-TO-DATE AVAILABLE AGE NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/my-release-ingress-nginx-controller 1/1 1 1 110s deployment.apps/nginx-ingress-controller 1/1 1 1 110s
NAME DESIRED CURRENT READY AGE NAME DESIRED CURRENT READY AGE
replicaset.apps/my-release-ingress-nginx-controller-695774d99c 1 1 1 110s replicaset.apps/nginx-ingress-controller-695774d99c 1 1 1 110s
``` ```
As nginx ingress is hungry for memory, let´s reduce the number of workers to 1: The nginx ingress global configuration can be modified as follows:
``` ```
kubectl -n ingress-nginx edit configmap my-release-ingress-nginx-controller kubectl -n ingress-nginx edit configmap ingress-nginx-controller
apiVersion: v1 apiVersion: v1
<<<ADD BEGINN>>> <<<ADD BEGINN>>>
data: data:
enable-ocsp: "true" enable-ocsp: "true"
use-gzip: "true"
worker-processes: "1" worker-processes: "1"
<<<ADD END>>> <<<ADD END>>>
kind: ConfigMap kind: ConfigMap
@ -301,13 +272,13 @@ kind: ConfigMap
``` ```
Finally the deployment needs to be restarted: Finally the deployment needs to be restarted:
`kubectl -n ingress-nginx rollout restart deployment my-release-ingress-nginx-controller` `kubectl -n ingress-nginx rollout restart deployment ingress-nginx-controller`
**If you are facing deployment problems like the following one** **If you are facing deployment problems like the following one**
``` ```
Error: UPGRADE FAILED: cannot patch "gitea-ingress-staging" with kind Ingress: Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://my-release-ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1beta1/ingresses?timeout=10s: context deadline exceeded Error: UPGRADE FAILED: cannot patch "gitea-ingress-staging" with kind Ingress: Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://nginx-ingress-controller-admission.ingress-nginx.svc:443/networking/v1beta1/ingresses?timeout=10s: context deadline exceeded
``` ```
A possible fix: `kubectl -n ingress-nginx delete ValidatingWebhookConfiguration my-release-ingress-nginx-admission` A possible fix: `kubectl -n ingress-nginx delete ValidatingWebhookConfiguration ingress-nginx-admission`
# Cert-Manager (references ingress controller) <a name="user-content-cert-manager"></a> # Cert-Manager (references ingress controller) <a name="user-content-cert-manager"></a>
## Installation <a name="user-content-cert-manager-install"></a> ## Installation <a name="user-content-cert-manager-install"></a>
@ -325,7 +296,7 @@ kubectl -n cert-manager get all
## Cluster-internal CA Issuer <a name="user-content-cert-manager-cluster-ca-issuer"></a> ## Cluster-internal CA Issuer <a name="user-content-cert-manager-cluster-ca-issuer"></a>
Docs: https://cert-manager.io/docs/configuration/ca/ Docs: https://cert-manager.io/docs/configuration/ca/
## Let´s Encrypt issuer <a name="user-content-cert-manager-le-issuer"></a> ## Let´s Encrypt HTTP-01 issuer <a name="user-content-cert-manager-le-issuer"></a>
Docs: https://cert-manager.io/docs/tutorials/acme/ingress/#step-6-configure-let-s-encrypt-issuer Docs: https://cert-manager.io/docs/tutorials/acme/ingress/#step-6-configure-let-s-encrypt-issuer
``` ```
ClusterIssuers are a resource type similar to Issuers. They are specified in exactly the same way, ClusterIssuers are a resource type similar to Issuers. They are specified in exactly the same way,
@ -338,7 +309,7 @@ lets-encrypt-cluster-issuers.yaml:
apiVersion: cert-manager.io/v1 apiVersion: cert-manager.io/v1
kind: ClusterIssuer kind: ClusterIssuer
metadata: metadata:
name: letsencrypt-staging-issuer name: letsencrypt-http01-staging-issuer
spec: spec:
acme: acme:
# You must replace this email address with your own. # You must replace this email address with your own.
@ -358,7 +329,7 @@ spec:
apiVersion: cert-manager.io/v1 apiVersion: cert-manager.io/v1
kind: ClusterIssuer kind: ClusterIssuer
metadata: metadata:
name: letsencrypt-prod-issuer name: letsencrypt-http01-prod-issuer
spec: spec:
acme: acme:
# The ACME server URL # The ACME server URL
@ -386,7 +357,7 @@ data:
apiVersion: cert-manager.io/v1 apiVersion: cert-manager.io/v1
kind: ClusterIssuer kind: ClusterIssuer
metadata: metadata:
name: letsencrypt-dns01-issuer name: letsencrypt-dns01-prod-issuer
spec: spec:
acme: acme:
email: user@example.com email: user@example.com
@ -411,9 +382,9 @@ spec:
`kubectl apply -f lets-encrypt-cluster-issuers.yaml` `kubectl apply -f lets-encrypt-cluster-issuers.yaml`
## Deploying a LE-certificate with ingress <a name="user-content-cert-manager-ingress"></a> ## Deploying a LE-certificate with ingress <a name="user-content-cert-manager-ingress"></a>
All you need is an `Ingress` resource of class `nginx` which references a ClusterIssuer (`letsencrypt-prod-issuer`) resource. All you need is an `Ingress` resource of class `nginx` which references a ClusterIssuer (`letsencrypt-http01-prod-issuer`) resource.
HTTP-01 solver (`cert-manager.io/cluster-issuer: "letsencrypt-prod-issuer"`): HTTP-01 solver (`cert-manager.io/cluster-issuer: "letsencrypt-http01-prod-issuer"`):
``` ```
apiVersion: networking.k8s.io/v1beta1 apiVersion: networking.k8s.io/v1beta1
kind: Ingress kind: Ingress
@ -423,7 +394,7 @@ metadata:
annotations: annotations:
# use the shared ingress-nginx # use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx" kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-prod-issuer" cert-manager.io/cluster-issuer: "letsencrypt-http01-prod-issuer"
spec: spec:
tls: tls:
- hosts: - hosts:
@ -438,7 +409,7 @@ spec:
serviceName: some-target-service serviceName: some-target-service
servicePort: some-target-service-port servicePort: some-target-service-port
``` ```
DNS-01 solver (`cert-manager.io/cluster-issuer: "letsencrypt-dns01-issuer"`): DNS-01 solver (`cert-manager.io/cluster-issuer: "letsencrypt-dns01-prod-issuer"`):
``` ```
apiVersion: networking.k8s.io/v1beta1 apiVersion: networking.k8s.io/v1beta1
kind: Ingress kind: Ingress
@ -448,7 +419,7 @@ metadata:
annotations: annotations:
# use the shared ingress-nginx # use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx" kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-dns01-issuer" cert-manager.io/cluster-issuer: "letsencrypt-dns01-prod-issuer"
spec: spec:
tls: tls:
- hosts: - hosts:
@ -538,7 +509,7 @@ spec:
## Troubleshooting <a name="user-content-cert-manager-troubleshooting"></a> ## Troubleshooting <a name="user-content-cert-manager-troubleshooting"></a>
Docs: https://cert-manager.io/docs/faq/acme/ Docs: https://cert-manager.io/docs/faq/acme/
ClusterIssuer runs in default namespace: ClusterIssuers are *visible* any namespaces:
``` ```
kubectl get clusterissuer kubectl get clusterissuer
kubectl describe clusterissuer <object> kubectl describe clusterissuer <object>
@ -555,7 +526,9 @@ kubectl -n <stage> get challenge
kubectl -n <stage> describe challenge <object> kubectl -n <stage> describe challenge <object>
``` ```
After successfull setup perform a TLS-test: `https://www.ssllabs.com/ssltest/index.html` After successfull setup perform a TLS-test:
* https://testssl.sh/ (`apt install testssl.sh`)
* https://www.ssllabs.com/ssltest/index.html
# HELM charts <a name="user-content-helm"></a> # HELM charts <a name="user-content-helm"></a>
Docs: Docs:
@ -690,13 +663,12 @@ NOTES:
``` ```
# Kubernetes in action <a name="user-content-kubernetes-in-action"></a> # Kubernetes in action <a name="user-content-kubernetes-in-action"></a>
## Running DaemonSets on `hostPort` <a name="user-content-running-daemonsets"></a> ## Running DaemonSets with `hostNetwork: true` <a name="user-content-running-daemonsets"></a>
* Docs: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ * [Docs: DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/)
* Good article: https://medium.com/stakater/k8s-deployments-vs-statefulsets-vs-daemonsets-60582f0c62d4 * [(Security) hints on using `hostNetwork`](https://kubernetes.io/docs/concepts/policy/pod-security-policy/#host-namespaces)
* [Pod´s DNS policy](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy)
In this case configuration of networking in context of services is not needed. This setup is suitable for scenarios where kubernetes nodes are already running with dual-IP-stack (IPv4 and IPv6) and the Pod needs IPv6 too, but k3s was deployed in ipv4-only mode. In this case the Pod can be deployed in the network namespace of the kubernetes node.
This setup is suitable for legacy scenarios where static IP-address are required and a NodePort service is not an alternative:
``` ```
kind: DaemonSet kind: DaemonSet
apiVersion: apps/v1 apiVersion: apps/v1
@ -713,45 +685,21 @@ spec:
labels: labels:
app: netcat-daemonset app: netcat-daemonset
spec: spec:
containers: hostNetwork: true
- command: dnsPolicy: ClusterFirstWithHostNet
- nc
- -lk
- -p
- "23456"
- -v
- -e
- /bin/true
env:
- name: DEMO_GREETING
value: Hello from the environment
image: dockreg-zdf.int.zwackl.de/alpine/latest/amd64:prod
imagePullPolicy: IfNotPresent
name: netcat-daemonset
ports:
- containerPort: 23456
hostPort: 23456
protocol: TCP
resources:
limits:
cpu: 500m
memory: 64Mi
requests:
cpu: 50m
memory: 32Mi
restartPolicy: Always restartPolicy: Always
securityContext: {} terminationGracePeriodSeconds: 10
terminationGracePeriodSeconds: 30 containers:
updateStrategy: - name: alpine-netcat-daemonset
rollingUpdate: image: alpine
maxUnavailable: 1 imagePullPolicy: IfNotPresent
type: RollingUpdate command: ["nc", "-lk", "-p", "23456", "-v", "-e", "/bin/true"]
``` ```
## Running StatefulSet with NFS storage <a name="user-content-running-statefulset-nfs"></a> ## Running StatefulSet with NFS storage <a name="user-content-running-statefulset-nfs"></a>
* https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ * [Docs: StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/)
* [NFS dynamic volume provisioning deployed](#pv-nfs) * [NFS dynamic volume provisioning deployed](#pv-nfs)
**Be careful:** *StatefulSets* are designed for stateful applications (like databases). To avoid split-brain scenarios StatefulSets behave as static as possible. If a node goes down, the StatefulSet controller will reschedule the pods to another nodes, which can meet the requirements! If you want to force a re-scheduling: StatefulSets are designed for stateful applications (like databases). To avoid split-brain scenarios StatefulSets behave as static as possible. If a node goes down, the StatefulSet controller will reschedule the pods to another node, that meets the required conditions! If you want to force a re-scheduling:
`kubectl delete pod web-1 --grace-period=0 --force` `kubectl delete pod web-1 --grace-period=0 --force`
More details on this can be found [here](https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/) More details on this can be found [here](https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/)
@ -824,7 +772,6 @@ spec:
`externalTrafficPolicy: Local` preserves the original client ip-address which is visible to the PODs. In any case (`DaemonSet` or `StatefulSet`) traffic remains on the Node which gets the traffic. In case of `StatefulSet` if more than one POD of a `ReplicaSet` is scheduled on the same Node, the workload gets balanced over all PODs on the same Node. `externalTrafficPolicy: Local` preserves the original client ip-address which is visible to the PODs. In any case (`DaemonSet` or `StatefulSet`) traffic remains on the Node which gets the traffic. In case of `StatefulSet` if more than one POD of a `ReplicaSet` is scheduled on the same Node, the workload gets balanced over all PODs on the same Node.
### Session affinity/persistence <a name="user-content-services-session-persistence"></a> ### Session affinity/persistence <a name="user-content-services-session-persistence"></a>
``` ```
apiVersion: v1 apiVersion: v1
@ -841,7 +788,7 @@ spec:
Session persistence (`None` by default) is only supported on client ip-address. Cookie-stickiness or stickiness on any other/higher layer is not supported yet. Session persistence (`None` by default) is only supported on client ip-address. Cookie-stickiness or stickiness on any other/higher layer is not supported yet.
## What happens if a node goes down? <a name="user-content-what-happens-node-down"></a> ## What happens if a node goes down? <a name="user-content-what-happens-node-down"></a>
If a node goes down kubernetes marks this node as *NotReady*, but nothing else: If a node goes down kubernetes marks this node as *NotReady*, but nothing else happens until [Pod tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration) take place, which are per default configured with a *timeout* of `300s` (5 minutes!).
``` ```
$ kubectl get node $ kubectl get node
NAME STATUS ROLES AGE VERSION NAME STATUS ROLES AGE VERSION
@ -858,7 +805,7 @@ web-2 1/1 Running 0 26
ds-test-c6xx8 1/1 Running 0 18m ds-test-c6xx8 1/1 Running 0 18m
ds-test-w45dv 1/1 Running 5 28h ds-test-w45dv 1/1 Running 5 28h
``` ```
Kubernetes supports [Pod-tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration), which are per default configured with a *timeout* of `300s` (5 minutes!). This means, that affected Pods will *remain* for a timespan of 300s on a *broken* node before eviction takes place Pod tolerations can be determined like this:
``` ```
$ kubectl -n <namespace> describe pod <pod-name> $ kubectl -n <namespace> describe pod <pod-name>
@ -868,7 +815,7 @@ Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists fo
[...] [...]
``` ```
To be more reactive Pod-tolerations can be configured as follows: To be more reactive to node failures Pod tolerations can be adjusted within the Pod template as follows:
``` ```
kind: Deployment or StatefulSet kind: Deployment or StatefulSet
apiVersion: apps/v1 apiVersion: apps/v1
@ -891,10 +838,10 @@ spec:
[...] [...]
``` ```
## Keep your cluster balanced <a name="user-content-keep-cluster-balanced"></a> ## Keeping the cluster balanced <a name="user-content-keep-cluster-balanced"></a>
Kubernetes, in first place, takes care of high availability, but not of well balance of pod/node. In first place, Kubernetes takes care of high availability, but not of well balance of pods per node.
In case of `Deployment` or `StatefulSet` a `topologySpreadConstraint` needs to be specified: In case of `Deployment` or `StatefulSet` a [`topologySpreadConstraint`](https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/) needs to be specified:
``` ```
kind: Deployment or StatefulSet kind: Deployment or StatefulSet
apiVersion: apps/v1 apiVersion: apps/v1
@ -909,7 +856,7 @@ spec:
topologySpreadConstraints: topologySpreadConstraints:
- labelSelector: - labelSelector:
matchLabels: matchLabels:
app: {{ .Chart.Name }}-{{ .Values.stage }} app: the-app
maxSkew: 1 maxSkew: 1
topologyKey: kubernetes.io/hostname topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule whenUnsatisfiable: DoNotSchedule