https://github.com/chillout2k/ExOTA-Milter/issues/40

INSTALL/README.md

@@ -0,0 +1,98 @@
+# How to install ExOTA-Milter
+
+#### Table of contents
+[docker-compose](#docker-compose)  
+[kubernetes](#kubernetes)  
+[systemd](#systemd)  
+
+## docker-compose <a name="docker-compose"/>
+```
+~/src/ExOTA-Milter/INSTALL/docker-compose$ docker-compose up
+[+] Running 2/2
+ ⠿ Network docker-compose_default           Created                                                                                                                                                                                     0.8s
+ ⠿ Container docker-compose-exota-milter-1  Created                                                                                                                                                                                     0.1s
+Attaching to docker-compose-exota-milter-1
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,503: INFO 140529821924168 Logger initialized
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,503: INFO 140529821924168 ENV[MILTER_NAME]: exota-milter
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,503: INFO 140529821924168 ENV[MILTER_SOCKET]: inet:4321@0.0.0.0
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_REJECT_MESSAGE]: CUSTOMIZE THIS! - Security policy violation!!
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_TMPFAIL_MESSAGE]: Service temporarily not available! Please try again later.
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_TRUSTED_AUTHSERVID]: dkimauthservid
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_DKIM_ALIGNMENT_REQUIRED]: True
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_DKIM_ENABLED]: True
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_X509_TRUSTED_CN]: mail.protection.outlook.com
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_X509_IP_WHITELIST]: ['127.0.0.1', '::1']
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,504: INFO 140529821924168 ENV[MILTER_X509_ENABLED]: True
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,505: INFO 140529821924168 ENV[MILTER_AUTHSERVID]: ThisAuthservID
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,505: INFO 140529821924168 ENV[MILTER_ADD_HEADER]: True
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,505: INFO 140529821924168 ENV[MILTER_POLICY_SOURCE]: file
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,505: INFO 140529821924168 ENV[MILTER_POLICY_FILE]: /data/exota-milter-policy.json
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,511: INFO 140529821924168 JSON policy backend initialized
+docker-compose-exota-milter-1  | 2022-06-06 21:54:04,511: INFO 140529821924168 Startup exota-milter@socket: inet:4321@0.0.0.0
+```
+
+## kubernetes <a name="kubernetes"/>
+```
+~/src/ExOTA-Milter/INSTALL/kubernetes$ kubectl apply -f 01_config-map.yaml
+configmap/exota-milter-policy-cmap created
+
+~/src/ExOTA-Milter/INSTALL/kubernetes$ kubectl apply -f 02_deployment.yaml
+deployment.apps/exota-milter created
+
+~/src/ExOTA-Milter/INSTALL/kubernetes$ kubectl apply -f 03_service.yaml
+service/exota-milter created
+```
+Check status of pods, replica-sets and service
+```
+~/src/ExOTA-Milter/INSTALL/kubernetes$ kubectl -n devel get all
+NAME                                READY   STATUS    RESTARTS   AGE
+pod/exota-milter-547dbccd8b-j69mn   1/1     Running   0          64s
+pod/exota-milter-547dbccd8b-7hl6c   1/1     Running   0          64s
+pod/exota-milter-547dbccd8b-k4ng8   1/1     Running   0          64s
+
+NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
+service/exota-milter   ClusterIP   10.43.78.163   <none>        4321/TCP   61s
+
+NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
+deployment.apps/exota-milter   3/3     3            3           64s
+
+NAME                                      DESIRED   CURRENT   READY   AGE
+replicaset.apps/exota-milter-547dbccd8b   3         3         3       65s
+```
+Get logs of one of the pods:
+```
+~/src/ExOTA-Milter/INSTALL/kubernetes$ kubectl -n devel logs exota-milter-547dbccd8b-7hl6c
+2022-06-06 21:57:03,515: INFO Logger initialized
+2022-06-06 21:57:03,515: INFO ENV[MILTER_NAME]: exota-milter
+2022-06-06 21:57:03,515: INFO ENV[MILTER_SOCKET]: inet:4321@127.0.0.1
+2022-06-06 21:57:03,515: INFO ENV[MILTER_REJECT_MESSAGE]: Security policy violation!!
+2022-06-06 21:57:03,515: INFO ENV[MILTER_TMPFAIL_MESSAGE]: Service temporarily not available! Please try again later.
+2022-06-06 21:57:03,515: INFO ENV[MILTER_TRUSTED_AUTHSERVID]: dkimauthservid
+2022-06-06 21:57:03,515: INFO ENV[MILTER_DKIM_ALIGNMENT_REQUIRED]: True
+2022-06-06 21:57:03,515: INFO ENV[MILTER_DKIM_ENABLED]: True
+2022-06-06 21:57:03,515: INFO ENV[MILTER_X509_TRUSTED_CN]: mail.protection.outlook.com
+2022-06-06 21:57:03,515: INFO ENV[MILTER_X509_IP_WHITELIST]: ['127.0.0.1', '::1']
+2022-06-06 21:57:03,515: INFO ENV[MILTER_X509_ENABLED]: True
+2022-06-06 21:57:03,516: INFO ENV[MILTER_AUTHSERVID]: some-auth-serv-id
+2022-06-06 21:57:03,516: INFO ENV[MILTER_ADD_HEADER]: True
+2022-06-06 21:57:03,516: INFO ENV[MILTER_POLICY_SOURCE]: file
+2022-06-06 21:57:03,516: INFO ENV[MILTER_POLICY_FILE]: /data/exota-milter-policy.json
+2022-06-06 21:57:03,516: INFO JSON policy backend initialized
+2022-06-06 21:57:03,516: INFO Startup exota-milter@socket: inet:4321@127.0.0.1
+```
+
+## systemd <a name="systemd"/>
+If you do not want to run the ExOTA-Milter in a containerized environment but directly as a systemd-unit/-service, first you´ll need to install all necessary python dependencies:
+```
+~/src/ExOTA-Milter/INSTALL/systemd# sudo pip3 install -r ../../requirements.txt
+Requirement already satisfied: authres==1.2.0 in /usr/local/lib/python3.8/dist-packages (from -r ../../requirements.txt (line 1)) (1.2.0)
+Requirement already satisfied: pymilter==1.0.4 in /usr/local/lib/python3.8/dist-packages (from -r ../../requirements.txt (line 2)) (1.0.4)
+Requirement already satisfied: ldap3 in /usr/local/lib/python3.8/dist-packages (from -r ../../requirements.txt (line 3)) (2.9.1)
+Requirement already satisfied: pyasn1>=0.4.6 in /usr/local/lib/python3.8/dist-packages (from ldap3->-r ../../requirements.txt (line 3)) (0.4.8)
+```
+Next you should be able to install the ExOTA-Milter as well as the systemd-stuff by running the `install.sh` script:
+```
+~/src/ExOTA-Milter/INSTALL/systemd$ sudo ./install.sh
+Created symlink /etc/systemd/system/multi-user.target.wants/exota-milter.service → /lib/systemd/system/exota-milter.service.
+```
+Use the `uninstall.sh` script to uninstall the ExOTA-Milter from your systemd environment. Files under `/etc/exota-milter/` (config and policy) are kept undeleted!

INSTALL/docker-compose/data/exota-milter-policy.json

@@ -0,0 +1,7 @@
+{
+  "example.com": {
+    "tenant_id": "abcd1234-18c5-45e8-88de-987654321cba",
+    "dkim_enabled": true,
+    "dkim_alignment_required": true
+  }
+}

INSTALL/docker-compose/docker-compose.yaml

@@ -0,0 +1,22 @@
+version: '2.4'
+services:
+  exota-milter:
+    image: chillout2k/exota-milter-amd64
+    restart: unless-stopped
+    environment:
+      LOG_LEVEL: 'debug'
+      MILTER_SOCKET: 'inet:4321@0.0.0.0'
+      MILTER_POLICY_FILE: '/data/exota-milter-policy.json'
+      MILTER_DKIM_ENABLED: 'True'
+      MILTER_DKIM_ALIGNMENT_REQUIRED: 'True'
+      MILTER_TRUSTED_AUTHSERVID: 'DKIMAuthservID'
+      MILTER_X509_ENABLED: 'True'
+      MILTER_X509_TRUSTED_CN: 'mail.protection.outlook.com'
+      MILTER_X509_IP_WHITELIST: '127.0.0.1,::1'
+      MILTER_ADD_HEADER: 'True'
+      MILTER_AUTHSERVID: 'ThisAuthservID'
+      MILTER_REJECT_MESSAGE: 'CUSTOMIZE THIS! - Security policy violation!!'
+    volumes:
+    - "./data/:/data/:ro"
+    ports:
+    - "127.0.0.1:4321:4321"

INSTALL/kubernetes/01_config-map.yaml

@@ -0,0 +1,15 @@
+---
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: exota-milter-policy-cmap
+  namespace: devel
+data:
+  exota-milter-policy: |
+    {
+      "example.com": {
+        "tenant_id": "abcd1234-18c5-45e8-88de-987654321cba",
+        "dkim_enabled": true,
+        "dkim_alignment_required": true
+      }
+    }

INSTALL/kubernetes/02_deployment.yaml

@@ -0,0 +1,82 @@
+---
+kind: Deployment
+apiVersion: apps/v1
+metadata:
+  namespace: devel
+  name: exota-milter
+  labels:
+    app: exota-milter
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: exota-milter
+  template:
+    metadata:
+      labels:
+        app: exota-milter
+    spec:
+      # Do not deploy more than one pods per node
+      topologySpreadConstraints:
+      - labelSelector:
+          matchLabels:
+            app: exota-milter
+        maxSkew: 1
+        topologyKey: kubernetes.io/hostname
+        whenUnsatisfiable: DoNotSchedule
+      # Pod eviction toleration overrides
+      tolerations:
+      - key: "node.kubernetes.io/unreachable"
+        operator: "Exists"
+        effect: "NoExecute"
+        tolerationSeconds: 30
+      - key: "node.kubernetes.io/not-ready"
+        operator: "Exists"
+        effect: "NoExecute"
+        tolerationSeconds: 30
+      restartPolicy: Always
+      terminationGracePeriodSeconds: 10
+      volumes:
+      - name: exota-milter-policy-volume
+        configMap:
+          name: exota-milter-policy-cmap
+          items:
+          - key: exota-milter-policy
+            path: 'exota-milter-policy.json'
+      containers:
+      - name: exota-milter
+        image: chillout2k/exota-milter-amd64
+        imagePullPolicy: Always
+        volumeMounts:
+        - mountPath: /data
+          name: exota-milter-policy-volume
+        startupProbe:
+          exec:
+            command: ["nc", "-v", "-w1", "127.0.0.1", "4321"]
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        env:
+        - name: LOG_LEVEL
+          value: 'info'
+        - name: MILTER_SOCKET
+          value: 'inet:4321@127.0.0.1'
+        - name: MILTER_POLICY_FILE
+          value: '/data/exota-milter-policy.json'
+        - name: MILTER_DKIM_ENABLED
+          value: 'True'
+        - name: MILTER_DKIM_ALIGNMENT_REQUIRED
+          value: 'True'
+        - name: MILTER_TRUSTED_AUTHSERVID
+          value: 'DKIMAuthservID'
+        - name: MILTER_X509_ENABLED
+          value: 'True'
+        - name: MILTER_X509_TRUSTED_CN
+          value: 'mail.protection.outlook.com'
+        - name: MILTER_X509_IP_WHITELIST
+          value: '127.0.0.1,::1'
+        - name: MILTER_ADD_HEADER
+          value: 'True'
+        - name: MILTER_AUTHSERVID
+          value: 'some-auth-serv-id'
+        - name: MILTER_REJECT_MESSAGE
+          value: 'Security policy violation!!'

INSTALL/kubernetes/03_service.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  namespace: devel
+  name: exota-milter
+spec:
+  selector:
+    app: exota-milter
+  ports:
+    - protocol: TCP
+      port: 4321
+      targetPort: 4321

INSTALL/systemd/exota-milter-policy.json

@@ -0,0 +1,7 @@
+{
+  "example.com": {
+    "tenant_id": "abcd1234-18c5-45e8-88de-987654321cba",
+    "dkim_enabled": true,
+    "dkim_alignment_required": true
+  }
+}

INSTALL/systemd/exota-milter.conf

@@ -0,0 +1,20 @@
+export LOG_LEVEL='info'
+export MILTER_SOCKET='inet:4321@127.0.0.1'
+export MILTER_POLICY_FILE='/etc/exota-milter/exota-milter-policy.json'
+export MILTER_DKIM_ENABLED='True'
+export MILTER_DKIM_ALIGNMENT_REQUIRED='True'
+export MILTER_TRUSTED_AUTHSERVID='DKIMAuthservID'
+export MILTER_X509_ENABLED='True'
+export MILTER_X509_TRUSTED_CN='mail.protection.outlook.com'
+export MILTER_X509_IP_WHITELIST='127.0.0.1,::1'
+export MILTER_ADD_HEADER='True'
+export MILTER_AUTHSERVID='some-auth-serv-id'
+#export MILTER_REJECT_MESSAGE='Security policy violation!!'
+
+# LDAP integration
+#export MILTER_POLICY_SOURCE=ldap
+#export MILTER_LDAP_SERVER_URI=ldaps://your.ldap.server
+#export MILTER_LDAP_SEARCH_BASE=ou=your-customer-domains,dc=example,dc=org
+#export MILTER_LDAP_QUERY='(domainNameAttr=%d)'
+#export MILTER_LDAP_BINDDN=uid=exota-milter,ou=apps,dc=example,dc=org
+#export MILTER_LDAP_BINDPW='$uPer§ecRet1!'

INSTALL/systemd/exota-milter.service

@@ -0,0 +1,9 @@
+[Unit]
+Description=ExOTA-Milter
+
+[Service]
+Restart=always
+ExecStart=/usr/loca/sbin/exota-milter.sh
+
+[Install]
+WantedBy=multi-user.target

INSTALL/systemd/exota-milter.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+if [ ! -e /etc/exota-milter/exota-milter.conf ]; then
+  echo "Missing /etc/exota-milter/exota-milter.conf!"
+  exit 1;
+fi
+
+if [ ! -e /etc/exota-milter/exota-milter-policy.json ]; then
+  echo "Missing /etc/exota-milter/exota-milter-policy.json!"
+  exit 1;
+fi
+
+. /etc/exota-milter/exota-milter.conf
+
+exec /usr/bin/python3 /usr/local/exota-milter/exota-milter.py 2>&1

INSTALL/systemd/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+
+if [ "$(id -u)" != "0" ]; then
+  echo "You must be root!"
+  exit 1
+fi
+
+install -d /usr/local/exota-milter/
+install ../../app/*.py /usr/local/exota-milter/
+install -m 750 exota-milter.sh /usr/local/sbin/exota-milter.sh
+install -d -m 660 /etc/exota-milter
+if [ -e /etc/exota-milter/exota-milter-policy.json ]; then
+  echo "Found existing /etc/exota-milter/exota-milter-policy.json - skipping"
+else
+  install -m 660 exota-milter-policy.json /etc/exota-milter/exota-milter-policy.json
+fi
+if [ -e /etc/exota-milter/exota-milter.conf ]; then
+  echo "Found existing /etc/exota-milter/exota-milter.conf - skipping"
+else
+  install -m 750 exota-milter.conf /etc/exota-milter/exota-milter.conf
+fi
+install -m 660 exota-milter.service /lib/systemd/system/exota-milter.service
+systemctl daemon-reload
+systemctl enable exota-milter.service

INSTALL/systemd/uninstall.sh

@@ -0,0 +1,14 @@
+#!/bin/sh
+
+if [ "$(id -u)" != "0" ]; then
+  echo "You must be root!"
+  exit 1
+fi
+
+systemctl disable exota-milter.service
+systemctl stop exota-milter.service
+rm -rf /usr/local/exota-milter/
+rm -f /usr/local/sbin/exota-milter.sh
+rm -f /lib/systemd/system/exota-milter.service
+systemctl daemon-reload
+echo "/etc/exota-milter/ was kept undeleted!"

README.md

@@ -144,3 +144,6 @@ Take a look [here](OCI/README.md)
 
 # How to test?
 First of all please take a look at how to set up the testing environment, which is described [here](tests/README.md)
+
+# How to install on docker/kubernetes/systemd?
+The installation procedure is documented [here](INSTALL/README.md)