LDAP policy backend

LDAP/README.md

@@ -1 +1,32 @@
-# ExOTA-Milter with LDAP backend
+# ExOTA-Milter with LDAP policy backend
+For small setups, with not so many domains, the JSON-file policy backend (default) may be sufficient. If you´re an email service provider (ESP), maintaining a lot of customer domains in a LDAP server, you may want use the LDAP backend instead.
+
+### Configuration
+To enable LDAP backend support you need to set up the following environment variables:
+```
+export MILTER_POLICY_SOURCE=ldap
+export MILTER_LDAP_SERVER_URI=ldaps://your.ldap.server
+export MILTER_LDAP_SEARCH_BASE=ou=your-customer-domains,dc=example,dc=org
+export MILTER_LDAP_QUERY='(domainNameAttr=%d)'
+export MILTER_LDAP_BINDDN=uid=exota-milter,ou=apps,dc=example,dc=org
+export MILTER_LDAP_BINDPW='$uPer§ecRet1!'
+```
+The `MILTER_LDAP_QUERY` variable requires a macro/placeholder **%d**, which identifies the domain name to search for in the LDAP tree.
+
+### Use EXoTA-Milter LDAP schema
+If you´re willing to use the ExOTA-Milter LDAP schema, you don´t need further configuration. Just feed your LDAP-server with the [ready to use schema file](exota-milter.schema) (auxiliary objectclass) and extend your customers domain objects with the following objectclass and attributes:
+
+Objectclass: `exotaMilterPolicy`  
+Attributes:
+* exotaMilterTenantId
+* exotaMilterDkimEnabled
+* exotaMilterDkimAlignmentRequired
+
+### Use your custom LDAP schema
+If you want to use an own custom LDAP schema with ExOTA-Milter you will have to set up the following environment variables as well:
+```
+export MILTER_LDAP_TENANT_ID_ATTR=your_custom_tenant_id_attr
+export MILTER_LDAP_DKIM_ENABLED_ATTR=your_custom_dkim_enabled_attr
+export MILTER_LDAP_DKIM_ALIGNMENT_REQUIRED_ATTR=your_custom_dkim_alignment_required_attr
+```
+Please make sure that your custom LDAP attributes are set up accordingly the **ExOTA-Milter** [LDAP schema](exota-milter.schema), otherwise your setup will not work as expected!

README.md

@@ -102,7 +102,7 @@ Further each Microsoft Exchange-Online tenant has a unique tenant-ID in form of
 X-MS-Exchange-CrossTenant-Id: <UUID-of-tenant>
 [...]
 ```
-At last the **ExOTA-Milter** needs an additional policy (JSON file), that provides a mapping of *sender-domain <-> tenant-id* and if DKIM-signatures must be taken under consideration or not. The JSON policy file itself looks like this:
+At last the **ExOTA-Milter** needs an additional policy (currently as JSON file or LDAP server), that provides a mapping of *sender-domain <-> tenant-id* and if DKIM-signatures must be taken under consideration or not. The JSON policy file itself looks like this:
 ```
 {
   "yad.onmicrosoft.com": {
@@ -112,8 +112,6 @@ At last the **ExOTA-Milter** needs an additional policy (JSON file), that provid
   }
 }
 ```
-Actually I´m also working on a LDAP-based version as policy backend.
-
 # The solution
 So, *how can an Exchange-Online user/tenant be identified by a third party smarthost?*
 
@@ -127,6 +125,9 @@ Finally it´s the combination of all of the above discussed aspects which may re
 
 ![Activity policy](http://www.plantuml.com/plantuml/png/5SKn3W8W30NGg-W1f8cZcuEZSN4tM8aq5ahAhyhjZMzvM-ciyIZXkgd0c0SYpv_q5DIunopErb4w4biZhg9gWVsBJj_BzRWxYw8ujJp_POQy1UisJ8LN6j7q1m00)
 
+# How about using LDAP as policy backend?
+For small setups, with not so many domains, the JSON-file policy backend (default) may be sufficient. If you´re an email service provider (ESP) maintaining a lot of customer domains in a LDAP server, you may want to use the LDAP backend instead. Details regarding the LDAP backend can be found [in the LDAP readme](LDAP/README.md).
+
 # How about a docker/OCI image?
 ## Using prebuilt images from [dockerhub](https://hub.docker.com/)
 * AMD64: https://hub.docker.com/r/chillout2k/exota-milter-amd64

app/exota-milter.py

@@ -52,6 +52,8 @@ g_milter_add_header = False
 g_milter_authservid = None
 # ENV[MILTER_LDAP_SERVER_URI]
 g_milter_ldap_server_uri = ''
+# ENV[MILTER_LDAP_RECEIVE_TIMEOUT]
+g_milter_ldap_receive_timeout = 5
 # ENV[MILTER_LDAP_BINDDN]
 g_milter_ldap_binddn = ''
 # ENV[MILTER_LDAP_BINDPW]
@@ -594,6 +596,15 @@ if __name__ == "__main__":
       logging.error("ENV[MILTER_LDAP_SERVER_URI] is mandatory!")
       sys.exit(1)
     g_milter_ldap_server_uri = os.environ['MILTER_LDAP_SERVER_URI']
+    if 'MILTER_LDAP_RECEIVE_TIMEOUT' in os.environ:
+      try:
+        g_milter_ldap_receive_timeout = int(os.environ['MILTER_LDAP_RECEIVE_TIMEOUT'])
+      except ValueError:
+        logging.error("ENV[MILTER_LDAP_RECEIVE_TIMEOUT] must be an integer!")
+        sys.exit(1)  
+    logging.info("ENV[MILTER_LDAP_RECEIVE_TIMEOUT]: {0}".format(
+      g_milter_ldap_receive_timeout
+    ))
     if 'MILTER_LDAP_BINDDN' not in os.environ:
       logging.info("ENV[MILTER_LDAP_BINDDN] not set! Continue...")
     else:
@@ -617,17 +628,18 @@ if __name__ == "__main__":
     if 'MILTER_LDAP_DKIM_ALIGNMENT_REQUIRED_ATTR' in os.environ:
       g_milter_ldap_dkim_alignment_required_attr = os.environ['MILTER_LDAP_DKIM_ALIGNMENT_REQUIRED_ATTR']
     try:
-      set_config_parameter("RESTARTABLE_SLEEPTIME", 2)
+      set_config_parameter("RESTARTABLE_SLEEPTIME", 1)
-      set_config_parameter("RESTARTABLE_TRIES", True)
+      set_config_parameter("RESTARTABLE_TRIES", 2)
       set_config_parameter('DEFAULT_SERVER_ENCODING', 'utf-8')
       set_config_parameter('DEFAULT_CLIENT_ENCODING', 'utf-8')
       server = Server(g_milter_ldap_server_uri, get_info=NONE)
       g_milter_ldap_conn = Connection(server,
         g_milter_ldap_binddn,
         g_milter_ldap_bindpw,
-        auto_bind=True, 
+        auto_bind = True, 
-        raise_exceptions=True,
+        raise_exceptions = True,
-        client_strategy='RESTARTABLE'
+        client_strategy = 'RESTARTABLE',
+        receive_timeout = g_milter_ldap_receive_timeout
       )
       logging.info("LDAP-Connection established")
       try:
@@ -644,7 +656,7 @@ if __name__ == "__main__":
         logging.error("Policy backend error: {0}".format(e.message))
         sys.exit(1)
     except LDAPException as e:
-      print("LDAP-Exception: " + traceback.format_exc())
+      print("LDAP-Exception: {0}".format(e))
       sys.exit(1)
   else:
     logging.debug("Unsupported backend: {0}!".format(g_milter_policy_source))

app/policy.py

@@ -46,6 +46,10 @@ class ExOTAPolicy():
       raise ExOTAPolicyInvalidException(
         "Policy must have a 'tenant_id' key!"
       )
+    if policy_dict['tenant_id'] is None:
+      raise ExOTAPolicyInvalidException(
+        "'tenant_id' needs a value!"
+      )
     for policy_key in policy_dict:
       if policy_key == 'tenant_id':
         try:
@@ -100,11 +104,11 @@ class ExOTAPolicyBackendJSON(ExOTAPolicyBackend):
               "Policy {0} is invalid: {1}".format(policy, e.message)
             ) from e
     except json.decoder.JSONDecodeError as e:
-      raise ExOTAPolicyException(
+      raise ExOTAPolicyBackendException(
         "JSON-error in policy file: " + str(e)
       ) from e
     except Exception as e:
-      raise ExOTAPolicyException(
+      raise ExOTAPolicyBackendException(
         "Error reading policy file: " + traceback.format_exc()
       ) from e
   
@@ -123,7 +127,7 @@ class ExOTAPolicyBackendJSON(ExOTAPolicyBackend):
       ) from e
 
 ########## LDAP
-class ExOTAPolicyBackendLDAP(ExOTAPolicyBackendJSON):
+class ExOTAPolicyBackendLDAP(ExOTAPolicyBackend):
   type = 'ldap'
   def __init__(self, ldap_config):
     try:
@@ -134,7 +138,7 @@ class ExOTAPolicyBackendLDAP(ExOTAPolicyBackendJSON):
       self.dkim_enabled_attr = ldap_config['ldap_dkim_enabled_attr']
       self.dkim_alignment_required_attr = ldap_config['ldap_dkim_alignment_required_attr']
     except Exception as e:
-      raise ExOTAPolicyException(
+      raise ExOTAPolicyBackendException(
         "An error occured while initializing LDAP backend: " + traceback.format_exc()
       ) from e
 
@@ -167,7 +171,7 @@ class ExOTAPolicyBackendLDAP(ExOTAPolicyBackendJSON):
             policy_dict['dkim_alignment_required'] = False
         ExOTAPolicy.check_policy(policy_dict)
         return ExOTAPolicy(policy_dict)
-      if len(self.conn.entries) > 1:
+      elif len(self.conn.entries) > 1:
         raise ExOTAPolicyInvalidException(
           "Multiple policies found for domain={0}!".format(from_domain)
         )
@@ -176,6 +180,4 @@ class ExOTAPolicyBackendLDAP(ExOTAPolicyBackendJSON):
           "Policy for domain={0} not found".format(from_domain)
         )
     except LDAPException as e:
-      raise ExOTAPolicyBackendException(
+      raise ExOTAPolicyBackendException(e) from e
-        "asdf"
-      ) from e

tests/miltertest.lua

@@ -10,7 +10,7 @@ if mt.conninfo(conn, "localhost", "::1") ~= nil then
   error "mt.conninfo() failed"
 end
 
-mt.set_timeout(3)
+mt.set_timeout(60)
 
 -- 5321.FROM
 if mt.mailfrom(conn, "envelope.sender@example.org") ~= nil then
@@ -57,7 +57,7 @@ end
 if mt.header(conn, "Authentication-RESULTS", "my-auth-serv-id;\n  dkim=pass header.d=yad.onmicrosoft.comx header.s=selector1-yad-onmicrosoft-com header.b=mmmjFpv8") ~= nil then
   error "mt.header(Subject) failed"  
 end
-if mt.header(conn, "Authentication-RESULTS", "my-auth-serv-id;\n  dkim=pass header.d=chillout2k.dex header.s=selector1-yad-onmicrosoft-com header.b=mmmjFpv8") ~= nil then
+if mt.header(conn, "Authentication-RESULTS", "my-auth-serv-id;\n  dkim=pass header.d=chillout2k.de header.s=selector1-yad-onmicrosoft-com header.b=mmmjFpv8") ~= nil then
   error "mt.header(Subject) failed"  
 end
 if mt.header(conn, "Authentication-Results", "my-auth-serv-id;\n  dkim=fail header.d=yad.onmicrosoft.com header.s=selector2-asdf header.b=mmmjFpv8") ~= nil then