spacefreak/pyquarantine-milter
1
0
Fork 0
Code Issues Pull Requests Releases Activity

massive refactoring of the source

Browse Source
This commit is contained in:
Thomas Oettli
2020-06-09 01:18:00 +02:00
parent d60ea5282c
commit 0651ceba62
7 changed files with 905 additions and 714 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
README.md
View File

@@ -2,7 +2,7 @@
A pymilter based sendmail/postfix pre-queue filter with the ability to add, remove and modify e-mail headers.
The project is currently in beta status, but it is already used in a productive enterprise environment which processes about a million e-mails per month.
The basic idea is to define rules with conditions and modifications which are processed when all conditions are true.
The basic idea is to define rules with conditions and actions which are processed when all conditions are true.
## Dependencies
Pymodmilter is depending on these python packages, but they are installed automatically if you are working with pip.
@@ -10,7 +10,7 @@ Pymodmilter is depending on these python packages, but they are installed automa
* [netaddr](https://github.com/drkjam/netaddr/)
## Installation
* Install pymodmilter with pip and copy the example configuration file.
* Install pymodmilter with pip and copy the example config file.
```sh
pip install pymodmilter
cp /etc/pymodmilter/pymodmilter.conf.example /etc/pymodmilter/pymodmilter.conf
@@ -18,96 +18,107 @@ cp /etc/pymodmilter/pymodmilter.conf.example /etc/pymodmilter/pymodmilter.conf
* Modify /etc/pymodmilter/pymodmilter.conf according to your needs.
## Configuration options
Pymodmilter uses a configuration file in JSON format. The options are described below. Make a copy of the [example configuration file](https://github.com/spacefreak86/pymodmilter/blob/master/docs/pymodmilter.conf.example) in the [docs](https://github.com/spacefreak86/pymodmilter/tree/master/docs) folder to start with.
Rules and modifications are processed in the given order.
Pymodmilter uses a config file in JSON format. The config file has to be JSON valid with the exception of allowed comment lines starting with **#**. The options are described below.
Rules and actions are processed in the given order.
### Global
The following global configuration options are optional:
* **socket**
The socket used to communicate with the MTA.
* **local_addrs**
A list of hosts and network addresses which are considered local. It is used to for the condition option [local](#Conditions). This option may be overriden within a rule object.
* **log**
Enable or disable logging. This option may be overriden by a rule or modification object.
Config options in **global** section:
* **socket** (optional)
The socket used to communicate with the MTA. If it is not specified in the config, it has to be set as command line option.
* **local_addrs** (optional)
A list of hosts and network addresses which are considered local. It is used to for the condition option [local](#Conditions).
Default: **::1/128, 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16**
* **loglevel** (optional)
Set the log level. This option may be overriden by any rule or action object. Possible values are:
* **error**
* **warning**
* **info**
* **debug**
Default: **info**
* **pretend** (optional)
Pretend actions, for test purposes. This option may be overriden by any rule or action object.
### Rules
The following configuration options are mandatory for each rule:
* **modifications**
A list of modification objects which are processed in the given order.
The following configuration options are optional for each rule:
* **name**
Config options for **rule** objects:
* **name** (optional)
Name of the rule.
* **conditions**
A list of conditions which all have to be true to process the rule.
* **local_addrs**
Default: **Rule #n**
* **actions**
A list of action objects which are processed in the given order.
* **conditions** (optional)
A list of conditions which all have to be true to process the actions.
* **loglevel** (optional)
As described above in the [Global](#Global) section.
* **log**
* **pretend** (optional)
As described above in the [Global](#Global) section.
* **pretend**
Just pretend to make the modifications, for test purposes.
### Modifications
The following configuration options are mandatory for each modification:
### Actions
Config options for **action** objects:
* **name** (optional)
Name of the action.
Default: **Action #n**
* **type**
Set the modification type. Possible values are:
Action type. Possible values are:
* **add_header**
* **del_header**
* **mod_header**
* **add_disclaimer**
* **conditions** (optional)
A list of conditions which all have to be true to process the action.
* **pretend** (optional)
Just pretend all actions of this rule, for test purposes.
* **loglevel** (optional)
As described above in the [Global](#Global) section.
The following configuration options are mandatory based on the modification type in use.
* **add_header**
Config options for **add_header** actions:
* **header**
Name of the header.
* **value**
Value of the header.
* **del_header**
Config options for **del_header** actions:
* **header**
Regular expression to match against header lines.
Regular expression to match against header names.
* **value** (optional)
Regular expression to match against the headers value.
* **mod_header**
Config options for **mod_header** actions:
* **header**
Regular expression to match against header lines.
* **search**
Regular expression to match against the value of header lines. You may use subgroups or named subgroups (python syntax) to include parts of the original value in the new value.
Regular expression to match against header names.
* **search** (optional)
Regular expression to match against header values. You may use subgroups or named subgroups (python syntax) to include parts of the original value in the new value.
* **value**
New value of the header.
* **add_disclaimer**
Config options for **add_disclaimer** actions:
* **action**
Action to perform with the disclaimer. Possible values are:
* append
* prepend
* **html_template**
Path to a file that contains the html representation of the disclaimer.
* **text_template**
Path to a file that contains the text representation of the disclaimer.
* **error_policy**
Set what should be done if the disclaimer could not be added (e.g. no body text found). Possible values are:
* **html_file**
Path to a file which contains the html representation of the disclaimer.
* **text_file**
Path to a file which contains the text representation of the disclaimer.
* **error_policy** (optional)
Set the error policy in case the disclaimer cannot be added (e.g. if no body part is present in the e-mail). Possible values are:
* wrap
The original e-mail will be attached to a new one containing the disclaimer.
A new e-mail body is generated with the disclaimer as body and the original e-mail attached.
* ignore
Ignore the error and do nothing.
* reject
Reject the e-mail.
Default: **wrap**
The following configuration options are optional for each modification:
* **name**
Name of the modification.
* **log**
As described above in the global object section.
### Conditions
The following condition options are optional:
* **local**
Config options for **conditions** objects:
* **local** (optional)
If set to true, the rule is only executed for e-mails originating from addresses defined in local_addrs and vice versa.
* **hosts**
* **hosts** (optional)
A list of hosts and network addresses for which the rule should be executed.
* **envfrom**
* **envfrom** (optional)
A regular expression to match against the evenlope-from addresses for which the rule should be executed.
* **envto**
* **envto** (optional)
A regular expression to match against all evenlope-to addresses. All addresses must match to fulfill the condition.
## Developer information

48
docs/pymodmilter.conf.example
View File

@@ -30,16 +30,23 @@
#
"local_addrs": ["127.0.0.0/8", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"],
# Option: log
# Option: loglevel
# Type: String
# Notes: Set loglevel for rules and actions.
# Value: [ error | warning | info | debug ]
#
"loglevel": "info",
# Option: pretend
# Type: Bool
# Notes: Enable or disable logging of rules and modifications.
# Notes: Just pretend to do the actions, for test purposes.
# Value: [ true | false ]
#
"log": true
"pretend": true
},
# Section: rules
# Notes: Rules and related modifications.
# Notes: Rules and related actions.
#
"rules": [
{
@@ -85,10 +92,10 @@
"envto": "^postmaster@.+$"
},
# Section: modifications
# Notes: Modifications of the rule.
# Section: actions
# Notes: Actions of the rule.
#
"modifications": [
"actions": [
{
# Option: name
# Type: String
@@ -116,7 +123,7 @@
# Notes: Value of the header.
# Value: [ VALUE ]
#
"value": "true",
"value": "true"
}, {
"name": "modify_subject",
@@ -127,7 +134,7 @@
# Notes: Regular expression to match against header lines (e.g. Subject: Test-Subject).
# Value: [ REGEX ]
#
"header": "^Subject:",
"header": "^Subject$",
# Option: search
# Type: String
@@ -151,7 +158,7 @@
# Notes: Regular expression to match against header lines (e.g. Subject: Test-Subject).
# Value: [ REGEX ]
#
"header": "^Received:"
"header": "^Received$"
}, {
"name": "add_disclaimer",
@@ -164,19 +171,19 @@
#
"action": "prepend",
# Option: html_template
# Option: html_file
# Type: String
# Notes: Path to a file that contains the html representation of the disclaimer.
# Notes: Path to a file which contains the html representation of the disclaimer.
# Value: [ FILE_PATH ]
#
"html_template": "/etc/pymodmilter/templates/disclaimer_html.template",
"html_file": "/etc/pymodmilter/templates/disclaimer_html.template",
# Option: text_template
# Option: text_file
# Type: String
# Notes: Path to a file that contains the text representation of the disclaimer.
# Notes: Path to a file which contains the text representation of the disclaimer.
# Value: [ FILE_PATH ]
#
"text_template": "/etc/pymodmilter/templates/disclaimer_text.template",
"text_file": "/etc/pymodmilter/templates/disclaimer_text.template",
# Option: error_policy
# Type: String
@@ -185,14 +192,7 @@
#
"error_policy": "wrap"
}
],
# Option: pretend
# Type: Bool
# Notes: Just pretend to do the modifications, for test purposes.
# Value: [ true | false ]
#
"pretend": true
]
}
]
}

2
docs/templates/disclaimer_html.template vendored
View File

@@ -6,4 +6,4 @@
</td>
</tr>
</table>
<br /><br />
<br/><br/>

650
pymodmilter/__init__.py
View File

@@ -14,28 +14,23 @@
__all__ = [
"make_header",
"replace_illegal_chars",
"actions",
"conditions",
"run",
"version",
"Modification",
"CustomLogger",
"Rule",
"ModifyMilter"]
import Milter
import logging
import logging.handlers
import re
from Milter.utils import parse_addr
from bs4 import BeautifulSoup
from copy import copy
from email.charset import Charset
from email.header import Header, decode_header
from email import message_from_binary_file
from email.message import MIMEPart
from email.policy import default as default_policy, SMTP
from io import BytesIO
from netaddr import IPAddress, IPNetwork, AddrFormatError
from pymodmilter.conditions import Conditions
def make_header(decoded_seq, maxlinelen=None, header_name=None,
@@ -60,524 +55,101 @@ def make_header(decoded_seq, maxlinelen=None, header_name=None,
return h
def replace_illegal_chars(string):
"""Replace illegal characters in header values."""
return string.replace(
"\x00", "").replace(
"\r", "").replace(
"\n", "")
class CustomLogger(logging.LoggerAdapter):
def process(self, msg, kwargs):
if "name" in self.extra:
msg = "{}: {}".format(self.extra["name"], msg)
if "qid" in self.extra:
msg = "{}: {}".format(self.extra["qid"], msg)
class Modification:
"""
Modification to implement certain modifications on e-mails.
if self.logger.getEffectiveLevel() != logging.DEBUG:
msg = msg.replace("\n", "").replace("\r", "")
Each modification function returns the necessary changes for ModifyMilter
so they can be applied to the email passing the MTA.
"""
def __init__(self, name, mod_type, log, **params):
self.logger = logging.getLogger(__name__)
self.logger.debug(f"initializing modification '{name}'")
self.name = name
self.log = log
# needs for each modification type
self.types = {
"add_header": {
"needs": ["headers"]},
"del_header": {
"needs": ["headers"]},
"mod_header": {
"needs": ["headers"]},
"add_disclaimer": {
"needs": ["headers", "data"]}}
if mod_type not in self.types:
raise RuntimeError(
f"{self.name}: invalid modification type '{mod_type}'")
self.mod_type = mod_type
try:
if mod_type == "add_header":
self.header = params["header"]
self.value = params["value"]
elif mod_type in ["del_header", "mod_header"]:
try:
self.header = re.compile(
params["header"],
re.MULTILINE + re.DOTALL + re.IGNORECASE)
except re.error as e:
raise RuntimeError(
f"{self.name}: unable to parse regex of "
f"option 'header': {e}")
if mod_type == "mod_header":
try:
self.search = re.compile(
params["search"],
re.MULTILINE + re.DOTALL + re.IGNORECASE)
except re.error as e:
raise RuntimeError(
f"{self.name}: unable to parse regex of "
f"option 'search': {e}")
self.value = params["value"]
elif mod_type == "add_disclaimer":
if params["action"] not in ["append", "prepend"]:
raise RuntimeError(
f"{self.name}: unknown action specified")
self.action = params["action"]
if params["error_policy"] not in ["wrap", "ignore", "reject"]:
raise RuntimeError(
f"{self.name}: unknown error_policy specified")
self.error_policy = params["error_policy"]
try:
with open(params["html_template"], "r") as f:
self.html = BeautifulSoup(f.read(), "html.parser")
body = self.html.find('body')
if body:
# just use content within the body tag if present
self.html = body
with open(params["text_template"], "r") as f:
self.text = f.read()
except IOError as e:
raise RuntimeError(f"unable to read template: {e}")
except KeyError as e:
raise RuntimeError(
f"{self.name}: mandatory configuration option not found: {e}")
def needs(self):
"""Return the needs of this modification to work."""
return self.types[self.mod_type]["needs"]
def add_header(self, qid, headers, header, value, pos=-1):
"""Add header to email."""
hdr = f"{header}: {value}"
if self.log:
self.logger.info(
f"{qid}: {self.name}: add_header: {hdr[0:70]}")
else:
self.logger.debug(
f"{qid}: {self.name}: add_header: {hdr}")
headers.append((header, value))
params = [header, value, pos]
return [("add_header", *params)]
def mod_header(self, qid, headers, header, search, replace):
"""Modify an email header."""
if isinstance(header, str):
header = re.compile(
header, re.MULTILINE + re.DOTALL + re.IGNORECASE)
if isinstance(search, str):
search = re.compile(
search, re.MULTILINE + re.DOTALL + re.IGNORECASE)
changes = []
index = 0
occurrences = {}
# iterate a copy of headers because headers may be modified
for name, value in headers.copy():
# keep track of the occurrence of each header
# needed by Milter.Base.chgheader
if name not in occurrences.keys():
occurrences[name] = 1
else:
occurrences[name] += 1
hdr = f"{name}: {value}"
if header.search(hdr):
new_value = search.sub(replace, value).strip()
if new_value == "":
self.logger.warning(
f"{qid}: {self.name}: mod_header: resulting value is "
f"empty, skip modification")
elif value != new_value:
old_hdr = hdr
hdr = f"{name}: {new_value}"
if self.log:
self.logger.info(
f"{qid}: {self.name}: mod_header: "
f"{old_hdr[0:70]}: {hdr[0:70]}")
else:
self.logger.debug(
f"{qid}: {self.name}: mod_header: "
f"(occ. {occurrences[name]}): {old_hdr}: "
f"{hdr}")
headers[index] = (name, new_value)
params = [name, new_value, occurrences[name]]
changes.append(("mod_header", *params))
index += 1
return changes
def del_header(self, qid, headers, header):
"""Delete an email header."""
if isinstance(header, str):
header = re.compile(
header, re.MULTILINE + re.DOTALL + re.IGNORECASE)
changes = []
index = 0
occurrences = {}
# iterate a copy of headers because headers may be modified
for name, value in headers.copy():
# keep track of the occurrence of each header,
# needed by Milter.Base.chgheader
if name not in occurrences.keys():
occurrences[name] = 1
else:
occurrences[name] += 1
hdr = f"{name}: {value}"
if header.search(hdr):
if self.log:
self.logger.info(
f"{qid}: {self.name}: del_header: "
f"{hdr[0:70]}")
else:
self.logger.debug(
f"{qid}: {self.name}: del_header: "
f"(occ. {occurrences[name]}): {hdr}")
del headers[index]
params = [name, "", occurrences[name]]
changes.append(("mod_header", *params))
index -= 1
occurrences[name] -= 1
index += 1
return changes
def add_disclaimer(self, qid, headers, fp, text_template, html_template,
error_policy):
"""Append or prepend a disclaimer to the email body."""
changes = []
fp.seek(0)
msg = message_from_binary_file(fp, policy=default_policy)
html_body = None
text_body = None
update_headers = False
try:
html_body = msg.get_body(preferencelist=("html"))
text_body = msg.get_body(preferencelist=("plain"))
except Exception as e:
self.logger.error(
f"{qid}: {self.name}: an error occured in "
f"email.message.EmailMessage.get_body: {e}")
if html_body is None and text_body is None:
if self.error_policy == "ignore":
self.logger.info(
f"{qid}: {self.name}: unable to find email body, "
f"ignore according to policy")
return changes
elif self.error_policy == "reject":
self.logger.info(
f"{qid}: {self.name}: unable to find email body, "
f"reject message according to policy")
return [
("reject", "Message rejected due to missing email body")]
self.logger.info(
f"{qid}: {self.name}: unable to find email body, "
f"wrapping original email in a new message envelope")
msg = MIMEPart()
msg.add_header("MIME-Version", "1.0")
msg.set_content(
"Please see the original email attached.")
msg.add_alternative(
"Please see the original email attached.",
subtype="html")
fp.seek(0)
msg.add_attachment(
fp.read(), maintype="plain", subtype="text",
filename=f"{qid}.eml")
html_body = msg.get_body(preferencelist=("html"))
text_body = msg.get_body(preferencelist=("plain"))
# content and mime headers may have to be updated because
# a new message has been created
update_headers = True
elif not msg.is_multipart():
# content and mime headers may have to be updated because
# we operate on a non-multipart email
update_headers = True
if text_body is not None:
if self.log:
self.logger.info(
f"{qid}: {self.name}: {self.action} text disclaimer")
else:
self.logger.debug(
f"{qid}: {self.name}: {self.action} text disclaimer")
text = text_body.get_content()
if self.action == "prepend":
text = f"{text_template}{text}"
else:
text = f"{text}{text_template}"
text_body.set_content(
text.encode(), maintype="text", subtype="plain")
text_body.set_param("charset", "UTF-8", header="Content-Type")
if html_body is not None:
if self.log:
self.logger.info(
f"{qid}: {self.name}: {self.action} html disclaimer")
else:
self.logger.debug(
f"{qid}: {self.name}: {self.action} html disclaimer")
soup = BeautifulSoup(html_body.get_content(), "html.parser")
body = soup.find('body')
if body:
# work within the body tag if it is present
soup = body
if self.action == "prepend":
soup.insert(0, copy(html_template))
else:
soup.append(html_template)
html_body.set_content(
str(soup).encode(), maintype="text", subtype="html")
html_body.set_param("charset", "UTF-8", header="Content-Type")
if update_headers:
for name, value in msg.items():
name_lower = name.lower()
if not name_lower.startswith("content-") and \
name_lower != "mime-version":
continue
defined = False
for n, v in headers:
if n.lower() == name_lower:
changes += self.mod_header(
qid, headers, f"^{n}:", ".*", value)
defined = True
break
if not defined:
changes += self.add_header(
qid, headers, name, value)
msg = msg.as_string(policy=SMTP).encode("ascii", errors="replace")
fp.seek(0)
fp.write(msg)
fp.truncate()
body_pos = msg.find(b"\r\n\r\n") + 2
changes.append(("mod_body", body_pos))
return changes
def execute(self, qid, headers, fp):
"""
Execute configured modification.
"""
changes = []
if self.mod_type == "add_header":
changes = self.add_header(
qid, headers, self.header, self.value)
elif self.mod_type == "mod_header":
changes = self.mod_header(
qid, headers, self.header, self.search, self.value)
elif self.mod_type == "del_header":
changes = self.del_header(
qid, headers, self.header)
elif self.mod_type == "add_disclaimer":
changes = self.add_disclaimer(
qid, headers, fp, self.text,
self.html, self.error_policy)
return changes
return msg, kwargs
class Rule:
"""
Rule to implement multiple modifications on emails based on conditions.
Rule to implement multiple actions on emails.
"""
def __init__(self, name, modifications, local_addrs, log, conditions={},
pretend=False):
self.logger = logging.getLogger(__name__)
if pretend:
self.name = f"{name} (pretend)"
else:
self.name = name
def __init__(self, name, local_addrs, conditions, actions, pretend=False,
loglevel=logging.INFO):
logger = logging.getLogger(name)
self.logger = CustomLogger(logger, {"name": name})
self.logger.setLevel(loglevel)
self.logger.debug(f"initializing rule '{self.name}'")
self.log = log
if logger is None:
logger = logging.getLogger(__name__)
self.logger = CustomLogger(logger, {"name": name})
self.conditions = Conditions(
local_addrs=local_addrs,
args=conditions,
logger=self.logger)
self.actions = actions
self.pretend = pretend
self._needs = []
self._local_addrs = []
try:
for addr in local_addrs:
self._local_addrs.append(IPNetwork(addr))
except AddrFormatError as e:
raise RuntimeError(
f"{self.name}: unable to parse entry of "
f"option local_addrs: {e}")
self.conditions = {}
for option, value in conditions.items():
if option == "local":
self.conditions[option] = value
self.logger.debug(
f"{self.name}: added condition: {option} = {value}")
elif option == "hosts":
self.conditions[option] = []
try:
for host in value:
self.conditions[option].append(IPNetwork(host))
except AddrFormatError as e:
raise RuntimeError(
f"{self.name}: unable to parse entry of "
f"condition '{option}': {e}")
self.logger.debug(
f"{self.name}: added condition: {option} = {value}")
elif option in ["envfrom", "envto"]:
try:
self.conditions[option] = re.compile(value, re.IGNORECASE)
except re.error as e:
raise RuntimeError(
f"{self.name}: unable to parse regex of "
f"condition '{option}': {e}")
self.logger.debug(
f"{self.name}: added condition: {option} = {value}")
self.modifications = []
for mod_idx, mod in enumerate(modifications):
params = {}
if "name" not in mod:
mod["name"] = f"Modification #{mod_idx}"
if self.name:
params["name"] = f"{self.name}: {mod['name']}"
else:
params["name"] = mod["name"]
if "log" in mod:
params["log"] = mod["log"]
else:
params["log"] = self.log
if "type" in mod:
params["mod_type"] = mod["type"]
else:
raise RuntimeError(
f"{params['name']}: mandatory config "
f"option 'type' not found")
for param in [
"header", "search", "value", "action", "html_template",
"text_template", "error_policy"]:
if param in mod:
params[param] = mod[param]
modification = Modification(**params)
for need in modification.needs():
for action in actions:
for need in action.needs():
if need not in self._needs:
self._needs.append(need)
self.modifications.append(modification)
self.logger.debug(
f"{self.name}: added modification: {mod['name']}")
self.logger.debug(
f"{self.name}: rule needs: {self._needs}")
self.logger.debug("needs: {}".format(", ".join(self._needs)))
def needs(self):
"""Return the needs of this rule."""
return self._needs
def ignore_host(self, host):
"""Check if host is ignored by this rule."""
ip = IPAddress(host)
def ignores(self, host=None, envfrom=None, envto=None):
args = {}
if "local" in self.conditions:
is_local = False
for addr in self._local_addrs:
if ip in addr:
is_local = True
break
if host is not None:
args["host"] = host
if is_local != self.conditions["local"]:
return True
if envfrom is not None:
args["envfrom"] = envfrom
if "hosts" in self.conditions:
# check if host is in list
for accepted in self.conditions["hosts"]:
if ip in accepted:
if envto is not None:
args["envto"] = envto
if self.conditions.match(args):
for action in self.actions:
if action.conditions.match(args):
return False
return True
return True
return False
def execute(self, milter, pretend=None):
"""Execute all actions of this rule."""
if pretend is None:
pretend = self.pretend
def ignore_envfrom(self, envfrom):
"""Check if envelope-from address is ignored by this rule."""
if "envfrom" in self.conditions:
if not self.conditions["envfrom"].search(envfrom):
return True
return False
def ignore_envto(self, envto):
"""Check if envelope-to address is ignored by this rule."""
if "envto" in self.conditions:
if not isinstance(envto, set):
envto = set(envto)
for to in envto:
if not self.conditions["envto"].search(to):
return True
return False
def execute(self, qid, headers, data):
"""Execute all modifications of this rule."""
changes = []
if self.log:
self.logger.info(f"{qid}: executing rule '{self.name}'")
else:
self.logger.debug(f"{qid}: executing rule '{self.name}'")
for mod in self.modifications:
self.logger.debug(f"{qid}: executing modification '{mod.name}'")
changes += mod.execute(qid, headers, data)
if self.pretend:
changes = []
return changes
for action in self.actions:
milter_action = action.execute(milter)
if milter_action is not None:
return milter_action
class ModifyMilter(Milter.Base):
"""ModifyMilter based on Milter.Base to implement milter communication"""
_rules = []
_loglevel = logging.INFO
@staticmethod
def set_rules(rules):
ModifyMilter._rules = rules
def set_loglevel(level):
ModifyMilter._loglevel = level
def __init__(self):
self.logger = logging.getLogger(__name__)
self.logger.setLevel(ModifyMilter._loglevel)
# save rules, it must not change during runtime
self.rules = ModifyMilter._rules.copy()
@@ -585,13 +157,10 @@ class ModifyMilter(Milter.Base):
self.logger.debug(
f"accepted milter connection from {hostaddr[0]} "
f"port {hostaddr[1]}")
ip = IPAddress(hostaddr[0])
# remove rules which ignore this host
for rule in self.rules.copy():
if rule.ignore_host(ip):
self.logger.debug(
f"host {hostaddr[0]} is ignored by rule '{rule.name}'")
if rule.ignores(host=hostaddr[0]):
self.rules.remove(rule)
if not self.rules:
@@ -605,10 +174,7 @@ class ModifyMilter(Milter.Base):
def envfrom(self, mailfrom, *str):
mailfrom = "@".join(parse_addr(mailfrom)).lower()
for rule in self.rules.copy():
if rule.ignore_envfrom(mailfrom):
self.logger.debug(
f"envelope-from {mailfrom} is ignored by "
f"rule '{rule.name}'")
if rule.ignores(envfrom=mailfrom):
self.rules.remove(rule)
if not self.rules:
@@ -628,10 +194,7 @@ class ModifyMilter(Milter.Base):
def data(self):
try:
for rule in self.rules.copy():
if rule.ignore_envto(self.recipients):
self.logger.debug(
f"envelope-to addresses are ignored by "
f"rule '{rule.name}'")
if rule.ignores(envto=[*self.recipients]):
self.rules.remove(rule)
if not self.rules:
@@ -641,17 +204,18 @@ class ModifyMilter(Milter.Base):
return Milter.ACCEPT
self.qid = self.getsymval('i')
self.logger.debug(f"{self.qid}: received queue-id from MTA")
self.headers = None
self.logger = CustomLogger(self.logger, {"qid": self.qid})
self.logger.debug("received queue-id from MTA")
self.fields = None
self.fp = None
for rule in self.rules:
if "headers" in rule.needs() and self.headers is None:
self.headers = []
if "fields" in rule.needs() and self.fields is None:
self.fields = []
if "data" in rule.needs() and self.fp is None:
if "body" in rule.needs() and self.fp is None:
self.fp = BytesIO()
if None not in [self.headers, self.fp]:
if None not in [self.fields, self.fp]:
break
except Exception as e:
@@ -663,21 +227,15 @@ class ModifyMilter(Milter.Base):
def header(self, name, value):
try:
# remove surrogates from value
value = value.encode(
errors="surrogateescape").decode(errors="replace")
if self.fp is not None:
self.fp.write(f"{name}: {value}\r\n".encode(
encoding="ascii", errors="replace"))
if self.headers is not None:
self.logger.debug(f"{self.qid}: received header: "
f"{name}: {value}")
if self.fields is not None:
# remove surrogates from value
value = value.encode(
errors="surrogateescape").decode(errors="replace")
self.logger.debug(f"received header: {name}: {value}")
header = make_header(decode_header(value), errors="replace")
value = str(header).replace("\x00", "")
self.logger.debug(
f"{self.qid}: decoded header: {name}: {value}")
self.headers.append((name, value))
self.logger.debug(f"decoded header: {name}: {value}")
self.fields.append((name, value))
except Exception as e:
self.logger.exception(
f"an exception occured in header function: {e}")
@@ -685,17 +243,6 @@ class ModifyMilter(Milter.Base):
return Milter.CONTINUE
def eoh(self):
try:
if self.fp is not None:
self.fp.write(b"\r\n")
except Exception as e:
self.logger.exception(
f"an exception occured in eoh function: {e}")
return Milter.TEMPFAIL
return Milter.CONTINUE
def body(self, chunk):
try:
if self.fp is not None:
@@ -709,43 +256,20 @@ class ModifyMilter(Milter.Base):
def eom(self):
try:
changes = []
for rule in self.rules:
changes += rule.execute(self.qid, self.headers, self.fp)
milter_action = rule.execute(self)
mod_body_pos = None
for mod_type, *params in changes:
if mod_type in ["add_header", "mod_header", "del_header"]:
header, value, occurrence = params
enc_value = replace_illegal_chars(
Header(s=value).encode())
if mod_type == "add_header":
self.logger.debug(f"{self.qid}: milter: add "
f"header: {header}: {enc_value}")
self.addheader(header, enc_value, occurrence)
else:
if enc_value == "":
self.logger.debug(
f"{self.qid}: milter: delete "
f"header (occ. {occurrence}): "
f"{header}")
else:
self.logger.debug(
f"{self.qid}: milter: modify "
f"header (occ. {occurrence}): "
f"{header}: {enc_value}")
if milter_action is not None:
if milter_action["action"] == "reject":
self.setreply("554", "5.7.0", milter_action["reason"])
return Milter.REJECT
self.chgheader(header, occurrence, enc_value)
elif mod_type == "mod_body":
mod_body_pos = params[0]
elif mod_type == "reject":
self.setreply("554", "5.7.0", params[0])
return Milter.REJECT
if milter_action["action"] == "accept":
return Milter.ACCEPT
if milter_action["action"] == "discard":
return Milter.DISCARD
if mod_body_pos is not None:
self.fp.seek(mod_body_pos)
self.logger.debug(f"{self.qid}: milter: replace body")
self.replacebody(self.fp.read())
except Exception as e:
self.logger.exception(
f"an exception occured in eom function: {e}")

456
pymodmilter/actions.py Normal file
View File

@@ -0,0 +1,456 @@
# PyMod-Milter is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# PyMod-Milter is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with PyMod-Milter. If not, see <http://www.gnu.org/licenses/>.
#
import logging
import re
from bs4 import BeautifulSoup
from collections import defaultdict
from copy import copy
from email.header import Header
from email.parser import BytesFeedParser
from email.message import MIMEPart
from email.policy import default as default_policy, SMTP
from pymodmilter import CustomLogger, Conditions
def _replace_illegal_chars(string):
"""Replace illegal characters in header values."""
return string.replace(
"\x00", "").replace(
"\r", "").replace(
"\n", "")
def add_header(field, value, milter, idx=-1, pretend=False,
logger=logging.getLogger(__name__)):
"""Add a mail header field."""
header = f"{field}: {value}"
if logger.getEffectiveLevel() == logging.DEBUG:
logger.debug(f"add_header: {header}")
else:
logger.info(f"add_header: {header[0:70]}")
if idx == -1:
milter.fields.append((field, value))
else:
milter.fields.insert(idx, (field, value))
if pretend:
return
encoded_value = _replace_illegal_chars(
Header(s=value).encode())
milter.logger.debug(f"milter: addheader: {field}[{idx}]: {encoded_value}")
milter.addheader(field, encoded_value, idx)
def mod_header(field, value, milter, search=None, pretend=False,
logger=logging.getLogger(__name__)):
"""Change the value of a mail header field."""
if not isinstance(field, re.Pattern):
field = re.compile(field, re.IGNORECASE)
if search is not None and not isinstance(search, re.Pattern):
search = re.compile(search, re.MULTILINE + re.DOTALL + re.IGNORECASE)
occ = defaultdict(int)
for idx, (f, v) in enumerate(milter.fields):
occ[f] += 1
if not field.match(f):
continue
if search is not None:
new_v = search.sub(value, v).strip()
else:
new_v = value.strip()
if new_v == v:
continue
if not new_v:
logger.warning(
f"mod_header: resulting value is empty, "
f"skip modification")
continue
header = f"{f}: {v}"
new_header = f"{f}: {new_v}"
if logger.getEffectiveLevel() == logging.DEBUG:
logger.debug(f"mod_header: {header}: {new_header}")
else:
logger.info(f"mod_header: {header[0:70]}: {new_header[0:70]}")
milter.fields[idx] = (f, new_v)
if pretend:
continue
encoded_value = _replace_illegal_chars(
Header(s=new_v).encode())
milter.logger.debug(
f"milter: chgheader: {f}[{occ[f]}]: {encoded_value}")
milter.chgheader(f, occ[f], encoded_value)
def del_header(field, milter, value=None, pretend=False,
logger=logging.getLogger(__name__)):
"""Delete a mail header field."""
if not isinstance(field, re.Pattern):
field = re.compile(field, re.IGNORECASE)
if value is not None and not isinstance(value, re.Pattern):
value = re.compile(value, re.MULTILINE + re.DOTALL + re.IGNORECASE)
idx = -1
occ = defaultdict(int)
# iterate a copy of milter.fields because elements may get removed
# during iteration
for f, v in milter.fields.copy():
idx += 1
occ[f] += 1
if not field.match(f):
continue
if value is not None and not value.search(v):
continue
header = f"{f}: {v}"
if logger.getEffectiveLevel() == logging.DEBUG:
logger.debug(f"del_header: {header}")
else:
logger.info(f"del_header: {header[0:70]}")
del milter.fields[idx]
if not pretend:
encoded_value = ""
milter.logger.debug(
f"milter: chgheader: {f}[{occ[f]}]: {encoded_value}")
milter.chgheader(f, occ[f], encoded_value)
idx -= 1
occ[f] -= 1
def _get_body_content(msg, body_type):
content = None
body_part = msg.get_body(preferencelist=(body_type))
if body_part is not None:
content = body_part.get_content()
return (body_part, content)
def _wrap_message(milter):
msg = MIMEPart()
msg.add_header("MIME-Version", "1.0")
msg.set_content(
"Please see the original email attached.")
msg.add_alternative(
"Please see the original email attached.",
subtype="html")
data = b""
for field, value in milter.fields:
encoded_value = _replace_illegal_chars(
Header(s=value).encode())
data += field.encode("ascii", errors="replace")
data += b": "
data += encoded_value.encode("ascii", errors="replace")
data += b"\r\n"
milter.fp.seek(0)
data += b"\r\n" + milter.fp.read()
msg.add_attachment(
data, maintype="plain", subtype="text",
filename=f"{milter.qid}.eml")
return msg
def _inject_body(milter, msg):
if not msg.is_multipart():
msg.make_mixed()
new_msg = MIMEPart()
new_msg.add_header("MIME-Version", "1.0")
new_msg.set_content("")
new_msg.add_alternative("", subtype="html")
new_msg.make_mixed()
for attachment in msg.iter_attachments():
new_msg.attach(attachment)
return new_msg
def add_disclaimer(text, html, action, policy, milter, pretend=False,
logger=logging.getLogger(__name__)):
"""Append or prepend a disclaimer to the mail body."""
milter.fp.seek(0)
fp = BytesFeedParser(policy=default_policy)
for field, value in milter.fields:
field_lower = field.lower()
if not field_lower.startswith("content-") and \
field_lower != "mime-version":
continue
logger.debug(
f"feed content header to message object: {field}: {value}")
encoded_value = _replace_illegal_chars(
Header(s=value).encode())
fp.feed(field.encode("ascii", errors="replace"))
fp.feed(b": ")
fp.feed(encoded_value.encode("ascii", errors="replace"))
fp.feed(b"\r\n")
fp.feed(b"\r\n")
logger.debug(f"feed body to message object: {field}: {value}")
fp.feed(milter.fp.read())
logger.debug("parse message")
msg = fp.close()
text_content = None
html_content = None
try:
try:
logger.debug("try to find a plain and/or html body part")
text_body, text_content = _get_body_content(msg, "plain")
html_body, html_content = _get_body_content(msg, "html")
if text_content is None and html_content is None:
raise RuntimeError()
except RuntimeError:
logger.info(
"message does not contain any body part, "
"inject empty plain and html body parts")
msg = _inject_body(milter, msg)
text_body, text_content = _get_body_content(msg, "plain")
html_body, html_content = _get_body_content(msg, "html")
if text_content is None and html_content is None:
raise RuntimeError("no message body present after injecting")
except Exception as e:
logger.warning(e)
if policy == "ignore":
logger.info(
f"unable to add disclaimer to message body, "
f"ignore error according to policy")
return
elif policy == "reject":
logger.info(
f"unable to add disclaimer to message body, "
f"reject message according to policy")
return [
("reject", "Message rejected due to error")]
logger.info("wrap original message in a new message envelope")
msg = _wrap_message(milter)
text_body, text_content = _get_body_content(msg, "plain")
html_body, html_content = _get_body_content(msg, "html")
if text_content is None and html_content is None:
raise Exception("no message body present after wrapping, "
"give up ...")
if text_content is not None:
logger.info(f"{action} text disclaimer")
if action == "prepend":
content = f"{text}{text_content}"
else:
content = f"{text_content}{text}"
text_body.set_content(
content.encode(), maintype="text", subtype="plain")
text_body.set_param("charset", "UTF-8", header="Content-Type")
if html_content is not None:
logger.info(f"{action} html disclaimer")
soup = BeautifulSoup(html_content, "html.parser")
body = soup.find('body')
if body:
soup = body
if action == "prepend":
soup.insert(0, copy(html))
else:
soup.append(html)
html_body.set_content(
str(soup).encode(), maintype="text", subtype="html")
html_body.set_param("charset", "UTF-8", header="Content-Type")
try:
logger.debug("serialize message as bytes")
data = msg.as_bytes(policy=SMTP)
except Exception as e:
logger.waring(
f"unable to serialize message as bytes: {e}")
try:
logger.warning("try to serialize message as string")
data = msg.as_string(policy=SMTP)
data = data.encode("ascii", errors="replace")
except Exception as e:
raise e
body_pos = data.find(b"\r\n\r\n") + 4
milter.fp.seek(0)
milter.fp.write(data[body_pos:])
milter.fp.truncate()
if pretend:
return
logger.debug("milter: replacebody")
milter.replacebody(data[body_pos:])
del data
fields = {
"mime-version": {
"field": "MIME-Version",
"value": msg.get("MIME-Version"),
"modified": False},
"content-type": {
"field": "Content-Type",
"value": msg.get("Content-Type"),
"modified": False},
"content-transfer-encoding": {
"field": "Content-Transfer-Encoding",
"value": msg.get("Content-Transfer-Encoding"),
"modified": False}}
for field, value in milter.fields.copy():
field_lower = field.lower()
if field_lower in fields and fields[field_lower]["value"] is not None:
mod_header(field=f"^{field}$", value=fields[field_lower]["value"],
milter=milter, pretend=pretend, logger=logger)
fields[field_lower]["modified"] = True
elif field_lower.startswith("content-"):
del_header(field=f"^{field}$", milter=milter,
pretend=pretend, logger=logger)
for field in fields.values():
if not field["modified"] and field["value"] is not None:
add_header(field=field["field"], value=field["value"],
milter=milter, pretend=pretend, logger=logger)
class Action:
"""Action to implement a pre-configured action to perform on e-mails."""
_types = {
"add_header": ["fields"],
"del_header": ["fields"],
"mod_header": ["fields"],
"add_disclaimer": ["fields", "body"]}
def __init__(self, name, local_addrs, conditions, action_type, args,
loglevel=logging.INFO, pretend=False):
logger = logging.getLogger(name)
self.logger = CustomLogger(logger, {"name": name})
self.logger.setLevel(loglevel)
self.conditions = Conditions(
local_addrs=local_addrs,
args=conditions,
logger=self.logger)
self.pretend = pretend
self._args = {}
if action_type not in self._types:
raise RuntimeError(f"invalid action_type '{action_type}'")
self._needs = self._types[action_type]
try:
if action_type == "add_header":
self._func = add_header
self._args["field"] = args["header"]
self._args["value"] = args["value"]
if "idx" in args:
self._args["idx"] = args["idx"]
elif action_type in ["mod_header", "del_header"]:
args["field"] = args["header"]
del args["header"]
regex_args = ["field"]
if action_type == "mod_header":
self._func = mod_header
self._args["value"] = args["value"]
regex_args.append("search")
elif action_type == "del_header" and "value" in args:
self._func = del_header
regex_args.append("value")
for arg in regex_args:
try:
self._args[arg] = re.compile(
args[arg],
re.MULTILINE + re.DOTALL + re.IGNORECASE)
except re.error as e:
raise RuntimeError(
f"unable to parse {arg} regex: {e}")
elif action_type == "add_disclaimer":
self._func = add_disclaimer
if args["action"] not in ["append", "prepend"]:
raise RuntimeError(f"invalid action '{args['action']}'")
self._args["action"] = args["action"]
if args["error_policy"] not in ["wrap", "ignore", "reject"]:
raise RuntimeError(f"invalid policy '{args['policy']}'")
self._args["policy"] = args["error_policy"]
try:
with open(args["html_file"], "r") as f:
html = BeautifulSoup(
f.read(), "html.parser")
body = html.find('body')
if body:
# just use content within the body tag if present
html = body
self._args["html"] = html
with open(args["text_file"], "r") as f:
self._args["text"] = f.read()
except IOError as e:
raise RuntimeError(f"unable to read template: {e}")
except KeyError as e:
raise RuntimeError(
f"mandatory argument not found: {e}")
def needs(self):
"""Return the needs of this action."""
return self._needs
def execute(self, milter, pretend=None):
"""Execute configured action."""
if pretend is None:
pretend = self.pretend
logger = CustomLogger(self.logger, {"qid": milter.qid})
return self._func(
milter=milter, pretend=pretend, logger=logger, **self._args)

134
pymodmilter/conditions.py Normal file
View File

@@ -0,0 +1,134 @@
# PyMod-Milter is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# PyMod-Milter is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with PyMod-Milter. If not, see <http://www.gnu.org/licenses/>.
#
import logging
import re
from netaddr import IPAddress, IPNetwork, AddrFormatError
class Conditions:
"""Conditions to implement conditions for rules and actions."""
def __init__(self, local_addrs, args, logger=None):
if logger is None:
logger = logging.getLogger(__name__)
self._local_addrs = []
self.logger = logger
self._args = {}
try:
for addr in local_addrs:
self._local_addrs.append(IPNetwork(addr))
except AddrFormatError as e:
raise RuntimeError(f"invalid address in local_addrs: {e}")
try:
if "local" in args:
logger.debug(f"condition: local = {args['local']}")
self._args["local"] = args["local"]
if "hosts" in args:
logger.debug(f"condition: hosts = {args['hosts']}")
self._args["hosts"] = []
try:
for host in args["hosts"]:
self._args["hosts"].append(IPNetwork(host))
except AddrFormatError as e:
raise RuntimeError(f"invalid address in hosts: {e}")
if "envfrom" in args:
logger.debug(f"condition: envfrom = {args['envfrom']}")
try:
self._args["envfrom"] = re.compile(
args["envfrom"], re.IGNORECASE)
except re.error as e:
raise RuntimeError(f"unable to parse envfrom regex: {e}")
if "envto" in args:
logger.debug(f"condition: envto = {args['envto']}")
try:
self._args["envto"] = re.compile(
args["envto"], re.IGNORECASE)
except re.error as e:
raise RuntimeError(f"unable to parse envto regex: {e}")
except KeyError as e:
raise RuntimeError(
f"mandatory argument not found: {e}")
def match(self, args):
if "host" in args:
ip = IPAddress(args["host"])
if "local" in self._args:
is_local = False
for addr in self._local_addrs:
if ip in addr:
is_local = True
break
if is_local != self._args["local"]:
self.logger.debug(
f"ignore host {args['host']}, "
f"condition local does not match")
return False
self.logger.debug(
f"condition local matches for host {args['host']}")
if "hosts" in self._args:
found = False
for addr in self._args["hosts"]:
if ip in addr:
found = True
break
if not found:
self.logger.debug(
f"ignore host {args['host']}, "
f"condition hosts does not match")
return False
self.logger.debug(
f"condition hosts matches for host {args['host']}")
if "envfrom" in args and "envfrom" in self._args:
if not self._args["envfrom"].match(args["envfrom"]):
self.logger.debug(
f"ignore envelope-from address {args['envfrom']}, "
f"condition envfrom does not match")
return False
self.logger.debug(
f"condition envfrom matches for "
f"envelope-from address {args['envfrom']}")
if "envto" in args and "envto" in self._args:
if not isinstance(args["envto"], list):
args["envto"] = [args["envto"]]
for envto in args["envto"]:
if not self._args["envto"].match(envto):
self.logger.debug(
f"ignore envelope-to address {args['envto']}, "
f"condition envto does not match")
return False
self.logger.debug(
f"condition envto matches for "
f"envelope-to address {args['envto']}")
return True

212
pymodmilter/run.py
View File

@@ -12,7 +12,6 @@
# along with PyMod-Milter. If not, see <http://www.gnu.org/licenses/>.
#
import Milter
import argparse
import logging
@@ -24,77 +23,103 @@ from re import sub
from pymodmilter import Rule, ModifyMilter
from pymodmilter.version import __version__ as version
from pymodmilter.actions import Action
def main():
"Run PyMod-Milter."
# parse command line
parser = argparse.ArgumentParser(
description="PyMod milter daemon",
formatter_class=lambda prog: argparse.HelpFormatter(
prog, max_help_position=45, width=140))
parser.add_argument(
"-c", "--config", help="Config file to read.",
default="/etc/pymodmilter/pymodmilter.conf")
parser.add_argument(
"-s",
"--socket",
help="Socket used to communicate with the MTA.",
default="")
parser.add_argument(
"-d",
"--debug",
help="Log debugging messages.",
action="store_true")
parser.add_argument(
"-t",
"--test",
help="Check configuration.",
action="store_true")
parser.add_argument(
"-v", "--version",
help="Print version.",
action="version",
version=f"%(prog)s ({version})")
args = parser.parse_args()
# setup logging
loglevel = logging.INFO
logname = "pymodmilter"
syslog_name = logname
if args.debug:
loglevel = logging.DEBUG
logname = f"{logname}[%(name)s]"
syslog_name = f"{syslog_name}: [%(name)s] %(levelname)s"
loglevels = {
"error": logging.ERROR,
"warning": logging.WARNING,
"info": logging.INFO,
"debug": logging.DEBUG
}
root_logger = logging.getLogger()
root_logger.setLevel(loglevel)
root_logger.setLevel(logging.DEBUG)
# setup console log
stdouthandler = logging.StreamHandler(sys.stdout)
stdouthandler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(message)s")
stdouthandler.setFormatter(formatter)
stdouthandler.setFormatter(
logging.Formatter("%(asctime)s - %(levelname)s: %(message)s"))
root_logger.addHandler(stdouthandler)
# setup syslog
sysloghandler = logging.handlers.SysLogHandler(
address="/dev/log", facility=logging.handlers.SysLogHandler.LOG_MAIL)
sysloghandler.setFormatter(
logging.Formatter("pymodmilter: %(message)s"))
root_logger.addHandler(sysloghandler)
logger = logging.getLogger(__name__)
if not args.debug:
logger.setLevel(logging.INFO)
try:
# read config file
logger.debug("parsing config file")
try:
with open(args.config, "r") as fh:
config = loads(
sub(r"(?m)^\s*#.*\n?", "", fh.read()))
config = sub(r"(?m)^\s*#.*\n?", "", fh.read())
config = loads(config)
except Exception as e:
for num, line in enumerate(config.splitlines()):
logger.error(f"{num+1}: {line}")
raise RuntimeError(
f"unable to parse config file: {e}")
logger.debug("preparing milter configuration ...")
# default values for global config if not set
if "global" not in config:
config["global"] = {}
if "loglevel" not in config["global"]:
config["global"]["loglevel"] = "info"
if args.debug:
loglevel = logging.DEBUG
else:
loglevel = loglevels[config["global"]["loglevel"]]
logger.setLevel(loglevel)
logger.debug("prepar milter configuration")
if "pretend" not in config["global"]:
config["global"]["pretend"] = False
if args.socket:
socket = args.socket
elif "socket" in config["global"]:
@@ -104,64 +129,110 @@ def main():
f"listening socket is neither specified on the command line "
f"nor in the configuration file")
if "local_addrs" not in config["global"]:
config["global"]["local_addrs"] = [
if "local_addrs" in config["global"]:
local_addrs = config["global"]["local_addrs"]
else:
local_addrs = [
"::1/128",
"127.0.0.0/8",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16"]
if "log" not in config["global"]:
config["global"]["log"] = True
if "pretend" not in config["global"]:
config["global"]["pretend"] = False
# check if mandatory sections are present in config
for section in ["rules"]:
if section not in config:
raise RuntimeError(
f"mandatory config section '{section}' not found")
if "rules" not in config:
raise RuntimeError(
f"mandatory config section 'rules' not found")
if not config["rules"]:
raise RuntimeError("no rules configured")
rules = []
# iterate configured rules
for rule_idx, rule in enumerate(config["rules"]):
params = {}
# set default values if not specified in config
if "name" in rule:
params["name"] = rule["name"]
else:
params["name"] = f"Rule #{rule_idx}"
logger.debug("initialize rules ...")
if "log" in rule:
params["log"] = rule["log"]
rules = []
for rule_idx, rule in enumerate(config["rules"]):
if "name" in rule:
rule_name = rule["name"]
else:
params["log"] = config["global"]["log"]
rule_name = f"Rule #{rule_idx}"
logger.debug(f"prepare rule {rule_name} ...")
if "actions" not in rule:
raise RuntimeError(
f"{rule_name}: mandatory config "
f"section 'actions' not found")
if not rule["actions"]:
raise RuntimeError("{rule_name}: no actions configured")
if args.debug:
rule_loglevel = logging.DEBUG
elif "loglevel" in rule:
rule_loglevel = loglevels[rule["loglevel"]]
else:
rule_loglevel = loglevels[config["global"]["loglevel"]]
if "pretend" in rule:
params["pretend"] = rule["pretend"]
rule_pretend = rule["pretend"]
else:
params["pretend"] = config["global"]["pretend"]
rule_pretend = config["global"]["pretend"]
if "local_addrs" in rule:
params["local_addrs"] = rule["local_addrs"]
else:
params["local_addrs"] = config["global"]["local_addrs"]
actions = []
for action_idx, action in enumerate(rule["actions"]):
if "name" in action:
action_name = action["name"]
else:
action_name = f"Action #{action_idx}"
if "conditions" in rule:
params["conditions"] = rule["conditions"]
if args.debug:
action_loglevel = logging.DEBUG
elif "loglevel" in action:
action_loglevel = loglevels[action["loglevel"]]
else:
action_loglevel = rule_loglevel
if "modifications" in rule:
params["modifications"] = rule["modifications"]
else:
raise RuntimeError(
f"{rule['name']}: mandatory config section "
f"'modifications' not found")
if "pretend" in action:
action_pretend = action["pretend"]
else:
action_pretend = rule_pretend
rules.append(Rule(**params))
if "type" not in action:
raise RuntimeError(
f"{rule_name}: {action_name}: mandatory config "
f"section 'actions' not found")
if "conditions" not in action:
action["conditions"] = {}
try:
actions.append(
Action(
name=action_name,
local_addrs=local_addrs,
conditions=action["conditions"],
action_type=action["type"],
args=action,
loglevel=action_loglevel,
pretend=action_pretend))
except RuntimeError as e:
logger.error(f"{action_name}: {e}")
sys.exit(253)
if "conditions" not in rule:
rule["conditions"] = {}
try:
rules.append(
Rule(
name=rule_name,
local_addrs=local_addrs,
conditions=rule["conditions"],
actions=actions,
loglevel=rule_loglevel,
pretend=rule_pretend))
except RuntimeError as e:
logger.error(f"{rule_name}: {e}")
sys.exit(254)
except RuntimeError as e:
logger.error(e)
@@ -171,27 +242,22 @@ def main():
print("Configuration ok")
sys.exit(0)
# change log format for runtime
formatter = logging.Formatter(
f"%(asctime)s {logname}: [%(levelname)s] %(message)s",
datefmt="%Y-%m-%d %H:%M:%S")
# setup console log for runtime
formatter = logging.Formatter("%(asctime)s - %(levelname)s: %(message)s")
stdouthandler.setFormatter(formatter)
# setup syslog
sysloghandler = logging.handlers.SysLogHandler(
address="/dev/log", facility=logging.handlers.SysLogHandler.LOG_MAIL)
sysloghandler.setLevel(loglevel)
formatter = logging.Formatter(f"{syslog_name}: %(message)s")
sysloghandler.setFormatter(formatter)
root_logger.addHandler(sysloghandler)
stdouthandler.setLevel(logging.DEBUG)
logger.info("pymodmilter starting")
ModifyMilter.set_rules(rules)
ModifyMilter.set_loglevel(loglevels[config["global"]["loglevel"]])
# register milter factory class
Milter.factory = ModifyMilter
Milter.set_exception_policy(Milter.TEMPFAIL)
if args.debug:
Milter.setdbg(1)
rc = 0
try:
Milter.runmilter("pymodmilter", socketname=socket, timeout=30)