smtplib: allow custom policy or use msg.policy in send_message

[Miksus]

BPO	47047
Nosy	@warsaw, @bitdancer, @Miksus

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-17.16:48:39.898>
labels = ['3.8', 'expert-email', '3.10', '3.11', '3.7', 'library', '3.9']
title = 'smtplib: allow custom policy or use msg.policy in send_message'
updated_at = <Date 2022-03-17.18:34:59.131>
user = 'https://github.com/Miksus'

bugs.python.org fields:

activity = <Date 2022-03-17.18:34:59.131>
actor = 'Miksus'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'email']
creation = <Date 2022-03-17.16:48:39.898>
creator = 'Miksus'
dependencies = []
files = []
hgrepos = []
issue_num = 47047
keywords = []
message_count = 2.0
messages = ['415428', '415436']
nosy_count = 3.0
nosy_names = ['barry', 'r.david.murray', 'Miksus']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue47047'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[Miksus]

The method smtplib.SMTP.send_message does not use the message's Policy if all of the from_addrs or to_addrs are not international. See: https://github.com/python/cpython/blob/v3.10.3/Lib/smtplib.py#L983 (unchanged in current main). The email.generator.BytesGenerator does not capture the email's policy as it was not passed to its init.

This has at least one notable setback: you cannot set the mangle_from to False meaning that the method will always turn "From ..." to ">From ..." in the plain text part (though often that is desirable). This is especially confusing as email library has the mangle_from as False by default for EmailMessages but smtplib.SMTP's send_message does not respect this by default.

The smtplib.SMTP.send_message has a mention about this in the docstring thus not entirely sure if intentional:

... Otherwise the generator is called without modifying the
    policy.

If we changed this line: https://github.com/python/cpython/blob/v3.10.3/Lib/smtplib.py#L983

from this:
g = email.generator.BytesGenerator(bytesmsg)

to this:
g = email.generator.BytesGenerator(bytesmsg, policy=msg.policy.clone()

smptlib's tests are passed but I suspect it's not that simple. The docstring mention indicates this is at some level intentional and I think the mangle_from needs to remain True as otherwise, it may cause security problems in existing code. Another option perhaps could be that the policy could be passed with the send_message and that is used if not None or we could have argument "msg_policy=False" that if True, the message's policy is used.

One could also think that this could be overcome by subclassing the SMTP. However, the logic is such deep in that it is not convenient.

So in short, the options I thought of:

• Have an argument "policy" in send_message to force usage of your own policy (safe option)
• Have an argument "msg_policy" (name debatable) in send_message and if True, the message's policy is always used (safe option)
• Use the message's policy always (unsafe, possibly breaking and causing security issues in existing code)