ssl module: add getter for SSL_CTX* and SSL*

[tiran]

BPO	43902
Nosy	@tiran, @njsmith, @zooba, @eighthave

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = None
created_at = <Date 2021-04-21.07:59:53.368>
labels = ['expert-SSL', 'type-feature', '3.10', '3.11']
title = 'ssl module: add getter for SSL_CTX* and SSL*'
updated_at = <Date 2022-03-20.14:22:54.275>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2022-03-20.14:22:54.275>
actor = 'eighthave'
assignee = 'christian.heimes'
closed = False
closed_date = None
closer = None
components = ['SSL']
creation = <Date 2021-04-21.07:59:53.368>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 43902
keywords = []
message_count = 8.0
messages = ['391498', '391530', '391533', '391534', '391535', '391536', '391539', '415606']
nosy_count = 4.0
nosy_names = ['christian.heimes', 'njs', 'steve.dower', 'eighthave']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue43902'
versions = ['Python 3.10', 'Python 3.11']

[tiran]

Python's ssl module exposes a limited and opinionated set of knobs to tune OpenSSL's behavior. Each new setter, getter, or function must be carefully design, tested, and documented. For each feature OpenSSL's C API must be converted into a Pythonic, self-explanatory interface.

I would like to give experts and power users an interface to set advanced options. libffi-based solutions like ctypes and cffi are obvious choices. For libffi to work, users need to be able to get the address of ssl.SSLContext()'s SSL_CTX pointer and the SSL* pointer of the internal _SSLSocket object.

While it's possible to use pointer arithmetic with id(ctx) + offset, I would like to add a more convenient way. Pointer arithmetic with ctypes is non-trivial. Users would have to rely on internal, private layout of PySSLContext and PySSLSocket struct. I'm considering two new methods ctx._ssl_ctx_addr and ssl._ssl_addr (names are tentative).

>>> import ssl, ctypes
>>> ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
>>> libssl = ctypes.CDLL("libssl.so.1.1")  # ssl._ssl.__file__ works, too
>>> libssl.SSL_CTX_set_ciphersuites(ctx._ssl_ctx_addr(), b"TLS_CHACHA20_POLY1305_SHA256")
1

Steve, Nathaniel, how do you like the idea in general? Do you have better ideas for function names?

[zooba]

Could we have the address exposed in a way that can only be passed back into ctypes? Or alternatively, doesn't function if ctypes is missing?

I don't like offering ways to get real memory addresses, especially for interesting objects. At the same time, I see the value of it being available raw for CFFI and other tools that can use it.

Maybe it would be better to expose it as ctx._call_with_ctypes("SSL_CTX_set_ciphersuites", b"TLS_CHACHA20_POLY1305_SHA256")? That also leaves the opportunity to special-case certain functions in the future.

[njsmith]

Funny, I was actually looking at this a bit last week, because I was trying to figure out if I could trick ssl into doing DTLS...

The two big problems I ran into are:

• for DTLS you need to instantiate the SSLContext with PROTOCOL_DTLS, and idk how you create a Python ssl._SSLMethod object for a random pointer.

• You need to somehow find the libssl symbols, and it's not clear how to do that in a safe and portable way.

The second issue seems like the big blocker. Your example does ctypes.CDLL("libssl.so.1.1"), but there's no guarantee that this will find the same libssl that the ssl module is linked against. (And if you get the wrong one then everything will probably explode.)

And like you say, ctypes.CDLL(ssl._ssl.__file__) also works, by taking advantage of how ELF shared libraries tend to reexport all the symbols they import, but... this is an ELF-specific thing. It doesn't work on Windows or macOS.

So to me the first question would be: is it possible to offer a utility function that reliably gives users a handle to the right libssl, on all systems and build configurations? If not then I don't see how this could work.

[tiran]

I don't want to import ctypes from the ssl module code.

PyCapsule could be a solution for the problem. Users would have to call PyCapsule_Import("_ssl.capsule") and PyCapsule_GetPointer() to access a struct with additional methods. It's a bit more work to implement and slightly harder to use, but safer than exposing raw memory addresses.

[tiran]

Funny, I was looking into the same issue with CDLL(). :)

The trick with ssl._ssl.__file__ may even break if users change sys.setdlopenflag() from RTLD_GLOBAL to RTLD_LOCAL. Static linking will also influence which symbols are available.

Python/dynload_shlib.c keeps a list of dlopen handles around, but there is no way to access the list of handles. Mmh ... tricky.