Support for passing in connection arg or existing SSL connection to asyncpg.connect()

[jackwotherspoon]

The Cloud SQL Python Connector would like to support database connections to Cloud SQL using asyncpg. The Cloud SQL connectors connect to a server side proxy that authorizes users based on a TLS client cert. In order to do this in asyncpg, we require the ability to configure the connection level SSL (outside of the database protocol) or pass in an existing connection (with its own SSL/TLS configuration).

For the pg8000 driver, we use the first option – their ssl_context argument allows us to pass in our pre-configured ssl.SSLContext object as long as the custom require_ssl attribute is set to False in order to skip the Postgres level SSL protocol . pg8000 code

For PyMySQL driver, we create the connection ahead of time, wrap it with our own SSL config, and pass it to the driver.

Would either of these options be suitable for asyncpg? Happy to provide more information or assist on this if needed. Thanks so much!

[elprans]

You can pass an arbitrary ssl.SSLContext as the ssl argument to connect()/create_pool().

[jackwotherspoon]

@elprans I need a way to skip the database level SSL/TLS exchange. pg8000 allows this to be done by passing in an ssl.SSLContext with a request_ssl attribute set to False. The arbitrary ssl.SSLContext for asyncpg does not natively have this functionality I believe. Would this be something that is suitable for asyncpg? It would allow us to help support asyncpg within the Cloud SQL Python Connector. I am happy to try and assist on it if needed. Thanks in advance.

[jackwotherspoon]

@elprans just wanted to follow up with a little more context for you.

The Cloud SQL Connector takes care of the SSL/TLS configuration independent of the database protocol. We need a way to skip the database level SSL/TLS exchange because requesting SSL fails since with Cloud SQL we're connecting to a server side proxy instead of directly to Postgres.

Happy to hop on a call to chat and discuss this more if need be. I'd also be willing to help out on the code too if we chat and agree on an implementation that you are comfortable with.

Let me know, thanks for the help and have a great day!

[elprans]

@jackwotherspoon, I see. I think it's not unreasonable to add the ability to perform a direct SSL connection instead of PostgreSQL STARTTLS mode. Perhaps as a tlsproxy=True argument to connect().

[jackwotherspoon]

That sounds reasonable to me as well. If you are able to provide me with a high level overview of main code files to update, I would be more than happy to attempt an initial pass at this and try to put up a PR?

Let me know what you think. Thanks for the updates, appreciate your help :)

[jackwotherspoon]

@elprans do you have any guidance on which parts of the _create_ssl_connection function specifically would need to be updated in order to accomplish the feat?

I will try and put up a PR if I can figure out which section of code to properly update.

Thanks in advance!

[jackwotherspoon]

@elprans just wanted to follow up and see if you could point me in the right direction for this :) I am more than happy to try and get started on the work if you can help scope out the changes you think are necessary and which files to update.

Let me know if this potentially possible, thanks so much!

[elprans]

In the case of tlsproxy=True you should not call _create_ssl_connection at all, since that's exactly the custom PostgreSQL STARTLS code we are trying to avoid. Instead, pass ssl=<ssl-context> to loop.create_connection() in __connect_addr.

[jackwotherspoon]

@elprans thanks for the assist, I have opened PR #923 that I tested with the Cloud SQL Python connector and it works! Just need an approver to run the tests workflow.

Thanks again for the help on this and have a great day.