Skip to content

PYTHON-3256 Obtain AWS credentials for CSFLE in the same way as for MONGODB-AWS #1035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .evergreen/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,9 @@ if [ -n "$TEST_ENCRYPTION" ]; then
python -c "import pymongocrypt; print('libmongocrypt version: '+pymongocrypt.libmongocrypt_version())"
# PATH is updated by PREPARE_SHELL for access to mongocryptd.

# Need aws dependency for On-Demand KMS Credentials.
python -m pip install '.[aws]'

# Get access to the AWS temporary credentials:
# CSFLE_AWS_TEMP_ACCESS_KEY_ID, CSFLE_AWS_TEMP_SECRET_ACCESS_KEY, CSFLE_AWS_TEMP_SESSION_TOKEN
. $DRIVERS_TOOLS/.evergreen/csfle/set-temp-creds.sh
Expand Down
3 changes: 2 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,8 @@ Wire protocol compression with zstandard requires `zstandard
$ python -m pip install "pymongo[zstd]"

Client-Side Field Level Encryption requires `pymongocrypt
<https://pypi.org/project/pymongocrypt/>`_::
<https://pypi.org/project/pymongocrypt/>`_ and
`pymongo-auth-aws <https://pypi.org/project/pymongo-auth-aws/>`_::

$ python -m pip install "pymongo[encryption]"

Expand Down
5 changes: 3 additions & 2 deletions doc/examples/encryption.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ Dependencies

To get started using client-side field level encryption in your project,
you will need to install the
`pymongocrypt <https://pypi.org/project/pymongocrypt/>`_ library
`pymongocrypt <https://pypi.org/project/pymongocrypt/>`_ and
`pymongo-auth-aws <https://pypi.org/project/pymongo-auth-aws/>`_ libraries
as well as the driver itself. Install both the driver and a compatible
version of pymongocrypt like this::
version of the dependencies like this::

$ python -m pip install 'pymongo[encryption]'

Expand Down
3 changes: 2 additions & 1 deletion doc/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ Wire protocol compression with zstandard requires `zstandard
$ python3 -m pip install "pymongo[zstd]"

:ref:`Client-Side Field Level Encryption` requires `pymongocrypt
<https://pypi.org/project/pymongocrypt/>`_::
<https://pypi.org/project/pymongocrypt/>`_ and
`pymongo-auth-aws <https://pypi.org/project/pymongo-auth-aws/>`_::

$ python3 -m pip install "pymongo[encryption]"

Expand Down
6 changes: 4 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -278,12 +278,14 @@ def build_extension(self, ext):
# https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.set_default_verify_paths
pyopenssl_reqs.append("certifi")

aws_reqs = ["pymongo-auth-aws<2.0.0"]

extras_require = {
"encryption": ["pymongocrypt>=1.3.0,<2.0.0"],
"encryption": ["pymongocrypt>=1.3.0,<2.0.0"] + aws_reqs,
"ocsp": pyopenssl_reqs,
"snappy": ["python-snappy"],
"zstd": ["zstandard"],
"aws": ["pymongo-auth-aws<2.0.0"],
"aws": aws_reqs,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's update the docs to say that pymongo-auth-aws is also required for encryption in these 3 places:

README.rst:135:  $ python -m pip install "pymongo[encryption]"
doc/examples/encryption.rst:30:  $ python -m pip install 'pymongo[encryption]'
doc/installation.rst:75:  $ python3 -m pip install "pymongo[encryption]"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the main point of the extra that the user doesn't need to concern themselves with the libraries used?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure but we should still document what libraries will be installed.

Copy link
Member

@ShaneHarvey ShaneHarvey Oct 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note I'm not saying to change those 3 lines in particular. Just the surrounding text. For example the readme says:

Client-Side Field Level Encryption requires `pymongocrypt
<https://pypi.org/project/pymongocrypt/>`_::

  $ python -m pip install "pymongo[encryption]"

But this is now incomplete because it doesn't mention pymongo-auth-aws.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

"srv": [], # PYTHON-3423 Removed in 4.3 but kept here to avoid pip warnings.
"tls": [], # PYTHON-2133 Removed in 4.0 but kept here to avoid pip warnings.
}
Expand Down
31 changes: 31 additions & 0 deletions test/test_encryption.py
Original file line number Diff line number Diff line change
Expand Up @@ -2304,6 +2304,37 @@ def run_test(self, src_provider, dst_provider):
self.assertEqual(decrypt_result2, "test")


# https://github.com/mongodb/specifications/blob/5cf3ed/source/client-side-encryption/tests/README.rst#on-demand-aws-credentials
class TestOnDemandAWSCredentials(EncryptionIntegrationTest):
def setUp(self):
super(TestOnDemandAWSCredentials, self).setUp()
self.master_key = {
"region": "us-east-1",
"key": ("arn:aws:kms:us-east-1:579766882180:key/89fcc2c4-08b0-4bd9-9f25-e30687b580d0"),
}

@unittest.skipIf(any(AWS_CREDS.values()), "AWS environment credentials are set")
def test_01_failure(self):
self.client_encryption = ClientEncryption(
kms_providers={"aws": {}},
key_vault_namespace="keyvault.datakeys",
key_vault_client=client_context.client,
codec_options=OPTS,
)
with self.assertRaises(EncryptionError):
self.client_encryption.create_data_key("aws", self.master_key)

@unittest.skipUnless(any(AWS_CREDS.values()), "AWS environment credentials are not set")
def test_02_success(self):
self.client_encryption = ClientEncryption(
kms_providers={"aws": {}},
key_vault_namespace="keyvault.datakeys",
key_vault_client=client_context.client,
codec_options=OPTS,
)
self.client_encryption.create_data_key("aws", self.master_key)


class TestQueryableEncryptionDocsExample(EncryptionIntegrationTest):
# Queryable Encryption is not supported on Standalone topology.
@client_context.require_no_standalone
Expand Down