ModuleNotFoundError: No module named 'google.auth'

[FreakTheMighty]

[REQUIRED] Step 2: Describe your environment

• Operating System version: amazonlinux Dockerfile
• Firebase SDK version: v2.12.0
• Library version: v2.12.0
• Firebase Product: firestore

[REQUIRED] Step 3: Describe the problem

pip install doesn't appear to install all transitive dependencies. As a result I'm getting missing module errors when I from firebase_admin import firestore

Traceback (most recent call last):
  File "lib/firebase_admin/firestore.py", line 22, in <module>
    from google.cloud import firestore # pylint: disable=import-error,no-name-in-module
  File "lib/google/cloud/firestore.py", line 18, in <module>
    from google.cloud.firestore_v1beta1 import __version__
  File "lib/google/cloud/firestore_v1beta1/__init__.py", line 20, in <module>
    from google.cloud.firestore_v1beta1 import types
  File "lib/google/cloud/firestore_v1beta1/types.py", line 18, in <module>
    from google.api_core.protobuf_helpers import get_messages
  File "lib/google/api_core/protobuf_helpers.py", line 21, in <module>
    from google.protobuf import field_mask_pb2
ModuleNotFoundError: No module named 'google.protobuf'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 4, in <module>
    from firebase_admin import firestore
  File "lib/firebase_admin/firestore.py", line 28, in <module>
    raise ImportError('Failed to import the Cloud Firestore library for Python. Make sure '
ImportError: Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.

Steps to reproduce:

I've created a dockerized reproduction of the problem:

1. git clone https://github.com/FreakTheMighty/FirestoreImportIssue.git
2. docker-compose run lambda bash test.sh
3. Note the traceback

Relevant Code:

from firebase_admin import firestore

[google-oss-bot]

Hmmm this issue does not seem to follow the issue template. Make sure you provide all the required information.

[google-oss-bot]

Hey there! I couldn't figure out what this issue is about, so I've labeled it for a human to triage. Hang tight.

[FreakTheMighty]

I see this is a duplicate of googleapis/google-cloud-python#5023

[FreakTheMighty]

This issue is driving me 🍌 🍌 🍌

I'm trying to follow the fix in the ☝️ issue, locking versions in a requirements file, but it feels like a game of wack-a-mole. My requirements now looks like:

firebase_admin==2.12.0
grpcio==1.13.0rc3
google-cloud-firestore==0.29.0
google-cloud-storage==1.10.0
google-auth==1.5.0
protobuf==3.6.0

And I now get this error:

Traceback (most recent call last):
  File "lib/firebase_admin/firestore.py", line 22, in <module>
    from google.cloud import firestore # pylint: disable=import-error,no-name-in-module
  File "lib/google/cloud/firestore.py", line 18, in <module>
    from google.cloud.firestore_v1beta1 import __version__
  File "lib/google/cloud/firestore_v1beta1/__init__.py", line 20, in <module>
    from google.cloud.firestore_v1beta1 import types
  File "lib/google/cloud/firestore_v1beta1/types.py", line 18, in <module>
    from google.api_core.protobuf_helpers import get_messages
  File "lib/google/api_core/protobuf_helpers.py", line 21, in <module>
    from google.protobuf import field_mask_pb2
ModuleNotFoundError: No module named 'google.protobuf'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 5, in <module>
    from firebase_admin import firestore
  File "lib/firebase_admin/firestore.py", line 28, in <module>
    raise ImportError('Failed to import the Cloud Firestore library for Python. Make sure '
ImportError: Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.

[hiranya911]

The only working solution I've found for this problem is simply installing google-cloud-firestore directly -- in addition to firebase-admin. I'm going to see whether taking a dependency on google-api-core[grpc] is going to fix this (which is the resolution we got from the google-auth team), but they also recommended dropping direct dependencies on requests and google-auth which seems seems strange, and is the reason why I'm reluctant to implement the change.

[FreakTheMighty]

@hiranya911 I tried installing both directly, but couldn't get it to work. What versions do you pin google-cloud-firestore to? As you can see from my requirements.txt file above ☝️ I've locked firestore to 0.29.0.

[Billcountry]

I'm experiencing the same issue. My packages are installed in the project directory lib.
The issue with exporting falls in when firebase-admin tries to import google.
If you have an installation of google_appengine the following module is imported instead:
<module 'google' from '/usr/lib/google-cloud-sdk/platform/google_appengine/google/__init__.py'>
while what we really expect is:
<module 'google.auth' from '/home/****/.local/lib/python2.7/site-packages/google/auth/init.pyc'>

[hiranya911]

@Billcountry you have a different problem stemming from the fact that Google Python libs use hierarchical packaging. See the discussion and the workarounds suggested in the following issue: googleapis/google-auth-library-python#169

[hiranya911]

It seems installation of grpcio currently breaks on Pypy: googleapis/google-cloud-python#5902

We cannot implement the proposed fix until that gets resolved.

[hiranya911]

Now that #229 is in place, we might be able to revisit this.

[hiranya911]

@FreakTheMighty can you try out the above fix in your test/development environment and see if that works? It seems to work fine in my local tox environment, as well as our Travis CI environment.