Open
Description
What happened?
When using litellm SDK version 1.48.7 like this:
from litellm import completion
import json
## GET CREDENTIALS
file_path = 'PATH_TO_JSON'
# Load the JSON file
with open(file_path, 'r') as file:
vertex_credentials = json.load(file)
# Convert to JSON string
vertex_credentials_json = json.dumps(vertex_credentials)
response = completion(
model="vertex_ai/gemini-pro",
messages=[{"content": "You are a good bot.","role": "system"}, {"content": "tell me poem on pasta","role": "user"}],
vertex_credentials=vertex_credentials_json,
vertex_project="my_project_id",
vertex_location="us-central1"
)
We are seeing error with stack trace:
Traceback (most recent call last):
File "/home/appuser/.local/lib/python3.11/site-packages/litellm/main.py", line 2280, in completion
model_response = vertex_chat_completion.completion( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/gemini/vertex_and_google_ai_studio_gemini.py", line 1208, in completion
_auth_header, vertex_project = self._ensure_access_token(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/vertex_llm_base.py", line 137, in _ensure_access_token
self._credentials, cred_project_id = self.load_auth(
^^^^^^^^^^^^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/vertex_llm_base.py", line 79, in load_auth
creds = identity_pool.Credentials.from_info(json_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/identity_pool.py", line 425, in from_info
return super(Credentials, cls).from_info(info, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/external_account.py", line 591, in from_info
return cls(
^^^^
File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/identity_pool.py", line 273, in __init__
raise exceptions.MalformedError(
google.auth.exceptions.MalformedError: Invalid Identity Pool credential_source field 'environment_id'
Whereas when I use vertex ai SDK itself like this with my same service account credentials file like this:
import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="PATH_TO_JSON"
os.environ["VERTEXAI_LOCATION"]="us-central1"
os.environ["VERTEXAI_PROJECT"]="my_project_id""
import vertexai
from vertexai.generative_models import GenerativeModel
vertexai.init(project="my_project_id"", location="us-central1")
model = GenerativeModel("gemini-pro")
response = model.generate_content(
"tell me poem on pasta"
)
print(response.text)
Then it works fine.
My service account file looks like this:
{
"type": "external_account",
"audience": "....",
"subject_token_type": "....",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/.....iam.gserviceaccount.com:generateAccessToken",
"token_url": "https://sts.googleapis.com/v1/token",
"credential_source": {
"environment_id": "aws1",
"region_url": "....",
"url": "....",
"regional_cred_verification_url": "...."
}
}
Relevant log output
No response
Twitter / LinkedIn details
No response