Bedrock API: Add option to configure API endpoint (#1018) #1063
Conversation
|
Any timeline for this to be merged? |
|
This change still hasn't been approved for merge. |
|
Hi @tzolov @markpollack it would be awesome if you guys could have a look at this. |
|
@tzolov @markpollack Any luck with this so far? It's a useful MR for a lot of users I'd reckon, given the concerns regarding data privacy and protection. |
| @@ -109,6 +110,21 @@ public AnthropicChatBedrockApi(String modelId, AwsCredentialsProvider credential | |||
| super(modelId, credentialsProvider, region, objectMapper, timeout); | |||
| } | |||
|
|
|||
| /** | |||
| * Create a new AnthropicChatBedrockApi instance using the provided credentials provider, region and object mapper. | |||
There was a problem hiding this comment.
I think we can also specify here that we require a new parameter, the endpoint override.
| @@ -113,6 +114,21 @@ public Anthropic3ChatBedrockApi(String modelId, AwsCredentialsProvider credentia | |||
| super(modelId, credentialsProvider, region, objectMapper, timeout); | |||
| } | |||
|
|
|||
| /** | |||
| * Create a new AnthropicChatBedrockApi instance using the provided credentials provider, region and object mapper. | |||
| @@ -108,6 +109,21 @@ public CohereChatBedrockApi(String modelId, AwsCredentialsProvider credentialsPr | |||
| super(modelId, credentialsProvider, region, objectMapper, timeout); | |||
| } | |||
|
|
|||
| /** | |||
| * Create a new CohereChatBedrockApi instance using the provided credentials provider, region and object mapper. | |||
| @@ -109,6 +110,22 @@ public CohereEmbeddingBedrockApi(String modelId, AwsCredentialsProvider credenti | |||
| super(modelId, credentialsProvider, region, objectMapper, timeout); | |||
| } | |||
|
|
|||
| /** | |||
| * Create a new CohereEmbeddingBedrockApi instance using the provided credentials provider, region and object | |||
| * @param region The AWS region to use. | ||
| * @param objectMapper The object mapper to use for JSON serialization and deserialization. | ||
| * @param timeout The timeout to use. | ||
| * @param endpointOverride The endpoint to use. |
There was a problem hiding this comment.
"The endpoint override to use."
| @@ -106,6 +107,21 @@ public LlamaChatBedrockApi(String modelId, AwsCredentialsProvider credentialsPro | |||
| super(modelId, credentialsProvider, region, objectMapper, timeout); | |||
| } | |||
|
|
|||
| /** | |||
| * Create a new LlamaChatBedrockApi instance using the provided credentials provider, region and object mapper. | |||
There was a problem hiding this comment.
Same small changes here and also all places similarly set below.
|
@szarpul Thank you for implementing this! I left some small comments, mostly about the "code documentation" aspect of the MR, that, otherwise, looks great to me and something that would be really, really useful for a lot of people to benefit from! If necessary, I can take these changes upon myself in terms of addressing the review comments and seeing if we can get this one past the finish line too! |
|
The chat models for bedrock API have been removed in Spring AI M6 since Amazon is encouraging people to switch to the new API and this reduced a great deal of maintenance burdon for us not having to support multiple vendor SDK/APIs. We did keep the embedding models, so that is something that is still relevant. |
|
Hey @bruno-oliveira is there something in this PR that can be reused? the embedding models perhaps? |
|
Hey @tzolov! Maybe!! I’d need to have a look! I recall indeed that we ended
up writing some custom code directly against the bedrock runtime api for
the embedding models but, maybe with the new update that can actually be
reused and we can get rid of our custom code 😄 I’ll have a look when time
is right and get back to you 💪
…On Wed, 9 Apr 2025 at 19:59, Christian Tzolov ***@***.***> wrote:
Hey @bruno-oliveira <https://github.com/bruno-oliveira> is there
something in this PR that can be reused? the embedding models perhaps?
—
Reply to this email directly, view it on GitHub
<#1063 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABEA53BD4D4LBCABZZN7V7T2YVNW3AVCNFSM6AAAAAB2M2QFX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJQGUZDINZRGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
*tzolov* left a comment (spring-projects/spring-ai#1063)
<#1063 (comment)>
Hey @bruno-oliveira <https://github.com/bruno-oliveira> is there
something in this PR that can be reused? the embedding models perhaps?
—
Reply to this email directly, view it on GitHub
<#1063 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABEA53BD4D4LBCABZZN7V7T2YVNW3AVCNFSM6AAAAAB2M2QFX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJQGUZDINZRGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
#1018
Add URL parameter to the constructor to set the
endpointOverrideinSdkClientBuilder.The URL can be null as the
endpointOverrideis also null by default.