|
| 1 | +[[ChatClient]] |
| 2 | += Chat Client API |
| 3 | + |
| 4 | +The `ChatClient` offers a fluent API for stateless interaction with an AI Model. It supports both a synchronous and reactive programming model. |
| 5 | + |
| 6 | +The fluent API has methods for building up the constituent parts of a `Prompt` that is passed to the AI model as input. |
| 7 | +The `Prompt` contains the instructional text to guide the AI model's output and behavior. From the API point of view, prompts consist of a collection of messages. |
| 8 | + |
| 9 | +The AI model processes two main types of messages: user messages, which are direct inputs from the user, and system messages, which are generated by the system to guide the conversation. |
| 10 | + |
| 11 | +These messages often contain template placeholders that are substituted at runtime based on user input to customize the response of the AI model to the user input. |
| 12 | + |
| 13 | +There are also Prompt options that can be specified., such as the name of the AI Model to generate content and the temperature setting that controls the randomness or creativity of the generated output. |
| 14 | + |
| 15 | +== Using an autoconfigured ChatClient.Builder |
| 16 | + |
| 17 | +In the most simple use case, Spring AI provides Spring Boot autoconfiguration, creating a prototype `ChatClient.Builder` bean for you to inject into your class. |
| 18 | +Here is a simple example of retrieving a String response to a simple user request. |
| 19 | + |
| 20 | +```java |
| 21 | +@RestController |
| 22 | +class MyController { |
| 23 | + |
| 24 | + private final ChatClient chatClient; |
| 25 | + |
| 26 | + public MyController(ChatClient.Builder chatClientBuilder) { |
| 27 | + this.chatClient = chatClientBuilder.build(); |
| 28 | + } |
| 29 | + |
| 30 | + @GetMapping("/ai") |
| 31 | + String generation(String userInput) { |
| 32 | + return this.chatClient.prompt() |
| 33 | + .user(userInput) |
| 34 | + .call() |
| 35 | + .content(); |
| 36 | + } |
| 37 | +} |
| 38 | +``` |
| 39 | + |
| 40 | +In this simple example, the user input sets the contents of the user message. The call method sends a request to the AI model, and the context method returns the AI model's response as a String. |
| 41 | + |
| 42 | + |
| 43 | +== Returing a `ChatResponse` |
| 44 | + |
| 45 | +The response from the AI model is a rich structure defined by the type ChatResponse. |
| 46 | +ChatResponse includes metadata about how the response was generated and can also contain multiple responses, known as generations, each with its own metadata. |
| 47 | +The metadata includes the number of tokens (each token is approximately 3/4 of a word) used to create the response. This information is important because hosted AI models charge based on the number of tokens used per request. |
| 48 | + |
| 49 | +An example to return the `ChatResponse` object that contains the metadata is shown below by invoking `chatResponse()` after the `call()` method. |
| 50 | + |
| 51 | + |
| 52 | +```java |
| 53 | +ChatResponse chatResponse = this.chatClient.prompt() |
| 54 | + .user("Tell me a joke") |
| 55 | + .call() |
| 56 | + .chatResponse(); |
| 57 | +``` |
| 58 | + |
| 59 | +== Returning an Entity |
| 60 | + |
| 61 | +You often want to return an entity class that is mapped from the returned String. The `entity` method provides this functionality. |
| 62 | + |
| 63 | +For example, given the Java record: |
| 64 | + |
| 65 | +```java |
| 66 | +record ActorFilms(String actor, List<String> movies) { |
| 67 | +} |
| 68 | +``` |
| 69 | + |
| 70 | +You can easily map the AI model's output to this record using the `entity` method, as shown below: |
| 71 | + |
| 72 | +```java |
| 73 | +ActorFilms actorFilms = chatClient.prompt() |
| 74 | + .user("Generate the filmography for a random actor.") |
| 75 | + .call() |
| 76 | + .entity(ActorFilms.class); |
| 77 | +``` |
| 78 | + |
| 79 | +There is also an overloaded `entity` method with the signature `entity(ParameterizedTypeReference<T> type)` that lets you specify types such as generic Lists. |
| 80 | + |
| 81 | +== Streaming Responses |
| 82 | + |
| 83 | +The `stream` lets you get an asynchronous response as shown below |
| 84 | +```java |
| 85 | +Flux<String> output = this.chatClient.prompt() |
| 86 | + .user("Tell me a joke") |
| 87 | + .stream() |
| 88 | + .content(); |
| 89 | +``` |
| 90 | + |
| 91 | +You can also stream the `ChatResponse` using the method `Flux<ChatResponse> chatResponse()`. |
| 92 | + |
| 93 | +== Using defaults and parameters |
| 94 | + |
| 95 | +It is often useful to create a `ChatClient` with default user and/or system text defined at design time. |
| 96 | +By design time, we mean creating it at application startup in a `@Configuration` class. |
| 97 | +You can then replace the user or system parameters in the main runtime code. |
| 98 | + |
| 99 | +Here is a simple example which defines the system text without using any parameters. |
| 100 | +The system text sets the context, instructions, and overall desired behavior for the model's response |
| 101 | + |
| 102 | +== Passing in user and system parameters |
0 commit comments