Skip to content

feat: support gemini #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,6 @@ PINECONE_HOST=

# Some examples are expensive to run, so we disable them by default
RUN_EXPENSIVE_EXAMPLES=false

# For using Gemini
GOOGLE_API_KEY=
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ LLM Chain categorizes two main types of models: **Language Models** and **Embedd
Language Models, like GPT, Claude and Llama, as essential centerpiece of LLM applications
and Embeddings Models as supporting models to provide vector representations of text.

Those models are provided by different **platforms**, like OpenAI, Azure, Replicate, and others.
Those models are provided by different **platforms**, like OpenAI, Azure, Google, Replicate, and others.

#### Example Instantiation

Expand All @@ -63,6 +63,7 @@ $embeddings = new Embeddings();
* [OpenAI's GPT](https://platform.openai.com/docs/models/overview) with [OpenAI](https://platform.openai.com/docs/overview) and [Azure](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) as Platform
* [Anthropic's Claude](https://www.anthropic.com/claude) with [Anthropic](https://www.anthropic.com/) as Platform
* [Meta's Llama](https://www.llama.com/) with [Ollama](https://ollama.com/) and [Replicate](https://replicate.com/) as Platform
* [Google's Gemini](https://gemini.google.com/) with [Google](https://ai.google.dev/) as Platform
* [Google's Gemini](https://gemini.google.com/) with [OpenRouter](https://www.openrouter.com/) as Platform
* [DeepSeek's R1](https://www.deepseek.com/) with [OpenRouter](https://www.openrouter.com/) as Platform
* Embeddings Models
Expand Down
28 changes: 28 additions & 0 deletions examples/chat-gemini-google.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<?php

use PhpLlm\LlmChain\Bridge\Google\Gemini;
use PhpLlm\LlmChain\Bridge\Google\PlatformFactory;
use PhpLlm\LlmChain\Chain;
use PhpLlm\LlmChain\Model\Message\Message;
use PhpLlm\LlmChain\Model\Message\MessageBag;
use Symfony\Component\Dotenv\Dotenv;

require_once dirname(__DIR__).'/vendor/autoload.php';
(new Dotenv())->loadEnv(dirname(__DIR__).'/.env');

if (empty($_ENV['GOOGLE_API_KEY'])) {
echo 'Please set the GOOGLE_API_KEY environment variable.'.PHP_EOL;
exit(1);
}

$platform = PlatformFactory::create($_ENV['GOOGLE_API_KEY']);
$llm = new Gemini(Gemini::GEMINI_2_FLASH);

$chain = new Chain($platform, $llm);
$messages = new MessageBag(
Message::forSystem('You are a pirate and you write funny.'),
Message::ofUser('What is the Symfony framework?'),
);
$response = $chain->call($messages);

echo $response->getContent().PHP_EOL;
32 changes: 32 additions & 0 deletions examples/image-describer-binary-gemini.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<?php

use PhpLlm\LlmChain\Bridge\Google\Gemini;
use PhpLlm\LlmChain\Bridge\Google\PlatformFactory;
use PhpLlm\LlmChain\Chain;
use PhpLlm\LlmChain\Model\Message\Content\Image;
use PhpLlm\LlmChain\Model\Message\Message;
use PhpLlm\LlmChain\Model\Message\MessageBag;
use Symfony\Component\Dotenv\Dotenv;

require_once dirname(__DIR__).'/vendor/autoload.php';
(new Dotenv())->loadEnv(dirname(__DIR__).'/.env');

if (empty($_ENV['GOOGLE_API_KEY'])) {
echo 'Please set the GOOGLE_API_KEY environment variable.'.PHP_EOL;
exit(1);
}

$platform = PlatformFactory::create($_ENV['GOOGLE_API_KEY']);
$llm = new Gemini(Gemini::GEMINI_1_5_FLASH);

$chain = new Chain($platform, $llm);
$messages = new MessageBag(
Message::forSystem('You are an image analyzer bot that helps identify the content of images.'),
Message::ofUser(
'Describe the image as a comedian would do it.',
new Image(dirname(__DIR__).'/tests/Fixture/image.jpg'),
),
);
$response = $chain->call($messages);

echo $response->getContent().PHP_EOL;
33 changes: 33 additions & 0 deletions examples/stream-google-gemini.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<?php

use PhpLlm\LlmChain\Bridge\Google\Gemini;
use PhpLlm\LlmChain\Bridge\Google\PlatformFactory;
use PhpLlm\LlmChain\Chain;
use PhpLlm\LlmChain\Model\Message\Message;
use PhpLlm\LlmChain\Model\Message\MessageBag;
use Symfony\Component\Dotenv\Dotenv;

require_once dirname(__DIR__).'/vendor/autoload.php';
(new Dotenv())->loadEnv(dirname(__DIR__).'/.env');

if (empty($_ENV['GOOGLE_API_KEY'])) {
echo 'Please set the GOOGLE_API_KEY environment variable.'.PHP_EOL;
exit(1);
}

$platform = PlatformFactory::create($_ENV['GOOGLE_API_KEY']);
$llm = new Gemini(Gemini::GEMINI_2_FLASH);

$chain = new Chain($platform, $llm);
$messages = new MessageBag(
Message::forSystem('You are a funny clown that entertains people.'),
Message::ofUser('What is the purpose of an ant?'),
);
$response = $chain->call($messages, [
'stream' => true, // enable streaming of response text
]);

foreach ($response->getContent() as $word) {
echo $word;
}
echo PHP_EOL;
60 changes: 60 additions & 0 deletions src/Bridge/Google/Gemini.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
<?php

declare(strict_types=1);

namespace PhpLlm\LlmChain\Bridge\Google;

use PhpLlm\LlmChain\Model\LanguageModel;

final readonly class Gemini implements LanguageModel
{
public const GEMINI_2_FLASH = 'gemini-2.0-flash';
public const GEMINI_2_PRO = 'gemini-2.0-pro-exp-02-05';
public const GEMINI_2_FLASH_LITE = 'gemini-2.0-flash-lite-preview-02-05';
public const GEMINI_2_FLASH_THINKING = 'gemini-2.0-flash-thinking-exp-01-21';
public const GEMINI_1_5_FLASH = 'gemini-1.5-flash';

/**
* @param array<string, mixed> $options The default options for the model usage
*/
public function __construct(
private string $version = self::GEMINI_2_PRO,
private array $options = ['temperature' => 1.0],
) {
}

public function getVersion(): string
{
return $this->version;
}

public function getOptions(): array
{
return $this->options;
}

public function supportsAudioInput(): bool
{
return false; // it does, but implementation here is still open
}

public function supportsImageInput(): bool
{
return true;
}

public function supportsStreaming(): bool
{
return true;
}

public function supportsStructuredOutput(): bool
{
return false; // it does, but implementation here is still open
}

public function supportsToolCalling(): bool
{
return false; // it does, but implementation here is still open
}
}
77 changes: 77 additions & 0 deletions src/Bridge/Google/GooglePromptConverter.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<?php

declare(strict_types=1);

namespace PhpLlm\LlmChain\Bridge\Google;

use PhpLlm\LlmChain\Model\Message\AssistantMessage;
use PhpLlm\LlmChain\Model\Message\Content\Image;
use PhpLlm\LlmChain\Model\Message\Content\Text;
use PhpLlm\LlmChain\Model\Message\MessageBagInterface;
use PhpLlm\LlmChain\Model\Message\MessageInterface;
use PhpLlm\LlmChain\Model\Message\Role;
use PhpLlm\LlmChain\Model\Message\UserMessage;

use function Symfony\Component\String\u;

final class GooglePromptConverter
{
/**
* @return array{
* contents: list<array{
* role: 'model'|'user',
* parts: list<array{inline_data?: array{mime_type: string, data: string}|array{text: string}}>
* }>,
* system_instruction?: array{parts: array{text: string}}
* }
*/
public function convertToPrompt(MessageBagInterface $bag): array
{
$body = ['contents' => []];

$systemMessage = $bag->getSystemMessage();
if (null !== $systemMessage) {
$body['system_instruction'] = [
'parts' => ['text' => $systemMessage->content],
];
}

foreach ($bag->withoutSystemMessage()->getMessages() as $message) {
$body['contents'][] = [
'role' => $message->getRole()->equals(Role::Assistant) ? 'model' : 'user',
'parts' => $this->convertMessage($message),
];
}

return $body;
}

/**
* @return list<array{inline_data?: array{mime_type: string, data: string}|array{text: string}}>
*/
private function convertMessage(MessageInterface $message): array
{
if ($message instanceof AssistantMessage) {
return [['text' => $message->content]];
}

if ($message instanceof UserMessage) {
$parts = [];
foreach ($message->content as $content) {
if ($content instanceof Text) {
$parts[] = ['text' => $content->text];
}
if ($content instanceof Image) {
$parts[] = ['inline_data' => [
'mime_type' => u($content->url)->after('data:')->before(';')->toString(),
'data' => u($content->url)->after('base64,')->toString(),
]];
}
}

return $parts;
}

return [];
}
}
128 changes: 128 additions & 0 deletions src/Bridge/Google/ModelHandler.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
<?php

declare(strict_types=1);

namespace PhpLlm\LlmChain\Bridge\Google;

use PhpLlm\LlmChain\Exception\RuntimeException;
use PhpLlm\LlmChain\Model\Message\MessageBagInterface;
use PhpLlm\LlmChain\Model\Model;
use PhpLlm\LlmChain\Model\Response\ResponseInterface as LlmResponse;
use PhpLlm\LlmChain\Model\Response\StreamResponse;
use PhpLlm\LlmChain\Model\Response\TextResponse;
use PhpLlm\LlmChain\Platform\ModelClient;
use PhpLlm\LlmChain\Platform\ResponseConverter;
use Symfony\Component\HttpClient\EventSourceHttpClient;
use Symfony\Contracts\HttpClient\Exception\ClientExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\DecodingExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\RedirectionExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\ServerExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
use Webmozart\Assert\Assert;

final readonly class ModelHandler implements ModelClient, ResponseConverter
{
private EventSourceHttpClient $httpClient;

public function __construct(
HttpClientInterface $httpClient,
#[\SensitiveParameter] private string $apiKey,
private GooglePromptConverter $promptConverter = new GooglePromptConverter(),
) {
$this->httpClient = $httpClient instanceof EventSourceHttpClient ? $httpClient : new EventSourceHttpClient($httpClient);
}

public function supports(Model $model, array|string|object $input): bool
{
return $model instanceof Gemini && $input instanceof MessageBagInterface;
}

/**
* @throws TransportExceptionInterface
*/
public function request(Model $model, object|array|string $input, array $options = []): ResponseInterface
{
Assert::isInstanceOf($input, MessageBagInterface::class);

$url = sprintf(
'https://generativelanguage.googleapis.com/v1beta/models/%s:%s',
$model->getVersion(),
$options['stream'] ?? false ? 'streamGenerateContent' : 'generateContent',
);

$generationConfig = ['generationConfig' => $options];
unset($generationConfig['generationConfig']['stream']);

return $this->httpClient->request('POST', $url, [
'headers' => [
'x-goog-api-key' => $this->apiKey,
],
'json' => array_merge($generationConfig, $this->promptConverter->convertToPrompt($input)),
]);
}

/**
* @throws TransportExceptionInterface
* @throws ServerExceptionInterface
* @throws RedirectionExceptionInterface
* @throws DecodingExceptionInterface
* @throws ClientExceptionInterface
*/
public function convert(ResponseInterface $response, array $options = []): LlmResponse
{
if ($options['stream'] ?? false) {
return new StreamResponse($this->convertStream($response));
}

$data = $response->toArray();

if (!isset($data['candidates'][0]['content']['parts'][0]['text'])) {
throw new RuntimeException('Response does not contain any content');
}

return new TextResponse($data['candidates'][0]['content']['parts'][0]['text']);
}

private function convertStream(ResponseInterface $response): \Generator
{
foreach ((new EventSourceHttpClient())->stream($response) as $chunk) {
if ($chunk->isFirst() || $chunk->isLast()) {
continue;
}

$jsonDelta = trim($chunk->getContent());

// Remove leading/trailing brackets
if (str_starts_with($jsonDelta, '[') || str_starts_with($jsonDelta, ',')) {
$jsonDelta = substr($jsonDelta, 1);
}
if (str_ends_with($jsonDelta, ']')) {
$jsonDelta = substr($jsonDelta, 0, -1);
}

// Split in case of multiple JSON objects
$deltas = explode(",\r\n", $jsonDelta);

foreach ($deltas as $delta) {
if ('' === $delta) {
continue;
}

try {
$data = json_decode($delta, true, 512, JSON_THROW_ON_ERROR);
} catch (\JsonException $e) {
dump($delta);
throw new RuntimeException('Failed to decode JSON response', 0, $e);
}

if (!isset($data['candidates'][0]['content']['parts'][0]['text'])) {
continue;
}

yield $data['candidates'][0]['content']['parts'][0]['text'];
}
}
}
}
Loading