Skip to content

VinF Hybrid Inference: consolidate onDeviceParams initialization #8969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Apr 24, 2025
Merged

VinF Hybrid Inference: consolidate onDeviceParams initialization #8969

merged 4 commits into from
Apr 24, 2025

Conversation

erikeldridge
Copy link

Problem statements:

  1. we were initializing LanguageModelCreateOptions repeatedly, which made it hard to reason about and maintain
  2. the VinF SDK allows some params, such as systemParam, to be set during construction and inference. We were partially implementing this pattern, which made the code confusing.

So, this change:

  1. defines a single default value for onDeviceParams and initializes onDeviceParams.expectedInput in a single place
  2. removes support for passing options via the request. We can add this as a standalone feature later if there's demand.

@erikeldridge erikeldridge requested a review from a team as a code owner April 23, 2025 23:50
@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Apr 23, 2025 •
edited
Loading

⚠️ No Changeset found

Latest commit: 2e83dca

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@erikeldridge erikeldridge mentioned this pull request Apr 23, 2025
Merged
@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 23, 2025 •
edited
Loading

Size Report 1

Affected Products

  • @firebase/vertexai

    TypeBase (a250559)Merge (12fbbdd)Diff
    browser46.0 kB46.0 kB-3 B (-0.0%)
    main47.1 kB47.1 kB-3 B (-0.0%)
    module46.0 kB46.0 kB-3 B (-0.0%)

  • firebase

    TypeBase (a250559)Merge (12fbbdd)Diff
    firebase-vertexai.js36.1 kB36.1 kB+17 B (+0.0%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/EiJxQ6XuFt.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 24, 2025 •
edited
Loading

Size Analysis Report 1

Affected Products

  • @firebase/vertexai

    • getGenerativeModel

      Size

      TypeBase (a250559)Merge (12fbbdd)Diff
      size26.1 kB26.2 kB+50 B (+0.2%)
      size-with-ext-deps45.1 kB45.2 kB+50 B (+0.1%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/6FGQjYaoEu.html

Base automatically changed from erikeldridge-vertex-backwardscompat to vaihi-exp-google-ai April 24, 2025 00:39
@erikeldridge erikeldridge requested a review from a team as a code owner April 24, 2025 00:39
@erikeldridge erikeldridge changed the base branch from vaihi-exp-google-ai to erikeldridge-vertex-backwardscompat April 24, 2025 00:41
gsiddh
gsiddh reviewed Apr 24, 2025
View reviewed changes
@erikeldridge erikeldridge changed the base branch from erikeldridge-vertex-backwardscompat to vaihi-exp-google-ai April 24, 2025 17:58
@erikeldridge erikeldridge merged commit fafa6a8 into vaihi-exp-google-ai Apr 24, 2025
32 of 34 checks passed
@erikeldridge erikeldridge deleted the erikeldridge-vertex-deviceparams branch April 24, 2025 18:15
gsiddh pushed a commit that referenced this pull request May 6, 2025
@pashanka @gsiddh
Complete Hybrid inference impl
5ddf24c 
Fix languageCode parameter in action_code_url (#8912)

* Fix languageCode parameter in action_code_url

* Add changeset

Vaihi add langmodel types. (#8927)

* Adding LanguageModel types. These are based off https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#full-api-surface-in-web-idl

* Adding LanguageModel types.

* Remove bunch of exports

* yarn formatted

* after lint

Define HybridParams (#8935)

Co-authored-by: Erik Eldridge <[email protected]>

Adding smoke test for new hybrid params (#8937)

* Adding smoke test for new hybrid params

* Use the existing name of the model params input

---------

Co-authored-by: Erik Eldridge <[email protected]>

Moving to in-cloud naming (#8938)

Co-authored-by: Erik Eldridge <[email protected]>

Moving to string type for the inference mode (#8941)

Define ChromeAdapter class (#8942)

Co-authored-by: Erik Eldridge <[email protected]>

VinF Hybrid Inference: Implement ChromeAdapter (rebased) (#8943)

Adding count token impl (#8950)

VinF Hybrid Inference #4: ChromeAdapter in stream methods (rebased) (#8949)

Define values for Availability enum (#8951)

VinF Hybrid Inference: narrow Chrome input type (#8953)

Add image inference support (#8954)

* Adding image based input for inference

* adding image as input to create language model object

disable count tokens api for on-device inference (#8962)

VinF Hybrid Inference: throw if only_on_device and model is unavailable (#8965)

VinF Hybrid Inference: update docs (#8970)

VinF Hybrid Inference: consolidate onDeviceParams initialization (#8969)

VinF Hybrid Inference: disable multi-turn support (#8973)

VinF Hybrid Inference: remove default expected input types (#8974)

VinF Hybrid Inference: set image (and text) as default input type (#8984)

VinF Hybrid Inference: log debug messages in conditional logic (#8992)
gsiddh pushed a commit that referenced this pull request May 13, 2025
@pashanka @gsiddh
Complete Hybrid inference impl
8a2dae2 
---------

Co-authored-by: Erik Eldridge <[email protected]>

Moving to in-cloud naming (#8938)

Co-authored-by: Erik Eldridge <[email protected]>

Moving to string type for the inference mode (#8941)

Define ChromeAdapter class (#8942)

Co-authored-by: Erik Eldridge <[email protected]>

VinF Hybrid Inference: Implement ChromeAdapter (rebased) (#8943)

Adding count token impl (#8950)

VinF Hybrid Inference #4: ChromeAdapter in stream methods (rebased) (#8949)

Define values for Availability enum (#8951)

VinF Hybrid Inference: narrow Chrome input type (#8953)

Add image inference support (#8954)

* Adding image based input for inference

* adding image as input to create language model object

disable count tokens api for on-device inference (#8962)

VinF Hybrid Inference: throw if only_on_device and model is unavailable (#8965)

VinF Hybrid Inference: update docs (#8970)

VinF Hybrid Inference: consolidate onDeviceParams initialization (#8969)

VinF Hybrid Inference: disable multi-turn support (#8973)

VinF Hybrid Inference: remove default expected input types (#8974)

VinF Hybrid Inference: set image (and text) as default input type (#8984)

VinF Hybrid Inference: log debug messages in conditional logic (#8992)

VinF Hybrid Inference: support structured output (#9009)
@firebase firebase locked and limited conversation to collaborators May 25, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@gsiddh gsiddh gsiddh left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@erikeldridge @google-oss-bot @gsiddh