Skip to content

VinF Hybrid Inference: consolidate onDeviceParams initialization #8969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 24, 2025

Conversation

erikeldridge
Copy link

Problem statements:

  1. we were initializing LanguageModelCreateOptions repeatedly, which made it hard to reason about and maintain
  2. the VinF SDK allows some params, such as systemParam, to be set during construction and inference. We were partially implementing this pattern, which made the code confusing.

So, this change:

  1. defines a single default value for onDeviceParams and initializes onDeviceParams.expectedInput in a single place
  2. removes support for passing options via the request. We can add this as a standalone feature later if there's demand.

@erikeldridge erikeldridge requested a review from a team as a code owner April 23, 2025 23:50
Copy link

changeset-bot bot commented Apr 23, 2025

⚠️ No Changeset found

Latest commit: 2e83dca

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 23, 2025

Size Report 1

Affected Products

  • @firebase/vertexai

    TypeBase (a250559)Merge (12fbbdd)Diff
    browser46.0 kB46.0 kB-3 B (-0.0%)
    main47.1 kB47.1 kB-3 B (-0.0%)
    module46.0 kB46.0 kB-3 B (-0.0%)
  • firebase

    TypeBase (a250559)Merge (12fbbdd)Diff
    firebase-vertexai.js36.1 kB36.1 kB+17 B (+0.0%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/EiJxQ6XuFt.html

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Apr 24, 2025

Size Analysis Report 1

Affected Products

  • @firebase/vertexai

    • getGenerativeModel

      Size

      TypeBase (a250559)Merge (12fbbdd)Diff
      size26.1 kB26.2 kB+50 B (+0.2%)
      size-with-ext-deps45.1 kB45.2 kB+50 B (+0.1%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/6FGQjYaoEu.html

Base automatically changed from erikeldridge-vertex-backwardscompat to vaihi-exp-google-ai April 24, 2025 00:39
@erikeldridge erikeldridge requested a review from a team as a code owner April 24, 2025 00:39
@erikeldridge erikeldridge changed the base branch from vaihi-exp-google-ai to erikeldridge-vertex-backwardscompat April 24, 2025 00:41
@erikeldridge erikeldridge changed the base branch from erikeldridge-vertex-backwardscompat to vaihi-exp-google-ai April 24, 2025 17:58
@erikeldridge erikeldridge merged commit fafa6a8 into vaihi-exp-google-ai Apr 24, 2025
32 of 34 checks passed
@erikeldridge erikeldridge deleted the erikeldridge-vertex-deviceparams branch April 24, 2025 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants