feat: Load the SchedulerConfig from a configuration file/text and make it easier to add plugins

[shmuelk]

This PR adds the ability to:

1. Define the SchedulerConfig to be used using yaml text either in-line as an argument or from a file
2. Add additional plugins with the need to modify existing code.

This PR implements issue #839.

This PR adds a configuration of the form:

plugin_definitions:
- name: lowQueue
  plugin_name: low-queue
  parameters:
    threshold: 10
profile_picker:
  plugin:
    plugin_name: all-profiles
scheduler_profiles:
- name: default
  plugins:
  - reference: lowQueue
  - plugin:
      plugin_name: prefix-cache
      parameters:
        hash-block-size: 32
    weight: 50
  - plugin:
      plugin_name: max-score

The plugin_definitions section enables a plugin instance to be used in multiple SchedulerProfiles, its use is optional. It is an array and the fields of each element are:

• name: provides a name by which the plugin instance can be referenced in the profile_picker and scheduler_profile sections.
• plugin_name: names the plugin to instantiate.
• parameters: passed unparsed to the named plugin's factory function when creating the instance.

The profile_picker section defines the ProfilePicker plugin to be used. Its fields are:

• reference: names a plugin instance by the name it was given in the plugin_definitions section.
• plugin: defines, in-line a plugin to instantiated. The plugin field has in turn the following sub-fields:
  • plugin_name: names the plugin to instantiate.
  • parameters: passed unparsed to the named plugin's factory function when creating the instance.
    Either the reference field or the plugin field should be specified.

The scheduler_profiles section defines the set of SchedulerProfiles that will be in the created SchedulerConfig. It is and array and each element has the following fields:

• name: provides a name for the SchedulerProfile
• plugins: The array of Plugins to be associated with this SchedulerProfile. Each element in this array has the following fields:
  • reference: names a plugin instance by the name it was given in the plugin_definitions section.
  • plugin: defines, in-line a plugin to instantiated. The plugin field has in turn the following sub-fields:
    • plugin_name: names the plugin to instantiate.
    • parameters: passed unparsed to the named plugin's factory function when creating the instance.
  • weight: the weight to be used if the plugin is a Scorer
    Either the reference field or the plugin field should be specified.

To enable the instantiation of plugins from the configuration file/text a singleton/static plugin registry was added. Each plugin was modified to add an init() function and a factory function. The init() function simply registers the plugin's factory function in the registry.

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	8582491
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6837161eb97f3d0008cc74da
😎 Deploy Preview	https://deploy-preview-881--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: shmuelk
Once this PR has been reviewed and has the lgtm label, please assign arangogutierrez for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @shmuelk. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ahg-g]

/ok-to-test

[k8s-ci-robot]

@shmuelk: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gateway-api-inference-extension-verify-main	8582491	link	true	/test pull-gateway-api-inference-extension-verify-main
pull-gateway-api-inference-extension-test-unit-main	8582491	link	true	/test pull-gateway-api-inference-extension-test-unit-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[ahg-g]

I recommend to treat this similar to CRDs so that we version it, and also place this under https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/api/config

See examples:

• https://github.com/kubernetes-sigs/lws/tree/main/api/config/v1alpha1
• https://github.com/kubernetes-sigs/jobset/tree/main/api/config/v1alpha1

[ahg-g]

Also, this need to be a generic config api for epp, we need to encapsulate the plugins configuration in its own struct so we can accommodate other parameters (most flags should move here basically)

[ahg-g]

Defining it similar to a CRD would allow us to use the decoder pkg from k8s, see this example: https://github.com/kubernetes-sigs/jobset/blob/a7a669590b3483715c193e0af53dbb5f4f066b03/pkg/config/config.go#L19

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.