Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: BenjaminBraunDev/gateway-api-inference-extension
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: tomatillo-and-multiverse/gateway-api-inference-extension
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 18 commits
  • 60 files changed
  • 2 contributors

Commits on Aug 29, 2025

  1. add latency predictor

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    6cd8444 View commit details
    Browse the repository at this point in the history

  2. add cv in model and update epp deployment

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    f4e5a72 View commit details
    Browse the repository at this point in the history

  3. bug fix

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    99b3788 View commit details
    Browse the repository at this point in the history

  4. track mape for predictions

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    7035855 View commit details
    Browse the repository at this point in the history

  5. add running queue size to metrics

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    9bbcb9f View commit details
    Browse the repository at this point in the history

  6. add xgboost regressor and update tpot sampling logic

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    1a492b6 View commit details
    Browse the repository at this point in the history

  7. emit predicted and actual ttft tpot in body

    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    fb9ab65 View commit details
    Browse the repository at this point in the history

  8. seperate servers for training and prediction 

    Add APIs for the instantiated plugins to the EPP Handle (kubernetes-sigs#1039) * Added plugin instance APIs to plugins.Handle Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * An implementation of the new plugins.Handle APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Moved all configuration loading code to new package Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Updates due to new and moved APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Cleanup of old configuration loading code Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> chore(deps): bump the kubernetes group with 6 updates (kubernetes-sigs#1050) Bumps the kubernetes group with 6 updates: | Package | From | To | | --- | --- | --- | | [k8s.io/api](https://github.com/kubernetes/api) | `0.33.1` | `0.33.2` | | [k8s.io/apiextensions-apiserver](https://github.com/kubernetes/apiextensions-apiserver) | `0.33.1` | `0.33.2` | | [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) | `0.33.1` | `0.33.2` | | [k8s.io/client-go](https://github.com/kubernetes/client-go) | `0.33.1` | `0.33.2` | | [k8s.io/code-generator](https://github.com/kubernetes/code-generator) | `0.33.1` | `0.33.2` | | [k8s.io/component-base](https://github.com/kubernetes/component-base) | `0.33.1` | `0.33.2` | Updates `k8s.io/api` from 0.33.1 to 0.33.2 - [Commits](kubernetes/api@v0.33.1...v0.33.2) Updates `k8s.io/apiextensions-apiserver` from 0.33.1 to 0.33.2 - [Release notes](https://github.com/kubernetes/apiextensions-apiserver/releases) - [Commits](kubernetes/apiextensions-apiserver@v0.33.1...v0.33.2) Updates `k8s.io/apimachinery` from 0.33.1 to 0.33.2 - [Commits](kubernetes/apimachinery@v0.33.1...v0.33.2) Updates `k8s.io/client-go` from 0.33.1 to 0.33.2 - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.33.1...v0.33.2) Updates `k8s.io/code-generator` from 0.33.1 to 0.33.2 - [Commits](kubernetes/code-generator@v0.33.1...v0.33.2) Updates `k8s.io/component-base` from 0.33.1 to 0.33.2 - [Commits](kubernetes/component-base@v0.33.1...v0.33.2) --- updated-dependencies: - dependency-name: k8s.io/api dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/apiextensions-apiserver dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/apimachinery dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/client-go dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/code-generator dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes - dependency-name: k8s.io/component-base dependency-version: 0.33.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kubernetes ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> remove datastore dependency from the scheduler (kubernetes-sigs#1049) * remove datastore dependency from the scheduler Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * added back comments on snapshotting pods from datastore before calling schedule Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * removed fake datastore from conformance scheduler test Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> Add subsetting logic for epp (kubernetes-sigs#981) feat: Added a factory function for the DecisionTree filter (kubernetes-sigs#1053) * Added a factory function for the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Added tests of the factory function of the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Registered the factory function of the DecisionTreeFilter Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Refactored the configuration loading Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> Adding pprof endpoints to metrics port (kubernetes-sigs#1069) feat: Add a context.Context to the plugins.HAndle interface (kubernetes-sigs#1076) * Added a context.Context to the plugins.Handle interface Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Changes due to changes in internal APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> * Changes to tests due to changes in internal APIs Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> --------- Signed-off-by: Shmuel Kallner <kallner@il.ibm.com> convert subset filter from a plugin to logic in director (kubernetes-sigs#1088) * convert subset filter from a plugin to logic in director Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * replace interface{} with any Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * make linter happy Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * address code review comments Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> chore(deps): bump golang.org/x/sync from 0.14.0 to 0.15.0 (kubernetes-sigs#1096) Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.14.0 to 0.15.0. - [Commits](golang/sync@v0.14.0...v0.15.0) --- updated-dependencies: - dependency-name: golang.org/x/sync dependency-version: 0.15.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Introduce plugins.TypedName to be used for Plugin base implementation (kubernetes-sigs#1086) * introduce TypedName to reduce boilerplate, modify plugins Signed-off-by: Etai Lev Ran <elevran@gmail.com> * implement GetTypedName() Signed-off-by: Etai Lev Ran <elevran@gmail.com> * Remove Type() and Name() from Plugin interface Signed-off-by: Etai Lev Ran <elevran@gmail.com> * use TypedName as private field, not embedded Signed-off-by: Etai Lev Ran <elevran@gmail.com> --------- Signed-off-by: Etai Lev Ran <elevran@gmail.com> move the conversion from pod metrics to scheduler pod representation one level up (kubernetes-sigs#1104) * move the converstion from pod metrics to scheduler pod representation one level up Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * minor change in helper func Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> handle picking multiple destinations in scheduling layer (kubernetes-sigs#1059) * implement multiple destination as the output of the scheduler Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * updated max score picker unit tests to cover multiple pods Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * imports Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> * unit-test fix Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> --------- Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com> refactor: 🔨 use the more explicit singular form (kubernetes-sigs#1129)
    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    d371f26 View commit details
    Browse the repository at this point in the history

  9. add latency predictor 

    put the predictor functions in director in a helper function add scores to reqcxt record prediction duration metrics add prefix cache score to model input slo based routing changes retreive request priority queue from the datastore update scoring logic
    @kaushikmitr @BenjaminBraunDev
    kaushikmitr authored and BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    60de70d View commit details
    Browse the repository at this point in the history

  10. better inital implemenation 

    Add scheduling profile, working state remove latencypredictor from director Move all latency prediction logic out of director and into scheduling profile. Make all Request/Response plugins take in RequestContext
    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    6583c6a View commit details
    Browse the repository at this point in the history

  11. progress towards fixing up merge conflicts from latency predictor merge

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    14fec2d View commit details
    Browse the repository at this point in the history

  12. More refactor progress, fixing and adding tests

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    84ad32b View commit details
    Browse the repository at this point in the history

  13. working state, latency prediction

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    56ba0d7 View commit details
    Browse the repository at this point in the history

  14. Clean up changes, remove unneeded files, working functionality withou… 

    …t latency flag and scheduling plugins
    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    3d3a39b View commit details
    Browse the repository at this point in the history

  15. Rebase cleanup, remove duplicate lines

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    5c0159e View commit details
    Browse the repository at this point in the history

  16. Integrate new alpha-beta slo scoring into scoring plugin

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    50e2aaa View commit details
    Browse the repository at this point in the history

  17. Fix prefix cache scoring for slo-aware routing

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    8b15b2c View commit details
    Browse the repository at this point in the history

  18. Add pycache or latency predictor to gitignore

    @BenjaminBraunDev
    BenjaminBraunDev committed Aug 29, 2025
    Configuration menu
    Copy the full SHA
    68d397f View commit details
    Browse the repository at this point in the history
Loading