Code

Add scorers support in scheduler#1

Code

Merged

mayabar merged 10 commits intoneuralmagic:devneuralmagic/gateway-api-inference-extension:devfrom

mayabar:mainmayabar/gateway-api-inference-extension:mainCopy head branch name to clipboard

Apr 17, 2025

Commits

Commits on Apr 10, 2025

• Add initial support for scorers, used as part of decision which pod will be the target for a request. Session affinity scorer added

  mayabar

  committed
  0c95c2a

  Copy full SHA for 0c95c2a

Commits on Apr 14, 2025

• Fixes in scores infrastructure & session aware scorer

  mayabar

  committed
  bde57da

  Copy full SHA for bde57da

• - Add cleanup for session->pod map

  Show description for 8f9785e
  mayabar

  committed
  8f9785e

  Copy full SHA for 8f9785e

• Export score and pod for external implementations

  mayabar

  committed
  52af66f

  Copy full SHA for 52af66f

Commits on Apr 15, 2025

• Rename session id header

  mayabar

  committed
  90e23bc

  Copy full SHA for 90e23bc

• Separate code of Scorer interface and scorer implementations + add scorerManager

  mayabar

  committed
  c946acc

  Copy full SHA for c946acc

• Remove vllmRequest from scoreTargets API since it exists in the context

  mayabar

  committed
  137ca09

  Copy full SHA for 137ca09

• Support negative score weights

  mayabar

  committed
  c42f72a

  Copy full SHA for c42f72a

• Fix fakeDataStore to be compatible with DataStore intereface

  mayabar

  committed
  a05a573

  Copy full SHA for a05a573

Commits on Apr 16, 2025

• - Check for nils in list of available pods in main scoring function of ScoreMng

  Show description for aca8e07
  mayabar

  committed
  aca8e07

  Copy full SHA for aca8e07