Skip to content

Conversation

@sgoedecke
Copy link
Contributor

@sgoedecke sgoedecke commented Sep 16, 2024

  • Use the updated models metadata API, which changes the shape of the models object
    • As a result of this, we've got to collect some extra data from the index endpoint response before fetching schema/details. No big deal, since it's already in-memory
  • Include the basic metadata about o1 models in the list
  • Make more resilient to typoed or missing model names by telling the model to just pick the closest one (e.g. so "Phi 3 mini" becomes "Phi-3-mini-128k-instruct")
  • Fixes an issue where we were passing params that were unsupported for some models, so executeModel was failing for models like Mistral-Nemo
@sgoedecke sgoedecke requested a review from JasonEtco September 16, 2024 03:51
Copy link
Contributor

@itsbagpack itsbagpack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One suggestion to ensure that version is a string and not a number

Co-authored-by: Yumin Wong <wongyumin@gmail.com>
@sgoedecke sgoedecke merged commit 601cc27 into main Sep 16, 2024
@sgoedecke sgoedecke deleted the sgoedecke/update-api-and-handle-wrong-model-names branch September 16, 2024 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants