- Notifications
You must be signed in to change notification settings - Fork 1.9k
Feat: add gpt 5 models and verbosity param #4086
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@sobychacko @markpollack updating the latest changes from 1a3ed95 Gpt-5 models: https://platform.openai.com/docs/models |
* image inputs, and produces text outputs. | ||
* <p> | ||
* Note: GPT-5 models require temperature=1.0 (default value). Custom temperature | ||
* values are not supported and will cause errors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this not the case anymore? In that case, we need to update the docs changed in this commit: https://github.com/spring-projects/spring-ai/pull/4068/files
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank youi for pointing out this one @sobychacko . it is the case for all gpt 5 models but not the gpt-5-chat model . https://community.openai.com/t/temperature-in-gpt-5-models/1337133/25
I have update documentation and added tests as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it mean that the gpt 5 models default to a value of temperature 1.0
? If so, I think we should make that clear in the docs or javadocs. From what I hear from you, the GPT 5 models completely ignore temperature and internally default to a value of 1.0
. Is that correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that is correct. Azure mentions also more params that are not supported, but I am am not sure if that info has to be on spring ai documentation as it is not a limitation of the framework but from the LLM model itself.
The following are currently unsupported with reasoning models:
temperature, top_p, presence_penalty, frequency_penalty, logprobs, top_logprobs, logit_bias, max_tokens
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GPT 5 models completely ignore temperature and internally default to a value of 1.0
I just wanted to point out that it is not technically correct. In my experiments when the temperature is set to any value other than 1.0
the OpenAI API actually throws an error. I.e. it does not ignore the temperature
parameter, instead it expects/forces it to always be 1.0
.
This is important difference because e.g. with the default auto-configuration the temperature
is 0.7
(see OpenAiChatProperties
). So, if the model is set to gpt-5-*
and the temperature
is not explicitly set to 1.0
, the default 0.7
is used in the API request and the ChatClient
throws an exception.
I'm not sure how this problem should be solved on the library level. Automatically set the temperature
to 1.0
when model is set to one of well-know model names? Probably this approach would be acceptable on OpenAiChatProperties
level, when it creates the options
object and sets the default temperature
.. It could set it to 1.0
(or skip setting it if it works too) if the model is one of well-known models that does not support custom temperature
.
- update gpt-5 tests - add verbosity parameter - udpate documentation and add tests Signed-off-by: Alexandros Pappas <apappascs@gmail.com>
37a5830
to 826175b
Compare - update gpt-5 tests - add verbosity parameter - udpate documentation and add tests Fixes spring-projects#4086 Auto-cherry-pick to 1.0.x Signed-off-by: Alexandros Pappas <apappascs@gmail.com>
- update gpt-5 tests - add verbosity parameter - udpate documentation and add tests Fixes spring-projects#4086 Auto-cherry-pick to 1.0.x Signed-off-by: Alexandros Pappas <apappascs@gmail.com>
Add gpt-5 models and update their tests