This repository was archived by the owner on Feb 11, 2025. It is now read-only.
structured outputs #33
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Contributor
langchain4j commented Aug 13, 2024
langchain4j commented - updated list of models
- added "strict" for tools
- redesigned json schema model
- added JSON_SCHEMA response format
- added "parallelToolCalls"
- added "refusal"
- added "required" tool choice mode
- fixed some tests
added 2 commits August 13, 2024 17:06
- updated list of models - added "strict" for tools - redesigned json schema model - added JSON_SCHEMA response format - added "parallelToolCalls" - added "refusal" - added "required" tool choice mode - fixed some tests
7 tasks
langchain4j pushed a commit to langchain4j/langchain4j that referenced this pull request Aug 14, 2024
## Issue Closes #1581 ## Change - OpenAI: added support for [Structured Outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/): - for tools - for json mode - Introduced new (still experimental) `ChatLanguageModel` API (which supports specifying json schema) ### OpenAI Structured Outputs for tools To enable Structured Outputs feature for tools, set `.strictTools(true)` when buidling the model: ```java OpenAiChatModel.builder() ... .strictTools(true) .build(), ``` Please note that this will automatically make all tool parameters mandatory (`required` in json schema) and set `additionalProperties=false` for each `object` in json schema. This is due to the current OpenAI limitations. ### OpenAI Structured Outputs for json mode To enable Structured Outputs feature for json mode, set `.responseFormat("json_schema")` and `.strictJsonSchema(true)` when buidling the model: ```java OpenAiChatModel.builder() ... .responseFormat("json_schema") .strictJsonSchema(true) .build(), ``` In this case `AiServices` will not append "You must answer strictly in the following JSON format: ..." string to the end of the last `UserMessage`, but will create a Json schema from the given POJO and pass it to the LLM. Please note that this works only when method return type is a POJO. If the return type is something else, (like an enum or a `List<String>`), the old behaviour is applied (with "You must answer strictly ..."). All return types will be supported in the near future. Please note that this feature is available now only for `gpt-4o-mini` and `gpt-4o-2024-08-06` models. ### Experimental `ChatLanguageModel` API This was drafted in #1261, but now it has to be rushed a bit in order to enable new Structured Outputs feature for OpenAI. A new method `ChatResponse chat(ChatRequest request)` was added into `ChatLanguageModel` which allows to specify messages, tools and response format (with json schema). In the future it will also support specifying model parameters like temperature. ## Upcoming Changes - Adopt new `ChatLanguageModel` API for Gemini - Adopt new `ChatLanguageModel` API for Azure OpenAI (once available) - Support Structured Outputs with all other method return types like `List<Pojo>` - Adopt new `JsonSchema` type for tools (instead of `ToolParameters`) Reated changes in openai4j: ai-for-java/openai4j#33 ## General checklist - [X] There are no breaking changes - [X] I have added unit and integration tests for my change - [X] I have manually run all the unit and integration tests in the module I have added/changed, and they are all green - [X] I have manually run all the unit and integration tests in the [core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core) and [main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j) modules, and they are all green <!-- Before adding documentation and example(s) (below), please wait until the PR is reviewed and approved. --> - [ ] I have added/updated the [documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs) - [ ] I have added an example in the [examples repo](https://github.com/langchain4j/langchain4j-examples) (only for "big" features) - [ ] I have added/updated [Spring Boot starter(s)](https://github.com/langchain4j/langchain4j-spring) (if applicable)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.