Make Tool Output dict conversion stricter to improve backward compatibility

[ihower]

Resolves #1930

Background

Before v0.4.0, all function tool outputs were converted to string.
Starting from v0.4.0 #1898, function tool outputs can now return a dict and the SDK attempts to convert into one of the ValidToolOutputPydanticModels = Union[ToolOutputText, ToolOutputImage, ToolOutputFileContent].

Problem

However, the current dict conversation is too permissive: returning any dict or list can trigger breaking behavior such as #1930.

For example, the following function output has no intention of returning an image:

{ "msg": "foobar" }

but it gets converted into:

{ "type": "input_image", "image_url": None, "file_id": None }

Similarly, this list:

[ { "product_id": 1 }, { "product_id": 2 } ]

is also converted into:

[ { "type": "input_image", "image_url": None, "file_id": None }, { "type": "input_image", "image_url": None, "file_id": None } ]

All of these cases cause API error:

Error code: 400 - {'error': {'message': "Missing mutually exclusive parameters: 'input[2].output[0]'. Ensure you are providing exactly one of: 'file_id' or 'image_url'.", 'type': 'invalid_request_error', 'param': 'input[2].output[0]', 'code': 'missing_mutually_exclusive_parameters'}} 

Solution

This PR refines the dict conversion logic to make _maybe_get_output_as_structured_function_output more accurate and stricter.

A dict will now only be converted into a structured output if both conditions are met:

1. The dict must include a valid type field

2. The dict must also contain the required content fields:

   • ToolOutputText: requires text
   • ToolOutputImage: requires at least one of image_url or file_id
   • ToolOutputFileContent: requires at least one of file_data, file_url, or file_id

If a dict does not meet these criteria, it will no longer be implicitly converted.
Instead, it will fall back to the previous behavior and be converted to a string (str()), consistent with pre-v0.4.0 behavior.

This aligns with the intended logic described in the original code comments.

Additionally, comprehensive unit tests have been added to verify which dict combinations should and should not be automatically converted.

Benefits

• Significantly improves backward compatibility and greatly reduces API errors caused by overly broad dict conversion.
• Only valid function output dict (with both type and required fields) will be converted automatically.
• For all other cases, behavior remains consistent with previous versions: the output will simply be converted by str().

[seratch]

Thanks for suggesting this! I will look into this tomorrow.

[seratch]

Thanks for fixing this (as always)!