- Notifications
You must be signed in to change notification settings - Fork 1.5k
Open
Labels
bugSomething isn't workingSomething isn't working
Milestone
Description
Initial Checks
- I confirm that I'm using the latest version of Pydantic AI
- I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue
Description
Bug Description
When using AWS Bedrock Claude Sonnet 4.5 with interleaved thinking enabled, the agent fails when attempting to use tools after the initial call. This appears to be a regression introduced between versions 1.0.1 and 1.0.15.
Error Message
botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use. Configuration
PydanticAI Version: 1.0.15
Model: Claude Sonnet 4.5 via AWS Bedrock
Provider: bedrock
Model Settings:
{ "max_tokens": 63000, "temperature": 1.0, "timeout": 300, "parallel_tool_calls": true, "bedrock_additional_model_requests_fields": { "anthropic_beta": [ "token-efficient-tools-2025-02-19", "interleaved-thinking-2025-05-14" ], "thinking": { "type": "enabled", "budget_tokens": 4096 } } }Steps to Reproduce
- Configure agent with AWS Bedrock Sonnet 4.5 and thinking enabled (settings above)
- Call agent with structured output:
agent.iter(prompt, output_type=MyOutputType) - Agent responds to first call with the error
Stack Trace
File ".venv/lib/python3.12/site-packages/pydantic_ai/models/instrumented.py", line 381, in request_stream async with self.wrapped.request_stream( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/anaconda3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/pydantic_ai/models/bedrock.py", line 289, in request_stream response = await self._messages_create(messages, True, settings, model_request_parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/pydantic_ai/models/bedrock.py", line 412, in _messages_create model_response = await anyio.to_thread.run_sync(functools.partial(self.client.converse_stream, **params)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/client.py", line 602, in _api_call return self._make_api_call(operation_name, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/context.py", line 123, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/client.py", line 1078, in _make_api_call raise error_class(parsed_response, operation_name) botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use.Expected Behavior
The agent should successfully use tools even when thinking is enabled, as it did in version 1.0.1.
Additional Context
- Regression: This worked correctly in PydanticAI version 1.0.1
- Consistent: Error occurs on every attempt with these settings
- Timing: Fails specifically when tools are required after the initial response
The error suggests that when PydanticAI forces tool usage (likely due to output_type requiring structured output or tool selection), it conflicts with Bedrock's thinking feature. This may be related to how tool_choice is set in the Bedrock API call when thinking is enabled.
Example Code
import asyncio from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModelSettings from pydantic_ai.models.bedrock import BedrockConverseModel # Define a simple tool def get_current_time() -> str: """Get the current time.""" from datetime import datetime return datetime.now().isoformat() # Define a simple structured output type for this example class SimpleOutput(BaseModel): """Simple structured output.""" summary: str timestamp: str # Model settings that trigger the bug model_settings = { "max_tokens": 63000, "temperature": 1.0, "timeout": 300, "parallel_tool_calls": True, "bedrock_additional_model_requests_fields": { "anthropic_beta": [ "token-efficient-tools-2025-02-19", "interleaved-thinking-2025-05-14" ], "thinking": { "type": "enabled", "budget_tokens": 4096 } } } # Create the Bedrock model model = BedrockConverseModel( model_name="us.anthropic.claude-sonnet-4-5-20250929-v1:0", settings=AnthropicModelSettings(**model_settings) ) # Create agent with tool agent = Agent( model=model, tools=[get_current_time], system_prompt="You are a helpful assistant. Use the get_current_time tool to provide accurate timestamps." ) async def main(): """Run the agent and trigger the error.""" # This prompt should trigger tool usage prompt = "What is the current time?" nodes = [] # Using iter() with output_type triggers the bug when tools are needed async with agent.iter(prompt, output_type=SimpleOutput) as agent_run: async for node in agent_run: nodes.append(node) result = agent_run.result.output print("\n\nResult:", result) if __name__ == "__main__": asyncio.run(main())Python, Pydantic AI & LLM client version
- **Python:** 3.12.7 - **Pydantic:** 2.11.7 - **PydanticAI:** 1.0.15 - **boto3:** 1.40.38 - **botocore:** 1.40.38 - **anthropic:** 0.69.0 Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working