Skip to content

[Bedrock] ValidationException: "Thinking may not be enabled when tool_choice forces tool use" with interleaved thinking on Sonnet 4.5 #3092

@sharon8811

Description

@sharon8811

Initial Checks

Description

Bug Description

When using AWS Bedrock Claude Sonnet 4.5 with interleaved thinking enabled, the agent fails when attempting to use tools after the initial call. This appears to be a regression introduced between versions 1.0.1 and 1.0.15.

Error Message

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use. 

Configuration

PydanticAI Version: 1.0.15
Model: Claude Sonnet 4.5 via AWS Bedrock
Provider: bedrock

Model Settings:

{ "max_tokens": 63000, "temperature": 1.0, "timeout": 300, "parallel_tool_calls": true, "bedrock_additional_model_requests_fields": { "anthropic_beta": [ "token-efficient-tools-2025-02-19", "interleaved-thinking-2025-05-14" ], "thinking": { "type": "enabled", "budget_tokens": 4096 } } }

Steps to Reproduce

  1. Configure agent with AWS Bedrock Sonnet 4.5 and thinking enabled (settings above)
  2. Call agent with structured output: agent.iter(prompt, output_type=MyOutputType)
  3. Agent responds to first call with the error

Stack Trace

File ".venv/lib/python3.12/site-packages/pydantic_ai/models/instrumented.py", line 381, in request_stream async with self.wrapped.request_stream( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/anaconda3/lib/python3.12/contextlib.py", line 210, in __aenter__ return await anext(self.gen) ^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/pydantic_ai/models/bedrock.py", line 289, in request_stream response = await self._messages_create(messages, True, settings, model_request_parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/pydantic_ai/models/bedrock.py", line 412, in _messages_create model_response = await anyio.to_thread.run_sync(functools.partial(self.client.converse_stream, **params)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/client.py", line 602, in _api_call return self._make_api_call(operation_name, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/context.py", line 123, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.12/site-packages/botocore/client.py", line 1078, in _make_api_call raise error_class(parsed_response, operation_name) botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use.

Expected Behavior

The agent should successfully use tools even when thinking is enabled, as it did in version 1.0.1.

Additional Context

  • Regression: This worked correctly in PydanticAI version 1.0.1
  • Consistent: Error occurs on every attempt with these settings
  • Timing: Fails specifically when tools are required after the initial response

The error suggests that when PydanticAI forces tool usage (likely due to output_type requiring structured output or tool selection), it conflicts with Bedrock's thinking feature. This may be related to how tool_choice is set in the Bedrock API call when thinking is enabled.

Example Code

import asyncio from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModelSettings from pydantic_ai.models.bedrock import BedrockConverseModel # Define a simple tool def get_current_time() -> str: """Get the current time.""" from datetime import datetime return datetime.now().isoformat() # Define a simple structured output type for this example class SimpleOutput(BaseModel): """Simple structured output.""" summary: str timestamp: str # Model settings that trigger the bug model_settings = { "max_tokens": 63000, "temperature": 1.0, "timeout": 300, "parallel_tool_calls": True, "bedrock_additional_model_requests_fields": { "anthropic_beta": [ "token-efficient-tools-2025-02-19", "interleaved-thinking-2025-05-14" ], "thinking": { "type": "enabled", "budget_tokens": 4096 } } } # Create the Bedrock model model = BedrockConverseModel( model_name="us.anthropic.claude-sonnet-4-5-20250929-v1:0", settings=AnthropicModelSettings(**model_settings) ) # Create agent with tool agent = Agent( model=model, tools=[get_current_time], system_prompt="You are a helpful assistant. Use the get_current_time tool to provide accurate timestamps." ) async def main(): """Run the agent and trigger the error.""" # This prompt should trigger tool usage prompt = "What is the current time?" nodes = [] # Using iter() with output_type triggers the bug when tools are needed async with agent.iter(prompt, output_type=SimpleOutput) as agent_run: async for node in agent_run: nodes.append(node) result = agent_run.result.output print("\n\nResult:", result) if __name__ == "__main__": asyncio.run(main())

Python, Pydantic AI & LLM client version

- **Python:** 3.12.7 - **Pydantic:** 2.11.7 - **PydanticAI:** 1.0.15 - **boto3:** 1.40.38 - **botocore:** 1.40.38 - **anthropic:** 0.69.0 

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions