Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
1d2a547
Make `end_strategy` also work for output tools, not just tools
Danipulok Nov 22, 2025
b3371e5
Fix tests
Danipulok Nov 22, 2025
c5bfc1b
Fix lints
Danipulok Nov 22, 2025
7856ce7
Add missing coverage
Danipulok Nov 22, 2025
52d6a4c
Update tests to use snapshot
Danipulok Nov 24, 2025
9a27360
Refactor duplicated code
Danipulok Nov 24, 2025
4a1a34e
Fix typecheck errors
Danipulok Nov 25, 2025
3a0fd30
Merge branch 'main' into feat/end-strategy
Danipulok Nov 25, 2025
95d979e
Merge branch 'main' into feat/end-strategy
Danipulok Nov 26, 2025
542a234
feat: Support exhaustive end_strategy for output tools with validation
Danipulok Nov 27, 2025
9841ee6
Fix comments
Danipulok Dec 3, 2025
c6f09e3
Add tests (WIP)
Danipulok Dec 3, 2025
8cdeb27
Move related tests to a `TestMultipleToolCalls` to mirror `test_agent…
Danipulok Dec 3, 2025
7fda323
Use `ErrorDetails` instead of raw dict
Danipulok Dec 3, 2025
4453fdc
Use `ErrorDetails` instead of raw dict
Danipulok Dec 3, 2025
0cb7ddd
Docs: remove unnecessary clarification
Danipulok Dec 3, 2025
1d8727e
Tests: fix `test_early_strategy_does_not_apply_to_tool_calls_without_…
Danipulok Dec 3, 2025
46356a9
Tests: remove unneeded `part_kind`
Danipulok Dec 3, 2025
c61a0de
Merge branch 'main' into feat/end-strategy
Danipulok Dec 3, 2025
5a7de7d
Tests: update comment for `test_early_strategy_with_external_tool_call`
Danipulok Dec 3, 2025
6e99104
Docs: update docs
Danipulok Dec 3, 2025
7cdcb6b
Coverage: trying to make it work
Danipulok Dec 4, 2025
f8d6200
Tests: add tests
Danipulok Dec 4, 2025
e1ea66f
Tests: add note about tests
Danipulok Dec 4, 2025
b5d5ae0
Tests: add note about tests
Danipulok Dec 4, 2025
5acaebb
Merge branch 'main' into feat/end-strategy
Danipulok Dec 9, 2025
0776b75
Tests: fix imports
Danipulok Dec 9, 2025
5323cb5
Update docs/tools-advanced.md
Danipulok Dec 9, 2025
d5af8bc
Update docs/output.md
Danipulok Dec 9, 2025
eb18933
Tests: compare with all messages, and not just the latest part
Danipulok Dec 9, 2025
7b62ffa
Tests: compare with all messages, and not just the latest part
Danipulok Dec 9, 2025
8b9abaf
Tests: make test similar with `test_agent`
Danipulok Dec 9, 2025
9ff103e
Tests: move `OutputType` to a global type
Danipulok Dec 9, 2025
4fed107
Tests: mirror `test_exhaustive_strategy_executes_all_tools`
Danipulok Dec 9, 2025
c6568d5
Tests: mirror `test_exhaustive_strategy_calls_all_output_tools`
Danipulok Dec 9, 2025
b7a641e
Tests: mirror `test_early_strategy_does_not_call_additional_output_to…
Danipulok Dec 9, 2025
07efa5b
Tests: mirror `test_exhaustive_strategy_invalid_first_valid_second_ou…
Danipulok Dec 9, 2025
ba5b212
Tests: mirror `test_exhaustive_strategy_valid_first_invalid_second_ou…
Danipulok Dec 9, 2025
e0b1662
Tests: mirror `test_early_strategy_with_final_result_in_middle`
Danipulok Dec 9, 2025
1b55376
Tests: mirror `test_exhaustive_raises_unexpected_model_behavior`
Danipulok Dec 9, 2025
74be45b
Tests: mirror `test_multiple_final_result_are_validated_correctly`
Danipulok Dec 9, 2025
cd49935
Tests: mirror `test_early_strategy_with_external_tool_call`
Danipulok Dec 9, 2025
29ceecb
Tests: mirror `test_early_strategy_with_deferred_tool_call`
Danipulok Dec 9, 2025
7c88c48
Docs: update docs
Danipulok Dec 9, 2025
636a646
Ruff: fix
Danipulok Dec 9, 2025
fb575d6
Docs: fix note
Danipulok Dec 9, 2025
60613bc
Tests: add `# pragma: no cover`
Danipulok Dec 9, 2025
31b62e6
Tests: replace `mark.xfail` with `mark.skip` to "fix" coverage
Danipulok Dec 9, 2025
ceabae5
Tests: add comments and assert to a tool
Danipulok Dec 9, 2025
817e662
Code: add `raise e`
Danipulok Dec 9, 2025
e100275
Code: remove `raise e`
Danipulok Dec 9, 2025
ea10210
Tests: add `@pytest\.mark\.skip` and `@pytest\.mark\.xfail` to skip c…
Danipulok Dec 9, 2025
aa16a41
Tests: add `test_exhaustive_strategy_with_tool_retry_and_final_result…
Danipulok Dec 9, 2025
d11b820
Docs: add mentioning of `end_strategy` in `Agent`
Danipulok Dec 11, 2025
b2fe5da
Merge branch 'pydantic:main' into feat/end-strategy
Danipulok Dec 11, 2025
0bf5414
Merge branch 'feat/end-strategy' of github.com:Danipulok/pydantic-ai …
Danipulok Dec 11, 2025
c4bf11b
Update docs/output.md
Danipulok Dec 11, 2025
f9c05de
Update docs/output.md
Danipulok Dec 11, 2025
06dd066
Update docs/tools-advanced.md
Danipulok Dec 12, 2025
cc01098
Merge branch 'pydantic:main' into feat/end-strategy
Danipulok Dec 12, 2025
109b11b
Tests: order tests better
Danipulok Dec 12, 2025
081da9a
Docs: improve `agents.md`
Danipulok Dec 12, 2025
aa37dcb
Docs: change `EndStrategy` docs
Danipulok Dec 12, 2025
ac391b3
Docs: change `Output Tool Calls` docs
Danipulok Dec 12, 2025
71691f5
Graph: change logic of handling failed output tools when final result…
Danipulok Dec 12, 2025
09799d4
Tests: add note about changing tests
Danipulok Dec 12, 2025
05af1d7
Tweak docs
DouweM Dec 12, 2025
aae0c1f
Update pydantic_ai_slim/pydantic_ai/agent/__init__.py
Danipulok Dec 12, 2025
a74a767
Graph: change line to `Output tool not used - output failed validation.`
Danipulok Dec 12, 2025
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,14 @@ print(repr(result.output))

1. If we were passing just `Fruit` and `Vehicle` without custom tool names, we could have used a union: `output_type=Fruit | Vehicle`. However, as `ToolOutput` is an object rather than a type, we have to use a list.

!!! note "Handling Multiple Output Tool Calls"
When a model calls multiple output tools in the same response, the agent's `end_strategy` parameter controls whether all output tool functions are executed or only the first one:

- `'early'` (default): Only the first output tool is executed, and additional output tool calls are skipped once a final result is found. This is the default behavior.
- `'exhaustive'`: All output tool functions are executed, even after a final result is found. The first valid output tool's result is used as the final output.

This parameter also applies to [function tools](tools.md), not just output tools.

_(This example is complete, it can be run "as is")_

#### Native Output
Expand Down
11 changes: 11 additions & 0 deletions docs/tools-advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,17 @@ If a tool requires sequential/serial execution, you can pass the [`sequential`][

Async functions are run on the event loop, while sync functions are offloaded to threads. To get the best performance, _always_ use an async function _unless_ you're doing blocking I/O (and there's no way to use a non-blocking library instead) or CPU-bound work (like `numpy` or `scikit-learn` operations), so that simple functions are not offloaded to threads unnecessarily.

### End Strategy

When a model returns multiple tool calls (either function tools or [output tools](output.md)), the agent's `end_strategy` parameter controls whether all tools are executed once a final result is found:

- **`'early'`** (default): Once a final result is found (from an output tool), all remaining tool calls (both function tools and output tools) are skipped.
- **`'exhaustive'`**: All tool calls are executed, even after a final result is found. The first valid output tool's result is used as the final output.

The `'exhaustive'` strategy is useful when tools have side effects (like logging, sending notifications, or updating metrics) that should always execute, regardless of whether a final result has been found.

For more details on how `end_strategy` applies to output tools, see [Handling Multiple Output Tool Calls](output.md#handling-multiple-output-tool-calls).

!!! note "Limiting tool executions"
You can cap tool executions within a run using [`UsageLimits(tool_calls_limit=...)`](agents.md#usage-limits). The counter increments only after a successful tool invocation. Output tools (used for [structured output](output.md)) are not counted in the `tool_calls` metric.

Expand Down
78 changes: 52 additions & 26 deletions pydantic_ai_slim/pydantic_ai/_agent_graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,10 @@
EndStrategy = Literal['early', 'exhaustive']
"""The strategy for handling multiple tool calls when a final result is found.

- `'early'`: Stop processing other tool calls once a final result is found
- `'exhaustive'`: Process all tool calls even after finding a final result
- `'early'`: Stop processing other tool calls (both function tools and output tools) once a final result is found
- `'exhaustive'`: Process all tool calls (both function tools and output tools) even after finding a final result

This applies to both function tools and output tools.
"""
DepsT = TypeVar('DepsT')
OutputT = TypeVar('OutputT')
Expand Down Expand Up @@ -833,35 +835,56 @@ async def process_tool_calls( # noqa: C901

# First, we handle output tool calls
for call in tool_calls_by_kind['output']:
if final_result:
if final_result.tool_call_id == call.tool_call_id:
part = _messages.ToolReturnPart(
tool_name=call.tool_name,
content='Final result processed.',
tool_call_id=call.tool_call_id,
)
else:
yield _messages.FunctionToolCallEvent(call)
part = _messages.ToolReturnPart(
tool_name=call.tool_name,
content='Output tool not used - a final result was already processed.',
tool_call_id=call.tool_call_id,
)
yield _messages.FunctionToolResultEvent(part)

# `final_result` can be passed into `process_tool_calls` from `Agent.run_stream`
# when streaming and there's already a final result
if final_result and final_result.tool_call_id == call.tool_call_id:
part = _messages.ToolReturnPart(
tool_name=call.tool_name,
content='Final result processed.',
tool_call_id=call.tool_call_id,
)
output_parts.append(part)
# Early strategy is chosen and final result is already set
elif ctx.deps.end_strategy == 'early' and final_result:
yield _messages.FunctionToolCallEvent(call)
part = _messages.ToolReturnPart(
tool_name=call.tool_name,
content='Output tool not used - a final result was already processed.',
tool_call_id=call.tool_call_id,
)
yield _messages.FunctionToolResultEvent(part)
output_parts.append(part)
# Early strategy is chosen and final result is not yet set
# Or exhaustive strategy is chosen
else:
try:
result_data = await tool_manager.handle_call(call)
except exceptions.UnexpectedModelBehavior as e:
ctx.state.increment_retries(
ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings
)
raise e # pragma: lax no cover
# If we already have a valid final result, don't fail the entire run
# This allows exhaustive strategy to complete successfully when at least one output tool is valid
if final_result:
# Just add a retry prompt part for the failed output tool
yield _messages.FunctionToolCallEvent(call)
retry_part = _messages.RetryPromptPart(
content=str(e.__cause__ or e),
tool_name=call.tool_name,
tool_call_id=call.tool_call_id,
)
output_parts.append(retry_part)
yield _messages.FunctionToolResultEvent(retry_part)
else:
# No valid result yet, so this is a real failure
ctx.state.increment_retries(
ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings
)
raise e
except ToolRetryError as e:
ctx.state.increment_retries(
ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings
)
# If we already have a valid final result, don't increment retries for invalid output tools
# This allows the run to succeed if at least one output tool returned a valid result
if not final_result:
ctx.state.increment_retries(
ctx.deps.max_result_retries, error=e, model_settings=ctx.deps.model_settings
)
yield _messages.FunctionToolCallEvent(call)
output_parts.append(e.tool_retry)
yield _messages.FunctionToolResultEvent(e.tool_retry)
Expand All @@ -872,7 +895,10 @@ async def process_tool_calls( # noqa: C901
tool_call_id=call.tool_call_id,
)
output_parts.append(part)
final_result = result.FinalResult(result_data, call.tool_name, call.tool_call_id)

# In both `early` and `exhaustive` modes, use the first output tool's result as the final result
if not final_result:
final_result = result.FinalResult(result_data, call.tool_name, call.tool_call_id)

# Then, we handle function tool calls
calls_to_run: list[_messages.ToolCallPart] = []
Expand Down
8 changes: 7 additions & 1 deletion pydantic_ai_slim/pydantic_ai/agent/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,13 @@ class Agent(AbstractAgent[AgentDepsT, OutputDataT]):

_name: str | None
end_strategy: EndStrategy
"""Strategy for handling tool calls when a final result is found."""
"""Strategy for handling tool calls when a final result is found.

- `'early'`: Stop processing other tool calls once a final result is found (default)
- `'exhaustive'`: Process all tool calls even after finding a final result

This applies to both function tools and output tools.
"""

model_settings: ModelSettings | None
"""Optional model request settings to use for this agents's runs, by default.
Expand Down
Loading
Loading