Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

Issue

Following the example in the doc summarize-old-messages, I run into an issue when using also tools with the Agent and OpenAI as provider.

The passed list of messages to the history_processor functions are including also generated messages (both ModelRequest and ModelResponse) during the run. If we follow the example which keeps only the latest message, the tool related Response is included but not the toll related Request which caused the OpenAI API to return HTTP error 400:

ModelHTTPError: status_code: 400, model_name: gpt-4o-mini, body: {'message': "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None}

Discussion

In general, we do not want to summarize messages produced during the run, at least, I was not expecting this behavior and I do not see reasons why we should do it. What I was expecting with this feature is to process only messages passed in the message_history of the run Agent method.

Doc example should be reconsidered

In addition, the example provided in the documentation is logically incorrect to me. It is making a summary of the first 10 messages in the list (which are the oldest), then it adds the most recent message. What about the messages between the index 10 and the last message ?

I think the example considered a very simple case where a history is passed and the first user prompt is evaluated by the agent (the user prompt is message[-1:] and the entire history is summarized, considering exactly 10 messaged in the history).

Example Code

from pydantic_ai import Agent, RunContext from pydantic_ai.messages import ModelMessage from pydantic_ai.models import ModelRequest, ModelResponse from pydantic_ai.messages import UserPromptPart, TextPart from dataclasses import dataclass @dataclass class MyContext: run_count: int = 0 history = [] # create 10 qa messages replicating the last one for qa in range(20): history.append(ModelRequest(parts=[UserPromptPart(content=f"Question {qa + 1}")])) history.append(ModelResponse(parts=[TextPart(content=f"Answer {qa + 1}")])) # Use a cheaper model to summarize old messages. summarize_agent = Agent( "openai:gpt-4o-mini", instructions=""" Summarize this conversation, omitting small talk and unrelated topics. Focus on the technical discussion and next steps. """, ) async def summarize_old_messages( ctx: RunContext[MyContext], messages: list[ModelMessage] ) -> list[ModelMessage]: # Summarize the oldest 10 messages if len(messages) > 10: ctx.deps.run_count += 1 oldest_messages = messages[:10] summary = await summarize_agent.run(message_history=oldest_messages) # Return the last message and the summary ############# DEBUG ############# print(f"Run count: {ctx.deps.run_count}") for msg in messages[-5:]: print(msg) ################################# return summary.new_messages() + messages[-1:] return messages agent = Agent("openai:gpt-4o-mini", history_processors=[summarize_old_messages]) # add a simple tool @agent.tool async def get_answer(ctx: RunContext[MyContext], question_number: int) -> str: return "There is no answer!" agent.run_sync( "What is the answer to question 5?", message_history=history, deps=MyContext() )

Python, Pydantic AI & LLM client version

Python 3.13 pydantic-ai-slim[openai]>=0.3.2