Python: ADR: Extend types

[eavanvalkenburg]

Motivation and Context

reopening because of detached fork.

Additional types to be added.

ImageGenerationTool is still a subject of discussion.

Description

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[Copilot]

Pull Request Overview

This PR proposes an ADR (Architecture Decision Record) to extend types and tools in the Agent Framework to support additional capabilities available in chat clients. The proposal evaluates new content types and abstracted tools needed for better interoperability between chat clients.

• Proposes HostedImageGenerationTool for image generation capabilities
• Decides against adding ComputerUseTool and BashTool due to implementation complexity
• Introduces CodeExecutionContent as a new content type for code interpreter responses

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The word 'API's' should use 'APIs' (without apostrophe) as it's a plural form, not possessive.

Suggested change

Multiple API's currently support Image Generation tools, such as OpenAI Responses [here](https://platform.openai.com/docs/api-reference/responses/create#responses-create-tools). And [Mistral](https://docs.mistral.ai/agents/connectors/image_generation/), For OpenAI and Mistral this can be enabled by adding: `{"type": "image_generation"}` to the tools list. You then get image data back in the response, inputs can be both text and images. Google so far, has only created specific models that can be prompted to return a generated image, no tool specified.

Multiple APIs currently support Image Generation tools, such as OpenAI Responses [here](https://platform.openai.com/docs/api-reference/responses/create#responses-create-tools). And [Mistral](https://docs.mistral.ai/agents/connectors/image_generation/), For OpenAI and Mistral this can be enabled by adding: `{"type": "image_generation"}` to the tools list. You then get image data back in the response, inputs can be both text and images. Google so far, has only created specific models that can be prompted to return a generated image, no tool specified.

[Copilot]

The word 'API's' should use 'APIs' (without apostrophe) as it's a plural form, not possessive.

Suggested change

	Multiple API's currently support Bash or Shell tools, such as OpenAI Responses [here](https://platform.openai.com/docs/api-reference/responses/create#responses-create-tools). And Anthropic Claude [here](https://docs.claude.com/en/docs/agents-and-tools/tool-use/bash-tool).
	Multiple APIs currently support Bash or Shell tools, such as OpenAI Responses [here](https://platform.openai.com/docs/api-reference/responses/create#responses-create-tools). And Anthropic Claude [here](https://docs.claude.com/en/docs/agents-and-tools/tool-use/bash-tool).

[Copilot]

The word 'API's' should use 'APIs' (without apostrophe) as it's a plural form, not possessive.

Suggested change

	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type
	- Con: only available in limited number of APIs currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type

[Copilot]

Should be 'an additional Content Type' instead of 'a additional Content Type'.

Suggested change

	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type
	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require an additional Content Type

[Copilot]

The word 'API's' should use 'APIs' (without apostrophe) as it's a plural form, not possessive.

Suggested change

	- Con: only available in limited number of API's currently.
	- Con: only available in limited number of APIs currently.

[Copilot]

The word 'API's' should use 'APIs' (without apostrophe) as it's a plural form, not possessive.

Suggested change

	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type
	- Con: only available in limited number of APIs currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type

[Copilot]

Should be 'an additional Content Type' instead of 'a additional Content Type'.

Suggested change

	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require a additional Content Type
	- Con: only available in limited number of API's currently and expectation would likely be to also include a local setup for it, which will be difficult to unify across platforms and providers. Would also require an additional Content Type

[Copilot]

Should be 'than a dict' instead of 'then a dict', and 'APIs' instead of 'API's'.

Suggested change

	- Pro: looks cleaner then a dict, would only supply `type` as a parameter, and even that might break for some API's.
	- Con: only marginally easier to use then a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Alternative names: GenericHostedTool, GenericTool
	2. HostedTool typeddict with total=False
	- Pro: looks cleaner then a dict, would only supply `type` as a parameter, and even that might break for some API's. Would make usage optional compared to a class.
	- Con: only marginally easier to use then a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Pro: looks cleaner than a dict, would only supply `type` as a parameter, and even that might break for some APIs.
	- Con: only marginally easier to use than a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Alternative names: GenericHostedTool, GenericTool
	2. HostedTool typeddict with total=False
	- Pro: looks cleaner than a dict, would only supply `type` as a parameter, and even that might break for some APIs. Would make usage optional compared to a class.
	- Con: only marginally easier to use than a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.

[Copilot]

Should be 'than a dict' instead of 'then a dict', and 'APIs' instead of 'API's'.

Suggested change

	- Pro: looks cleaner then a dict, would only supply `type` as a parameter, and even that might break for some API's.
	- Con: only marginally easier to use then a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Alternative names: GenericHostedTool, GenericTool
	2. HostedTool typeddict with total=False
	- Pro: looks cleaner then a dict, would only supply `type` as a parameter, and even that might break for some API's. Would make usage optional compared to a class.
	- Con: only marginally easier to use then a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Pro: looks cleaner than a dict, would only supply `type` as a parameter, and even that might break for some APIs.
	- Con: only marginally easier to use than a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.
	- Alternative names: GenericHostedTool, GenericTool
	2. HostedTool typeddict with total=False
	- Pro: looks cleaner than a dict, would only supply `type` as a parameter, and even that might break for some APIs. Would make usage optional compared to a class.
	- Con: only marginally easier to use than a dict, and does not provide any additional type safety or discoverability. Not a full abstraction of the tools, so not a lot of added value.