Remove meta requirements for mocked tools in Agent Evals

[samedson]

Description

Prevents tools from being blocked by signing in and having a project set. Fixes the init eval test

[gemini-code-assist]

Summary of Changes

Hello @samedson, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request streamlines the agent evaluation process by making mocked tools more flexible. It removes unnecessary authentication and project-related meta requirements, preventing tools from being blocked during tests. This change enhances the reliability and ease of running evaluation tests, particularly by resolving issues in the init eval test, and improves debugging capabilities through updated logging configurations.

Highlights

• Mocked Tool Requirements Removed: The pull request introduces a mechanism to remove meta requirements such as authentication (requiresAuth), project dependency (requiresProject), and optional project directory (optionalProjectDir) for mocked tools used in agent evaluations. This ensures that mocked tools can run without being blocked by these checks.
• Logging Improvements: The logging path for agent evaluation mocks has been adjusted to point to a 'home' directory within the evaluation environment, rather than the user's Desktop. Additionally, file logging (ENABLE_FILE_LOGGING) is now enabled by default for easier debugging.
• Evaluation Environment Consistency: The directory created for user-specific files within the evaluation runner has been renamed from user to home to align with how the HOME environment variable is typically set and used in these evaluation contexts.
• Test Fix: These changes specifically address and fix an issue with the init eval test, allowing it to pass correctly by removing the blocking meta requirements.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request removes metadata requirements for mocked tools in agent evaluations, which simplifies testing by avoiding the need for sign-in or project setup. The changes involve modifying the tool objects to disable these requirements and updating file paths for consistency. My review includes a couple of suggestions for improvement: one to avoid object mutation for better predictability and another to reconsider enabling file logging by default.

[samedson]

Hold off on reviewing for now. Cleo and I talked and we might take a different approach

[joehan]

Hold off on reviewing for now. Cleo and I talked and we might take a different approach

Missed this, but will come back later