Add vmcp e2e tests using yardstick MCP server #2778
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This adds two new e2e test files for VirtualMCPServer: 1. virtualmcp_yardstick_base_test.go: - Tests basic multi-backend aggregation using yardstick - Verifies prefix conflict resolution - Tests tool listing and tool calls through vMCP - Uses deterministic yardstick echo tool for reliable testing 2. virtualmcp_aggregation_filtering_test.go: - Tests tool filtering per backend workload - Verifies that filtered tools from one backend appear - Verifies that empty filter excludes all tools from a backend - Tests that filtered tools can still be called Yardstick is a deterministic MCP server designed for testing that provides an "echo" tool returning the input - perfect for verifying data flow through vMCP aggregation and composite tools. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@ ## main #2778 +/- ## ========================================== + Coverage 56.34% 56.36% +0.02% ========================================== Files 319 319 Lines 30887 30887 ========================================== + Hits 17402 17409 +7 + Misses 11999 11986 -13 - Partials 1486 1492 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The previous test incorrectly assumed that an empty filter `[]string{}` would exclude all tools from a backend. In reality, empty/nil filter means "allow all tools" (no filtering applied). This fix uses a non-matching filter `["nonexistent_tool"]` as a workaround to exclude all tools from backend2, since yardstick only exposes the "echo" tool. Added TODO comments referencing issue #2779 which proposes adding an `excludeAll` option for proper tool exclusion. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> 
github-actions bot added size/L 
yrobla approved these changes Nov 28, 2025
Contributor
| Hello, i have concerns about this code. so anonymous auth: type: "anonymous" - its like no authentication. Anyone can connect????
|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Summary
virtualmcp_yardstick_base_test.go: Tests basic multi-backend aggregation with prefix conflict resolutionvirtualmcp_aggregation_filtering_test.go: Tests tool filtering per backend workloadWhy Yardstick?
Yardstick is a deterministic MCP server designed specifically for testing:
echotool that returns the inputghcr.io/stackloklabs/yardstick/yardstick-server:0.0.2Test Coverage
Base Test (
virtualmcp_yardstick_base_test.go)Filtering Test (
virtualmcp_aggregation_filtering_test.go)aggregation.tools[].filterconfigurationTest plan
task thv-operator-e2e-testto execute these tests (requires Kind cluster)task lint🤖 Generated with Claude Code