A framework for testing MCP (Model Context Protocol) client and server implementations against the specification.

[!WARNING] This repository is a work in progress and is unstable. Join the conversation in the #conformance-testing-wg in the MCP Contributors discord.

npx @modelcontextprotocol/conformance client --command "tsx examples/clients/typescript/test1.ts" --scenario initialize

# Run all server scenarios (default) npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp # Run a single scenario npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --scenario server-initialize

npx @modelcontextprotocol/conformance list

The conformance test framework validates MCP implementations by:

For Clients:

1. Starting a test server for the specified scenario
2. Running the client implementation with the test server URL
3. Capturing MCP protocol interactions
4. Running conformance checks against the specification
5. Generating detailed test results

For Servers:

1. Connecting to the running server as an MCP client
2. Sending test requests and capturing responses
3. Running conformance checks against server behavior
4. Generating detailed test results

npx @modelcontextprotocol/conformance client --command "<client-command>" --scenario <scenario-name> [options]

Options:

• --command - The command to run your MCP client (can include flags)
• --scenario - The test scenario to run (e.g., "initialize")
• --timeout - Timeout in milliseconds (default: 30000)
• --verbose - Show verbose output

The framework appends the server URL as the final argument to your command.

npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]

Options:

• --url - URL of the server to test
• --scenario <scenario> - Test scenario to run (e.g., "server-initialize". Runs all available scenarios by default

Client Testing - Results are saved to results/<scenario>-<timestamp>/:

• checks.json - Array of conformance check results with pass/fail status
• stdout.txt - Client stdout output
• stderr.txt - Client stderr output

Server Testing - Results are saved to results/server-<scenario>-<timestamp>/:

• checks.json - Array of conformance check results with pass/fail status

• examples/clients/typescript/test1.ts - Valid MCP client (passes all checks)
• examples/clients/typescript/test-broken.ts - Invalid client missing required fields (fails checks)

• initialize - Tests MCP client initialization handshake
  • Validates protocol version
  • Validates clientInfo (name and version)
  • Validates server response handling
• tools-call - Tests tool invocation
• auth/basic-dcr - Tests OAuth Dynamic Client Registration flow
• auth/basic-metadata-var1 - Tests OAuth with authorization metadata

Run npx @modelcontextprotocol/conformance list --server to see all available server scenarios, including:

• server-initialize - Tests server initialization and capabilities
• tools-list - Tests tool listing endpoint
• tools-call-* - Various tool invocation scenarios
• resources-* - Resource management scenarios
• prompts-* - Prompt management scenarios

See src/runner/DESIGN.md for detailed architecture documentation.

• Runner (src/runner/) - Orchestrates test execution and result generation
  • client.ts - Client testing implementation
  • server.ts - Server testing implementation
  • utils.ts - Shared utilities
  • index.ts - Public API exports
• CLI (src/index.ts) - Command-line interface using Commander.js
• Scenarios (src/scenarios/) - Test scenarios with expected behaviors
• Checks (src/checks/) - Conformance validation functions
• Types (src/types.ts) - Shared type definitions

1. Create a new directory in src/scenarios/<scenario-name>/
2. Implement the Scenario interface with start(), stop(), and getChecks()
3. Register the scenario in src/scenarios/index.ts

See src/scenarios/initialize/ for a reference implementation.