Allow WWW-Authenticate header to be accessed by client

[cliffhall]

Summary of Changes

This pull request enhances the server's proxy functionality by enabling the correct forwarding of WWW-Authenticate headers from the backend to the client. This ensures that clients receive appropriate authentication challenges. The changes involve capturing the header during transport creation, exposing it through the transport object, and applying it to the proxy's outgoing responses. A significant refactoring was also undertaken to improve the robustness and maintainability of the fetch interception mechanism and error handling, directly addressing critical feedback regarding potential race conditions and type safety.

Highlights

• Authentication Header Forwarding: Implemented logic to intercept and forward the WWW-Authenticate header from backend (MCP server) responses to the client, ensuring proper authentication challenges are propagated.
• Localized Fetch Interception: Refactored the createTransport function to use a localized interceptingFetch mechanism, which is passed directly to the SSEClientTransport and StreamableHTTPClientTransport constructors. This change addresses critical race condition concerns by avoiding global modification of globalThis.fetch.
• Enhanced Transport Creation Return: The createTransport function now returns an object containing both the Transport instance and any captured WWW-Authenticate header, allowing for its propagation to the client.
• Centralized Header Handling: Introduced new helper functions, maybeSetAuthHeader and setAuthHeaderFromError, to centralize and reuse the logic for setting the WWW-Authenticate header on proxy responses, particularly in error scenarios, improving code maintainability and reducing duplication.

Changelog

• server/src/index.ts
  • Added maybeSetAuthHeader helper function to conditionally set the WWW-Authenticate header on an Express response.
  • Added setAuthHeaderFromError helper function to safely extract and set the WWW-Authenticate header from an error object, including type checks for robustness.
  • Modified the createTransport function signature to return an object containing both the Transport instance and an optional authHeader.
  • Implemented a local interceptingFetch within createTransport to capture the WWW-Authenticate header from 401 responses, passing this custom fetch to SSEClientTransport and StreamableHTTPClientTransport to avoid global side effects.
  • Updated the app.post route for StreamableHTTP connections to utilize the new createTransport return value and the maybeSetAuthHeader and setAuthHeaderFromError helpers.
  • Updated the app.get route for STDIO connections to utilize the new createTransport return value and the maybeSetAuthHeader and setAuthHeaderFromError helpers.
  • Updated the app.get route for SSE connections to utilize the new createTransport return value and the maybeSetAuthHeader and setAuthHeaderFromError helpers.

Motivation and Context

When a 401 Unauthorized error is returned from a server, the WWW-Authenticate header should be included in the response, advertising the HTTP authentication methods (or challenges) that might be used to gain access.

By adding this header name to the Access-Control-Expose-Headers header in the server response, it should tell the browser to expose that header to scripts, so that the client can extract the authentication method or challenge and react appropriately.

However, even with the above change, we can see that with a server that is actually returning a WWW-Authenticate header with a 401, the header isn't making back to the client via the proxy.

There are two transports involved; Client <-> Proxy and Proxy <-> Server. We also need to bridge the headers across the transports in the proxy, not just the error message.

How Has This Been Tested?

User @MOmarMiraj has been helping with a server he has that returns WWW-Authenticate header on 401.

Breaking Changes

Nope.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

• This fixes WWW-Authenticate headers should be passed through #168

[jescalan]

In case it's useful at all, I've got a demo MCP server that returns 401/www-authenticate here! I have tested this with a client that handles auth properly according to spec and it's working. Would be great if the inspector worked this way 👀

[cliffhall]

In case it's useful at all, I've got a demo MCP server that returns 401/www-authenticate here! I have tested this with a client that handles auth properly according to spec and it's working. Would be great if the inspector worked this way 👀

Only problem is I don't have a Clerk application. If you don't mind, though, keep an eye on this PR and when it gets merged, raise a flag here if it doesn't work with your server.

[MOmarMiraj]

Hey Folks,

Just wanted to know what is currently blocking this MR and if needed how I can help. This would be awesome to get merged in considering the latest spec.

[cliffhall]

Hey Folks,

Just wanted to know what is currently blocking this MR and if needed how I can help. This would be awesome to get merged in considering the latest spec.

Waiting for SDK support. @pcarleton can probably elucidate the gap.

[MOmarMiraj]

Hmm do you mean modelcontextprotocol/typescript-sdk#503 (this is server side) and this is client side modelcontextprotocol/typescript-sdk#416 unless im missing something.

[cliffhall]

@pcarleton mentioned this one when I asked in Discord. But I'm not certain that keeps 401 from reaching the client, it's just related to scopes. So I'll look into this one again today.

ALSO: A big problem Is I don't have an in-project server that returns a 401 with this header for testing.

[MOmarMiraj]

@pcarleton mentioned this one when I asked in Discord. But I'm not certain that keeps 401 from reaching the client, it's just related to scopes. So I'll look into this one again today.

ALSO: A big problem Is I don't have an in-project server that returns a 401 with this header for testing.

I am in the process of building one.. if you can get the MR up I can try testing!

[cliffhall]

ALSO: A big problem Is I don't have an in-project server that returns a 401 with this header for testing.

I am in the process of building one.. if you can get the MR up I can try testing!

Thanks @MOmarMiraj but I don't want to merge this without testing, there might be more to do. However, in order to test this PR, you could

• Fork this repo
• Clone your fork locally, and assuming you didn't rename it, do:
  • git clone https://github.com/MOmarMiraj/inspector.git
• From your local project folder add my repo as a remote:
  • git add remote cliffhall https://github.com/cliffhall/mcp-inspector.git
• See the branches on my repo
  • fetch cliffhall
  • git checkout -b pass-www-authenticate-headers cliffhall/pass-www-authenticate-headers to check out my PR branch
• Install deps
  • npm install
• Build the project
  • npm run build
• Launch the inspector with my changes
  • npm run start
• Point it at your server and see what happens in the network tab of the browser's devtools window. Does the header make it through?

[MOmarMiraj]

I see in the request that it is exposed and when I try to start the server.. it immediately runs the OAuth flow and tries to fetch the .well-known/oauth-protected-resource etc.. So I believe it is working but I can't find the actual 401 unauthorized in the console.

When I change the status code to 200 but send the WWW-Authenticate header, it doesn't start the OAuth flow.. so from my testing looks like it goes through!

[cliffhall]

I see in the request that it is exposed and when I try to start the server.. it immediately runs the OAuth flow and tries to fetch the .well-known/oauth-protected-resource etc.. So I believe it is working but I can't find the actual 401 unauthorized in the console.

When I change the status code to 200 but send the WWW-Authenticate header, it doesn't start the OAuth flow.. so from my testing looks like it goes through!

@MOmarMiraj This report is sort of confusing. You never see the 401 at the client?

[MOmarMiraj]

Ahh sorry for the confusing report, I do see the 401 and then it continues with the OAuth Flow.

See attached picture as you can see at the top with

Error POSTing to endpoint (HTTP 401) 

and then the OAuth flow starts.

[cliffhall]

Ahh sorry for the confusing report, I do see the 401 and then it continues with the OAuth Flow.

See attached picture as you can see at the top with

Error POSTing to endpoint (HTTP 401) 

@MOmarMiraj Can you show the output both when you do and when you don't have these changes in place for comparison?

Open the network tab of devtools and select any request to view its response headers directly, without relying on error console output. this is what we really want to verify.

E.g, ...

[MOmarMiraj]

This is the image with no WWW-Authenticate

and this is the image with WWW-Authenticate

[cliffhall]

This is the image with no WWW-Authenticate ...
and this is the image with WWW-Authenticate ...

Ok, right. But if you notice, the actual header you're looking at is Access-Control-Expose-Headers and its value is WWW-Authenticate. This is good and what I would expect; it says to expose any actual WWW-Authenticate header to the client for processing.

What I need to see now, is the headers of a request that returned a 401 response. In that case, we want to see WWW-Authenticate on the left side of the picture - an actual header. And the same call when this code is not in place should not show that header. Try again and examine whichever call was outputting the 401 error message in the console in your comment above.

[MOmarMiraj]

Hm.. for some reason I can't find the 401 in the browser client inspector and I just see a 200 for the actual /mcp endpoint call. I tried doing curl requests and I can see it better:

and this is when I pass an actual token

[cliffhall]

Hm.. for some reason I can't find the 401 in the browser client inspector and I just see a 200 for the actual /mcp endpoint call. I tried doing curl requests and I can see it better

Ok this is super useful. We can see that the server is actually returning a WWW-Authenticate header with a 401, but that isn't making back to the client via the proxy.

There are two transports involved; Client <-> Proxy and Proxy <-> Server. This report tells me we need to do more to bridge the headers across the transports in the proxy.

[cliffhall]

@MOmarMiraj I've had a crack at a fix. Can you test it again?

[MOmarMiraj]

@cliffhall Just tried it and still similar issue, don't see the 401 on the network side of things.

[MOmarMiraj]

Also what I noticed, the MCP inspector doesn't extract the URL from the WWW-Authenticate header. No matter the value of the WWW-Authenticate header, the protected resource URL is always the URL of my MCP server + the .well-known/oauth-protected-resource which doesn't abide by the MCP spec. I haven't looked into if this is more of SDK issue or inspector issue.

[cliffhall]

@cliffhall Just tried it and still similar issue, don't see the 401 on the network side of things.

It would be great to see the response header list there. The screenshot says there are 10 headers, but the header list is closed

[cliffhall]

Also what I noticed, the MCP inspector doesn't extract the URL from the WWW-Authenticate header.

First we have to get the header. We're working on that part.

[MOmarMiraj]

@cliffhall Just tried it and still similar issue, don't see the 401 on the network side of things.

It would be great to see the full response header list there. I trust you but it would still be nice to see that is missing and not just below the fold. I'm going to have to build this into an in-project server for testing soon.

the protected resource URL is always the URL of my MCP server + the .well-known/oauth-protected-resource which doesn't abide by the MCP spec.

Your screenshot doesn't support this statement; it doesn't talk about the URL.

The second paragraph speaking about the server metadata URL. If we look inside the RFC pointed out in the MCP spec, you can see it talks about the full URL.

[cliffhall]

Response headers. Can you show them? You posted the request headers.

Your screenshot doesn't support this statement; it doesn't talk about the URL.
Also, my bad on that part I amended my comment

[MOmarMiraj]

oops lol sorry wrong screen shot.. Here you go is the response headers

[cliffhall]

Ok, thanks @MOmarMiraj for your help. I'm going to have to figure out a good way to test this locally.

[MOmarMiraj]

I believe the issue lies in the SDK not accepting a custom fetch function. I believe you are correctly overriding it but inside the StreamableHTTPClient class in the MCP TS SDK, they don't call the custom fetch implementation and always defaults to the native TS one.

Heres an issue about it as well modelcontextprotocol/typescript-sdk#476

[cliffhall]

I believe the issue lies in the SDK not accepting a custom fetch function. I believe you are correctly overriding it but inside the StreamableHTTPClient class in the MCP TS SDK, they don't call the custom fetch implementation and always defaults to the native TS one.

Heres an issue about it as well modelcontextprotocol/typescript-sdk#476

Thanks for this insight @MOmarMiraj!

[cliffhall]

Waiting on modelcontextprotocol/typescript-sdk#721

[cliffhall]

@MOmarMiraj could you try this again? You'll need to run npm install again, and should be using sdk version 1.15.0, which has the support for custom fetch that this PR uses to capture and bridge the www-authenticate header to the client.

[MOmarMiraj]

I updated to the latest SDK and pulled ur branch and still not getting that WWW-Authenticate header across.. I still think the fetch isn't working correctly. I try to debug from inside that function and I do not receive my debug statement...

From the proxy, we get a 401 unauthorized but for some reason the actual POST /mcp endpoint call gives a 200?

[MOmarMiraj]

I realized the way we create the StreamableHTTPTransport is incorrect. We should be overriding the fetch function like so:

 const transport = new StreamableHTTPClientTransport( new URL(query.url as string), { requestInit: { headers, }, fetch: interceptingFetch, }, ); 

as the constructor for StreamableHTTPTransport expects this:

 constructor( url: URL, opts?: StreamableHTTPClientTransportOptions, ) { this._url = url; this._resourceMetadataUrl = undefined; this._requestInit = opts?.requestInit; this._authProvider = opts?.authProvider; this._fetch = opts?.fetch; this._sessionId = opts?.sessionId; this._reconnectionOptions = opts?.reconnectionOptions ?? DEFAULT_STREAMABLE_HTTP_RECONNECTION_OPTIONS; } 

which currently we are doing opts?.requestInit?.fetch.

When I changed it, I can see that the auth header gets set correctly but not showing up in console still.

[cliffhall]

which currently we are doing opts?.requestInit?.fetch.

When I changed it, I can see that the auth header gets set correctly but not showing up in console still.

Good catch, @MOmarMiraj. Updated the location of the fetch in transport options. But are you actually throwing an error from your server or just setting the header?

[MOmarMiraj]

Good catch, @MOmarMiraj. Updated the location of the fetch in transport options. But are you actually throwing an error from your server or just setting the header?
I am throwing the error and the header which from my earlier curl request you can see.

I believe the issue is that when setting up the proxy, everything goes smoothly which is why I get a 200 OK on the request and then when it tries to actually ping the server it gets the 401.