That's because many people don't understand how the index directive actually works. If the request URI maps to a physical directory under the web server root, and a suitable index file is found in that directory, the index directive performs an internal redirect to the same URI with the index file name appended. This behavior is explicitly described in the documentation:
It should be noted that using an index file causes an internal redirect, and the request can be processed in a different location. For example, with the following configuration:
location = / { index index.html; } location / { ... }
a / request will actually be processed in the second location as /index.html.
Thus, if the incoming request is for / and certificate authentication fails, nginx passes request processing to your named location @client_cert_fail. There, the index directive finds an index file client_cert_fail.html in the /var/www directory and performs an internal redirect to /client_cert_fail.html. At that point, the request is re-evaluated, and the best matched prefix or regex location block is selected to serve the new URI. For requests other than /, you're likely to get a 404 Not Found error while the request is still being processed within the named location, since the original request URI remains unchanged.
You can easily fix this by rewriting any request URI to /client_cert_fail.html inside your named location:
location @client_cert_fail { root /var/www; rewrite ^ /client_cert_fail.html break; }
If you want to allow the client_cert_fail.html file to load additional assets, you can change the above configuration as follows:
location @client_cert_fail { root /var/www; try_files $uri /client_cert_fail.html =404; }
The assets must, of course, be located under the same /var/www directory or its subdirectories. (The solution was originally authored by Richard Smith.)
An addition to the answer, addressing the OP's ongoing questions from the comments:
This perfectly explains why the config using index doesn't work! However, using an empty location @client_cert_fail {} block also allows clients to bypass client authentication. Is this because @client_cert_fail doesn't terminate the request and nginx is just looking for the next location block than can satisfy the request?
A request can leave the location it was initially assigned to only through an internal rewrite. This can happen implicitly in two main cases:
- Through the
index directive (either inherited from a higher level or by implying the default index.html), if an index file is found in the directory mapped from the request URI and web server root. - Through an
error_page directive, if an error occurs during processing the request. However, this is less likely here, since you're already handling a 495/496 error. For this to happen, recursive_error_pages on; must be explicitly enabled.
Looking at the error log it shows me that when I leave the location @client_cert_fail {} block empty, it's generating an internal redirect to /usr/share/nginx/html/index.html which is then redirected to /index.html?. I'm not entirely sure why that happens (the redirect from /usr/share/nignx/html/index.html to /index.html?), but it confirms what you said: the location block is left by an index directive that must be implicit because I don't have an index directive anywhere in my nginx config.
You're not interpreting the error log entries quite correctly.
Both index and root directives have their defaults. For the index directive, the default is:
index index.html;
Unfortunately, there's no way in nginx to completely disable the index directive, even though this is actually desired in some cases (see this thread for an example). Having a directive like noindex would really be helpful (I've even considered writing a patch for the nginx index module to add such a directive).
The root directive defaults to the html directory, relative to the prefix compile-time option value. You can check the actual prefix value for your nginx binary using the nginx -V command. The output will contain a line like:
configure arguments: --prefix=...
For most packaged nginx distributions, the prefix is usually either /var/www or /usr/share/nginx, as in your case.
Back to your error log: nginx checks for the presence of the default index file index.html under the default document root /usr/share/nginx/html, finds it, and performs an internal redirect to the URI /index.html?, appending the (empty) query string to the rewritten URI. This behavior — preserving the query string for the rewritten URI — is particularly important when a PHP file such as index.php is specified as an index file using the index directive.
I suppose I should put additional checks (like if ($ssl_client_verify != SUCCESS) {}) in sensitive location blocks.
This is exactly what you should try to avoid by all means. An if directive, when used within a location block, has nothing to do with the traditional if ... then construction from imperative programming languages. The internal implementation of the if directive declared in the server and location contexts is completely different, and If is Evil... when used in the location context (if you're curious, you can read about how it actually works here).
You only need to ensure the request won't be internally rewritten when processed by your @client_cert_fail named location. Both provided examples guarantee this won't happen due to the index directive. In the first example, any request is rewritten to /client_cert_fail.html, and the index module does not process any request where the request URI does not end with a slash. In the second example, the try_files directive ensures the requested URI maps to a physical file under the /var/www directory, or it will be rewritten to /client_cert_fail.html otherwise. This directive cannot pass a request that maps to a directory to the next request processing phase, where the index module is actually invoked, due to the fact that the $uri/ parameter is not present.
