3

i'm trying to use lighttpd as a simple "alive" server for some nodes i have running. However, the script I'm running throws 500 errors non-deterministically, and i can't find them anywhere.

Here's my debugging configuration:

server.document-root = "/var/www/health_checker" server.port = 44445 server.modules += ("mod_auth", "mod_rewrite", "mod_accesslog", "mod_cgi", "mod_redirect", "mod_status" ) cgi.assign = ( ".sh" => "/bin/sh" ) url.rewrite-once = ( "^/healthz" => "/healthz.sh", "^/.*" => "/livez.sh" ) server.error-handler-404 = "/livez.sh" server.errorlog = "/home/aronchick/temperror.log" server.breakagelog = "/home/aronchick/tempcgi.log" server.accesslog = "/home/aronchick/tempaccess.log" debug.log-request-header = "enable" debug.log-request-header-on-error = "enable" debug.log-response-header = "enable" debug.log-file-not-found = "enable" debug.log-request-handling = "enable" debug.log-condition-handling = "enable" debug.log-condition-cache-handling = "enable" debug.log-ssl-noise = "enable" debug.log-timeouts = "enable"

And here's an example of the non-determinism:

aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh { 'hostname': 'myservice-vm-0.us-east4-c.c.myservice-production.internal', 'date': '2022-06-19 19:38:37.445066814+00:00', 'ip': 'xx.xx.xx.xx'} aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> </body> </html> aronchick@myservice-vm-0:~$ aronchick@myservice-vm-0:~$ curl 0.0.0.0:44445/livez.sh { 'hostname': 'myservice-vm-0.us-east4-c.c.myservice-production.internal', 'date': '2022-06-19 19:38:37.445066814+00:00', 'ip': 'xx.xx.xx.xx'} aronchick@myservice-vm-0:~$ ps aux

I can't even figure out where the 500 errors are being written. Here's the /home/aronchick/temperror.log:

2022-06-19 19:29:37: (connections.c.771) fd:8 rqst: GET / HTTP/1.1 2022-06-19 19:29:37: (connections.c.771) fd:8 rqst: Host: 127.0.0.1:44445 2022-06-19 19:29:37: (connections.c.771) fd:8 rqst: User-Agent: curl/7.81.0 2022-06-19 19:29:37: (connections.c.771) fd:8 rqst: Accept: */* 2022-06-19 19:29:37: (connections.c.771) fd:8 rqst: 2022-06-19 19:29:37: (response.c.407) -- parsed Request-URI 2022-06-19 19:29:37: (response.c.409) Request-URI : / 2022-06-19 19:29:37: (response.c.411) URI-scheme : http 2022-06-19 19:29:37: (response.c.413) URI-authority : 127.0.0.1:44445 2022-06-19 19:29:37: (response.c.415) URI-path (clean): / 2022-06-19 19:29:37: (response.c.417) URI-query : 2022-06-19 19:29:37: (response.c.407) -- parsed Request-URI 2022-06-19 19:29:37: (response.c.409) Request-URI : /livez.sh 2022-06-19 19:29:37: (response.c.411) URI-scheme : http 2022-06-19 19:29:37: (response.c.413) URI-authority : 127.0.0.1:44445 2022-06-19 19:29:37: (response.c.415) URI-path (clean): /livez.sh 2022-06-19 19:29:37: (response.c.417) URI-query : 2022-06-19 19:29:37: (response.c.495) -- logical -> physical 2022-06-19 19:29:37: (response.c.497) Doc-Root : /var/www/health_checker 2022-06-19 19:29:37: (response.c.499) Basedir : /var/www/health_checker 2022-06-19 19:29:37: (response.c.501) Rel-Path : /livez.sh 2022-06-19 19:29:37: (response.c.503) Path : /var/www/health_checker/livez.sh 2022-06-19 19:29:37: (response.c.522) -- handling subrequest 2022-06-19 19:29:37: (response.c.524) Path : /var/www/health_checker/livez.sh 2022-06-19 19:29:37: (response.c.526) URI : /livez.sh 2022-06-19 19:29:37: (response.c.528) Pathinfo : 2022-06-19 19:29:37: (response.c.164) fd:8 resp: HTTP/1.1 500 Internal Server Error 2022-06-19 19:29:37: (response.c.164) fd:8 resp: Content-Type: text/html 2022-06-19 19:29:37: (response.c.164) fd:8 resp: Content-Length: 365 2022-06-19 19:29:37: (response.c.164) fd:8 resp: Date: Sun, 19 Jun 2022 19:29:37 GMT 2022-06-19 19:29:37: (response.c.164) fd:8 resp: Server: lighttpd/1.4.63 2022-06-19 19:29:37: (response.c.164) fd:8 resp:
Improve this question
1
  • on http 500 errors, the stderr output from CGI scripts is logged to server.breakagelog (source) Commented Apr 23 at 6:39

1 Answer 1

Reset to default
2

If your script fails to return a response, then lighttpd returns 500 Internal Server Error.

The CGI script is probably not following the CGI specification.

? Does your CGI script do something like:

printf "Status: 200\nContent-Type: application/json\n\n"

before printing { 'hostname': 'myservice-vm-0.us-east4-c.c.myservice-production.internal', 'date': '2022-06-19 19:38:37.445066814+00:00', 'ip': 'xx.xx.xx.xx'} ?

Improve this answer
2
  • Unfortunately I moved on to using nginx because I didn’t figure this out. I’m marking this as the answer as it sounds correct, but ymmv. Commented Jun 26, 2022 at 10:36
  • Great! Can confirm this was my problem. Commented Jun 28, 2023 at 11:33

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.