0

I have a perl and shell script that process a bunch of data and output results as they happen, like:

5 processed in 0.58 seconds. 10 processed in 0.79 seconds. ... 150 processed in 0.65 seconds. DONE!

etc.

However, when I visit the URL with Chrome, it shows nothing, waits until the script finishes, then shows the entire output at once. Is there a way to configure nginx to show each line output as it happens? I used to be able to do this with Apache.

I put gzip off in the nginx config, thinking this could nginx waiting to compress the text before sending it to the client, but that did not help.

I also tried buffer-flushing techniques in the scripts themselves, like these lines at the top of my Perl scripts:

use IO::Handle; $| ++; STDERR->autoflush(1); STDOUT->autoflush(1); print "Content-type: text/plain\n\n";

Relevant parts of the nginx config look like this

 ssl on; root /my/path/cgi ; proxy_read_timeout 900s ; fastcgi_read_timeout 900s ; fastcgi_request_buffering off ; gzip off ; location ~ \.pl|cgi$ { try_files $uri =404; gzip off; fastcgi_pass unix:/var/run/fcgiwrap.socket; include /etc/nginx/fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; }

Any ideas?

Improve this question
2
  • It might be that fcgiwrap implements some buffering. Commented Aug 25, 2020 at 6:34
  • Yeah from the looks of it, seems to be no way to turn off buffering for fastcgi. Instead, I set up an apache server listening on localhost with ExecCGI enabled for the cgi directory and AddHandler cgi-script specified, then I changed my nginx to proxy_pass the .pl and .cgi files to the local apache port. This worked. Apache does no caching of cgi output, and outputs the lines as they happen to the client. Commented Aug 25, 2020 at 18:18

1 Answer 1

Reset to default
0

You need to tell nginx to not buffer the response from the FastCGI server, with fastcgi_buffering off;.

From the docs:

When buffering is disabled, the response is passed to a client synchronously, immediately as it is received. nginx will not try to read the whole response from the FastCGI server. The maximum size of the data that nginx can receive from the server at a time is set by the fastcgi_buffer_size directive.

Which you may also need to tweak.

But in general, a plain HTML web page is probably not the best interface for whatever it is you're doing.

Improve this answer
2
  • Sadly this did not help. Same result. I also checked that this is not a browser issue by doing an HTTP GET on a linux shell and watching the raw output show nothing until the script finishes. Commented Aug 24, 2020 at 23:13
  • Also just tried adding http header X-Accel-Buffering: no to the output of the cgi script, also did not help. Docs for that nginx.com/resources/wiki/start/topics/examples/x-accel/… Commented Aug 24, 2020 at 23:37

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.