0

We have an internal app and I want to have the apache httpd server create new log files based on each of the client ips. In the documentation I see you can have multiple logs.

http://httpd.apache.org/docs/2.2/logs.html

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common CustomLog logs/referer_log "%{Referer}i -> %U" CustomLog logs/agent_log "%{User-agent}i" 

but what I really want is something like this.

CustomLog logs/%h-access_log common 

where the %h is extrapolated into the ip address i.e. 1.1.1.1-access_log

I imagine there's a mod_perl way to do this but just wondering if anyone knows a way to do something like this through standard configuration before I write a module.

2
  • Are you sure you can not achieve things another way, like postprocessing Apache log files ? Because having one logfile per client IP (are you thinking about IPv6 too ?) may mean a lot of (possibly tiny) files in the same directory, and various filesystems will have trouble managing tens or hundreds of thousands of files in the same directory. You may also hit the limit on the number of open files per (Apache) process. Commented Mar 11, 2017 at 19:43
  • Since it's an internal application we might have 1000 different servers posting to it at the most. Some of the servers post millions of records a day and some post 2 or 3. We need them to be separated so we can rotate the larger ones as necessary so we have at least the last 1000 transactions in the ip specific log file. Commented Mar 12, 2017 at 21:22

1 Answer 1

5
+50

Use the Apache log pipe feature: logs can be sent to an outside process. That process will have full latitude to filter things as it wants and create files as needed. See http://httpd.apache.org/docs/2.2/logs.html#piped

You will just need to write the small script that parse the log input to extract the IP address and forward the line to the appropriate file.

Otherwise if you sort of control the amount of clients, you could define all needed files beforehand, and then using conditional logging, through specific variables, see this example at the same URL:

# Mark requests from the loop-back interface SetEnvIf Remote_Addr "127\.0\.0\.1" dontlog # Mark requests for the robots.txt file SetEnvIf Request_URI "^/robots\.txt$" dontlog # Log what remains CustomLog logs/access_log common env=!dontlog 

But even with your latest comment, I still think postprocessing is a better route. YMMV.

4
  • I'd rather not have to spawn a process for each request. (I'm assuming that is what log piping does.) But if I don't get another answer before the bounty expires then I'll award it to you. Commented Mar 13, 2017 at 21:06
  • 2
    No the process is started once and gets data on STDIN continously. As written in the webpage : " Apache will start the piped-log process when the server starts, and will restart it if it crashes while the server is running. (This last feature is why we can refer to this technique as "reliable piped logging".)" Commented Mar 13, 2017 at 21:07
  • Also note: "As with conditional logging, piped logs are a very powerful tool, but they should not be used where a simpler solution like off-line post-processing is available." :-) Commented Mar 13, 2017 at 21:08
  • Didn't realized I didn't award the bounty when I marked this as the correct answer. Commented Mar 18, 2017 at 21:35

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.