1

I have servers with huge daily size of log files ~ 1GB. I want to do daily logrotate at midnight. The problem is for each rotated file missing first 17-25 minutes after midnight and same file contains 17-25 minutes data for next day.

My question is how to do properly logrotate for each day and data for that day must be from midnight to midnight.

Server has a few services and each service has 100MB-1,5GB daily file as apache2, lighttpd, ftp, etc.

less /var/log/lighttpd/access.log.1

np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abc.xml HTTP/1.1" 2422 200 "domainexample/1.3 CFNetwork/596.2.3 Darwin/12.2.0 (x86_64) (iMac9%2C1)" "-" np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abc.xml HTTP/1.1" 2422 200 "domainexample/1.3 CFNetwork/596.2.3 Darwin/12.2.0 (x86_64) (iMac12%2C1)" "-" np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abr.xml HTTP/1.1" 0 304 "hhh%20Radio%20Umee/616 CFNetwork/548.1.4 Darwin/11.0.0" "-" np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abr.xml HTTP/1.1" 2433 200 "NobexRadio" "-" np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abr.xml HTTP/1.1" 2433 200 "domainexample/1.3 CFNetwork/520.4.3 Darwin/11.4.0 (x86_64) (iMac11%2C2)" "-"

tail /var/log/lighttpd/access.log.1

np.domainexample.co.uk 11.22.33.44 - [17/Oct/2012:00:21:56 +0100] "GET /nowplaying/aim_abr.xml HTTP/1.1" 2121 200 "domainexample/1.3 CFNetwork/520.5.1 Darwin/11.4.2 (x86_64) (iMac12%2C2)" "-" exam.as5623.net 11.22.33.44 - [17/Oct/2012:00:21:56 +0100] "GET /oconnell/20121012170055.mp3 HTTP/1.1" 78118 206 "AppleCoreMedia/1.0.0.10A403 (iPod; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" exam.as5623.net 11.22.33.44 - [17/Oct/2012:00:21:56 +0100] "GET /oconnell/20121001174703.mp3 HTTP/1.1" 78120 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_us)" "-" exam.as5623.net 11.22.33.44 - [17/Oct/2012:00:21:56 +0100] "GET /frankskinner/20120905121827.mp3 HTTP/1.1" 78117 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" np.domainexample.co.uk 11.22.33.44 - [17/Oct/2012:00:21:56 +0100] "GET /nowplaying/aim_ab8.xml HTTP/1.1" 0 304 "hhh%2080s/616 CFNetwork/609 Darwin/13.0.0" "-"

bzless /var/log/lighttpd/access.log.2.bz2

exam.as5623.net 11.22.33.44 - [15/Oct/2012:00:17:03 +0100] "GET /dave_gorman/20121014130335.mp3 HTTP/1.1" 78117 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_us)" "-" np.domainexample.co.uk 11.22.33.44 - [15/Oct/2012:00:17:03 +0100] "GET /nowplaying/aim_ab8.xml HTTP/1.1" 2190 200 "hhh/1 CFNetwork/520.4.3 Darwin/11.4.0 (x86_64) (iMac12%2C1)" "-" exam.as5623.net 11.22.33.44 - [15/Oct/2012:00:17:03 +0100] "GET /frankskinner/20121013132200.mp3 HTTP/1.1" 284941 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" exam.as5623.net 11.22.33.44 - [15/Oct/2012:00:17:03 +0100] "GET /dave_gorman/20121014130335.mp3 HTTP/1.1" 78117 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_us)" "-" exam.as5623.net 11.22.33.44 - [15/Oct/2012:00:17:03 +0100] "GET /dave_gorman/20121014130335.mp3 HTTP/1.1" 78117 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_us)" "-"

bztail /var/log/lighttpd/access.log.2.bz2

np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abr.xml HTTP/1.1" 2433 200 "hhh/1.3 CFNetwork/454.12.4 Darwin/10.8.0 (i386) (MacBookPro5%2C5)" "-" exam.as5623.net 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /dave_gorman/20121014130335.mp3 HTTP/1.1" 41486 206 "AppleCoreMedia/1.0.0.10A403 (iPod; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" exam.as5623.net 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /frankskinner/20121013132200.mp3 HTTP/1.1" 78377 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" np.domainexample.co.uk 11.22.33.44 - [16/Oct/2012:00:18:20 +0100] "GET /nowplaying/aim_abc.xml HTTP/1.1" 0 304 "Classic%20Rock/616 CFNetwork/609 Darwin/13.0.0" "-"

bzless /var/log/lighttpd/access.log.3.bz2

np.domainexample.co.uk 11.22.33.44 - [14/Oct/2012:00:19:03 +0100] "GET /nowplaying/aim_ab0.xml HTTP/1.1" 0 304 "hhh%2080s/616 CFNetwork/609 Darwin/13.0.0" "-" exam.as5623.net 11.22.33.44 - [14/Oct/2012:00:19:03 +0100] "GET /oconnell/20120920182425.mp3 HTTP/1.1" 78118 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_us)" "-" np.domainexample.co.uk 11.22.33.44 - [14/Oct/2012:00:19:03 +0100] "GET /nowplaying/aim_ab0.xml HTTP/1.1" 0 304 "hhh%2080s/616 CFNetwork/548.1.4 Darwin/11.0.0" "-" np.domainexample.co.uk 11.22.33.44 - [14/Oct/2012:00:19:03 +0100] "GET /nowplaying/aim_abc.xml HTTP/1.1" 0 304 "hhh%20Radio%20Umee/616 CFNetwork/609 Darwin/13.0.0" "-" np.domainexample.co.uk 11.22.33.44 - [14/Oct/2012:00:19:03 +0100] "GET /nowplaying/aim_ab9.xml HTTP/1.1" 0 304 "hhh%20Radio%20Umee/616 CFNetwork/548.1.4 Darwin/11.0.0" "-"

bztail /var/log/lighttpd/access.log.3.bz2

np.domainexample.co.uk 11.22.33.44 - [15/Oct/2012:00:17:02 +0100] "GET /nowplaying/ab7.xml HTTP/1.1" 1295 200 "PHP/5.2.6" "-" np.domainexample.co.uk 11.22.33.44 - [15/Oct/2012:00:17:02 +0100] "GET /tracks/7869.xml HTTP/1.1" 1998 200 "PHP/5.2.6" "-" exam.as5623.net 11.22.33.44 - [15/Oct/2012:00:17:02 +0100] "GET /frankskinner/20121013132200.mp3 HTTP/1.1" 230649 206 "AppleCoreMedia/1.0.0.10A403 (iPhone; U; CPU OS 6_0 like Mac OS X; en_gb)" "-" exam.as5623.11.22.33.44 - [15/Oct/2012:00:17:02 +0100] "GET /frankskinner/20121013132200.mp3 HTTP/1.1" 78377 206 "AppleCoreMedia/1.0.0.10A403 (iPad; U; CPU OS 6_0 like Mac OS X; en_us)" "-" np.domainexample.co.uk 11.22.33.44 - [15/Oct/2012:00:17:02 +0100] "GET /nowplaying/abx.xml HTTP/1.1" 314 200 "PHP/5.2.6" "-"

cat /etc/logrotate.d/lighttpd

/var/log/lighttpd/*.log { daily missingok rotate 999 compresscmd /bin/bzip2 compressext .bz2 compress delaycompress notifempty copytruncate sharedscripts postrotate if [ -f /var/run/lighttpd.pid ]; then \ if [ -x /usr/sbin/invoke-rc.d ]; then \ invoke-rc.d lighttpd force-reload > /dev/null; \ else \ /etc/init.d/lighttpd force-reload > /dev/null; \ fi; \ fi; endscript }

ls /etc/logrotate.d/

apache2 apt aptitude dpkg lighttpd ocsinventory-agent ppp rsyslog.disabled syslog-ng vsftpd

Cheers,

E

Improve this question

1 Answer 1

Reset to default
0

Do you really need from midnight-to-midnight without any log-entry of the previous/following day?

Then you need to do post-processing of the log-files where you cut'n'paste your older logs to the right borders.

If you could time the run of your logrotation better, you are always running in race conditions between rotating all you logfiles and users accessing your servers at the same time, so for perfect one-day-logfiles you need post processing...

Improve this answer
1
  • > Do you really need from midnight-to-midnight without any log-entry of > the previous/following day? Yes We have 999 log rotated files. Problem is only files contains data with offset 15-25 min from midnight current day and after midnight next day. We need daily log rotated file with data from 00:00:00 to 23-59:59 compressed in GZ format. Best, E Commented Nov 30, 2012 at 14:13

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.