rsyslog from Heroku drain creates empty log files

Question

I'm sending logs from my Heroku app to an rsyslog server, but the resulting log files seem to come up empty. The rsyslog configuration for receiving remote messages is as follows:

$template RemoteDailyLog,"/var/log/remote/%hostname%/%$year%/%$month%/%$day%.log" :fromhost-ip, !isequal, "127.0.0.1" -?RemoteDailyLog & ~ 

My complete rsyslog configuration is available in this paste.

This configuration appears to create the directories correctly. I see the Heroku app's logging hostname (of the form "d.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx") appear in /var/log on the rsyslog host, which implies that log messages are successfully making it to the logging daemon, but the resulting logfiles are zero-size.

I'm guessing the issue is with rsyslog, rather than Heroku, but I'm not sure where to look next.

Answer 1

After examining the server's /var/log/syslog, I discovered that rsyslog was having problems opening dynamic files. It turns out that the default Ubuntu 10.04 LTS config for rsyslog is not adequate for dynamic (template-based) log files. The proposed patch, due to Richard Fleming, is as follows:

--- /etc/rsyslog.conf.orig 2009-11-17 11:21:10.874573462 -0500 +++ /etc/rsyslog.conf 2009-11-17 12:45:15.604573200 -0500 @@ -42,9 +42,11 @@ $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 +$DirOwner syslog +$DirGroup adm $Umask 0022 $PrivDropToUser syslog -$PrivDropToGroup syslog +$PrivDropToGroup adm # # Include all config files in /etc/rsyslog.d/ 

For more information, see: https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/484336