(This is ported from old blog.)

I had been frustrated for probably 15 years or so about the logging output of syslong on my FreeBSD server.

I did not do much to fix it. It is something you can ignore, but it was frustrating nonetheless.

Last night, after seeing

Sep 23 23:08:22 nile imap[95035]: IOERROR: opening /servers/cyrus/imap/user_deny.db: No such file or directory
Sep 23 23:08:22 nile imap[73920]: IOERROR: opening /servers/cyrus/imap/user_deny.db: No such file or directory
Sep 23 23:08:22 nile last message repeated 23 times

in /var/log/debug.log, cluttered by this logging, I had enough. I need to take the control back from all thouse wacky syslogging.

in /usr/ports/sysutils/, there are a few alternatives, and I decided the first one I see, syslog-ng.

The result was a great success. So, if you want to tidy up the /var/log/*, I hope this helps.

Here is what I wanted to do.

  1. I don’t need to change much from what it is now.
  2. I, however want all of email related message to go into /var/log/maillog
  3. That means that I have to filter out the entries based on the application name, not the syslog facility.

I use Cyrus IMAP server and Cyrus does not fall into “mail” syslog facility, therefore, the log entry goes all sorts of places. This has been annoying me for over a decade. (and I’m lazy enough to not do anything about it.)

Using portinstall -PP sysutils/syslog-ngI’m ready to go. I made the following config file, /usr/local/etc/syslog-ng.conf.

It’s pretty close to the sample file, but here is my tweak.

  • I don’t want any of Cyrus logs to go into any of logs except /var/log/maillog.
  • I do want to see lmtp logging in /var/log/maillog.

So, I created a few filters.

filter f_not_cyrus { not program("(imap.*|ctl_cyrusdb|master|lmtp)"); };
filter f_cyrus { program("(imap.*|ctl_cyrusdb|master|lmtp)"); };
filter f_lmtp { program("lmtp"); };

f_not_cyrus is used to block the Cyrus application to send the log to /var/log/message.

log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_kern); filter(f_debug); destination(messages); };
log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_lpr); filter(f_info); destination(messages); };

This reads – for all evetns, add entry to “message” only when the application is not cyurs “filter(f_not_cyrus)”.

f_not_cyrus is shown above. It is dead simple.

The rest is that, I want to log the Cyrus and all mail related logging to /var/log/maillog.

log { source(events); filter(f_cyrus); filter(f_notice); destination(maillog); };
log { source(events); filter(f_lmtp); destination(maillog); };
log { source(events); filter(f_mail); destination(maillog); };

So, this reads, if it’s Cyrus, and notice or above, log. If you are lmtp, log. All “mail” facility logging to into maillog.

Again, this is really simple, yet I don’t know how I can do this with the syslog that comes with FreeBSD.

Here is the entire syslog-ng.conf file.


@version:3.0
#
# options
#
options { long_hostnames(off); flush_lines(0); };

#
# sources
#
source events {
unix-dgram("/var/run/log");
unix-dgram("/var/run/logpriv" perm(0600));
udp();
internal();
file("/dev/klog");
};

#
# destinations
#
destination messages { file("/var/log/messages"); };
destination security { file("/var/log/security"); };
destination authlog { file("/var/log/auth.log"); };
destination maillog { file("/var/log/maillog"); };
destination lpd-errs { file("/var/log/lpd-errs"); };
destination xferlog { file("/var/log/xferlog"); };
destination cron { file("/var/log/cron"); };
destination debuglog { file("/var/log/debug.log"); };
destination consolelog { file("/var/log/console.log"); };
destination all { file("/var/log/all.log"); };
destination newscrit { file("/var/log/news/news.crit"); };
destination newserr { file("/var/log/news/news.err"); };
destination newsnotice { file("/var/log/news/news.notice"); };
destination slip { file("/var/log/slip.log"); };
destination ppp { file("/var/log/ppp.log"); };
destination console { file("/dev/console"); };
destination allusers { usertty("*"); };
destination loghost { udp("loghost" port(514)); };

#
# log facility filters
#
filter f_auth { facility(auth); };
filter f_authpriv { facility(authpriv); };
filter f_not_authpriv { not facility(authpriv); };
filter f_console { facility(console); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_ftp { facility(ftp); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail); };
filter f_not_mail { not facility(mail); };
filter f_news { facility(news); };
filter f_security { facility(security); };
filter f_user { facility(user); };
filter f_uucp { facility(uucp); };
filter f_local0 { facility(local0); };
filter f_local1 { facility(local1); };
filter f_local2 { facility(local2); };
filter f_local3 { facility(local3); };
filter f_local4 { facility(local4); };
filter f_local5 { facility(local5); };
filter f_local6 { facility(local6); };
filter f_local7 { facility(local7); };

#
# log level filters
#
filter f_emerg { level(emerg); };
filter f_alert { level(alert..emerg); };
filter f_crit { level(crit..emerg); };
filter f_err { level(err..emerg); };
filter f_warning { level(warning..emerg); };
filter f_notice { level(notice..emerg); };
filter f_info { level(info..emerg); };
filter f_debug { level(debug..emerg); };
filter f_is_debug { level(debug); };

#
# program filters
#
filter f_ppp { program("ppp"); };
filter f_slip { program("startslip"); };

filter f_cyrus { program("(imap.*|ctl_cyrusdb|master|lmtp|cyr_*|tls_prune)"); };
filter f_not_cyrus { not filter(f_cyrus); };
filter f_lmtp { program("lmtp"); };

filter f_afpd { program("afpd"); };
filter f_not_afpd { not filter(f_afpd); };

#
# *.err;kern.warning;auth.notice;mail.crit /dev/console
#
log { source(events); filter(f_not_cyrus); filter(f_err); destination(console); };
log { source(events); filter(f_not_cyrus); filter(f_kern); filter(f_warning); destination(console); };
log { source(events); filter(f_not_cyrus); filter(f_auth); filter(f_notice); destination(console); };
log { source(events); filter(f_not_cyrus); filter(f_mail); filter(f_crit); destination(console); };

#
# *.notice;authpriv.none;kern.debug;lpr.info;news.err /var/log/messages
#
log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_notice); filter(f_not_authpriv); destination(messages); };
log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_kern); filter(f_debug); destination(messages); };
log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_lpr); filter(f_info); destination(messages); };
log { source(events); filter(f_not_cyrus); filter(f_not_mail); filter(f_news); filter(f_err); destination(messages); };

#
# security.* /var/log/security
#
log { source(events); filter(f_security); destination(security); };

#
# auth.info;authpriv.info /var/log/auth.log
log { source(events); filter(f_auth); filter(f_info); destination(authlog); };
log { source(events); filter(f_authpriv); filter(f_info); destination(authlog); };

#
# mail.* /var/log/maillog
#
log { source(events); filter(f_cyrus); filter(f_notice); destination(maillog); };
log { source(events); filter(f_lmtp); destination(maillog); };
log { source(events); filter(f_mail); destination(maillog); };

#
# lpr.info /var/log/lpd-errs
#
log { source(events); filter(f_lpr); filter(f_info); destination(lpd-errs); };

#
# ftp.info /var/log/xferlog
#
log { source(events); filter(f_ftp); filter(f_info); destination(xferlog); };

#
# cron.* /var/log/cron
#
log { source(events); filter(f_cron); destination(cron); };

#
# *.=debug /var/log/debug.log
# except imap, afpd
#
log { source(events); filter(f_is_debug); filter(f_not_cyrus); filter(f_not_afpd); destination(debuglog); };

#
# *.emerg *
#
log { source(events); filter(f_emerg); destination(allusers); };

#
# uncomment this to log all writes to /dev/console to /var/log/console.log
# console.info /var/log/console.log
#
#log { source(events); filter(f_console); filter(f_info); destination(consolelog); };

#
# uncomment this to enable logging of all log messages to /var/log/all.log
# touch /var/log/all.log and chmod it to mode 600 before it will work
# *.* /var/log/all.log
#
#log { source(events); destination(all); };

#
# uncomment this to enable logging to a remote loghost named loghost
# *.* @loghost
#
#log { source(events); destination(loghost); };

#
# uncomment these if you're running inn
# news.crit /var/log/news/news.crit
# news.err /var/log/news/news.err
# news.notice /var/log/news/news.notice
#
#log { source(events); filter(f_news); filter(f_crit); destination(newscrit); };
#log { source(events); filter(f_news); filter(f_err); destination(newserr); };
#log { source(events); filter(f_news); filter(f_notice); destination(newsnotice); };

#
# !startslip
# *.* /var/log/slip.log
#
log { source(events); filter(f_slip); destination(slip); };

#
# !ppp
# *.* /var/log/ppp.log
#
log { source(events); filter(f_ppp); destination(ppp); };

It worked for a day, and the logs all stop. I scratched my head. After another round of googling, the syslogd process needs a singal, but the default installation of syslog-ng creates a pid file as “/var/run/syslog-ng.pid”, not “/var/run/syslog.pid” which FreeBSD’s log rotation look for. You can change either the log rotation, or syslog-ng’s setting, and I took the syslog-ng’s setting route. I put following lines into /etc/rc.conf

#
# syslogd is replaced with syslog-ng
#
syslogd_enable="NO"
#
syslog_ng_enable="YES"
syslog_ng_config="-u root"
syslog_ng_pid="/var/run/syslog.pid"
This really cleaned up the logging. No more annoying warnings from Cyrus, and I can really tell what’s going on with fetchmail/exim/cyrus imap combination by just looking at /var/log/maillog.
I probably want to tweak some more, like some of Cyrus utilities are still logged in /var/log/messages, but overall, my frustration level is now minimul. It was a good couple of hours of investment to finally solve the frustration of 15 years.