The cronolog time bomb
I got another Sunday afternoon error notification from $ORK today, this about a high-profile web server running out of disk.
The Apache configuration has this (paths tweaked to protect the guilty):
CustomLog "|/usr/sbin/cronolog -S /var/log/httpd/site-name_access_log \
/var/log/httpd/%Y/%m/%d/site-name_access_log" combined
Pros and cons of cronolog:
The Apache configuration has this (paths tweaked to protect the guilty):
CustomLog "|/usr/sbin/cronolog -S /var/log/httpd/site-name_access_log \
/var/log/httpd/%Y/%m/%d/site-name_access_log" combined
Pros and cons of cronolog:
- CON: this is a ticking time bomb, keeping around uncompressed log files indefinitely
- CON: uses external cronolog utility
- CON: requires additional unspecified homebrew process to remove old logs (find + delete maybe?)
- CON: two additional open pipes per vhost (one for access, one for error)
- PRO: log files are broken down by day
- CON: log files are broken down by day (does anyone really need this?)
- CON: no builtin support for log compression
Pros and cons of using logrotate:
- CON: uses external logrotate utility
- PRO: can rotate by date or size
- PRO: can mail sysadmin on rotation
- PRO: can manage the number of backup files kept
- PRO: can manage the size of backup files kept
- PRO: options to label log backups with dates or integers
Conclusions
I'm not going to candy-coat it--I got an email from a system in danger today because it was using cronolog with no real log rotation, and /var/ was 95% full due to log files being kept around forever.
I'd call it at best irresponsible and at worst malicious to do this to your coworkers. Unless you plan on working a lot more than 40 hours a week, chances are the failures are going to happen when you are off work. This violates one of my cardinal rules of work:
I'd call it at best irresponsible and at worst malicious to do this to your coworkers. Unless you plan on working a lot more than 40 hours a week, chances are the failures are going to happen when you are off work. This violates one of my cardinal rules of work:
Don't make your coworkers suffer.
Links
- https://serverfault.com/questions/20337/cronolog-vs-logrotate
- https://github.com/fordmason/cronolog
- https://linux.die.net/man/8/logrotate
- https://github.com/logrotate/logrotate
- https://stackoverflow.com/questions/44048/best-way-to-rotate-apache-log-files
- https://serverfault.com/questions/411763/compress-logs-rotated-with-cronolog
Comments