Blogumé part deux: Log Rotation

I spent the morning at work:

1. rotating neglected logs on our production web and database servers
2. setting up logrotate scripts to keep the logs fresh
   

Oddly enough, I can't seem to find a good online definition of "log rotation" for linking. Wikipedia apparently has none, and the online dictionaries are all failing me, so here's one I came across at http://blog.loglogic.com/2007/05/logging_glossary_log_rotation/:

Log Rotation

A log management process where logs are moved, renamed, or over written for system resource management and archival.

Rotation is usually based on time or size. Rotation usually includes some form of log identification. Log identification may take a number of forms, all intended to represent the current log, next current log, and so on up to some number of logs. When reaching a limit to the number of logs, the oldest log is removed before the current log is created.

Why is this important? Because without regular log rotation:

• Your logs will grow too big to use. In my environment, a file bigger than about 3 GB is useless. The file I was trying to examine was over 30 GB. That's 30,000,000,000 bytes of logs.
  
• If totally neglected, your logs can fill their partition. At best logging stops; at worst the application itself stops (typically around 3 AM some Sunday morning).

Log rotation is such a common task that utilities (e.g. logrotate) exist, with options and hooks to handle all sorts of general log rotation scenarios. If nothing else, I've made it more likely that when I need to look at a log file in the future that there's actually something there I can work with.

Update: just checked on the state of the logs on the production webservers, a bit more than a week after doing this. It's so much better.