Bug 51081

Summary: [PATCH] RotateLogs.exe with less granular time formatted names keep growing log files thereby exhausting disk space
Product: Apache httpd-2 Reporter: Thangaraj AntonyCrouse <thangaraj>
Component: Other ModulesAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED INVALID    
Severity: normal CC: thangaraj
Priority: P2 Keywords: PatchAvailable
Version: 2.2.17   
Target Milestone: ---   
Hardware: PC   
OS: Windows Server 2003   
Attachments: Fix for RotateLogs.exe to overwrite logs having less granular time formatted names

Description Thangaraj AntonyCrouse 2011-04-18 22:11:59 UTC
Created attachment 26900 [details]
Fix for RotateLogs.exe to overwrite logs having less granular time formatted names

Problem:
RotateLogs.exe with less granular time formatted names keep growing thereby exhausting disk space. 


Reproduction Steps:

1. Edit httpd.conf file to contain following lines

LogLevel debug
ErrorLog  "|| bin/rotatelogs.exe logs/error-%Z.log 1M"


2. Run Apache for few hours.

Actual Result-  Log file size grows beyond 1 MB

Expected Result- Log files should not grow beyond 1 MB, it should get overwitten.

Solution:
Reset nAppend flag in RotateLogs.c such that size based rotation will overwrite existing file thereby honoring the given Size limit as stated in http://httpd.apache.org/docs/2.2/programs/rotatelogs.html
Copying HTML exerpt here: When using strftime(3) filename formatting, be sure the log file format has enough granularity to produce a different file name each time the logs are rotated. Otherwise rotation will overwrite the same file instead of starting a new one. For example, if logfile was "/var/logs/errorlog.%Y-%m-%d" with log rotation at 5 megabytes, but 5 megabytes was reached twice in the same day, the same log file name would be produced and log rotation would keep writing to the same file.
Comment 1 William A. Rowe Jr. 2011-04-19 11:15:14 UTC
I'd vote -1 on this patch.

rotatelogs is designed to be nondestructive, and you propose to add destructive
behavior to the log records.
Comment 2 Eric Covener 2011-04-19 13:03:58 UTC
(In reply to comment #1)
> I'd vote -1 on this patch.
> 
> rotatelogs is designed to be nondestructive, and you propose to add destructive
> behavior to the log records.

this alarmed me as well, if you don't add reasonable granularity to the filename, letting it grow until someone notices is better than truncating it periodically.
Comment 3 Thangaraj AntonyCrouse 2011-04-19 13:27:44 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > I'd vote -1 on this patch.
> > 
> > rotatelogs is designed to be nondestructive, and you propose to add destructive
> > behavior to the log records.
> this alarmed me as well, if you don't add reasonable granularity to the
> filename, letting it grow until someone notices is better than truncating it
> periodically.

Appreciate your comments.

This path provides the best of both worlds - non-destructive and destructive modes of operation. 

If users don't like destructive mode - they can use large file size option.
If users like me don't want Server disk space to be exhausted and prefer log overwriting they can use small file size as option with less granular time format (as mentioned in docs: http://httpd.apache.org/docs/2.2/programs/rotatelogs.html).

If Rotatelogs is meant to be non-destructive always - I'm trying to understand then why do we even have filesize option?
Comment 4 William A. Rowe Jr. 2011-04-19 13:36:01 UTC
> If Rotatelogs is meant to be non-destructive always - I'm trying to understand
> then why do we even have filesize option?

rotatelogs file size option is intended to allow the user to roll over at a given
limit, for archival purposes; not to destroy the log.  I would suggest that if
you want a destructive / sampling behavior as opposed to a record of operations,
then a new utility would be more appropriate than modifying rotatelogs.

Based on two committers negative feedback to destructive operation, I'm going
to mark this INVALID for now as the documentation is very clear, but further
httpd developer comments are welcome.

"When using strftime(3) filename formatting, be sure the log file format has enough granularity to produce a different file name each time the logs are rotated. Otherwise rotation will overwrite the same file instead of starting a new one. For example, if logfile was /var/logs/errorlog.%Y-%m-%d with log rotation at 5 megabytes, but 5 megabytes was reached twice in the same day, the same log file name would be produced and log rotation would keep writing to the same file."