Bug 47575 - GZipping large files stops logging
Summary: GZipping large files stops logging
Status: NEEDINFO
Alias: None
Product: Log4j - Now in Jira
Classification: Unclassified
Component: Appender (show other bugs)
Version: unspecified
Hardware: PC Linux
: P2 normal
Target Milestone: ---
Assignee: log4j-dev
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-07-24 14:54 UTC by Derek Libby
Modified: 2009-08-04 20:50 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Derek Libby 2009-07-24 14:54:52 UTC
The following block:

      synchronized (this) {
        //
        //   if a previous async task is still running
        //}
        if (lastRolloverAsyncAction != null) {
          //
          //  block until complete
          //
          lastRolloverAsyncAction.close();

          //
          //    or don't block and return to rollover later
          //
          //if (!lastRolloverAsyncAction.isComplete()) return false;
        }

from RolingFileAppender.java means that the next LoggingEvent which triggers a TriggerPolicy::isTriggeringEvent after the gzip compression has begun will cause all logging threads to block.

The:

    if (lastRolloverAsyncAction != null) {

check should not be performed until the NEXT rollover is called for.  That is after:

    if (rollover != null) {
Comment 1 Curt Arnold 2009-08-04 20:30:44 UTC
FYI: This is regarding o.a.l.rolling.RFA in the extras companion,

Seems like it should be very rare for the stock rolling policies to return null for rollover().  Moving the block for completion of async tasks until after if (rollover != null) would seem to insignificantly delay the blocking,

There is some commented out code indicating a developer considered failing the rollover instead of blocking.  That would seem to be a more desirable approach.  Maybe it could be configurable?

Would appreciate knowing more about your situation and the benefit you saw from changing the location of the block.
Comment 2 Derek Libby 2009-08-04 20:50:32 UTC
We had accumulated a large log file approximately 10GB in size.  Just using gzip on the (Linux) box in question required about 2 minutes to compress the file.  The rollover check happens then next time a logging event comes in and the check interval has been exceeded which for our purposes meant essentially the next time the check interval was exceeded (1000ms I believe).  All of our threads then blocked until the GZip operation was complete causing the system to stop responding for that interval.

I didn't actually move the block in question, but the only time it makes sense to block on lastRolloverAsyncAction is when its value is about to change which will not be until the next log roll is required which was midnight in our application.