Bug 46941

Summary: Sub Level Logging Technique
Product: Log4j - Now in Jira Reporter: Roger Tannous <roger77_lb>
Component: OtherAssignee: log4j-dev <log4j-dev>
Status: NEW ---    
Severity: enhancement CC: roger77_lb
Priority: P1    
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   

Description Roger Tannous 2009-03-31 02:49:22 UTC
While improving code for a logger at hand I found it useful to define my own levels to be suited to every application I have. That is, different logging levels defined for different applications.

Moreover, I needed to be able to log a certain type of events no matter what the chosen level was (see logger.transparentLevels below).

At the beginning, I created a new config parameter as:

logger.referenceDescendingLevels=err,inf,deb

which means that err (error) > inf (information) > deb (debug)

which is practically the same story as Apache Log4J' Basic Selection Rule.

To be able to log CDR (Call Detail Records) type events, I have created a new config parameter as: 

logger.transparentLevels=cdr

(we can add / modify / displace levels in both parameters according to our needs)

The name 'transparent' was chosen to indicate that the levels listed in logger.transparentLevels are transparent to the Selection Rule ! A request with a level listed here passes the Selection Rule no matter what other application logging settings are.


Till now, we have defined a Reference list and a Transparent List of levels, we just need one thing, a chosen log level which will define the current logging settings of our application:

logger.chosenLogLevel=inf.5


For the currently defined reference logging levels, values could be as listed below:

err (same as err.0)
inf (same as inf.0)
deb (same as deb.0)
err.2 (any number)
inf.3 (any number)
deb.8 (any number)


I will explain what is the .5 for (in the logger.chosenLogLevel config) below:

The next interesting feature I've added is to be able to use Sub Levels into each level in an arbitrary fashion using a simple approach: the numbering system !

logger.chosenLogLevel=deb.4 means:
log all log requests (Logger.Log("deb.4", "This is your log request");) till level deb and sub level 4
Examples: 
     1. a request with deb.10 will be ignored.
     2. a request with deb.3 / deb.2 / deb.1 / deb / inf / inf.2 / inf.10000, etc. / err.4, etc. / err will certainly be served.


Note that the technique illustrated here can be used in a way to enable debugging for an application deployed on a live server, and be able to do further debugging (for instance, by specifying logger.chosenLogLevel=deb.7) when a problem is to be debugged. Of course, software developers are required to include adequate logging requests wherever useful it might be.

Of course, software developers are invited to plan their logging software architecture so that they define which 'things' are to be considered at which 'level', 'sub level'; and code their sub levels in the most appropriate way, such as to use 'sub level' numbers in a meaningful way. 

For this purpose, I propose to use SubLevel classes where useful sub levels are defined as:

public static int sub_level_live = 0;

public static int sub_level_deb = 0;

public static int sub_level_deb_transactions_summary = 1;

public static int sub_level_deb_transactions_full = 2;

public static int sub_level_deb_show_low_level_handshaking = 3;


Those sub levels definitions should be suitable for your application logic.

Can this be implemented in Log4J ? If it already can be done, how is that ? Can you provide a clear example please ?

If not, what about adding it to Log4J's TODO list ?

Best Regards,
Roger Tannous
Comment 1 Roger Tannous 2009-04-02 05:36:28 UTC
(In reply to comment #0)

The 'requirement' is modified (improved) as described below:


logger.referenceDescendingLevels=err,general_info,(connection_status,handshaking),deb

As you can see, we can put sets of levels into parentheses to indicate that we wish the 'grouped levels' to be considered in the same 'logical level'.

For the example above, connection_status and handshaking both belong to the same 'logical level' 2 (levels are 0 based in my implementation).

Of course, we can include more than one group of parentheses like in the below:

logger.referenceDescendingLevels=a,(b,c,d),e,(f,g),h,i

LEVEL     LOGICAL_LEVEL
a         0
b         1
c         1
d         1
e         2
f         3
g         3
h         4
i         5


Furthermore, we can include levels in "full format" into this config value, but this has one drawback, continue reading...

logger.referenceDescendingLevels=a,b,(c,d),(e,e.1,e.2,e.3),e.4,e.5,f

Once e,e.1,e.2,e.3 are defined in 'logical level' 3, we are obliged to state all other available levels belonging to the level named 'e' in "full format" (currently: e.4 and e.5) to prevent them from being missed !
Comment 2 Curt Arnold 2009-04-02 19:28:08 UTC
This really seems like you are trying to use levels to accomplish what the logger hierarchy was designed to do.  Many people make the assumption that since a common pattern for logger names is to follow class names that is the only way they can be used.  Logger names are, in general, are "topics" and you can use any system that works to give you the type of control that you'd want.

If you are trying to assign a "topic" (network, database, etc) to a level, then you not messing up the "topic" or "audience" concept up with the "severity" concept.  Logger names are really flexible and you can likely accomplish you goals without nearly as many complications.
Comment 3 Ralph Goers 2009-04-02 22:20:35 UTC
I would suggest that you take a look at the Marker support in SLF4J and Logback and see if that would meet your needs. If so I would suggest asking for that feature in a future release instead of sub levels.
Comment 4 Roger Tannous 2009-04-03 02:57:42 UTC
(In reply to comment #2)
> This really seems like you are trying to use levels to accomplish what the
> logger hierarchy was designed to do.  Many people make the assumption that
> since a common pattern for logger names is to follow class names that is the
> only way they can be used.  Logger names are, in general, are "topics" and you
> can use any system that works to give you the type of control that you'd want.
> 
> If you are trying to assign a "topic" (network, database, etc) to a level, then
> you not messing up the "topic" or "audience" concept up with the "severity"
> concept.  Logger names are really flexible and you can likely accomplish you
> goals without nearly as many complications.


Using sub-levels is quite different from the standard logger hierarchy in that a new sub level can be introduced without being defined (because it's already a numeric value !)

For example, if we have already defined our logging by: 

logger.referenceDescendingLevels=a,b,(c,d),e,f

We can use in the code: 

Logger.Log("e.11", "log text here");

without having to define sub-level 11 for logging level 'e' anywhere.
(<start_of_parenthesis>
I'm using here a direct String value as: "e.11", but in reality, you can have a class where levels are defined like: 
public static String LOG_LEVEL_DATABASE_CONNECTION = "e";
public static String LOG_LEVEL_DATABASE_QUERY = "f";

then use: 

Logger.Log(LOG_LEVEL_DATABASE_CONNECTION + ".11", "DB connection logging here");
// Remember, I call this Logger.Log(...); function call a " log request " or a " logging request "; you'll find this name in the text below.
<end_of_parenthesis>
)


Additionally, suppose you have defined your levels and your application is up and running, then after a certain time you need a new logging "level" and need to separate its logging from the rest of the 'logs'; without the proposed technique, you have to 'insert' your new level _in_between_ existing ones. Doesn't this mean that you have to do additional work as you'll have to modify all " logging requests " that are usually "below" the newly introduced one ?


At the contrary, using the proposed technique, all you have to do is add the " logging requests " and add your new entry ANYWHERE in the logger.referenceDescendingLevels like:

logger.referenceDescendingLevels=a,b,MY_NEW_LEVEL,(c,d),e,f

Now within MY_NEW_LEVEL, you can use as many sub-levels as you need without having to define them here.


Finally, in the proposed technique, the method does not oblige you to abide by a certain rule as how to design levels and sub-levels. Most importantly, it doesn't tell sub-levels are strictly to be considered as severity ! This can be regarded as your own application design issue.

You can use sub-levels for one application as a severity indicator; for others as a sub-category, or even as a dummy number that you plan to deny logging for later after you test your new feature set.



Remember that we can do this:

logger.referenceDescendingLevels=a,b,MY_NEW_LEVEL,MY_NEW_LEVEL.1,MY_NEW_LEVEL.2,(c,d),e,f

Here is an example scenario:

Suppose MY_NEW_LEVEL is newly introduced so we can do:

logger.referenceDescendingLevels=a,b,MY_NEW_LEVEL,(c,d),e,f

supposing:
logger.chosenLogLevel=e

and the following " logging requests " are all allowed:

Logger.Log(MY_NEW_LEVEL, "your log text here");
Logger.Log(MY_NEW_LEVEL + ".0", "your log text here"); // same as first line
Logger.Log(MY_NEW_LEVEL + ".1", "your log text here");
Logger.Log(MY_NEW_LEVEL + ".2", "your log text here");
Logger.Log(MY_NEW_LEVEL + ".3", "your log text here");

You do your tests, then want to restrict logging beyond MY_NEW_LEVEL.2 for the logging level MY_NEW_LEVEL. All you'll have to do is:

logger.referenceDescendingLevels=a,b,MY_NEW_LEVEL,MY_NEW_LEVEL.1,MY_NEW_LEVEL.2,(c,d),e,f

supposing:
logger.chosenLogLevel=e

As a result, the following " logging requests " are allowed:

Logger.Log(MY_NEW_LEVEL, "your log text here");
Logger.Log(MY_NEW_LEVEL + ".0", "your log text here"); // same as first line
Logger.Log(MY_NEW_LEVEL + ".1", "your log text here");
Logger.Log(MY_NEW_LEVEL + ".2", "your log text here");

But this one is not:

Logger.Log(MY_NEW_LEVEL + ".3", "your log text here");

For this example value, the following " logging requests " for an undefined level is not allowed (i.e., they don't generate an error, but are simply not logged):

Logger.Log("x", "your log text here");
Logger.Log("x.5", "your log text here");

The idea in the example is that " logging requests " with levels that are not defined anywhere will not generate any error, they're simply ignored.