A potential problem for webapps using the java.util.logging API is that loggers are usually global over the entire JVM, rather than per-context as we might prefer. Say you have a single Tomcat server with two webapps. Each webapp includes it's own copy of foo.jar in WEB-INF/lib. foo.jar uses a standard idiom for naming loggers: package com.acme.foo; import java.util.logging.*; public class SomeClass { public static final Logger logger = Logger.getInstance(SomeClass.class.getName()); ... } So each webapp has it's own classloader with it's own copy of SomeClass. The problem is that both classes end up with a reference to the same Logger instance. If both webapps attach their own handler to the Logger, their handler will receive log events from the other webapp. Say each webapp has it's own log file handler, you'll see every log message written to *both* the log files. There's nothing you can do in the webapp or handler to fix this, as there's no way to tell which context a particular LogRecord object came from. Log4j avoids this issue completely, as the entire logging system is contained within the servlet context's ClassLoader. Java logging is loaded via the system classloader, however, so loggers end up being global to the JVM. I think it is fair to class this as a failing on the Java logging system when using it inside any sort of container (serlvet, EJB, or other). I can only find one small reference to this issue in the docs. The javadoc for the constructor of LogManager says: "Protected constructor. This is protected so that container applications (such as J2EE containers) can subclass the object. It is non-public as it is intended that there only be one LogManager object, whose value is retrieved by calling Logmanager.getLogManager." Which seems to indicate that Sun expect servlet containers to provide their own LogManager implementation that provides the behaviour the container wants. In Tomcat's case, I think the desired behaviour is for Loggers to be instantiated per servlet context. So, I have written a LogManager implementation that does this. Is basically splits the Logger namespace up based on the classloader returned by Thread.currentThread().getContextClassLoader(). This seems to do exactly what I want under Tomcat 5.0.x. Please see the attached source. The problem is that the standard LogManager can only be replaced at JVM startup. A webapp can't do it. The procedure is to put the replacement LogManager in a jar, add that jar to the system classpath, and specify the LogManager's class name in the system property "java.util.logging.manager". I suggest that Tomcat should use this replacement LogManager, as having per-context loggers provides much better behaviour inside a container.
Created attachment 14036 [details] Replacement for java.util.logging.LogManager Attaching source.
Ok, the weak hash map should allow avoiding leaking memory (hopefully). The default bare bones implementation of java.util.logging, as provided by Sun in its JDK, is rather limited for containers. It lacks a good file based logger, for example. So your class is one part of a container-friendly java.util.logging implementation. This is not Tomcat specific, and at the moment, we have not decided to provide such an implementation. Note: we don't have anything against comments in sources ;)
Yeah, java.util.logging is very bare bones, and probably very few apps running inside Tomcat use it. But the fact remains that some web apps will use it, perhaps providing their own more sophisticated handlers. I think it makes sense for any container to provide the best support possible for apps using standard APIs, even if those APIs aren't perfect.
I didn't say your code will not be accepted eventually. I just said that at the moment nothing had been decided. I think we would also need a Logger which would do file rolling (code from the old FileLogger could be reused). At the moment, using java.logging is probably good enough in development (you have nothing to install, and it's easy to configure when you need to enable debugging), but not much more.
I'm -0.5 on this: I don't want anything to do with a logging implementation. Leave it java.util.logging, log4j, and commons-logging to work things out.
Make sense. I suppose an implementation of java.logging for servers would be its own separate project.
Thanks for your interest in this. I'm not sure if I made the problem clear, however. I'm not asking for Tomcat to change the way its own logging works or anything like that. I'm not asking for Tomcat to provide better handlers for the JDK logging system. The problem I'm trying to solve is this. If you have two webapps that both use JDK logging internally, then they both share the same set of Logger instances. This means that if they use the same names for the loggers (which would be the case if they both make use of the same support library, for example) then their log messages get mixed up together. The log messages from the second webapp will be sent to the handlers of the first webapp and vice versa. Note that the code actually doing the logging is probably not even aware that it's running inside a container. This is arguably a (minor) security issue as well, as a rogue webapp can grab references to the loggers used by other webapps and screw with them. Using this replacement LogManager does nothing more than isolate Loggers to the servlet context that creates them, rather than having them global to the entire VM which is the default. It's still up to the webapp to provide appropriate log filters and handlers for what it wants to do. The reason for my request is that, due to the design of the JDK logging system, this can only be addressed by the container. Without this, there's nothing a correctly written webapp can do to prevent other webapps sending their messages to its logs (accidently or otherwise). Now, you can legitamately argue that JDK logging is badly designed and you don't see why Tomcat should have to fix its mistakes. Fair enough. But at the end of the day, correctly written webapps that happen to use JDK logging may not work too well when they're run inside Tomcat. I think my biggest obstacle to getting this in is that 99.9999% of webapps running inside Tomcat use Log4j. I think I'm the only person in the world who needs this :-)
I'd imagine you're not the only one. If it makes you feel better, we on the log4j-dev list have been dealing with this issue for months now, in preparation for log4j 1.3. Of course, there's nothing preventing you from doing your own Tomcat build with your patch, if you haven't already done so. Whether you've done this already or not, another good idea would be to post your issue on the tomcat- user mailing list: maybe you'll get other creative solutions.
This is the subject of a separate project. Please don't start chiming out "security" in the hope your patch is integrated in Tomcat: it will not (and it is obviously not a security issue). The only reason I'm keeping this open is that so someone can find it more easily for an eventual commons-foobar project.
(In reply to comment #9) > This is the subject of a separate project. Please don't start chiming out > "security" in the hope your patch is integrated in Tomcat: it will not (and it > is obviously not a security issue). The only reason I'm keeping this open is > that so someone can find it more easily for an eventual commons-foobar project. The point I was trying to demonstrate is that this is an issue of context isolation as much as logging. Both the spec and Tomcat go to some lengths to ensure that webapps running in the same JVM can be isolated from each other. I just see this as another aspect of that isolation; the fact it's related to logging is almost incidental. But as you don't think it's appropriate for Tomcat, I'll stop bugging you now :-)
I would suggest switching to Log4j 1.3 or the UGLI API ( http://logging.apache.org/log4j/docs/ugli.html ), of which there are implementations for Log4j, JDK1.4 logging, simple system.out logging, and NOP. You decide which implementation by providing it in the classpath and never change the API you reference. Kind of like commons-logging, but without the brittleness of the discovery system. Anyway, if you use Log4j as the implementation (directly or via UGLI-Log4j), you can use a repository selector ( http://www.qos.ch/logging/sc.jsp ) which will effectively separate container logging from application logging using a single instance of Log4j (in common/lib). Actually, you can share logging by giving two apps the same logger repository name to use. Read the Javadoc for more info... http://cvs.apache.org/viewcvs.cgi/logging-log4j/src/java/org/apache/log4j/selector/ContextJNDISelector.java?rev=1.18&view=markup Jake
Strong -1. As I have said a few times, I find Ceki's more intent on splitting the logging market for his own benefit rather than try to adhere or participate to any standards. Since you seem to participate in those efforts, how about encouraging taking the interoperability route, which would be beneficial to everyone involved, and stop pushing stupid intiatives like this ugly one ?
Remy, First, I didn't ask for a vote. Second, your comments are *always* "Strong", so you can skip the redundancy and just comment. Third, the core of my comment was about using a Log4j repository selector which specifically addresses the comments about separating logging per application in the presence of a VM wide logging implementation. UGLI was an aside that maybe could have been left out, but I thought I'd mention it. Sorry to ruffle your feathers! But since you seem to have focused on the non-core part of the comment, I'll bite. What do you suggest then? Continue using the commons-logging? IMO, that's not a solution and I'm not alone in this thought (one example of many below)... Commons Logging was my fault http://radio.weblogs.com/0122027/2003/08/15.html UGLI provides for a common logging API, avoids the brittleness of commons-logging discovery, allows for a tiny jar to be compiled against and/or distributed with any given project, and allows one to use any logging implementation at runtime including Log4j, JDK1.4 logging, System.out and NOP. More can be added. Does your objection merely have to do with a personality conflict with Ceki and, by proxy, myself? Any problems with Yoav who, to my knowledge, hasn't voiced objections to UGLI and is a Log4j developer (sorry Yoav, I don't mean to bring you into this, just pointing out a possible inconsistency here)? If not, I find your motivations suspect. If I'm just way off on the above, is it that the UGLI API is not javax.ugli, but org.apache.ugli? Any constructive thoughts on making UGLI or some other concept more of a community process? You mention the "interoperability route". Exactly what you do mean here? What's not "interoperable" about UGLI? If you think I'm clueless, how about giving me a clue and be constructive yourself "which would be beneficial to everyone involved". Jake
The article pointed is completely inaccurate. It's not my fault if the original author did not know how his tool could be improved and used properly. Commons-logging specific issues are minimal these days, so I don't see anything to fix here. This bug is about limitations with the default implementation of java.util.logging. -1 for ugli. We're obviously going to continue using commons-logging (which does more or less the same as ugli does; again, I note willingness to avoid participating in non log4j initiatives). I have no personality conflict with Ceki, other than the fact that he chose to split the logging API, rather than try to contribute and enhance standards. That is his choice, but as you can probably understand, this created difficulties for users like me, which now have multiple incompatible APIs on hand, with no merge on the horizon, and forced me into using useless wrappers (like commons-logging), which have their own individual issues, to overcome this. So it's a professional conflict, if you'd like ;) UGLI is yet another fork attempt by Ceki of a well accepted API, rather than try to contibute or enhance the said API. It's just lame, I have nothing else to say.
> I have no personality conflict with Ceki, other than the fact that he chose to > split the logging API, rather than try to contribute and enhance standards. Log4j joined Apache in late 1999. Not only is calling log4j a fork of java.util.logging factually inaccurate, it is also insulting. As for participation by log4j community, have a look at pages 85 and 86 of the JSR-47 specification: 5.4 Changes between 0.71 and 0.80 Adopted some changes suggested by the Apache log4j community: - There is now a Logger.getParent method to allow you to find the nearest extant ancestor for a given Logger. To help implement this, a Logger.setParent method has been added. - Loggers may now inherit their effective level from their parent. If a Logger’s level is set to null then it will effectively inherit its parent’s level (recursively up the tree). This effective level is the level that will be used for all output checks. As a side effect of this change, the LogManager.getLevel and LogManager.setLevel methods have been removed, as they are now largely redundant and potentially confusing. - default Loggers will now also send their output to their parent’s Handlers (recursively up the tree) - default Loggers will now also send their output to their parent’s Handlers (recursively up the tree). As a side effect of this, the old "global handlers" has become obsolete. The global handlers are now replaced by the Handlers for the root Logger in the namespace. So the LogManager methods addGlobalHandler, removeGlobalHandler, getGlobalHandlers, removeAllGobalHandlers, publish, and flush have been removed. The Logger methods getUseGlobalHandlers and setUseGlobalHandlers have been replaced with the methods getUseParentHandlers and setUseParentHandlers. The LogManager "handlers" is now used to define the initial handlers for the root Logger. - a Logger does not have a resource bundle name defined, then it will inherit the resource bundle name defined for its parent, recursively up the tree. - There is a new class ErrorManager for handling errors during logging output. An ErrorManager may be added to a Handler using Handler.setErrorManager and retrieved using Handler.getErrorManager. When an output error occurs on the Handler the associated ErrorManager will be called to process the error. This has replaced the previous mechanism based on Handler.setException and Handler.getException, and those two methods have been removed. Note that some of these contributions are not insignificant. > UGLI is yet another fork attempt by Ceki of a well accepted API, > rather than try to contibute or enhance the said API. It's just lame, > I have nothing else to say. UGLI was developed in order to support logging generated by log4j components internally. (In log4j 1.3, log4j components use log4j itself when logging internally generated messages.) In case the log4j component cannot get a handle to a valid LoggingRepository, it must fallback to another implementation which is where UGLI comes in. Bugs caused by JCL's discovery mechanism do much harm to log4j's reputation. UGLI avoids these bugs with a simpler and more robust approach. I chose to ignore Remy's other more personal comments which are unbecoming of an ASF member.
Please reread my comments. I am calling ugli a fork of commons-logging, which is accurate. It would be much more productive to enhance commons-logging instead (as long as API compatibility is preserved). The code is very small (as is ugli's code). log4j predates java.logging, of course, but it's been years now, and there have been no efforts or progress to support it. I think it's safe to say that few people care about whatever design issues might exist with java.logging (I am fairly certain it is good enough, overall, since the small subset which is commons-logging is good enough for us right now), and they would have been very grateful to avoid having to use stuff like commons-logging. I am also fairly certain usage of log4j would be even higher, and, on a personal standpoint, I would be a happy camper :) Obviously, contributions to java.logging original design are nice, but in the end it does not matter if log4j does not support it (or if java.logging had failed altogether, which, unfortunately or not, is not the case). I do not actually care about java.logging, all I want is to not have 2 different incompatible logging APIs at the end. Many people apparently think the same way, as they are now standardizing on commons-logging to get unified configuration for all components. Since the API problem is at the root of nearly all of the issues we've had (CL issues, etc), I think the way this could have been avoided is fairly clear, and you could have avoided whatever all the bugs that were caused by the use of commons-logging. Anyway, I'd like to point out this bug report was about enhancements to java.logging support in Tomcat, not about replacing whatever logging API we use with log4j or (that's new) one of its "child" APIs. This is a recurrent "candid" proposal, and is getting old.
Remy, That's a lot of words for having "nothing else to say", especially when you are the one who strayed off in to this discussion when all I was trying to do was comment about context isolation, albeit from Log4j's perspective, which I don't think is exactly off-topic in this report even if it is about JDK 1.4 logging. In any case, can you address why Tomcat 5.0.xx and 5.5.x ship with commons-logging-api.jar and not commons-logging.jar? Why is the former necessary? As I understand it, the only diffence is the exclusion of "org.apache.commons.logging.impl.Log4JLogger" and "org.apache.commons.logging.impl.AvalonLoggersupport". Is it a technical decision or a philisophical one? If a technical one, is the reason to avoid commons-logging discovery issues if Log4j or Avalon is somewhere in a child classloader visible to discovery, but invisible to the server, and therefore, to avoid NoClassDefFoundError's? You say commons-logging isn't broken, but doesn't this just typify the problem with commons-logging? Don't let me put words in your mouth, but I can't come up with any other rational reason for confusing the situation with two nearly identical, but differently functioning, jars from the commons-logging project. Why would Log4j want to be involved in something already so broken other than making sure we generally interoperate with it (which we did for the commons-logging-1.0.4 release, so please stop claiming that the Log4j is some rogue uncooperative project). BTW, how is a logging alternative like UGLI (which, as Ceki explained, is an interal fallback logging implementation that happens to double as generic API) considered a commons-logging "fork"? My understanding of a "fork" is starting with an existing API and modifying it so it is incompatible with the original. UGLI was written from scratch and not based on commons-logging so it is quite literally not a "fork" but an "alternative". That is, unless you consider any similarily functioning software as a "fork", which would make competing application servers "forks" or XOM a "fork" of "JDOM". They are "alternatives", not "forks". If you are talking about unifying alternatives, are you saying that the commons-logging project would accept moving to an UGLI-like concept and get rid of the discovery? I remain unconvinced. So, start the dialog and convince me and everyone else. If you are unwilling to do this after making clear the consternation you have over a lack of a unified logging API, then you are just blowing hot air. Let's see some action from someone with a lot of pull in the Jakarta sphere. We're all waiting. Jake
The design I have in mind is that the commons-logging implementation for the logger you're using must reside in the same classloader hierarchy as the logger implementation. This is why the -api JAR is used. As java.logging is available in the root classloader, so does its commons-logging impl, and it is located next to it in the classloader hierarchy (from Tomcat perspective; actually, it's one layer above it). Given the size of both projects (a couple hundred of lines of code each, it seems), UGLI is a fork of commons-logging. It demonstrates unwilligness to work with anyone else's API, and willingness to split the logging APIs again. If this is not your intention, then fine, but I need to see positive results. I am not involved in commons-logging development, and thus I view the situation as a user, and care only about the results. BTW, the log4j commons-logging implementation was developed by Costin, Craig, and Robert, with only one really simple patch from Ceki, contributed for commons-logging 1.0.4 (removing the need for a log4j 1.2 compatibility flag). I remember attempts by Ceki to torpedo commons-logging when the project was started, however. So I don't call this contributing to commons-logging, sorry. Do you have any more FUD for us to enjoy ? Basically, I now consider the existence of the log4j API harmful at this point. The loggers implementation, etc, are all good, but I hope you understand that if the thin API layer at the top was standardized, everyone would benefit. -1 for UGLI.
This is not getting anywhere constructive.
> The design I have in mind is that the commons-logging implementation > for the logger you're using must reside in the same classloader > hierarchy as the logger implementation. This is why the -api JAR is > used. As java.logging is available in the root classloader, so does > its commons-logging impl, and it is located next to it in the > classloader hierarchy (from Tomcat perspective; actually, it's one > layer above it). Remy, In plain English, you are admitting that JCL's discovery mechanism does not work correctly except for java.util.logging, correct? JCL is supposed to bridge between logging APIs, including log4j. However, in practice it only bridges for java.util.logging and break for other implementations residing outside the JDK. Isn't this the plain truth? Given the above, I don't know how you dare fling accusations at log4j's direction and claim that JCL is problem-free. Moreover, your contempt of a fellow Apache project is shocking.
I don't see where or why I would admit anything. You claim on #19 that this is not constructive, and on #20 you come back for more. This does not make any sense to me, but here's more anyway :) Why I like commons-logging: - When packaged right, it works ok - A huge amount of people are now using its API, therefore enabling unified container wide logging configuration - Logging integration for people who embed Tomcat (as my company is doing) I do not care at all about discovery or whatever as long as it works well enough, and I did not look at the implementation details inside commons-logging either. I would have less contempt for log4j if, for example, it had implemented the java.util.logging API as a layer on top of its own API (and shipped the said layer in its official JAR, of course). I don't think anybody would get hurt by this, and this would be a productive first step towards unification. It would remove the need for commons-logging in the next Tomcat version. We would use java.util.logging, and, most likely, would be shipping a minimal log4j JAR with only a rotatable file logger (obviously, nothing can be decided at this point, and it's merely an idea). So you would get more users, while our default logging setup would be better. BTW, I don't see anything wrong with having "contempt of a fellow Apache project". The amount of ASF folks who have written than Tomcat sucked in one way or the other is absolutely staggering ;) Besides, the amount of Apache code is now so huge, with some prjects more or less competing with each other, that I don't see how I could love all of them. And, in log4j, I only actually dislike the log4j API here.
"When packaged right" JCL will only use java.util.logging, isn't that a fact? Remy, have your *ever* used Tomcat with log4j in a production environment? I guess that the people using commons-logging probably do not realize how badly JCL is broken. Given that you yourself admit not having being familiar with the JCL implementation, you apparently do not understand that JCL is much more complicated than UGLI.
(In reply to comment #22) > "When packaged right" JCL will only use java.util.logging, isn't that > a fact? Remy, have your *ever* used Tomcat with log4j in a production > environment? I use Tomcat with log4j in production all the time. It works very well, when packaged right ;-). > I guess that the people using commons-logging probably do not realize > how badly JCL is broken. Given that you yourself admit not having > being familiar with the JCL implementation, you apparently do not > understand that JCL is much more complicated than UGLI. Of course JCL is much more complicated than UGLI, since UGLI expects you to create a properties file on the CP, and is useless if you forget (you don't even get log4j :). However, IMHO, that complication is necessary for logging in a container environment. I'm also -1 on having UGLI Tomcat code, and +0 for having an optional example of j.u.l.LogManager.
What makes you think that UGLI needs a properties file on the classpath? How did you come up with that? As for packaged right, do you have an example of what you mean by that?
(In reply to comment #24) > What makes you think that UGLI needs a properties file on the > classpath? How did you come up with that? http://cvs.apache.org/viewcvs.cgi/logging- log4j/src/java/org/apache/ugli/LoggerFactory.java?rev=1.2&view=markup > As for packaged right, do you have an example of what you mean by that? http://jakarta.apache.org/tomcat/faq/logging.html#commonsLoggingConfiguration
I am still +1 for using LogManager, but where do we put it ? We would also need companion loggers for java.util.logging to be a viable solution, such as a rotatable logger. Yoav didn't want us to have anything to do with logging inside Tomcat (I think he's right), as he mentioned in #5 when this bug report was still on topic. This leads to the need of a full fledged project to host the code, or at least a commons component.
The ugli.properties file is not specified on the class path. It is bundled with each version of ugli-xxx.jar. In a class loader following the child-first delegation model (as in WebAppClassLoader in Tomcat) the factory used will be the correct one whereas in a class loader following the parent-first delegation model the factory used will be that of the parent. It's not perfect but at least UGLI won't blow up and crash your application. Moreover, the intent is to perform the factory binding at compile time.
(In reply to comment #26) >This > leads to the need of a full fledged project to host the code, or at least a > commons component. Commons would be a good place for it, since it isn't really Tomcat specific. It would also let you learn the joys of svn ;-). Otherwise I'd vote for j-t-c so that it is available for TC 3&4 as well. Putting it in catalina/modules also works for me.
I think I will try Commons to experiment with svn. svn first experience is rough, since it has all sorts of folders all over the place, and I have no idea how to import code ;) It's going to be fun :) Anyway, I will propose a new component named commons-juli, which will contain as its initial code: - the replacement log manager - a bare bones rotatable file logger Those two will make a bare bones container friendly mini-JAR. We can then either ship that to provide decent java.util.logging defaults, or give instructions on how to install. I expect additional loggers to start appearing, but I'll keep a bare bones JAR containing only the basics. BTW, juli stands for Java Util Logging Implementation. I thought it was cute.
(In reply to comment #29) > Anyway, I will propose a new component named commons-juli, which will contain as > its initial code: > - the replacement log manager > - a bare bones rotatable file logger > Those two will make a bare bones container friendly mini-JAR. We can then either > ship that to provide decent java.util.logging defaults, or give instructions on > how to install. Cool, that sounds great. Sorry about the lack of comments in the code. Let me know if you need me to clarify anything. Much of the code is re-implementing stuff from the standard LogManager, which unfortunately wasn't written with easy subclassing in mind.
I have expanded the code provided to provide configurability using properties file (like java.util.logging has), and the code is now (likely temporarily) located in j-t-c/juli. Thanks.