Bug 12428

This is a major problem for us porting our application.  We have a menu system
which stays the same for all the users.  Based on the role and if we have a
principal or not, the menu changes with more or less options.

I don't see anything in the 2.3 spec that precludes the way that Tomcat handles 
this.  The 2.4 spec is a bit more ambiguous, so I'm going to have to try to get 
a clarification from the expert-group before marking this as INVALID.

As a work-around, try using a simple Filter something like:
   public void doFilter(ServletRequest req, ServletResponse res, FilterChain 
chain)
      throws IOException, ServletException {
      HttpServletRequest request = (HttpServletRequest)req;
      Principal userPrin = request.getUserPrincipal();
      if(userPrin == null) {
          HttpSession session = request.getSession(true);
          Principal myPrin = (Principal)session.getAttribute
("com.myfirm.MyPrincipal");
          if(myPrin != null) {
            req = new MyAuthRequest(myPrin);
          }
      } else {
          HttpSession session = request.getSession(true);
          session.setAttribute("com.myfirm.MyPrincipal", userPrin);
      }
      chain.doFilter(req, res);
   }
   static class MyAuthRequest extends HttpServletRequestWrapper {
      Principal myPrin;
      MyAuthRequest(Principal prin) {
      myPrin = prin;
      }
      public Principal getUserPrincipal() {
         return myPrin;
      }
   }

This works, but only in the context of the WEB container.  If my WEB application
calls down to my entity beans, I am out of luck.  I don't know of a way to pass
this "context" down to the entity beans without having to implicitly pass a user
parameter to ALL the functions which would normally get a principal from the
context.

By the way, this is happening with JBOSS 3.2.2RC3 which has TomCat version 4.1
bundled with it (I am not sure which one).

In my application, additional features become available when user becomes a
member.  Thus all of my code is based on:

a) Do I have a principal, no? then GUEST access otherwise b)
b) If role X is enabled, allow functions X1...Xn

Not having principal in the public pages, does not allow me to check the role.  

I noticed that I am not the only one who is having an issue with the way this
works.  Is there a way to make a parameter, that when set, would pass the
principal to the public pages.  If the parameter is not set, then don't pass the
principal as it's now?  Since the spec is so ambiguous, it makes migration to
TomCat/JBOSS combo from Weblogic, Orion or other app servers difficult.

Thanks.

Created attachment 8384 [details]
Filter attempting to store principal in the session

I tried your suggestion, code attached, but it did not work.  Turns out that the
filter is getting called pretty late in the game.  By the time the filter is
called, a null Principal was already seen and configured in to the "context".  I
need to work at the Pipeline/Valve level to get this to work.  

Ya  - this is a problem but tomcat is compatible with the spec. (So its a spec
problem) The pricipal only needs to be set on protected URLs.

doh - wrong status

fix status to INVALID, not FIXED. (Sorry for the extra emails)

*** Bug 29537 has been marked as a duplicate of this bug. ***

I've been reading again the servlet 2.3 specification, and, actually, i don't 
see in it anything that give the opposite position, ie always return the 
principal when one has been authenticated, when the requested url is protected 
or not. Further, Tomcat 4 behaves as expected (i mean, i expect), which is, i 
think, the 2.3 implementation. What about the 2.4 version, which is the base 
for the new Tomcat 5 ?...

More, what about the 'isUserInRole' ? Does it follow the same rule ? How a 
simple menu page could take decision according to identity or roles of the 
authenticated user, and show or hide links for example, even if this page 
itself is not protected ?

Thanks for your precisions.

I truly think this is a wrong interpretation of the spec. From the JavaDoc of
HttpServletRequest:

"Returns a java.security.Principal object containing the name of the current
authenticated user. If the user has not been authenticated, the method returns
null."

This clearly states that the getUserPrincipal()-method should only return null
when the user has not been authenticated. There is no exception to this rule, as
earlier comments would suggest.

Clearly it would not be against the spec to always return the principal when
authentication has been done wether or not the viewed resource is protected or
not. This is clearly needed for many web-applications.

There is a an exception.

<spec-quote>
SRV.12.9 Default Policies
By default, authentication is not needed to access resources. Authentication is
needed for requests for a web resource collection only when specified by the
deployment descriptor.
</spec-quote>

Hence the user is only authenticated for protected resources.

I appreciate the application design issues raised above, but it remains the 
case that TC4 is compliant with the spec.

That's fine.  You are right, there is no need to Authenticate for non protected
resources, however, once authenticated and the Principal is available it should
be returned regardless if authenticated or un-authenticated resource is being
accessed.

The underlying issue here is how BASIC authentication works. With BASIC
authentication, the user is required to authenticate with every request and the
browser helpfully caches the user name and password and re-uses them with
subsequent requests.

The spec requires that authentication only takes place if a resource is
protected. Therefore, for an unprotected resource no BASIC authentication takes
place even if the browser sends the credentials. In turn, this means that
getUserPrincipal() will return null since the user has not been authenticated.

One work-around is to use FORM authentication. In this scheme, the user is
authenticated once and the Principal added to the session. This authenticated
Principal remains available whilst the session is valid regardless of whether an
individual request requires authentication.

I have considered modifying the BASIC authentication implementation so a user is
always authenticated if the present credentials but:
- this would violate the spec
- the behaviour if the authentication fails is undefined (because the spec
obviously doesn't define behaviour that violates the spec)

Therefore I am going to resolve this as INVALID since any other behaviour is a
spec violation.

As a final comment I do not like that this means that application behaviour
varies with the authentication scheme specified in web.xml but this is a direct
side-effect of the differences in per-request and per-session authentication.

(In reply to comment #14)
Unless there have been changes in the latest version of TomCat, I have always
been using Form Authentication.  So the description of this problem is IN the
context of Form Authentication.  Even after being authenticated via a Form
authentication method, the getUserPrincipal still returns NULL for those
resources which are not protected.  

I worked around this problem, however, by creating my own Valve and storing the
authenticated user somewhere I can get it later.  Later, I ensure that this user
is correctly setup for the JBOSS call to the EJB.

I disagree with the assesment of this bug. Tomcat's behaviour is based on the
following:

<spec-quote>
SRV.12.9 Default Policies
By default, authentication is not needed to access resources. Authentication is
needed for requests for a web resource collection only when specified by the
deployment descriptor.
</spec-quote>

Authentication can also be provided when it is not mandated by the deployment
descriptor. The spec quote doesn't say that authentication is forbidden when it
is not specified in the deployment descriptor.

When authentication is provided, no matter how and why, the documentation of the
getUserPrincipal method applies.

The use case for this is any situation where authorisation is based on
application data. When some application logic has found that unauthenticated
access to a resource is not allowed it can require authentication and reconsider
its access control descision. The status code 401 can also be returned by the
application. How would you otherwise implement RFC 3744 for example?

Having looked at this again, I agree that authenticating the user, if
credentials are present, for a non-protected resource would not be in violation
of the spec. However, the question remains - what to do if that authentication
fails?

Anyway, I am changing this to an enhancement request since the current behaviour
is spec compliant. As ever, patches for review will be welcomed.

When the authentication fails the server can return a 401, because the
spontaneously provided Authorization header is wrong (RFC 2617 section 1.2).
Since the server didn't require authentication for the method, the User Agent
would have volunteered it, perhaps trying to get in and call other methods for
which authentication is required. After having received the 401, the User Agent
can continue interacting with the server unauthenticated. In this scenario the
server should always check a provided Authorization header, even if the method
doesn't require authentication.

Evaluating whether the current behaviour is compliant with the spec or not
depends. The starting point is the specification of the
HttpServletRequest.getUserPrincipal method. Looking at that alone makes the
behaviour non-compliant. Including SRV.12.9 makes it more difficult. Does
SRV.12.9 apply in this case? In don't think so, because it says nothing about
spontaneous authentication, which is allowed.

Created attachment 20857 [details]
Make authentication independent of security constraints

The specification of the getUserPrincipal doesn't say the availability of
principal information depends on the existance of security constraints in the
deployment descriptor.

The patch causes the login configuration to be installed unconditionally and
checks if authentication is provided in the request regardless of the existance
of security constraints. Credentials may have been provided spontaneously or
after a 401 response comming from the servlet.

Note that this is also the way WebLogic behaves.

This patch is for Tomcat 5.5.23 and not Tomcat 4 for which the bug was created.

Please provide a text version of your patch in diff -u form.

Created attachment 20859 [details]
org.apache.catalina.authenticator.AuthenticatorBase

Created attachment 20860 [details]
org.apache.catalina.startup.ContextConfig

Having looked at this further this is no need for a patch. Tomcat has the necessary functionality to do this. You just need to ensure that a) the application is using sessions and b) that the authenticators are configured to cache the authenticated Principal in the session.

A recent enhancement to Tomcat 7 (the alwaysUseSession attribute) will make this even easier. On earlier versions, ensure a session exists before the authentication takes place. Depending on circumstances that might require a valve.

Marking this as WONTFIX since the patch isn't going to be applied.

The other advantage of this approach is that the handling of fail unprompted authentications does not need to be considered. There were issues with complying with RFC2617 with that approach and it couldn't possible work with DIGEST auth.

Sessions don't solve the problem and doesn't make it comply to the spec. The container is not the only party that can decide if authentication is necessary. The application can do this too. Even if no credentials were provided spontaneously by the client, the application could set the status code to 401. The client would then reissue the request with credentials and the application couldn't anything else but return 401 again, because the principal is not passed through by Tomcat as the resource is not declared as protected in web.xml.

What were the issues with RFC 2617?

The fact that it wouldn't work for DIGEST authentication is not relevant. We're talking about a valid scenario for Basic authentication.

This is a grey area of the specification. My reading of the various specs remains that Tomcat is spec compliant. I have added this to my list of things to ask the Servlet EG to clarify in 3.next 

I believe that a web application's fundamental behaviour should not change just by changing the authentication mechanism. That DIGEST can't work with pre-emptive authentication is a significant concern.

The scope of the feature is also important. This is do-able as previously described with container managed authentication. Once the application starts to get involved, things get more complex. However there is a way to do this in Servlet 3.0. The application can call request.authenticate() but it needs to make sure it checks the return code and stops any 401 going back to the client. The application will also need to handle any IllegalStateExcpetions if the response has already been committed.

The RFC2617 issue was mainly that a failed authentication SHOULD result in a 401 response and this feature requires that there is no 401 else the application could end up prevent a user from accessing a page for which no authentication is required. The SHOULD does give some leeway (it isn't a MUST) but I'm not convinced there is a good enough reason to ignore the spec here.

There are two issues here. The first is pre-emptive authentication. RFC 2617 section 1.2 is very clear on this:

" A user agent that wishes to authenticate itself with an origin
   server--usually, but not necessarily, after receiving a 401
   (Unauthorized)--MAY do so by including an Authorization header field
   with the request."

And further:

"If the origin server does not wish to accept the credentials sent
   with a request, it SHOULD return a 401 (Unauthorized) response. The
   response MUST include a WWW-Authenticate header field containing at
   least one (possibly new) challenge applicable to the requested
   resource."

This means the origin server must always deal with this, no matter the technology being used. The fact that a method is not declared as being protected doesn't play any role.

The second issue is that according to you the application should not be able to trigger the challenge-reponse mechanism by setting the status code to 401. Can you give one piece of evidence in the servlet spec for this? Are there restrictions in the HttpServletRequest.setStatus() method that say the application can't use certain status codes? Here is what the servlet spec says about programmatic security:

"Programmatic security is used by security aware applications when declarative security alone is not sufficient to express the security model of the application."

Declarative security is fine, but doesn't prohibit programmatic security and programmatic security doesn't require declarative security to be present. They are simply not related. According to you they are. Can you tell me why?

The patch shows how trivial it is to implement this correctly. Would harm be done if it were implemented? Which spec would be violated in that case?

Your stance makes it impossible to implement WebDAV ACLs (RFC 3744). A user can change access control for a resource from protected to public. The former requires authentication prior to access control, while the latter doesn't.

see comment #26

Since this is the last open Tomcat 4 bug and a) I would like to make the Tomcat 4 bugs read-only and b) any changes as a result of this bug will be in Tomcat 7 onwards, I am moving this issue to Tomcat 7.

This has been fixed in 7.0.x and will be included in 7.0.12 onwards. It is disable by default but can be enabled on a per context basis.

To address some of the points raised in comment 26:

I don't believe RFC2617 or the Servlet specification are sufficiently clear to enable preemptive authentication by default. They are open to interpretation and my reading of them is that there are ambiguities in the language that defines the server response in such a scenario.

I did not say that an application cannot trigger authentication by returning a 401 response. My point was that if the application uses the Servlet 3.0 API to manually implement preemptive authentication, it needs to consider what to do if that authentication fails when the client has requested a non-protected resource. Returning a 401 in that case strikes me as the wrong thing to do but that goes back to the ambiguity in the Servlet spec and RFC2617.

Regarding programmatic security and declarative security, the relationship between them is set out in section 13 if the servlet spec. The point I was making was that if an application uses both declarative security and programmatic security and the programmatic security performs actions normally handled by declarative security (e.g. sending and processing authentication headers) then you need to be careful to ensure that the two do not interfere.

To summarise the current position:
- with alwaysUseSession and cache enabled on an authenticator, the authenticated user name and principal will be available to all requests once the user has accessed a protected resource
- with preemptiveAuthentication enabled on an authenticator, the authenticated user name and principal will always be available

Fascinating reading. My question would be: why does anyone want to have a resource that doesn't need authentication (no security-constraint) but then checks the authentication status, anyway (calls getPrincipal, isUserInRole, etc.)?

If this "bug" was really ruining anyone's day, isn't it a simple matter of providing an aliased URL to the same resource that /is/ protected by a security-constrains and always sending authenticated users to /that/ URL instead?

Glad the 9-year saga is over...

@Mark The relevant specs are crystal clear. If you think there is any room for interpretation then you should provide proof of how your interpretation can be constructed. At present we still don't know what that is.

You wonder what a "non-protected" servlet should do when the provided credentials are wrong. That is simple, it should do nothing, because the container will have returned a 401, which it should always do when the credentials are wrong. That is because there is no response code for reporting wrong credentials.

Where can there be interference? All involved parties, container and servlet, should comply with the specifications. When the container imposes a security constraint because it was declared in the application, the servlet won't see anything about it.

@Chris You don't seem to understand the difference between declarative and programmatic protection. An alternative URL doesn't provide prgrammatic protection. Study the WebDAV ACL specification and you will see it is impossible to implement it without this bug being fixed.