Bug 64011 - JNDIRealm no longer authenticates to LDAP
Summary: JNDIRealm no longer authenticates to LDAP
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 8.5.50
Hardware: PC All
: P2 regression (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-12-17 13:44 UTC by Mike Lothian
Modified: 2020-01-04 13:48 UTC (History)
1 user (show)



Attachments
Working logs (16.23 KB, text/plain)
2020-01-02 14:21 UTC, Mike Lothian
Details
Broken logs (34.17 KB, text/plain)
2020-01-02 14:21 UTC, Mike Lothian
Details
Revert (10.35 KB, text/plain)
2020-01-02 16:41 UTC, Mike Lothian
Details
Patch against tag 8.5.50 (1.91 KB, patch)
2020-01-03 09:44 UTC, Michael Osipov
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Lothian 2019-12-17 13:44:41 UTC
This works in 8.5.49 and stops working in 8.5.50

jaas.conf:

ORDS {
   com.sun.security.auth.module.Krb5LoginModule required
    doNotPrompt=true
    principal="HTTP/computer@REALM.COM"
    useKeyTab=true
    keyTab="krb5.keytab"
    storeKey=true;
};

ords.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Context>
  <Valve className="org.apache.catalina.authenticator.SpnegoAuthenticator"
      loginConfigName="ORDS"
  />
 <Realm className="org.apache.catalina.realm.JNDIRealm"
   connectionURL="ldap://ldap1:3268"
   alternateURL="ldap://ldap2:3268"
   userSearch="(sAMAccountName={0})"
   userBase="DC=realm,DC=com"
   userSubtree="true"
   roleSearch="(member={0})"
   roleBase="DC=realm,DC=com"
   roleName="CN"
   roleSubtree="true"
   allRolesMode="authOnly"
   spnegoDelegationQop="auth"
   stripRealmForGss="true"
   authentication="none"
   referrals="ignore"
   useDelegatedCredential="true"
 />
</Context>

This now gives the following error:

17-Dec-2019 13:16:30.754 SEVERE [https-jsse-nio-10843-exec-3] org.apache.catalina.realm.JNDIRealm.getPrincipal Exception performing authentication
        javax.naming.NamingException: [LDAP: error code 1 - 000004DC: LdapErr: DSID-0C090A4C, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v3839 ]; remaining name 'DC=realm,DC=com'
                at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3194)
                at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3100)
                at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2891)
                at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1846)
                at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769)
                at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
                at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
                at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
                at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:267)
                at org.apache.catalina.realm.JNDIRealm.getUserBySearch(JNDIRealm.java:1693)
                at org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1528)
                at org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1456)
                at org.apache.catalina.realm.JNDIRealm.getPrincipal(JNDIRealm.java:2354)
                at org.apache.catalina.realm.JNDIRealm.getPrincipal(JNDIRealm.java:2280)
                at org.apache.catalina.realm.JNDIRealm.getPrincipal(JNDIRealm.java:2260)
                at org.apache.catalina.realm.RealmBase.getPrincipal(RealmBase.java:1283)
                at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
                at org.apache.catalina.authenticator.SpnegoAuthenticator$AuthenticateAction.run(SpnegoAuthenticator.java:344)
                at org.apache.catalina.authenticator.SpnegoAuthenticator$AuthenticateAction.run(SpnegoAuthenticator.java:329)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:360)
                at org.apache.catalina.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:243)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
                at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:609)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:810)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1623)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.lang.Thread.run(Thread.java:748)
Comment 1 Michael Osipov 2019-12-17 14:19:47 UTC
The error is pretty obvious, the credentials aren't delegated but you have set useDelegatedCredential="true". Moreover, spnegoDelegationQop="auth" is invalid. According to the RFC 4752 it must be either auth-int or auth-conf. Can you enable debug logging? I want to see it from here: https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/realm/RealmBase.java#L474-L509
Comment 2 Mike Lothian 2019-12-17 16:15:30 UTC
I got the info from here:

https://tomcat.apache.org/tomcat-8.5-doc/config/realm.html

spnegoDelegationQop This attribute should be a comma-separated list of values selected from auth-conf, auth-int and auth

Removing useDelegatedCredential="true" also doesn't work with 8.5.50 but does with 8.5.49

Which debug options should be switched on?
Comment 3 Michael Osipov 2019-12-18 12:19:49 UTC
(In reply to Mike Lothian from comment #2)
> I got the info from here:
> 
> https://tomcat.apache.org/tomcat-8.5-doc/config/realm.html
> 
> spnegoDelegationQop This attribute should be a comma-separated list of
> values selected from auth-conf, auth-int and auth

The Java SASL GSSAPI mech has a bug which makes this possible, otherwise it should fail: https://bugs.openjdk.java.net/browse/JDK-8160818

> Removing useDelegatedCredential="true" also doesn't work with 8.5.50 but
> does with 8.5.49
> 
> Which debug options should be switched on?

You need to enable FINEST for org.apache.catalina.realm and org.apache.catalina.authenticator.
Comment 4 Mike Lothian 2020-01-02 14:21:17 UTC
Created attachment 36940 [details]
Working logs
Comment 5 Mike Lothian 2020-01-02 14:21:40 UTC
Created attachment 36941 [details]
Broken logs
Comment 6 Mike Lothian 2020-01-02 14:22:17 UTC
This was with:

org.apache.catalina.realm.level = FINEST
org.apache.catalina.authenticator.level = FINEST

Added to logging.properties
Comment 7 Remy Maucherat 2020-01-02 15:04:25 UTC
There are no changes in JNDIRealm in 8.5.50, so given the stack and error message a Tomcat issue is not the most likely explanation at the moment.
You should continue investigating without assuming it is a Tomcat issue, for example you have a meaningful error message with a key "DSID-0C090A4C" that you can google and the results could be relevant to you.
Comment 8 Michael Osipov 2020-01-02 15:20:53 UTC
Can you please also enable debug log on authenticators and also provide a tcpdump of the unsuccessful and successful dump. If you don't want to disclose the dump, plase send to my apache email address, if nesessary use my PGP public key to encrypt it.

I would also like to see BER debug from LDAP provider with property com.sun.jndi.ldap.trace.ber bug this required a OutputStream to be set which is not possible via context.xml. I will be happy with the former mentioned for now.
Comment 9 Mike Lothian 2020-01-02 15:59:16 UTC
Hi

I bisected between 8.5.49 and 8.5.50 and this was the first bad commit:

commit 12b857227b2671c9c871aa324cf5fc25c5d53c9a
Author: Michael Osipov <michaelo@apache.org>
Date:   Wed Aug 21 23:23:19 2019 +0200

    BZ 63681: Introduce RealmBase#authenticate(GSSName, GSSCredential) and friends

It doesn't cleanly revert from 8.5.50

Would you mind providing the logging.properties strings for the options you'd like me to enable - org.apache.catalina.authenticator.level = FINEST was on last time

I don't have root access on this machine, but I'll see if I can get a tcpdump from either another machine or if I can get escalated privileges
Comment 10 Mike Lothian 2020-01-02 16:41:19 UTC
Created attachment 36942 [details]
Revert

This was my attempt at reverting the problem commit and does get things working for me again
Comment 11 Michael Osipov 2020-01-02 20:21:19 UTC
(In reply to Mike Lothian from comment #9)
> Hi
> 
> I bisected between 8.5.49 and 8.5.50 and this was the first bad commit:
> 
> commit 12b857227b2671c9c871aa324cf5fc25c5d53c9a
> Author: Michael Osipov <michaelo@apache.org>
> Date:   Wed Aug 21 23:23:19 2019 +0200
> 
>     BZ 63681: Introduce RealmBase#authenticate(GSSName, GSSCredential) and
> friends
> 
> It doesn't cleanly revert from 8.5.50

I already assumed that it could be one of my changes, but I do not yet understand why.
 
> Would you mind providing the logging.properties strings for the options
> you'd like me to enable - org.apache.catalina.authenticator.level = FINEST
> was on last time
> 
> I don't have root access on this machine, but I'll see if I can get a
> tcpdump from either another machine or if I can get escalated privileges

No need for now. I am still trying to wrap my head around the failure. I will get back to you tomorrow after understanding the issue or if I still need more data.
Comment 12 Michael Osipov 2020-01-02 21:53:25 UTC
Looked into it. Something is fishy because the issue should have failed earlier. Will know more tomorrow.

I have the slight feeling that the changed triggered an bug previously present in the code.
Comment 13 Michael Osipov 2020-01-02 23:01:10 UTC
I have found the bug. Apologies, this was an oversight although I have read the diff many times before merging into master.

Can add a modified JAR to your installation to verify potential fix? I will provide one. Alternatively a patch if you want to compile yourself.
Comment 14 Mike Lothian 2020-01-02 23:54:54 UTC
If you could post the patch I'll test it tomorrow when I'm in the office, thanks
Comment 15 Michael Osipov 2020-01-03 09:44:45 UTC
Created attachment 36946 [details]
Patch against tag 8.5.50

I personally don't like the strip operation because it is flawed in several ways:

* It modifies the username which should be at most at the discretion of a realm implemenation
* It assumes to be email style w/o inspecting the OID
* It is true by default

You might be better off with my specialized SPNEGO and Active Directory realm for Tomcat. It works with tens of domains and cross-forest suppor too.

Waiting for your test results.
Comment 16 Mike Lothian 2020-01-03 09:51:21 UTC
That works :D

Do you have more info on your SPNEGO realm? Does it allow fallback to another type too?
Comment 17 Michael Osipov 2020-01-03 10:02:14 UTC
(In reply to Mike Lothian from comment #16)
> That works :D

Did you already try? Can I go ahead and commit that?

> Do you have more info on your SPNEGO realm? Does it allow fallback to
> another type too?

http://tomcatspnegoad.sourceforge.net/

The Tomcat SPNEGO realm is based on my donation years ago. My external project is more sophistictaed and better suited for AD than the JNDI realm.
Comment 18 Mike Lothian 2020-01-03 10:12:13 UTC
Yes, I tested it on the system I was using yesterday

I'll take a look at your project, but I think we might be too far along with using JNDIRealm to change at this point. If the documentation is better though I'll make a case for it
Comment 19 Michael Osipov 2020-01-03 10:15:18 UTC
(In reply to Mike Lothian from comment #18)
> Yes, I tested it on the system I was using yesterday
> 
> I'll take a look at your project, but I think we might be too far along with
> using JNDIRealm to change at this point. If the documentation is better
> though I'll make a case for it

Sure, if you think the documentation is short, just let me know. I have been using this for almost 10 years on the largest commercial AD installation worldwide.
Comment 20 Michael Osipov 2020-01-04 13:48:35 UTC
Fixed in:
- master for 9.0.31 onwards
- 8.5.x for 8.5.51 onwards
- 7.0.x for 7.0.100 onwards