We found that mod_ldap unable to work through firewalls, because ldap session keepalive not implemented in it. There is not always possible to solve keepalive on the server side, because of operation policies, too far from client etc., but we need it because of ldap connection pooling. So mod_ldap client keepalive should be implemented. We patched the code (## line) to implement a workaround for the missing functionality: ./modules/ldap/util_ldap.c: static void uldap_connection_close(util_ldap_connection_t *ldc) { /* * QUESTION: * * Is it safe leaving bound connections floating around between the * different modules? Keeping the user bound is a performance boost, * but it is also a potential security problem - maybe. * * For now we unbind the user when we finish with a connection, but * we don't have to... */ uldap_connection_unbind(ldc); ## mod by RZS /* mark our connection as available for reuse */ #if APR_HAS_THREADS apr_thread_mutex_unlock(ldc->lock); #endif }
Does that just disable the connection pooling?
(In reply to comment #1) > Does that just disable the connection pooling? Yes. mod_ldap unusable in firewalled environment, so better to disable pooling. I know, this is not a full solution, it would be better if the behaviour could be configurable.
(In reply to comment #2) > (In reply to comment #1) > > Does that just disable the connection pooling? > > Yes. mod_ldap unusable in firewalled environment, so better to disable pooling. > I know, this is not a full solution, it would be better if the behaviour could > be configurable. When the connection is severed by a firewall, what does your LDAP SDK return when Apache goes to use the connection?
It tries to use the dropped tcp connection, so it waits a long time (for tcp timeout). It is unacceptable long. What dou you mean "your LDAP SDK"?
(In reply to comment #2) > (In reply to comment #1) > > Does that just disable the connection pooling? > > Yes. mod_ldap unusable in firewalled environment, so better to disable pooling. > I know, this is not a full solution, it would be better if the behaviour could > be configurable. Other options: - implement some sort of "check" logic when a connection is requested from the pool. If it doesn't return in a reasonable amount of time (re-use the connection time out time or add another parameter) then unbind the connection and rebind - implement connection-max-age and/or connection-max-idle parameters so that if a connection is requested from the pool and it has been connected/idle for "too long" then unbind/rebind
The cleanest fix is the ldap library enabling the tcp keepalive option. Recent version of OpenLDAP do this. Under Linux, you can tune the interval between keepalive probes with /proc/sys/net/ipv4/tcp_keepalive_time (default is two hours). Reducing this to a value below the firewall's state timeout should fix the problem. You should try to find out if your OS / your ldap library supports this. If it doesn't, you should bug the vendor to implement it. Another fix is to configure the firewall to send tcp reset packets instead of silently dropping connections. Apart from that, allowing to set a timeout for ldap searches in Apache httpd would be a good idea.
(In reply to comment #5) > Other options: > - implement some sort of "check" logic when a connection is requested from the > pool. If it doesn't return in a reasonable amount of time (re-use the > connection time out time or add another parameter) then unbind the connection > and rebind Something like this is now implemented in trunk in r898102: There is a new timeout value for search/bind. If the timeout is reached, the operation is retried after unbind/rebind (at least for the initial bind after an old connection is taken from the pool). Note that at least with openldap, you can influence mod_ldap with /etc/ldap.conf and $HOME/.ldaprc (especially with the TIMEOUT parameter). Httpd 2.2.x will return 500 if TIMEOUT triggers, but this may still be better than the child waiting for > 15 minutes.
This issue may also occur when the LDAP server is behind a load balancer which does TCP session timeouts. In large production environments, a load balancer in front of a pool of LDAP servers is typical. While leaving a connection in the "bind" state open for performance seems like a good thing, it isn't practical except in limited environments. I suggest a directive which would turn off the bound connection pool completely. I would also suggest that it be the default for the connection pool to be disabled.
trunk now has a configuration switch LDAPConnectionPoolTTL which can be set to zero to always unbind, or set to a positive # to unbind connections right before they're reused if they're too old. The default behavior is not changed. Since all we do is unbind, this may be backportable.
(In reply to R. Gilligan from comment #8) > This issue may also occur when the LDAP server is behind a load balancer > which does TCP session timeouts. Hi Gilligan Is it easy to reproduce this issue? I don't use apache 2.2.11 but I have apache 2.2.3; 2.2.21; 2.4.6 I cannot reproduce this issue with these version. Does it happen with every http request?
There are now search and bind timeouts and LDAPConnectionPoolTTL (including zero)