Bug 59092

Summary: Shutdown of ISAPI filter leads to infinite wait and hangs whole IIS
Product: Tomcat Connectors Reporter: Thomas Reiser <thomas.reiser>
Component: isapiAssignee: Tomcat Developers Mailing List <dev>
Status: RESOLVED DUPLICATE    
Severity: critical CC: thomas.reiser
Priority: P2    
Version: 1.2.41   
Target Milestone: ---   
Hardware: PC   
OS: All   
Attachments: Process Explorer screenshot for IIS worker process
Proposed fix

Description Thomas Reiser 2016-03-01 11:53:29 UTC
Created attachment 33614 [details]
Process Explorer screenshot for IIS worker process

When there are done changes in the IIS frontend (e.g. adjustment of rewrite rules), it seems like IIS is restarting all applied filters and so it will also restart the ISAPI filter.

We have experienced that the ISAPI filter no more starts up after it was terminated. After having a look at the Process Explorer, we found that the IIS worker process is blocked because of WaitForeSingleObjectEx in GetFilterVersion:

jk_isapi_plugin.c, line 1487:
BOOL WINAPI GetFilterVersion(PHTTP_FILTER_VERSION pVer)
{
    BOOL rv = TRUE;
    ULONG http_filter_revision = HTTP_FILTER_REVISION;

    pVer->dwFilterVersion = pVer->dwServerFilterVersion;

    if (pVer->dwFilterVersion > http_filter_revision) {
        pVer->dwFilterVersion = http_filter_revision;
    }
    WaitForSingleObject(init_cs, INFINITE);
    if (!is_inited) {
        rv = initialize_extension();
    }
    ReleaseMutex(init_cs);
    if (iis_info.major < 5 || (iis_info.major == 5 && iis_info.minor < 1)) {
        SetLastError(ERROR_OLD_WIN_VERSION);
        return FALSE;
    }
    pVer->dwFlags = SF_NOTIFY_ORDER_HIGH |
                    SF_NOTIFY_SECURE_PORT |
                    SF_NOTIFY_NONSECURE_PORT |
                    SF_NOTIFY_LOG |
                    iis_info.filter_notify_event;

    StringCbCopy(pVer->lpszFilterDesc, SF_MAX_FILTER_DESC_LEN, (VERSION_STRING));
    return rv;
}

This code seams to be correct, it waits infinite time to get the lock for init_cs and releases it after the extension was initialized. But in our case, init_cs is STILL locked as it was NOT released when the filter was terminated.

If you have a look at the method TerminateFilter at line 2363 (also jk_isapi_plugin.c), it also calls WaitForSingleObject for init_cs but at the end it calls ReleaseMutex for the pointer of init_cs and not for init_cs itself:

WaitForSingleObject(init_cs, INFINITE);
...
ReleaseMutex(&init_cs);


I am not 100% sure, but I believe that this is the cause why we cannot restart the filter without causing an infinite IIS hang.


ISAPI debug log:
[Tue Mar 01 11:29:36.039 2016] [6888:6180] [debug] handle_notify_event::jk_isapi_plugin.c (1908): Filter started
[Tue Mar 01 11:29:36.042 2016] [6888:6180] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1170): Attempting to map URI '/XXXXX.test/templates/XXXX/gfx/favicon.ico' from 16 maps
[Tue Mar 01 11:29:36.043 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/services/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.045 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/jsonapi/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.047 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/WEB-INF/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.053 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/it/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.055 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/fr/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.058 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/no/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.060 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/nl/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.062 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/de/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.064 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/cs/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.066 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/sk/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.068 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/en/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.069 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/*.svg=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.072 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/*.jsp=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.073 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/services/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.075 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/jsonapi/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.077 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/WEB-INF/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.083 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/it/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.085 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/fr/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.087 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/vg/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.089 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/no/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.092 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/nl/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.093 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/de/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.095 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/cs/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.097 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/sk/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.100 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/en/*=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.103 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/*.svg=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.106 2016] [6888:6180] [debug] find_match::jk_uri_worker_map.c (984): Attempting to map context URI '/*.jsp=ajp13' source 'uriworkermap'
[Tue Mar 01 11:29:36.108 2016] [6888:6180] [debug] handle_notify_event::jk_isapi_plugin.c (2133): [/templates/XXXX/gfx/favicon.ico] is not a servlet url
[Tue Mar 01 11:29:36.854 2016] [6888:4000] [info] TerminateFilter::jk_isapi_plugin.c (2383): Jakarta/ISAPI/isapi_redirector/1.2.40 stopping
[Tue Mar 01 11:29:36.900 2016] [6888:4000] [debug] close_workers::jk_worker.c (217): close_workers will destroy worker ajp13
[Tue Mar 01 11:29:36.902 2016] [6888:4000] [debug] ajp_destroy::jk_ajp_common.c (3091): up to 250 endpoints to close
[Tue Mar 01 11:29:36.904 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 716
[Tue Mar 01 11:29:36.905 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 716 [127.0.0.1:56160 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.908 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 716 [127.0.0.1:56160 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.910 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 716 [127.0.0.1:56160 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.912 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 752
[Tue Mar 01 11:29:36.916 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 752 [127.0.0.1:56161 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.918 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 752 [127.0.0.1:56161 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.920 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 752 [127.0.0.1:56161 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.922 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 808
[Tue Mar 01 11:29:36.923 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 808 [127.0.0.1:56162 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.925 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 808 [127.0.0.1:56162 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.927 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 808 [127.0.0.1:56162 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.930 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 816
[Tue Mar 01 11:29:36.931 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 816 [127.0.0.1:56163 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.933 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 816 [127.0.0.1:56163 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.935 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 816 [127.0.0.1:56163 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.937 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 824
[Tue Mar 01 11:29:36.940 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 824 [127.0.0.1:56164 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.945 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 824 [127.0.0.1:56164 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.948 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 824 [127.0.0.1:56164 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.951 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket 740
[Tue Mar 01 11:29:36.953 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (863): About to shutdown socket 740 [127.0.0.1:56165 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.955 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (934): shutting down the read side of socket 740 [127.0.0.1:56165 -> 127.0.0.1:8010]
[Tue Mar 01 11:29:36.956 2016] [6888:4000] [debug] jk_shutdown_socket::jk_connect.c (945): Shutdown socket 740 [127.0.0.1:56165 -> 127.0.0.1:8010] and read 0 lingering bytes in 0 sec.
[Tue Mar 01 11:29:36.958 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.960 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.962 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.964 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.966 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.967 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.969 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.971 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.974 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.976 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.978 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.980 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.982 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.984 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.986 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.988 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.990 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.991 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.993 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.995 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.997 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:36.998 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.000 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.001 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.003 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.005 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.007 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.009 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.011 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.012 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.013 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.015 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.016 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.018 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.021 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.022 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.025 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.027 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.028 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.030 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.032 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.033 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.035 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.037 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.039 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.041 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.043 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.045 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.047 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.048 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.051 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.053 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.055 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.056 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.058 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.059 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.062 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.063 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.065 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.067 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.068 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.071 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.072 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.074 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.076 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.078 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.080 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.083 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.084 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.086 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.088 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.090 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.092 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.093 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.095 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.098 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.100 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.102 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.104 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.106 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.107 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.109 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.111 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.113 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.117 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.119 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.121 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.123 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.125 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.126 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.128 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.130 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.132 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.133 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.135 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.136 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.138 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.141 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.142 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.144 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.145 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.148 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.149 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.151 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.152 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.154 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.156 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.159 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.160 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.162 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.164 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.166 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.168 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.170 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.172 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.174 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.175 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.177 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.179 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.181 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.183 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.185 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.187 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.189 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.192 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.193 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.195 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.197 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.200 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.202 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.203 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.205 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.208 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.210 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.211 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.213 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.215 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.217 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.219 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.221 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.222 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.223 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.226 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.227 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.230 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.231 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.233 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.235 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.237 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.238 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.240 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.242 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.245 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.246 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.249 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.251 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.253 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.254 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.256 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.258 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.259 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.261 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.263 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.265 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.267 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.269 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.271 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.273 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.275 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.277 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.279 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.281 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.283 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.284 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.287 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.289 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.291 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.292 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.294 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.297 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.299 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.301 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.302 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.305 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.306 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.309 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.311 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.312 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.314 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.316 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.318 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.321 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.323 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.324 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.326 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.328 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.329 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.332 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.334 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.336 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.338 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.339 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.341 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.343 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.345 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.347 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.348 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.350 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.352 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.354 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.355 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.357 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.359 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.361 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.362 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.364 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.366 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.368 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.370 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.371 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.374 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.376 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.378 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.380 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.382 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.383 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.386 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.387 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.389 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.391 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.393 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.395 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.397 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.399 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.401 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.403 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.405 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.406 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.408 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.410 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.412 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.413 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.415 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.418 2016] [6888:4000] [debug] ajp_close_endpoint::jk_ajp_common.c (828): (ajp13) closing endpoint with socket -1 (socket shutdown)
[Tue Mar 01 11:29:37.420 2016] [6888:4000] [debug] jk_shm_close::jk_shm.c (360): Closed shared memory Global\JK_XXXX_TEST_3_JAKARTA_ISAPI_REDIRECT childs=1
Comment 1 Thomas Reiser 2016-03-01 12:03:45 UTC
Created attachment 33615 [details]
Proposed fix
Comment 2 Matthew Reiter 2016-03-07 15:14:53 UTC
This is the same defect as https://bz.apache.org/bugzilla/show_bug.cgi?id=58813 (Incoming requests hang after a website using the ISAPI connector is restarted).
Comment 3 Nolan 2016-05-07 06:24:12 UTC
(In reply to Thomas Reiser from comment #1)
> Created attachment 33615 [details]
> Proposed fix

I have verified this patch corrects the issue in 59334, 58813, 59092
Comment 4 Mark Thomas 2016-09-12 19:07:28 UTC

*** This bug has been marked as a duplicate of bug 58813 ***