I ran gdb against the hung process and got this

0xb7b69f8e in read () from /lib/libc.so.6
(gdb) where
#0  0xb7b69f8e in read () from /lib/libc.so.6
#1  0xb7c4b967 in apr_socket_recv () from /usr/local/apache2/lib/libapr-1.so.0
#2  0xb7d72403 in socket_bucket_read () from /usr/local/apache2/lib/libaprutil-1.so.0
#3  0x0808133e in ap_core_input_filter ()
#4  0x0808f6e0 in ap_get_brigade ()
#5  0x080dba77 in ap_http_filter ()
#6  0x0808f6e0 in ap_get_brigade ()
#7  0x080daad9 in check_pipeline_flush ()
#8  0x080dac09 in ap_process_request ()
#9  0x080d7616 in ap_process_http_connection ()
#10 0x0808b452 in ap_run_process_connection ()
#11 0x0808b89b in ap_process_connection ()
#12 0x08114a92 in child_main ()
#13 0x08114c84 in make_child ()
#14 0x08114ce3 in startup_children ()
#15 0x08115173 in ap_mpm_run ()
#16 0x0806c6ed in main ()

Trying to figure out if a different code path is used when cross compiled

Started working fine when i added  ac_cv_o_nonblock_inherited=no in configure script

    CFLAGS="$(TARGET_CFLAGS) -DBIG_SECURITY_HOLE " \
    ac_cv_file__dev_zero=yes ac_cv_func_setpgrp_void=yes
apr_cv_process_shared_works=yes apr_cv_mutex_robust_shared=no
apr_cv_tcp_nodelay_with_cork=yes apr_cv_mutex_recursive=yes
ap_cv_void_ptr_lt_long=no ac_cv_sizeof_struct_iovec=8 ac_cv_struct_rlimit=yes \
ac_cv_o_nonblock_inherited=no \
    ./configure \
    --with-mpm=prefork \
    --with-included-apr \
    --target=$(GNU_TARGET_NAME) \
    --host=$(GNU_TARGET_NAME) \
    --build=$(GNU_HOST_NAME) \
    --prefix=/usr/local/apache2 \
    );

It would be nice to have it documented or have a cleaner fix for this.

I ran into another symptom of this problem when cross-compiling 2.2.22 for Linux on PowerPC.  

When configuring a directory to require client certificates, like this:

Directory "/ramdisk/www/cert_area">
   SSLVerifyDepth 1
   SSLVerifyClient require

the renegotiation hangs every time.  The call stack looks like this:

#0  0x0fcfeae8 in __read_nocancel () from /usr/lib/libpthread.so.0
#1  0x0fc41678 in apr_socket_recv
#2  0x0fcbf9d0 in socket_bucket_read 
#3  0x1006792c in ap_core_input_filter
#4  0x10072d0c in ap_get_brigade
#6  0x0fec5a30 in BIO_read
#7  0x0ffb6b9c in ssl3_read_n
#8  0x0ffb7960 in ssl3_read_bytes
#9  0x0ffb5444 in ssl3_read
#10 0x0ffc7b64 in SSL_read
#11 0x10099920 in ssl_io_input_read
#12 0x1009a588 in ssl_io_filter_input
#13 0x10072d0c in ap_get_brigade 
#14 0x1009c318 in ssl_hook_Access
#15 0x10064e64 in ap_run_access_checker
#16 0x100669f8 in ap_process_request_internal
#17 0x100a9a38 in ap_process_request
#18 0x100a6e88 in ap_process_http_connection
#19 0x1006f70c in ap_run_process_connection
#20 0x100c38e4 in child_main
#21 0x100c3a90 in make_child
#22 0x100c4354 in ap_mpm_run
#23 0x1005693c in main

I verified in apr_socket_recv/socket_bucket_read that it expects the socket read to be non-blocking, but it blocks.

This is the higher-level functionality that is trying to be accomplished:

/* Perform a speculative (and non-blocking) read from the connection
 * filters for the given request, to determine whether there is any
 * pending data to read.  Return non-zero if there is, else zero. */ static int has_buffered_data(request_rec *r) {
    apr_bucket_brigade *bb;
    apr_off_t len;
    apr_status_t rv;
    int result;
    
    bb = apr_brigade_create(r->pool, r->connection->bucket_alloc);
    
    rv = ap_get_brigade(r->connection->input_filters, bb, AP_MODE_SPECULATIVE,
                        APR_NONBLOCK_READ, 1);    <<<< HANGS HERE

This was a nasty debug.  I ended up building the same version of Apache natively for Unbuntu where I did not find the problem, and then compared the build configuration.

Is there a better way to handle this compilation flag than just defaulting it to true for cross compiles?  If it isn't possible to automatically determine the proper value for cross compiles at configuration time, could it be defaulted to an invalid value that would force the user to set ac_cv_o_nonblock_inherited prior to running configure?

I have cross compiled and installed Apache 2.4.4 and 2.4.9 for my cusotm Linux Board.

Whenever I tried to listen on multiple listen ports like
Listen 80
Listen 8080
at that time, HTTP request goes into hang state and can not come out for that state.
It works find with single Listen Directive for my linux custom board.

I have also added following flags in configuration script of httpd to solve that issue but still failed to execute HTTP request.

ac_cv_file__dev_zero=yes ac_cv_func_setpgrp_void=yes
apr_cv_process_shared_works=yes apr_cv_mutex_robust_shared=no
apr_cv_tcp_nodelay_with_cork=yes apr_cv_mutex_recursive=yes
ap_cv_void_ptr_lt_long=no ac_cv_sizeof_struct_iovec=8 ac_cv_struct_rlimit=yes \
ac_cv_o_nonblock_inherited=no



I have also debug httpd process using starce and found that read system call goe into blocking state which never returns.

$ ./strace -p 1811
Process 1811 attached
restart_syscall(<... resuming interrupted call ...>) = 0
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 0 (Timeout)
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 0 (Timeout)
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 1 ([{fd=4, revents=POLLIN}])
accept(4, {sa_family=AF_INET6, sin6_port=htons(40827), inet_pton(AF_INET6, "::ffff:192.168.0.45", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 11
fcntl64(11, F_GETFD)                    = 0
fcntl64(11, F_SETFD, FD_CLOEXEC)        = 0
semop(950296, {{0, 1, SEM_UNDO}}, 1)    = 0
gettimeofday({1405747420, 372062}, NULL) = 0
getsockname(11, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "::ffff:192.168.0.183", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
gettimeofday({1405747420, 374162}, NULL) = 0
gettimeofday({1405747420, 374976}, NULL) = 0
read(11, "GET / HTTP/1.1\r\nHost: 192.168.0."..., 8000) = 289
gettimeofday({1405747420, 377850}, NULL) = 0
gettimeofday({1405747420, 380080}, NULL) = 0
gettimeofday({1405747420, 382529}, NULL) = 0
gettimeofday({1405747420, 384638}, NULL) = 0
gettimeofday({1405747420, 386798}, NULL) = 0
gettimeofday({1405747420, 388903}, NULL) = 0
gettimeofday({1405747420, 390971}, NULL) = 0
gettimeofday({1405747420, 393006}, NULL) = 0
gettimeofday({1405747420, 395390}, NULL) = 0
gettimeofday({1405747420, 397474}, NULL) = 0
stat64("/usr/local/apache2/htdocs/", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
stat64("/usr/local/apache2/htdocs/index.html", {st_mode=S_IFREG|0777, st_size=45, ...}) = 0
open("/usr/local/apache2/htdocs/index.html", O_RDONLY|O_CLOEXEC) = 12
open("/etc/localtime", O_RDONLY)        = 13
fstat64(13, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
fstat64(13, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a01d000
read(13, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 127
_llseek(13, -11, [116], SEEK_CUR)       = 0
read(13, "\n<GMT-8>-8\n", 4096)         = 11
close(13)                               = 0
munmap(0x2a01d000, 4096)                = 0
gettimeofday({1405747420, 441452}, NULL) = 0
read(11, 

Please let me know if anyone has idea about this issue or any work arounds for that.

This bug appears to be related to bug 56727.

I have already tried with all possible compilation flags as well as some other configuration in httpd compilation process to listen multiple ports but still failed to send HTTP request.

Then I have started debugging task ftom httpd source code and found that request is stuck in check_pipeline() function of http_request.c file.

There is one function call 
ap_get_brigade(c->input_filters, bb, AP_MODE_SPECULATIVE,APR_NONBLOCK_READ, 1) in check_pipeline() function which never returns after sending HTTP request with multiple Listen Directive.

ap_get_brigade() function defined in server/util_filter.c file

AP_DECLARE(apr_status_t) ap_get_brigade(ap_filter_t *next,
                                        apr_bucket_brigade *bb,
                                        ap_input_mode_t mode,
                                        apr_read_type_e block,
                                        apr_off_t readbytes)
{
    if (next) {
        return next->frec->filter_func.in_func(next, bb, mode, block,
                                               readbytes);
    }
    return AP_NOBODY_READ;
}

So, This above function call never returns and blocked forever from check_pipeline() function call after sending HTTP request on multiple port Listen.

Please let me know if any one has idea about this.

I have debugged ap_get_brigade() function in details from httpd source code and found that it executes ap_core_input_filter() function from core_filters.c file.

Then, I have looked into ap_core_input_filter() function definition and found that HTTP request execution with multiple listen ports stuck after calling apr_input_read() function which is called from ap_core_input_filter() API.

After that, I have looked for apr_read_bucket() function definition which is defined as read() system call in apr_buckets.h file of Apr-Util Package.

#define apr_bucket_read(e,str,len,block) (e)->type->read(e, str, len, block)

I have also downgraded and tested Apr and Apr-Util packages from 1.5.X to 1.4.X but still failed to execute HTTP request on multiple listen ports.

Please let me know if any one knows about this issue.

I have also found while debugging apr_bucket_read() function that read() system call executed from apr_socket_recv() function (network_io/unix/sendrecv.c file of Apr Package) blocks HTTP request after listening on multiple ports. 


apr_status_t apr_socket_recv(apr_socket_t *sock, char *buf, apr_size_t *len)
{
    apr_ssize_t rv;
    apr_status_t arv;

    if (sock->options & APR_INCOMPLETE_READ) {
        sock->options &= ~APR_INCOMPLETE_READ;
        goto do_select;
    }

    do {
        rv = read(sock->socketdes, buf, (*len));  ---> Never return from Here
    } while (rv == -1 && errno == EINTR);


That apr_socket_recv() function is called from socket_bucket_read() function of Apr-Util source package (buckets/apr_buckets_socket.c file).

And that socket_bucket_read() function is called from ap_get_brigade() API (util_filters.c file) of http source package to read data from bucket.

So, this issue is caused while reading data from bucket from httpd->Apr-Util->Apr when sending HTTP request on multiple Listen Ports.

We have also checked NON_Block Mode using apr_is_option_set() function and it gives that APR mode is set as NON_Block Mode but still HTTP request never returns from that read() system call of apr_socket_recv() function.

Please let me know if any one has idea about this.

We have debugged code of APR and found that there is one macro defined 
#define APR_O_NONBLOCK_INHERITED 1
which creates problem to create inherited socket as non block from listener socket while requesting HTTP page in multiple Listen Ports.

So, We have set ac_cv_o_nonblock_inherited=no in configuration options of APR Package and tested HTTP request on multiple Listen ports as well as enabling SSL Modules which works fine without any issue.

Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.