Bug 5380 - Using DOMParser results in hung threads in multi-threaded environment
Summary: Using DOMParser results in hung threads in multi-threaded environment
Status: NEW
Alias: None
Product: Xerces-J
Classification: Unclassified
Component: DTD (show other bugs)
Version: 1.4.3
Hardware: Sun Solaris
: P3 major
Target Milestone: ---
Assignee: Xerces-J Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2001-12-12 08:08 UTC by Shane Riddell
Modified: 2004-11-16 19:05 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shane Riddell 2001-12-12 08:08:47 UTC
Calls to parse() on a DOMParser appear to randomly block forever, preventing 
the calling thread from doing any more work.

This problem has been observed to occurr in a heavily multi-threaded 
application.

I have replicated it in a small test case (included below) on both Win2k and 
Solaris.  The following logs are from a Solaris run under JDK 1.3.1_01 using 
Xerces 1.4.3.  (I have also tried 1.4.4 and observed the same problem.)

The text.xml can be any xml file whose DOCTYPE references an external DTD 
served from an http server.  (I have not observed the problem when the DTD is 
on the local file system, only when it is on an http server separate from the 
server running the test.)

The problem occurs if xml validation is on, or if validation is off, but 
external DTD loading is still on.  

The only reliable way I have found to prevent the problem is to disable both 
validation and external DTD loading (features 
http://xml.org/sax/features/validation and 
http://apache.org/xml/features/nonvalidating/load-external-dtd both set to 
false).

I suspect the problem may be related to bug 1965, as it always seems to occurr 
after the test harness encounters random parse failures.

The following is the output from a brief test run of XmlTest.  After the output 
log indicated that a thread appeared to have stalled, I issued a kill to allow 
the shutdown hook to run and gracefully shutdown the VM.  However, the VM never 
exited, and continued to consume about 50% of the cpu.

Using "kill -QUIT" on the process yielded the following in my log:

Thread Thread[Thread-104,5,main] appears to have stalled
Thread Thread[Thread-75,5,main] appears to have stalled

Full thread dump:

"Thread-150" prio=5 tid=0xa3778 nid=0xa6 waiting on monitor 
[0xe3281000..0xe32819e0]
	at java.lang.Thread.sleep(Native Method)
	at XmlTest$1.run(XmlTest.java:85)

"SIGTERM handler" daemon prio=10 tid=0xa3648 nid=0xa5 waiting on monitor 
[0xe3381000..0xe33819e0]
	at java.lang.Object.wait(Native Method)
	at java.lang.Thread.join(Thread.java:930)
	at java.lang.Thread.join(Thread.java:983)
	at java.lang.Shutdown.runHooks(Shutdown.java:133)
	at java.lang.Shutdown.sequence(Shutdown.java:168)
	at java.lang.Shutdown.exit(Shutdown.java:213)
	at java.lang.Terminator$1.handle(Terminator.java:38)
	at sun.misc.Signal$1.run(Signal.java:198)
	at java.lang.Thread.run(Thread.java:484)

"Thread-104" prio=5 tid=0x1ea788 nid=0x74 waiting for monitor entry 
[0xe6180000..0xe61819e0]
	at sun.net.ProgressData.update(ProgressData.java:103)
	at sun.net.www.MeteredStream.justRead(MeteredStream.java:45)
	at sun.net.www.MeteredStream.skip(MeteredStream.java:81)
	at sun.net.www.http.KeepAliveStream.close(KeepAliveStream.java:64)
	at sun.net.www.MeteredStream.justRead(MeteredStream.java:40)
	at sun.net.www.MeteredStream.read(MeteredStream.java:69)
	at org.apache.xerces.utils.ChunkyByteArray.read
(ChunkyByteArray.java:131)
	at org.apache.xerces.readers.UTF8Reader.fillCurrentChunk
(UTF8Reader.java:2762)
	at org.apache.xerces.readers.UTF8Reader.slowLoadNextByte
(UTF8Reader.java:152)
	at org.apache.xerces.readers.UTF8Reader.lookingAtValidChar
(UTF8Reader.java:292)
	at org.apache.xerces.framework.XMLDTDScanner.scanComment
(XMLDTDScanner.java:972)
	at org.apache.xerces.framework.XMLDTDScanner.scanDecls
(XMLDTDScanner.java:1418)
	at org.apache.xerces.framework.XMLDocumentScanner.scanDoctypeDecl
(XMLDocumentScanner.java:2147)
	at org.apache.xerces.framework.XMLDocumentScanner.access$0
(XMLDocumentScanner.java:2100)
	at 

org.apache.xerces.framework.XMLDocumentScanner$PrologDispatcher.dispatch
(XMLDocumentScanner.java:831)
	at org.apache.xerces.framework.XMLDocumentScanner.parseSome
(XMLDocumentScanner.java:381)
	at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1098)
	at XmlTest.run(XmlTest.java:33)

"Thread-75" prio=5 tid=0x1d7748 nid=0x57 runnable [0xe7e80000..0xe7e819e0]
	at java.util.Observable.clearChanged(Observable.java:171)
	at java.util.Observable.notifyObservers(Observable.java:138)
	at sun.net.ProgressData.update(ProgressData.java:124)
	at sun.net.www.MeteredStream.justRead(MeteredStream.java:45)
	at sun.net.www.MeteredStream.skip(MeteredStream.java:81)
	at sun.net.www.http.KeepAliveStream.close(KeepAliveStream.java:64)
	at org.apache.xerces.utils.ChunkyByteArray.close
(ChunkyByteArray.java:205)
	at org.apache.xerces.readers.UTF8Reader.fillCurrentChunk
(UTF8Reader.java:2770)
	at org.apache.xerces.readers.UTF8Reader.slowLoadNextByte
(UTF8Reader.java:152)
	at org.apache.xerces.readers.UTF8Reader.lookingAtValidChar
(UTF8Reader.java:292)
	at org.apache.xerces.framework.XMLDTDScanner.scanComment
(XMLDTDScanner.java:972)
	at org.apache.xerces.framework.XMLDTDScanner.scanDecls
(XMLDTDScanner.java:1418)
	at org.apache.xerces.framework.XMLDocumentScanner.scanDoctypeDecl
(XMLDocumentScanner.java:2147)
	at org.apache.xerces.framework.XMLDocumentScanner.access$0
(XMLDocumentScanner.java:2100)
	at 

org.apache.xerces.framework.XMLDocumentScanner$PrologDispatcher.dispatch
(XMLDocumentScanner.java:831)
	at org.apache.xerces.framework.XMLDocumentScanner.parseSome
(XMLDocumentScanner.java:381)
	at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1098)
	at XmlTest.run(XmlTest.java:33)

"Signal Dispatcher" daemon prio=10 tid=0xb1a08 nid=0xa waiting on monitor 
[0..0xfae81a48]

"Finalizer" daemon prio=5 tid=0xadf60 nid=0x7 waiting on monitor 
[0xfaf81000..0xfaf819e0]
	at java.lang.Object.wait(Native Method)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:108)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:123)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:162)

"Reference Handler" daemon prio=5 tid=0xac638 nid=0x6 waiting on monitor 
[0xfe281000..0xfe2819e0]
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:420)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:110)

"main" prio=5 tid=0x29170 nid=0x1 waiting on monitor [0xffbee000..0xffbeeeb4]
	at java.lang.Thread.sleep(Native Method)
	at XmlTest.main(XmlTest.java:106)

"VM Thread" prio=5 tid=0xab9d8 nid=0x4 runnable 

"VM Periodic Task Thread" prio=10 tid=0xafda8 nid=0x8 waiting on monitor 
"Suspend Checker Thread" prio=10 tid=0xafe60 nid=0x9 runnable 


Showing that the stalled threads 75 and 104.  Every time I have run this (and 
in my real application as well) has shown a similar stack trace.

The following is the code I used to replicate the problem (based on the test 
from bug report 1965):

import javax.xml.parsers.*;
import org.xml.sax.XMLReader;
import org.xml.sax.*;
import org.apache.xerces.parsers.DOMParser;
import java.io.*;
import java.util.*;

public class XmlTest extends Thread {
  private boolean _stop = false;
  private String _input;
  private boolean _isRunning;
  private static boolean _keepRunning = true;

  private static int _threadCount = 150;
  private static XmlTest _threads[] = new XmlTest[_threadCount];
  private static Hashtable _threadMap = new Hashtable();


  public XmlTest(String input) {
    _input = input;
  }

  public void run() {
    int i = 0;
    _isRunning = true;
    //SAXParser parser = parserFactory.newSAXParser();
    //XMLReader reader = parser.getXMLReader();
    

    while (!_stop) {
      try {
          DOMParser parser = new DOMParser();
          parser.parse( new InputSource( new StringReader( _input ) ) );
          
      } catch (Exception ex) {
        ex.printStackTrace( System.out );
      } finally {
          _threadMap.put( this, new Long( System.currentTimeMillis() ) );
      }
    }
    _threadMap.remove( this );
    _isRunning = false;
  }

    public boolean isRunning() {
        return _isRunning;
    }

    public static void main(String[] args) throws Exception {
        
        File file = new File( "test.xml" );
        BufferedReader reader = new BufferedReader( new FileReader( file ) );
        StringBuffer input = new StringBuffer();
        while (reader.ready()) {
            input.append( reader.readLine() );
        }
    
    
        for (int i = 0; i < _threads.length; i++) {
            _threads[i] = new XmlTest( input.toString() );
            _threadMap.put( _threads[i], new Long( System.currentTimeMillis
() ) );
            _threads[i].start();
        }
        System.out.println( "started " + _threads.length + " threads" );
        BufferedReader in = new BufferedReader( new InputStreamReader( 
System.in ) );

        Runtime.getRuntime().addShutdownHook( new Thread() {
                    public void run() {

                        System.out.println( "stopping threads" );
                        for (int i = 0; i < _threads.length; i++) {
                            _threads[i]._stop = true;
                        }
                    
                        boolean isRunning = true;
                        while (isRunning) {
                            isRunning = false;
                            for (int i = 0; i < _threads.length; i++) {
                                if (_threads[i].isRunning()) {
                                    isRunning = true;
                                } else {
                                    //System.out.println( "Thread " + i + " 
stopped" );
                                }
                            }
                            try { Thread.currentThread().sleep( 5000 ); } catch 
(InterruptedException e) {}
                        }

                        _keepRunning = false;
                        System.out.println( "All threads stopped" );
                    }
                }
            );



        while (_keepRunning) {
            Hashtable map = (Hashtable) _threadMap.clone();
            for (Enumeration e = map.keys(); e.hasMoreElements();) {
                Object key = e.nextElement();
                Long lastTime = (Long) map.get( key );
                if (System.currentTimeMillis()-lastTime.longValue() > 
2*60*1000) {
                    System.out.println( "Thread " + key + " appears to have 
stalled" );
                }
            }

            try { Thread.currentThread().sleep( 30*1000 ); } catch 
(InterruptedException e) {}

        }

        

            
    }
}