This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 79407 - Deadlock in TaskThreadGroupGCTest
Summary: Deadlock in TaskThreadGroupGCTest
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Petr Nejedly
URL: http://beetle.czech:8080/unittest/Tes...
Keywords: TEST
Depends on:
Blocks:
 
Reported: 2006-06-29 09:48 UTC by Jaroslav Tulach
Modified: 2008-12-22 17:46 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
The statistic of the failure (6.72 KB, text/html)
2006-06-29 09:50 UTC, Jaroslav Tulach
Details
the test deadlocked (9.75 KB, text/plain)
2006-07-11 10:56 UTC, Jaroslav Tulach
Details
Log with thread dump (18.01 KB, text/plain)
2006-09-12 09:51 UTC, Jaroslav Tulach
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jaroslav Tulach 2006-06-29 09:48:45 UTC
CallbackSystemActionTest does not finish all the time which is impossible as 
the test has timeout in itself and has to finish after each 5s.


Please find the root cause of the failure
Comment 1 Jaroslav Tulach 2006-06-29 09:50:23 UTC
Created attachment 31500 [details]
The statistic of the failure
Comment 2 Jaroslav Tulach 2006-06-30 10:20:29 UTC
Why xtest does not provide link to workdir when the test does not finish? The 
workdir is there, created, but the xtest does not link to it. That is stange.
Comment 3 Jiri Skrivanek 2006-06-30 12:49:24 UTC
Because XTest is checking whether workdir was used or not in
TestListener.endTest(). And it is never called when test crashes.
Comment 4 Jaroslav Tulach 2006-07-10 09:21:43 UTC
I am not saying the xtest has to be fixed, but I need to find out what is 
happening with the failing test runs. Does anyone has access to those 
machines, can you run the tests by hand? Attach thread dump, etc.?
Comment 5 Jaroslav Tulach 2006-07-11 09:42:05 UTC
Next failure:
http://beetle.czech:8080/unittest/TestCase.jsp?issues=3735
Comment 6 Jaroslav Tulach 2006-07-11 10:56:41 UTC
Created attachment 31722 [details]
the test deadlocked
Comment 7 Jaroslav Tulach 2006-07-11 10:59:10 UTC
Ok, I've managed to simulate the problem. There is a deadlock between two(!) 
finalizer threads and AWT one.

However shall not the xtest time the test out, generate thread dump and report 
it?
Comment 8 Petr Nejedly 2006-07-13 10:29:16 UTC
I have similar problem, my test randomly failed with "Suite is not finished":
http://ffjqa.czech/automatedtests/xtest/netbeans_5.5/200607130000/qa-unit/qa-linux-s1/testrun_060713-052259/testbag_6/htmlresults/suites/TEST-org.openide.explorer.view.TreeView48993Test.html

I can't evaluate the failure w/o additional info, I need a thread dump from that
state as well.
Comment 9 Jiri Skrivanek 2006-07-18 10:05:37 UTC
XTest generates thread dump only when run in IDE mode. I will try to look at
possibility to generate thread dump when running in jvm mode.
Comment 10 Jaroslav Tulach 2006-08-14 14:20:45 UTC
Another failure:
http://beetle.czech:8080/unittest/TestCase.jsp?issues=2053
Comment 11 Jaroslav Tulach 2006-08-15 13:06:06 UTC
I have added basic ability to generate thread dumps to the NbTestCase itself. 
But this may not be enough as the java usually deadlocks itself and then it 
prints nothing.

--
IDE: [15.8.06 14:02] Committing "NB JUnit" started
Checking in test/unit/src/org/netbeans/junit/TimeOutHasToPrintLogTest.java;
/shared/data/ccvs/repository/xtest/nbjunit/test/unit/src/org/netbeans/junit/TimeOutHasToPrintLogTest.java,v  
<--  TimeOutHasToPrintLogTest.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/org/netbeans/junit/Log.java;
/shared/data/ccvs/repository/xtest/nbjunit/src/org/netbeans/junit/Log.java,v  
<--  Log.java
new revision: 1.10; previous revision: 1.9
done
Checking in src/org/netbeans/junit/NbTestCase.java;
/shared/data/ccvs/repository/xtest/nbjunit/src/org/netbeans/junit/NbTestCase.java,v  
<--  NbTestCase.java
new revision: 1.50; previous revision: 1.49
Comment 12 Jiri Skrivanek 2006-08-30 09:18:10 UTC
Fixed. Not thread dump is printed out and process is killed when xtest.timeout
expires.

Checking in
plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JVMExecuteWatchdog.java;
/cvs/xtest/plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JVMExecuteWatchdog.java,v
 <--  JVMExecuteWatchdog.java
initial revision: 1.1
done
Checking in
plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JVMTestRunnerTask.java;
/cvs/xtest/plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JVMTestRunnerTask.java,v
 <--  JVMTestRunnerTask.java
new revision: 1.8; previous revision: 1.7
done
Checking in
plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JUnitTestRunnerLauncher.java;
/cvs/xtest/plugins_src/jvm/src/org/netbeans/xtest/plugin/jvm/JUnitTestRunnerLauncher.java,v
 <--  JUnitTestRunnerLauncher.java
new revision: 1.6; previous revision: 1.5
Comment 13 Petr Nejedly 2006-09-12 09:12:53 UTC
One of my test cases is still failing on timeout w/o any thread dump provided.
See (internal) http://beetle.czech:8080/unittest/TestCase.jsp?issues=2641
Comment 14 Jiri Skrivanek 2006-09-12 09:27:01 UTC
Please, look at latest results. A thread dump produced by NbTestCase is here:

http://beetle.czech/automatedtests/xtest/netbeans_dev/200609091800/qa-unit/qa-sol10-s5_1/testrun_060910-220556/logs/core_execution_unit.log

A real thread dump would be generated if a test case is locked but timeout is
not set in NbTestCase.
Comment 15 Jaroslav Tulach 2006-09-12 09:47:27 UTC
This seems to be fixed, I can see the thread dump. Looks like a deadlock in 
AWT on solaris...
Comment 16 Jaroslav Tulach 2006-09-12 09:48:23 UTC
Reassigning to Petr to fix.
Comment 17 Jaroslav Tulach 2006-09-12 09:51:15 UTC
Created attachment 33804 [details]
Log with thread dump
Comment 18 Jaroslav Tulach 2006-09-12 09:51:52 UTC
Imho, the problematic part is:

[jvmTestRunner]     sun.misc.Unsafe.park:-2
[jvmTestRunner]     java.util.concurrent.locks.LockSupport.park:158
[jvmTestRunner]     
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt:712
[jvmTestRunner]     
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued:743
[jvmTestRunner]     
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire:1079
[jvmTestRunner]     
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock:186
[jvmTestRunner]     java.util.concurrent.locks.ReentrantLock.lock:262
[jvmTestRunner]     sun.awt.SunToolkit.awtLock:245
[jvmTestRunner]     sun.awt.X11.XWM.removeSizeHints:720
[jvmTestRunner]     sun.awt.X11.XDecoratedPeer.setVisible:920
[jvmTestRunner]     sun.awt.X11.XFramePeer.setVisible:275

Petr, do you have more insight to what is going on?
Comment 19 Petr Nejedly 2006-09-12 12:19:46 UTC
I don't understand few things now.
First, it was really far from obvious to get from the failure to the build log,
where the thread dump indeed was. Why isn't the thread dump part of the failure
message? Well, it is generated just a line before triggering the test failure
anyway! (org.netbeans.junit.NbTestCase$1Guard.waitFinished:212)
And when we're at it, can the infractructure list the threads hierarchically
(i.e. according to their nested thread groups)? It could help me diagnose this
issue by making sure there is no runaway thread in the execution ThreadGroup...

Second, if I don't specify the timeout, it would provide real (SIGQUIT) thread
dump after some default timeout? How long is that timeout?

Third, I really don't understand what's going on in the log from 20060909 (it
might as well still be processing), but the latest log from 20060911 looks like
the engine doesn't recoginzed the processing is over, which may be due to wrong
synchronization is TaskThreadGroup.waitFor;

I'll try to fix the synchronization problem, just to be sure, while I think it
is mostly hypothetical.

Thanks for wide cooperation.

Comment 20 Petr Nejedly 2006-09-12 14:56:58 UTC
Imroved synchronization:
core/execution/src/org/netbeans/core/execution/TaskThreadGroup.java,v1.7

Jirko, please see my previous comment. I haven't realize you're not on CC anymore.
Comment 21 Jiri Skrivanek 2006-09-12 15:33:58 UTC
ad 1) It is a question for Jarda who added thread dumping feature to NbTestCase.
ad 2) Yes. xtest.timeout=2400000 (40 minutes)
Comment 22 Jaroslav Tulach 2006-09-13 09:33:17 UTC
ad1) go on, write a test and better impl in NbTestCase if you want.
Comment 23 Petr Nejedly 2006-09-18 13:53:22 UTC
Failed again w/o visible reason.
I'll improve the deadlock reporting infrastructure.
Comment 24 Petr Nejedly 2006-09-19 15:42:19 UTC
Better thread dump reporting implemented:

xtest/nbjunit/src/org/netbeans/junit/Log.java,v1.11
xtest/nbjunit/src/org/netbeans/junit/NbTestCase.java,v1.55
xtest/nbjunit/test/unit/src/org/netbeans/junit/TimeOutHasToPrintLogTest.java,v1.3

Jirko, can you ensure it gets deployed quickly onto the testing machines, please?
Comment 25 Petr Nejedly 2006-09-27 09:57:48 UTC
Test failed again. The failure is caused by AWT shudwown thread sneaking into
execution thread group.
Comment 26 Petr Nejedly 2006-09-27 11:11:38 UTC
Forced the AWT shutdown thread to stay in proper thread group and not leak into
the execution TG.
core/execution/test/unit/src/org/netbeans/core/execution/TaskThreadGroupGCTest.java,v1.7
Comment 27 Jaroslav Tulach 2006-10-03 12:02:24 UTC
 
 
testTTGGC:
      java.lang.reflect.InvocationTargetException
        at java.awt.EventQueue.invokeAndWait(EventQueue.java:851)
        at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1257)
        at 
org.netbeans.core.execution.TaskThreadGroupGCTest.setUp(TaskThreadGroupGCTest.java:46)
        at org.netbeans.junit.NbTestCase.runBare(NbTestCase.java:274)
        at junit.framework.TestResult$1.protect(TestResult.java:106)
        at junit.framework.TestResult.runProtected(TestResult.java:124)
        at junit.framework.TestResult.run(TestResult.java:109)
        at junit.framework.TestCase.run(TestCase.java:120)
        at org.netbeans.junit.NbTestCase.run(NbTestCase.java:161)
        at junit.framework.TestSuite.runTest(TestSuite.java:230)
        at junit.framework.TestSuite.run(TestSuite.java:225)
        at 
org.netbeans.xtest.testrunner.JUnitTestRunner.runTests(JUnitTestRunner.java:173)
        at 
org.netbeans.xtest.testrunner.JUnitTestRunner.runTests(JUnitTestRunner.java:124)
        at 
org.netbeans.xtest.plugin.jvm.JUnitTestRunnerLauncher.main(JUnitTestRunnerLauncher.java:60)
Caused by: java.lang.IllegalArgumentException: null owner window
        at java.awt.Window.ownedInit(Window.java:414)
        at java.awt.Window.<init>(Window.java:346)
        at java.awt.Dialog.<init>(Dialog.java:222)
        at java.awt.Dialog.<init>(Dialog.java:177)
        at 
org.netbeans.core.execution.TaskThreadGroupGCTest$1.run(TaskThreadGroupGCTest.java:48)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:199)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:461)
        at 
java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:242)
        at 
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:163)
        at 
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:157)
        at 
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:149)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:110)

   
Comment 28 Petr Nejedly 2006-10-10 09:50:27 UTC
AWT doesn't like null-parent dialogs:
core/execution/test/unit/src/org/netbeans/core/execution/TaskThreadGroupGCTest.java,v1.8