Bug 58382 - Data races inside org.apache.catalina.tribes.membership.Membership
Summary: Data races inside org.apache.catalina.tribes.membership.Membership
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 8
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 8.0.x-trunk
Hardware: PC Linux
: P2 normal (vote)
Target Milestone: ----
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-12 16:02 UTC by Yilong Li
Modified: 2015-09-15 12:46 UTC (History)
0 users


Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Yilong Li 2015-09-12 16:02:28 UTC 
Reported by RV-Predict (a dynamic race detector) when running the test suite:
Data race on field java.util.HashMap.$state: {{{
    Concurrent write in thread T189 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:167)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:525)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:176)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:79)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:251)
        at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:471)
        at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:383)
    T189 is created by T185
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent read in thread T1 (locks held: {Monitor@2b72a0b3, Monitor@16307bd4, Monitor@22551e94})
 ---->  at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:106)
        - locked Monitor@22551e94 at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:472)
        - locked Monitor@16307bd4 at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:463)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.stop(ChannelInterceptorBase.java:171)
        at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:444)
        - locked Monitor@2b72a0b3 at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.tearDown(TestNonBlockingCoordinator.java:127)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033)
    T1 is the main thread
}}} 

Data race on field java.util.HashMap.$state: {{{
    Concurrent write in thread T53 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:167)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:525)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:176)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:79)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:251)
        at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:471)
        at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:383)
    T53 is created by T32
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent read in thread T55 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:167)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:525)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:176)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:79)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:251)
        at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:471)
        at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:383)
    T55 is created by T32
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)
}}} 

Data race on field java.util.HashMap.$state: {{{
    Concurrent read in thread T250 (locks held: {Monitor@2c4e42fe, Monitor@ebeabe7})
 ---->  at org.apache.catalina.tribes.membership.Membership.expire(Membership.java:200)
        - locked Monitor@ebeabe7 at org.apache.catalina.tribes.membership.Membership.expire(Membership.java:n/a)
        at org.apache.catalina.tribes.membership.McastServiceImpl.checkExpired(McastServiceImpl.java:455)
        - locked Monitor@2c4e42fe at org.apache.catalina.tribes.membership.McastServiceImpl.checkExpired(McastServiceImpl.java:454)
        at org.apache.catalina.tribes.membership.McastServiceImpl.send(McastServiceImpl.java:512)
        at org.apache.catalina.tribes.membership.McastServiceImpl.send(McastServiceImpl.java:487)
        at org.apache.catalina.tribes.membership.McastServiceImpl$SenderThread.run(McastServiceImpl.java:575)
    T250 is created by T165
        at org.apache.catalina.tribes.membership.McastServiceImpl.start(McastServiceImpl.java:280)

    Concurrent write in thread T185 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:167)
        at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:376)
        at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:356)
        at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:541)
    T185 is created by T165
        at org.apache.catalina.tribes.membership.McastServiceImpl.start(McastServiceImpl.java:269)
}}} 

Data race on field java.util.HashMap.$state: {{{
    Concurrent read in thread T250 (locks held: {Monitor@2c4e42fe, Monitor@ebeabe7})
 ---->  at org.apache.catalina.tribes.membership.Membership.expire(Membership.java:201)
        - locked Monitor@ebeabe7 at org.apache.catalina.tribes.membership.Membership.expire(Membership.java:n/a)
        at org.apache.catalina.tribes.membership.McastServiceImpl.checkExpired(McastServiceImpl.java:455)
        - locked Monitor@2c4e42fe at org.apache.catalina.tribes.membership.McastServiceImpl.checkExpired(McastServiceImpl.java:454)
        at org.apache.catalina.tribes.membership.McastServiceImpl.send(McastServiceImpl.java:512)
        at org.apache.catalina.tribes.membership.McastServiceImpl.send(McastServiceImpl.java:487)
        at org.apache.catalina.tribes.membership.McastServiceImpl$SenderThread.run(McastServiceImpl.java:575)
    T250 is created by T165
        at org.apache.catalina.tribes.membership.McastServiceImpl.start(McastServiceImpl.java:280)

    Concurrent write in thread T185 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:167)
        at org.apache.catalina.tribes.membership.McastServiceImpl.memberDataReceived(McastServiceImpl.java:376)
        at org.apache.catalina.tribes.membership.McastServiceImpl.receive(McastServiceImpl.java:356)
        at org.apache.catalina.tribes.membership.McastServiceImpl$ReceiverThread.run(McastServiceImpl.java:541)
    T185 is created by T165
        at org.apache.catalina.tribes.membership.McastServiceImpl.start(McastServiceImpl.java:269)
}}}
Comment 1 Yilong Li 2015-09-12 16:57:22 UTC 
There are also many races on Membership.member:
Data race on field org.apache.catalina.tribes.membership.Membership.members: {{{
    Concurrent write in thread T66 (locks held: {Monitor@56b4bad2, Monitor@1a149173})
 ---->  at org.apache.catalina.tribes.membership.Membership.addMember(Membership.java:154)
        - locked Monitor@1a149173 at org.apache.catalina.tribes.membership.Membership.addMember(Membership.java:147)
        at org.apache.catalina.tribes.membership.Membership.memberAlive(Membership.java:125)
        - locked Monitor@56b4bad2 at org.apache.catalina.tribes.membership.Membership.memberAlive(Membership.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:514)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:509)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberAdded(TcpFailureDetector.java:141)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberAdded(DomainFilterInterceptor.java:65)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberAdded(ChannelCoordinator.java:245)
        at org.apache.catalina.tribes.membership.McastService.memberAdded(McastService.java:461)
        at org.apache.catalina.tribes.membership.McastServiceImpl$2.run(McastServiceImpl.java:397)
    T66 is created by T33
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent read in thread T47 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.hasMembers(Membership.java:225)
        at org.apache.catalina.tribes.membership.Membership.getMembers(Membership.java:250)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator$CoordinationEvent.<init>(NonBlockingCoordinator.java:779)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:516)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:509)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberAdded(TcpFailureDetector.java:141)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberAdded(DomainFilterInterceptor.java:65)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberAdded(ChannelCoordinator.java:245)
        at org.apache.catalina.tribes.membership.McastService.memberAdded(McastService.java:461)
        at org.apache.catalina.tribes.membership.McastServiceImpl$2.run(McastServiceImpl.java:397)
    T47 is created by T33
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)
}}} 

Data race on field org.apache.catalina.tribes.membership.Membership.members: {{{
    Concurrent write in thread T70 (locks held: {Monitor@56be514e, Monitor@20e11a66})
 ---->  at org.apache.catalina.tribes.membership.Membership.addMember(Membership.java:154)
        - locked Monitor@20e11a66 at org.apache.catalina.tribes.membership.Membership.addMember(Membership.java:147)
        at org.apache.catalina.tribes.membership.Membership.memberAlive(Membership.java:125)
        - locked Monitor@56be514e at org.apache.catalina.tribes.membership.Membership.memberAlive(Membership.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:514)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberAdded(NonBlockingCoordinator.java:509)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberAdded(TcpFailureDetector.java:141)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberAdded(DomainFilterInterceptor.java:65)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberAdded(ChannelInterceptorBase.java:87)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberAdded(ChannelCoordinator.java:245)
        at org.apache.catalina.tribes.membership.McastService.memberAdded(McastService.java:461)
        at org.apache.catalina.tribes.membership.McastServiceImpl$2.run(McastServiceImpl.java:397)
    T70 is created by T32
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent read in thread T105 (locks held: {Monitor@a2c0d51})
 ---->  at org.apache.catalina.tribes.membership.Membership.getMember(Membership.java:232)
        at org.apache.catalina.tribes.util.Arrays.diff(Arrays.java:160)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.mergeOnArrive(NonBlockingCoordinator.java:306)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.processCoordMessage(NonBlockingCoordinator.java:320)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.messageReceived(NonBlockingCoordinator.java:498)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:81)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailureDetector.java:117)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:81)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.messageReceived(DomainFilterInterceptor.java:48)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:81)
        at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:262)
        at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:242)
        at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:211)
        at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:102)
        - locked Monitor@a2c0d51 at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:n/a)
    T105 is created by T23
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)
}}} 

Data race on field org.apache.catalina.tribes.membership.Membership.members: {{{
    Concurrent read in thread T189 (locks held: {Monitor@32aaa2ff})
 ---->  at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:170)
        - locked Monitor@32aaa2ff at org.apache.catalina.tribes.membership.Membership.removeMember(Membership.java:168)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:525)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:176)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:79)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:251)
        at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:471)
        at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:383)
    T189 is created by T185
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent write in thread T1 (locks held: {Monitor@2b72a0b3, Monitor@16307bd4, Monitor@22551e94})
 ---->  at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:107)
        - locked Monitor@22551e94 at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:472)
        - locked Monitor@16307bd4 at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:463)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.stop(ChannelInterceptorBase.java:171)
        at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:444)
        - locked Monitor@2b72a0b3 at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.tearDown(TestNonBlockingCoordinator.java:127)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033)
    T1 is the main thread
}}} 

Data race on field org.apache.catalina.tribes.membership.Membership.members: {{{
    Concurrent read in thread T189 (locks held: {})
 ---->  at org.apache.catalina.tribes.membership.Membership.hasMembers(Membership.java:225)
        at org.apache.catalina.tribes.membership.Membership.getMembers(Membership.java:250)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator$CoordinationEvent.<init>(NonBlockingCoordinator.java:779)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:528)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:176)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:79)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:93)
        at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:251)
        at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:471)
        at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:383)
    T189 is created by T185
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:1010)

    Concurrent write in thread T1 (locks held: {Monitor@2b72a0b3, Monitor@16307bd4, Monitor@22551e94})
 ---->  at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:107)
        - locked Monitor@22551e94 at org.apache.catalina.tribes.membership.Membership.reset(Membership.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:472)
        - locked Monitor@16307bd4 at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.stop(NonBlockingCoordinator.java:463)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.stop(ChannelInterceptorBase.java:171)
        at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:444)
        - locked Monitor@2b72a0b3 at org.apache.catalina.tribes.group.GroupChannel.stop(GroupChannel.java:n/a)
        at org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.tearDown(TestNonBlockingCoordinator.java:127)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182)
        at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033)
    T1 is the main thread
}}}
Comment 2 Mark Thomas 2015-09-15 12:46:30 UTC 
Lots to fix in that class.

Fixed in trunk and 8.0.x for 8.0.27 onwards.