Author |
Message |
08/15/2008 03:34:13
|
Shake
master
Joined: 01/05/2008 07:27:49
Messages: 84
Offline
|
Hi,
I suddently got that dead lock (after several weeks without any problems).
But I don't really have an idea where the problem might be. So does anyone has any suggestions?
I killed all clients but one, but the dead lock still existed...
Code:
"Database Server" prio=10 tid=0x090c4c00 nid=0x2c9 in Object.wait() [0x511ad000..0x511adf40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:294)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.lock(ThreadLockManagerImpl.java:46)
at com.tc.object.tx.ClientTransactionManagerImpl.begin(ClientTransactionManagerImpl.java:187)
at com.tc.object.bytecode.ManagerImpl.begin(ManagerImpl.java:337)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:507)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:495)
at com.tc.object.bytecode.ManagerUtil.monitorEnter(ManagerUtil.java:485)
at java.util.concurrent.locks.ReentrantReadWriteLock$DsoLock.lock(ReentrantReadWriteLock/java:35)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock/java)
at eu.cedarsoft.history.ElementsCollection.getElements(ElementsCollection.java:110)
Full stack traces:
Code:
"ConnectionEstablisher" daemon prio=10 tid=0x09133c00 nid=0x500f in Object.wait() [0x50d5a000..0x50d5b040]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.tc.util.concurrent.TCFuture.get(TCFuture.java:88)
- locked <0x576161f8> (a com.tc.net.protocol.transport.MessageTransportStatus)
at com.tc.util.concurrent.TCFuture.get(TCFuture.java:64)
at com.tc.net.protocol.transport.ClientMessageTransport.waitForSynAck(ClientMessageTransport.java:219)
at com.tc.net.protocol.transport.ClientMessageTransport.handShake(ClientMessageTransport.java:213)
at com.tc.net.protocol.transport.ClientMessageTransport.reconnect(ClientMessageTransport.java:262)
at com.tc.net.protocol.transport.ClientConnectionEstablisher.reconnect(ClientConnectionEstablisher.java:153)
at com.tc.net.protocol.transport.ClientConnectionEstablisher.access$100(ClientConnectionEstablisher.java:29)
at com.tc.net.protocol.transport.ClientConnectionEstablisher$AsyncReconnect.run(ClientConnectionEstablisher.ja
va:274)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.id.LastGivenExternalId. 19618574 25149
781" prio=10 tid=0x08c39400 nid=0x2d2 in Object.wait() [0x500ad000..0x500adec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57dad348> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dad348> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dad348> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contact.StaffGroup. 1109242 24081726"
prio=10 tid=0x09786800 nid=0x2d1 in Object.wait() [0x500fe000..0x500fef40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57db6ce0> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db6ce0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db6ce0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.appointment.AbstractCalendar. 2458662
22103325" prio=10 tid=0x09786400 nid=0x2d0 in Object.wait() [0x5075c000..0x5075cdc0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db7eb0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db7eb0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: eu.cedarsoft.business.contact.City. 22670902
14444682" prio=10 tid=0x08d81400 nid=0x2cf in Object.wait() [0x507ad000..0x507ade40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db8e20> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db8e20> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: eu.cedarsoft.business.Bank. 20725235 21230798
" prio=10 tid=0x08d81000 nid=0x2ce in Object.wait() [0x507fe000..0x507ff0c0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dba248> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dba248> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contact.AvrContact. 26811705 15864550"
prio=10 tid=0x09018400 nid=0x2cd in Object.wait() [0x50dad000..0x50dae140]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbb1c8> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbb1c8> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contracts.Contract. 7045653 24132223"
prio=10 tid=0x09017c00 nid=0x2cc in Object.wait() [0x50dfe000..0x50dfefc0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbc5e0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbc5e0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contracts.ContractBasket. 8117065 7013
877" prio=10 tid=0x092cf800 nid=0x2cb in Object.wait() [0x5115c000..0x5115d040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbd8e8> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbd8e8> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"MySQL Statement Cancellation Timer" daemon prio=10 tid=0x09da7800 nid=0x2ca in Object.wait() [0x52050000..0x52050ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x578b4440> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x578b4440> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Database Server" prio=10 tid=0x090c4c00 nid=0x2c9 in Object.wait() [0x511ad000..0x511adf40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:294)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.lock(ThreadLockManagerImpl.java:46)
at com.tc.object.tx.ClientTransactionManagerImpl.begin(ClientTransactionManagerImpl.java:187)
at com.tc.object.bytecode.ManagerImpl.begin(ManagerImpl.java:337)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:507)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:495)
at com.tc.object.bytecode.ManagerUtil.monitorEnter(ManagerUtil.java:485)
at java.util.concurrent.locks.ReentrantReadWriteLock$DsoLock.lock(ReentrantReadWriteLock/java:35)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock/java)
at eu.cedarsoft.history.ElementsCollection.getElements(ElementsCollection.java:110)
at eu.cedarsoft.gdao.caching.CachingDao.findAll(CachingDao.java:120)
at eu.cedarsoft.gdao.AbstractGenericDao.getElements(AbstractGenericDao.java:25)
at eu.cedarsoft.history.DelegatingObservableObjectAccess.getElements(DelegatingObservableObjectAccess.java:25)
at de.avr.AbstractObjectManager.getContracts(AbstractObjectManager.java:920)
at de.avr.CommitAllAction.execute(CommitAllAction.java:38)
at de.avr.ConsoleServer.run(ConsoleServer.java:66)
at de.avr.DatabaseServer$3.run(DatabaseServer.java:84)
at java.lang.Thread.run(Thread.java:619)
"Thread-11" daemon prio=10 tid=0x08bd1c00 nid=0x2bf in Object.wait() [0x51566000..0x51567040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.management.remote.protocol.terracotta.TunnelingEventHandler.accept(TunnelingEventHandler.java:84)
- locked <0x575a6868> (a com.tc.management.remote.protocol.terracotta.TunnelingEventHandler)
at com.tc.management.remote.protocol.terracotta.TunnelingMessageConnectionServer.accept(TunnelingMessageConnec
tionServer.java:31)
at com.sun.jmx.remote.generic.SynchroMessageConnectionServerImpl.accept(SynchroMessageConnectionServerImpl.jav
a:47)
at javax.management.remote.generic.GenericConnectorServer$Receiver.run(GenericConnectorServer.java:340)
"Timer-3" daemon prio=10 tid=0x08bd4400 nid=0x2be in Object.wait() [0x515b7000..0x515b7ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57613590> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x57613590> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=10 tid=0x51e42000 nid=0x2bd runnable [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=10 tid=0x51e37000 nid=0x2bc waiting on condition [0x00000000..0x51689418]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x51e37c00 nid=0x2bb waiting on condition [0x00000000..0x5170a498]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x51e5b800 nid=0x2ba waiting on condition [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"WorkerThread(client_coordination_stage,0)" daemon prio=10 tid=0x51e5a400 nid=0x2b9 in Object.wait() [0x517cb000..0x51
7cc140]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a6780> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_response_stage,0)" daemon prio=10 tid=0x51e59400 nid=0x2b8 in Object.wait() [0x5181c000..0x5181cfc0
]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a74f8> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(hydrate_message_stage,0)" daemon prio=10 tid=0x51e58000 nid=0x2b7 in Object.wait() [0x5186d000..0x5186e0
40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7318> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_transaction_complete_stage,0)" daemon prio=10 tid=0x51e57000 nid=0x2b6 in Object.wait() [0x518be
000..0x518beec0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7e68> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(dmi_stage,0)" daemon prio=10 tid=0x51e67400 nid=0x2b5 in Object.wait() [0x5190f000..0x5190ff40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a8ef8> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(batch_txn_ack_stage,0)" daemon prio=10 tid=0x51e66400 nid=0x2b4 in Object.wait() [0x51960000..0x51960dc0
]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7a70> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_statistics_response_stage,0)" daemon prio=10 tid=0x51e65000 nid=0x2b3 in Object.wait() [0x519b1000.
.0x519b1e40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a8618> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(jmxremote_tunnel_stage,0)" daemon prio=10 tid=0x51e62800 nid=0x2b2 in Object.wait() [0x51a02000..0x51a03
0c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7638> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_object_stage,0)" daemon prio=10 tid=0x51e61800 nid=0x2b1 in Object.wait() [0x51a53000..0x51a5414
0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a78e0> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_root_id_stage,0)" daemon prio=10 tid=0x51e10800 nid=0x2b0 in Object.wait() [0x51aa4000..0x51aa4f
c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7778> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(object_id_request_response_stage,0)" daemon prio=10 tid=0x51e0fc00 nid=0x2af in Object.wait() [0x51af500
0..0x51af6040]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7d28> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_statistics_enable_disable_stage,0)" daemon prio=10 tid=0x51e2b400 nid=0x2ae in Object.wait() [0x51b
46000..0x51b46ec0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7c00> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_transaction_stage,0)" daemon prio=10 tid=0x51e23000 nid=0x2ad in Object.wait() [0x51b97000..0x51
b97f40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7200> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"Reaper" daemon prio=10 tid=0x51e28800 nid=0x2ab in Object.wait() [0x51c39000..0x51c39e40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x575a52b8> (a java.lang.ref.ReferenceQueue$Lock)
at com.tc.object.ClientObjectManagerImpl$2.run(ClientObjectManagerImpl.java:1200)
"H2 Log Writer STATISTICS-BUFFER" daemon prio=10 tid=0x51e1f000 nid=0x2aa sleeping[0x51cba000..0x51cbb0c0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.h2.store.WriterThread.run(WriterThread.java:147)
"H2 File Lock Watchdog /home/avr/avr/live/databaseServer/statistics-127.0.1.1/statistics-buffer.lock.db" daemon prio=1
0 tid=0x51e0d400 nid=0x2a9 sleeping[0x51d0c000..0x51d0c140]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.h2.store.FileLock$1.run(FileLock.java:198)
at java.lang.Thread.run(Thread.java:619)
"com.tc.util.ToggleableReferenceManager$QueueProcessor" daemon prio=10 tid=0x51e08400 nid=0x2a7 in Object.wait() [0x51
d5c000..0x51d5d040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x575a63f0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at com.tc.util.ToggleableReferenceManager$QueueProcessor.run(ToggleableReferenceManager.java:129)
"DSO Lock Object.wait() timer" daemon prio=10 tid=0x51e03c00 nid=0x2a6 in Object.wait() [0x51dad000..0x51dadec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.runGC(ClientLockManagerImpl.java:121)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl$LockGCTask.run(ClientLockManagerImpl.java:631)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
"RemoteTransactionManager Flusher" daemon prio=10 tid=0x51e00800 nid=0x2a5 in Object.wait() [0x51dfe000..0x51dfef40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.tx.RemoteTransactionManagerImpl.waitUntilRunning(RemoteTransactionManagerImpl.java:440)
at com.tc.object.tx.RemoteTransactionManagerImpl.getCompletedTransactionIDLowWaterMark(RemoteTransactionManage
rImpl.java:401)
- locked <0x575a5380> (a java.lang.Object)
at com.tc.object.tx.RemoteTransactionManagerImpl.access$100(RemoteTransactionManagerImpl.java:43)
at com.tc.object.tx.RemoteTransactionManagerImpl$RemoteTransactionManagerTimerTask.run(RemoteTransactionManage
rImpl.java:461)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
"HealthChecker" daemon prio=10 tid=0x08379800 nid=0x2a4 sleeping[0x51f5d000..0x51f5ddc0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at java.lang.Thread.sleep(Thread.java:298)
at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:24)
at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:16)
at com.tc.net.protocol.transport.ConnectionHealthCheckerImpl$HealthCheckerMonitorThreadEngine.run(ConnectionHe
althCheckerImpl.java:183)
at java.lang.Thread.run(Thread.java:619)
"TCComm Main Selector Thread" daemon prio=10 tid=0x08373800 nid=0x2a3 runnable [0x51fae000..0x51faee40]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:184)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x57615e10> (a sun.nio.ch.Util$1)
- locked <0x57615e00> (a java.util.Collections$UnmodifiableSet)
- locked <0x57614690> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
at com.tc.net.core.CoreNIOServices.selectLoop(CoreNIOServices.java:338)
at com.tc.net.core.CoreNIOServices.run(CoreNIOServices.java:61)
"SampledCounterManager Timer" daemon prio=10 tid=0x08516400 nid=0x2a2 in Object.wait() [0x51fff000..0x520000c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x57614098> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"startlevel job thread" daemon prio=10 tid=0x08bbe400 nid=0x2a0 in Object.wait() [0x520b5000..0x520b5fc0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.knopflerfish.framework.Queue.removeWait(Queue.java:108)
- locked <0x574bdc98> (a org.knopflerfish.framework.Queue)
at org.knopflerfish.framework.StartLevelImpl.run(StartLevelImpl.java:170)
at java.lang.Thread.run(Thread.java:619)
"Finalizer" daemon prio=10 tid=0x080f2400 nid=0x29f in Object.wait() [0x52720000..0x52721040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x570b38c0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x080f1c00 nid=0x29e in Object.wait() [0x52771000..0x52771ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x570b38e0> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x08058c00 nid=0x29a in Object.wait() [0xb7e2a000..0xb7e2b218]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x576f7408> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:423)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at de.avr.DatabaseServer.main(DatabaseServer.java:110)
- locked <0x576f7408> (a java.lang.Object)
|
|
|
08/15/2008 11:18:50
|
amiller
ophanim
Joined: 08/29/2007 09:05:48
Messages: 722
Location: St. Louis, MO
Offline
|
What version of Terracotta?
What's in your tc-config?
What is happening on the Terracotta server? Is it doing anything? Do you have a server thread dump?
|
Alex Miller (Terracotta Engineer) - Want to post to the forums? Sign up here |
|
|
08/15/2008 12:57:57
|
teck
seraphim
Joined: 05/24/2006 15:03:25
Messages: 1128
Offline
|
I second amiller's questions...
From those stack traces it looks like your process has been disconnected from the terracotta server. So some possibilities are that the server is no longer running, or there is a partition in the network, or the server is actively refusing reconnection.
Info from the server (thread dump and terracotta log file) should provide more information.
THe terracotta log file on client side might have some information as well. In particular I'm wondering if our health check decided that the TC server was down
|
Tim Eck (terracotta engineer)
|
|
|
08/15/2008 13:21:45
|
Shake
master
Joined: 01/05/2008 07:27:49
Messages: 84
Offline
|
I am using terracotta-2.6.2.
I think it shouldn't be a network problem since the server and client run on the same host...
Sometimes I get some "Unable to find communications stack" messages. Don't know why. But I didn't care until now.
It shouldn't be caused by an old client because the Terracotta server has been up for several weeks now...
The terracotta server was still running (could quit it with Ctrl-C).
tc-config.xml:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/config http://www.terracotta.org/schema/terracotta-4.xsd">
<servers>
<server host="192.168.123.100">
<data>/home/avr/avr/live/data</data>
<logs>/home/avr/avr/live/logs</logs>
<dso>
<persistence>
<mode>temporary-swap-only</mode>
</persistence>
</dso>
</server>
<!--server host="192.168.0.158">
<data>/tmp/avr/data</data>
<logs>/tmp/avr/logs</logs>
<dso>
<persistence>
<mode>temporary-swap-only</mode>
</persistence>
</dso>
</server-->
</servers>
<clients>
<modules>
<module name="tim-annotations" version="1.0.3" />
<module name="tim-hibernate-3.2.5" version="1.1.2" />
</modules>
<dso>
<debugging>
<instrumentation-logging>
<!--<class>true</class>-->
<!--<hierarchy>true</hierarchy>-->
<!--<locks>true</locks>-->
<transient-root>true</transient-root>
<roots>true</roots>
<distributed-methods>true</distributed-methods>
</instrumentation-logging>
<runtime-logging>
<lock-debug>false</lock-debug>
<!--<field-change-debug>true</field-change-debug>-->
<!--<wait-notify-debug>true</wait-notify-debug>-->
<!--<distributed-method-debug>true</distributed-method-debug>-->
<!--<new-object-debug>true</new-object-debug>-->
<non-portable-dump>false</non-portable-dump>
</runtime-logging>
<runtime-output-options>
<auto-lock-details>true</auto-lock-details>
<caller>true</caller>
<full-stack>false</full-stack>
</runtime-output-options>
</debugging>
</dso>
<logs>logs/terracotta/client/%h/%D</logs>
</clients>
<application>
<dso>
<locks>
<!--<autolock>-->
<!--<method-expression>* eu.cedarsoft..*.*(..)</method-expression>-->
<!--<lock-level>write</lock-level>-->
<!--</autolock>-->
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.invokeNullable(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback()</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.acknowledge(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run()</method-expression>
<lock-level>write</lock-level>
</autolock>
<!-- Useage of synchronized statements -->
<autolock>
<method-expression>* de.avr.imports.access.AccessImporter.addContactInformations(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.ui.dialogs.AbstractPropertiesDialog.onFinish(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.imports.HsbcImporter.importInsuredPersons(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.ui.FormPage.commit()</method-expression>
<lock-level>write</lock-level>
</autolock>
</locks>
<distributed-methods>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryDeleted(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryAdded(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryChanged(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryAdded(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryRemoved(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryChanged(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryRemoved(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryAdded(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryChanged(Object)
</method-expression>
<method-expression>void eu.cedarsoft.utils.ChangeListenerSupport.changed(..)</method-expression>
<!-- property change support -->
<method-expression>void java.beans.PropertyChangeSupport.fire*(..)</method-expression>
</distributed-methods>
<instrumented-classes>
<include>
<class-expression>eu.cedarsoft..*..*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr..*..*</class-expression>
<honor-transient>true</honor-transient>
</include>
<!-- Anonymous inner classes -->
<include>
<class-expression>de.avr.AbstractObjectManager$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr.contracts.offer.PurchaseOffer$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr.contracts.offer.PurchaseOfferProvision$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<!-- Hibernate -->
<include>
<class-expression>org.hibernate.collection.Persistent*</class-expression>
</include>
<include>
<class-expression>org.hibernate.collection.AbstractPersistentCollection</class-expression>
</include>
<!-- Exceptions -->
<include>
<class-expression>org.springframework..*Exception</class-expression>
</include>
<include>
<class-expression>org.hibernate.*Exception</class-expression>
</include>
<include>
<class-expression>org.hibernate.exception.*</class-expression>
</include>
<include>
<class-expression>java.sql.*Exception</class-expression>
</include>
<include>
<class-expression>java.lang.*Exception</class-expression>
</include>
<!-- Joda time -->
<include>
<class-expression>org.joda.time..*</class-expression>
<honor-transient>false</honor-transient>
</include>
<!-- Property Change support -->
<include>
<class-expression>java.beans.PropertyChangeSupport</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>java.beans.PropertyChangeEvent</class-expression>
<honor-transient>true</honor-transient>
</include>
</instrumented-classes>
<additional-boot-jar-classes>
<include>java.beans.PropertyChangeSupport</include>
<include>java.beans.PropertyChangeEvent</include>
<include>java.sql.SQLException</include>
<include>java.sql.BatchUpdateException</include>
<include>java.lang.NullPointerException</include>
</additional-boot-jar-classes>
</dso>
</application>
</tc:tc-config>
Filename |
terracotta-client.log |
Download
|
Description |
Client log |
Filesize |
7582 Kbytes
|
Downloaded: |
399 time(s) |
Filename |
terracotta-server.log.1 |
Download
|
Description |
Server log |
Filesize |
590 Kbytes
|
Downloaded: |
254 time(s) |
|
|
|
08/15/2008 13:51:20
|
teck
seraphim
Joined: 05/24/2006 15:03:25
Messages: 1128
Offline
|
So the client does think the server is dead:
2008-08-15 11:35:34,473 [HealthChecker] INFO com.tc.net.protocol.transport.ConnectionHealthCheckerImpl. DSO Client - avrserver.local:9510 is DEAD
Of course the server wasn't dead (I can see that from the log, and you have confirned that fact as well).
So the question is why was it unresponsive beyond the timeout period for the health check? Usually the problem is a long GC in the terracotta server. Are you running the terracotta server with the stock start-tc-server.sh script, or have you changed it to increase the heap size? Any chance the host where all this is running had to do any swapping?
I recommend monitoring the heap for the terracota server process with -verbose:gc or jstat to make more data in case this happens again
|
Tim Eck (terracotta engineer)
|
|
|
08/15/2008 14:05:04
|
teck
seraphim
Joined: 05/24/2006 15:03:25
Messages: 1128
Offline
|
I forget to mention that you can disable the health check function if you don't ever want clients to disconnect themselves automatically.
-Dcom.tc.l1.healthcheck.l2.ping.enabled=false
|
Tim Eck (terracotta engineer)
|
|
|
08/15/2008 15:02:53
|
steve
ophanim
Joined: 05/24/2006 14:22:53
Messages: 619
Offline
|
One small correction... Long GC in the server should not trip up the Health check. We can detect that. It's long GC's in the client that can lead to confusion for the health checker (we have some plans to look at that over
the next few months).
|
Want to post to this forum? Join the Terracotta Community |
|
|
08/16/2008 08:57:27
|
Shake
master
Joined: 01/05/2008 07:27:49
Messages: 84
Offline
|
Thanks for your help.
Yes - I have increased the memory for the TC server. And I will try the command line parameter "-Dcom.tc.l1.healthcheck.l2.ping.enabled=false".
Hope it helps....
|
|
|
|