Author |
Message |
|
Thanks for your help.
Yes - I have increased the memory for the TC server. And I will try the command line parameter "-Dcom.tc.l1.healthcheck.l2.ping.enabled=false".
Hope it helps....
|
|
|
I am using terracotta-2.6.2.
I think it shouldn't be a network problem since the server and client run on the same host...
Sometimes I get some "Unable to find communications stack" messages. Don't know why. But I didn't care until now.
It shouldn't be caused by an old client because the Terracotta server has been up for several weeks now...
The terracotta server was still running (could quit it with Ctrl-C).
tc-config.xml:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/config http://www.terracotta.org/schema/terracotta-4.xsd">
<servers>
<server host="192.168.123.100">
<data>/home/avr/avr/live/data</data>
<logs>/home/avr/avr/live/logs</logs>
<dso>
<persistence>
<mode>temporary-swap-only</mode>
</persistence>
</dso>
</server>
<!--server host="192.168.0.158">
<data>/tmp/avr/data</data>
<logs>/tmp/avr/logs</logs>
<dso>
<persistence>
<mode>temporary-swap-only</mode>
</persistence>
</dso>
</server-->
</servers>
<clients>
<modules>
<module name="tim-annotations" version="1.0.3" />
<module name="tim-hibernate-3.2.5" version="1.1.2" />
</modules>
<dso>
<debugging>
<instrumentation-logging>
<!--<class>true</class>-->
<!--<hierarchy>true</hierarchy>-->
<!--<locks>true</locks>-->
<transient-root>true</transient-root>
<roots>true</roots>
<distributed-methods>true</distributed-methods>
</instrumentation-logging>
<runtime-logging>
<lock-debug>false</lock-debug>
<!--<field-change-debug>true</field-change-debug>-->
<!--<wait-notify-debug>true</wait-notify-debug>-->
<!--<distributed-method-debug>true</distributed-method-debug>-->
<!--<new-object-debug>true</new-object-debug>-->
<non-portable-dump>false</non-portable-dump>
</runtime-logging>
<runtime-output-options>
<auto-lock-details>true</auto-lock-details>
<caller>true</caller>
<full-stack>false</full-stack>
</runtime-output-options>
</debugging>
</dso>
<logs>logs/terracotta/client/%h/%D</logs>
</clients>
<application>
<dso>
<locks>
<!--<autolock>-->
<!--<method-expression>* eu.cedarsoft..*.*(..)</method-expression>-->
<!--<lock-level>write</lock-level>-->
<!--</autolock>-->
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.invokeNullable(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback()</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport.acknowledge(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run()</method-expression>
<lock-level>write</lock-level>
</autolock>
<!-- Useage of synchronized statements -->
<autolock>
<method-expression>* de.avr.imports.access.AccessImporter.addContactInformations(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.ui.dialogs.AbstractPropertiesDialog.onFinish(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.imports.HsbcImporter.importInsuredPersons(..)</method-expression>
<lock-level>write</lock-level>
</autolock>
<autolock>
<method-expression>* de.avr.ui.FormPage.commit()</method-expression>
<lock-level>write</lock-level>
</autolock>
</locks>
<distributed-methods>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryDeleted(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryAdded(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.CollectionSupport.entryChanged(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryAdded(Object)</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryRemoved(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.HistoryListenerSupport.notifyEntryChanged(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryRemoved(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryAdded(Object)
</method-expression>
<method-expression>void eu.cedarsoft.history.ContinuousEntriesInformation.notifyEntryChanged(Object)
</method-expression>
<method-expression>void eu.cedarsoft.utils.ChangeListenerSupport.changed(..)</method-expression>
<!-- property change support -->
<method-expression>void java.beans.PropertyChangeSupport.fire*(..)</method-expression>
</distributed-methods>
<instrumented-classes>
<include>
<class-expression>eu.cedarsoft..*..*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr..*..*</class-expression>
<honor-transient>true</honor-transient>
</include>
<!-- Anonymous inner classes -->
<include>
<class-expression>de.avr.AbstractObjectManager$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr.contracts.offer.PurchaseOffer$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>de.avr.contracts.offer.PurchaseOfferProvision$*</class-expression>
<honor-transient>true</honor-transient>
</include>
<!-- Hibernate -->
<include>
<class-expression>org.hibernate.collection.Persistent*</class-expression>
</include>
<include>
<class-expression>org.hibernate.collection.AbstractPersistentCollection</class-expression>
</include>
<!-- Exceptions -->
<include>
<class-expression>org.springframework..*Exception</class-expression>
</include>
<include>
<class-expression>org.hibernate.*Exception</class-expression>
</include>
<include>
<class-expression>org.hibernate.exception.*</class-expression>
</include>
<include>
<class-expression>java.sql.*Exception</class-expression>
</include>
<include>
<class-expression>java.lang.*Exception</class-expression>
</include>
<!-- Joda time -->
<include>
<class-expression>org.joda.time..*</class-expression>
<honor-transient>false</honor-transient>
</include>
<!-- Property Change support -->
<include>
<class-expression>java.beans.PropertyChangeSupport</class-expression>
<honor-transient>true</honor-transient>
</include>
<include>
<class-expression>java.beans.PropertyChangeEvent</class-expression>
<honor-transient>true</honor-transient>
</include>
</instrumented-classes>
<additional-boot-jar-classes>
<include>java.beans.PropertyChangeSupport</include>
<include>java.beans.PropertyChangeEvent</include>
<include>java.sql.SQLException</include>
<include>java.sql.BatchUpdateException</include>
<include>java.lang.NullPointerException</include>
</additional-boot-jar-classes>
</dso>
</application>
</tc:tc-config>
|
|
|
Hi,
I suddently got that dead lock (after several weeks without any problems).
But I don't really have an idea where the problem might be. So does anyone has any suggestions?
I killed all clients but one, but the dead lock still existed...
Code:
"Database Server" prio=10 tid=0x090c4c00 nid=0x2c9 in Object.wait() [0x511ad000..0x511adf40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:294)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.lock(ThreadLockManagerImpl.java:46)
at com.tc.object.tx.ClientTransactionManagerImpl.begin(ClientTransactionManagerImpl.java:187)
at com.tc.object.bytecode.ManagerImpl.begin(ManagerImpl.java:337)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:507)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:495)
at com.tc.object.bytecode.ManagerUtil.monitorEnter(ManagerUtil.java:485)
at java.util.concurrent.locks.ReentrantReadWriteLock$DsoLock.lock(ReentrantReadWriteLock/java:35)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock/java)
at eu.cedarsoft.history.ElementsCollection.getElements(ElementsCollection.java:110)
Full stack traces:
Code:
"ConnectionEstablisher" daemon prio=10 tid=0x09133c00 nid=0x500f in Object.wait() [0x50d5a000..0x50d5b040]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.tc.util.concurrent.TCFuture.get(TCFuture.java:88)
- locked <0x576161f8> (a com.tc.net.protocol.transport.MessageTransportStatus)
at com.tc.util.concurrent.TCFuture.get(TCFuture.java:64)
at com.tc.net.protocol.transport.ClientMessageTransport.waitForSynAck(ClientMessageTransport.java:219)
at com.tc.net.protocol.transport.ClientMessageTransport.handShake(ClientMessageTransport.java:213)
at com.tc.net.protocol.transport.ClientMessageTransport.reconnect(ClientMessageTransport.java:262)
at com.tc.net.protocol.transport.ClientConnectionEstablisher.reconnect(ClientConnectionEstablisher.java:153)
at com.tc.net.protocol.transport.ClientConnectionEstablisher.access$100(ClientConnectionEstablisher.java:29)
at com.tc.net.protocol.transport.ClientConnectionEstablisher$AsyncReconnect.run(ClientConnectionEstablisher.ja
va:274)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.id.LastGivenExternalId. 19618574 25149
781" prio=10 tid=0x08c39400 nid=0x2d2 in Object.wait() [0x500ad000..0x500adec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57dad348> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dad348> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dad348> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contact.StaffGroup. 1109242 24081726"
prio=10 tid=0x09786800 nid=0x2d1 in Object.wait() [0x500fe000..0x500fef40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57db6ce0> (a java.util.LinkedList)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db6ce0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db6ce0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.appointment.AbstractCalendar. 2458662
22103325" prio=10 tid=0x09786400 nid=0x2d0 in Object.wait() [0x5075c000..0x5075cdc0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db7eb0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db7eb0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: eu.cedarsoft.business.contact.City. 22670902
14444682" prio=10 tid=0x08d81400 nid=0x2cf in Object.wait() [0x507ad000..0x507ade40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57db8e20> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57db8e20> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: eu.cedarsoft.business.Bank. 20725235 21230798
" prio=10 tid=0x08d81000 nid=0x2ce in Object.wait() [0x507fe000..0x507ff0c0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dba248> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dba248> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contact.AvrContact. 26811705 15864550"
prio=10 tid=0x09018400 nid=0x2cd in Object.wait() [0x50dad000..0x50dae140]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbb1c8> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbb1c8> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contracts.Contract. 7045653 24132223"
prio=10 tid=0x09017c00 nid=0x2cc in Object.wait() [0x50dfe000..0x50dfefc0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbc5e0> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbc5e0> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"AsyncWorkerThread AsynchroniousDao-CallbackCaller for HibernateDao for: de.avr.contracts.ContractBasket. 8117065 7013
877" prio=10 tid=0x092cf800 nid=0x2cb in Object.wait() [0x5115c000..0x5115d040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLock.waitForLock(ClientLock.java:619)
- locked <0x57dbd8e8> (a java.util.LinkedList)
at com.tc.object.lockmanager.impl.ClientLock.wait(ClientLock.java:353)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.wait(ClientLockManagerImpl.java:361)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.wait(ThreadLockManagerImpl.java:54)
at com.tc.object.tx.ClientTransactionManagerImpl.wait(ClientTransactionManagerImpl.java:227)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:416)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at eu.cedarsoft.async.AsyncCallSupport.waitForNextCallback(AsyncCallSupport.java:158)
- locked <0x57dbd8e8> (a java.util.LinkedList)
at eu.cedarsoft.async.AsyncCallSupport$AsyncWorker.run(AsyncCallSupport.java:255)
at java.lang.Thread.run(Thread.java:619)
"MySQL Statement Cancellation Timer" daemon prio=10 tid=0x09da7800 nid=0x2ca in Object.wait() [0x52050000..0x52050ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x578b4440> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x578b4440> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Database Server" prio=10 tid=0x090c4c00 nid=0x2c9 in Object.wait() [0x511ad000..0x511adf40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:294)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ThreadLockManagerImpl.lock(ThreadLockManagerImpl.java:46)
at com.tc.object.tx.ClientTransactionManagerImpl.begin(ClientTransactionManagerImpl.java:187)
at com.tc.object.bytecode.ManagerImpl.begin(ManagerImpl.java:337)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:507)
at com.tc.object.bytecode.ManagerImpl.monitorEnter(ManagerImpl.java:495)
at com.tc.object.bytecode.ManagerUtil.monitorEnter(ManagerUtil.java:485)
at java.util.concurrent.locks.ReentrantReadWriteLock$DsoLock.lock(ReentrantReadWriteLock/java:35)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock/java)
at eu.cedarsoft.history.ElementsCollection.getElements(ElementsCollection.java:110)
at eu.cedarsoft.gdao.caching.CachingDao.findAll(CachingDao.java:120)
at eu.cedarsoft.gdao.AbstractGenericDao.getElements(AbstractGenericDao.java:25)
at eu.cedarsoft.history.DelegatingObservableObjectAccess.getElements(DelegatingObservableObjectAccess.java:25)
at de.avr.AbstractObjectManager.getContracts(AbstractObjectManager.java:920)
at de.avr.CommitAllAction.execute(CommitAllAction.java:38)
at de.avr.ConsoleServer.run(ConsoleServer.java:66)
at de.avr.DatabaseServer$3.run(DatabaseServer.java:84)
at java.lang.Thread.run(Thread.java:619)
"Thread-11" daemon prio=10 tid=0x08bd1c00 nid=0x2bf in Object.wait() [0x51566000..0x51567040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.management.remote.protocol.terracotta.TunnelingEventHandler.accept(TunnelingEventHandler.java:84)
- locked <0x575a6868> (a com.tc.management.remote.protocol.terracotta.TunnelingEventHandler)
at com.tc.management.remote.protocol.terracotta.TunnelingMessageConnectionServer.accept(TunnelingMessageConnec
tionServer.java:31)
at com.sun.jmx.remote.generic.SynchroMessageConnectionServerImpl.accept(SynchroMessageConnectionServerImpl.jav
a:47)
at javax.management.remote.generic.GenericConnectorServer$Receiver.run(GenericConnectorServer.java:340)
"Timer-3" daemon prio=10 tid=0x08bd4400 nid=0x2be in Object.wait() [0x515b7000..0x515b7ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x57613590> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x57613590> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=10 tid=0x51e42000 nid=0x2bd runnable [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=10 tid=0x51e37000 nid=0x2bc waiting on condition [0x00000000..0x51689418]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x51e37c00 nid=0x2bb waiting on condition [0x00000000..0x5170a498]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x51e5b800 nid=0x2ba waiting on condition [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"WorkerThread(client_coordination_stage,0)" daemon prio=10 tid=0x51e5a400 nid=0x2b9 in Object.wait() [0x517cb000..0x51
7cc140]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a6780> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_response_stage,0)" daemon prio=10 tid=0x51e59400 nid=0x2b8 in Object.wait() [0x5181c000..0x5181cfc0
]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a74f8> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(hydrate_message_stage,0)" daemon prio=10 tid=0x51e58000 nid=0x2b7 in Object.wait() [0x5186d000..0x5186e0
40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7318> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_transaction_complete_stage,0)" daemon prio=10 tid=0x51e57000 nid=0x2b6 in Object.wait() [0x518be
000..0x518beec0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7e68> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(dmi_stage,0)" daemon prio=10 tid=0x51e67400 nid=0x2b5 in Object.wait() [0x5190f000..0x5190ff40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a8ef8> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(batch_txn_ack_stage,0)" daemon prio=10 tid=0x51e66400 nid=0x2b4 in Object.wait() [0x51960000..0x51960dc0
]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7a70> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_statistics_response_stage,0)" daemon prio=10 tid=0x51e65000 nid=0x2b3 in Object.wait() [0x519b1000.
.0x519b1e40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a8618> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(jmxremote_tunnel_stage,0)" daemon prio=10 tid=0x51e62800 nid=0x2b2 in Object.wait() [0x51a02000..0x51a03
0c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7638> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_object_stage,0)" daemon prio=10 tid=0x51e61800 nid=0x2b1 in Object.wait() [0x51a53000..0x51a5414
0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a78e0> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_root_id_stage,0)" daemon prio=10 tid=0x51e10800 nid=0x2b0 in Object.wait() [0x51aa4000..0x51aa4f
c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7778> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(object_id_request_response_stage,0)" daemon prio=10 tid=0x51e0fc00 nid=0x2af in Object.wait() [0x51af500
0..0x51af6040]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7d28> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(lock_statistics_enable_disable_stage,0)" daemon prio=10 tid=0x51e2b400 nid=0x2ae in Object.wait() [0x51b
46000..0x51b46ec0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7c00> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"WorkerThread(receive_transaction_stage,0)" daemon prio=10 tid=0x51e23000 nid=0x2ad in Object.wait() [0x51b97000..0x51
b97f40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source)
- locked <0x575a7200> (a java.lang.Object)
at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29)
at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)
"Reaper" daemon prio=10 tid=0x51e28800 nid=0x2ab in Object.wait() [0x51c39000..0x51c39e40]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x575a52b8> (a java.lang.ref.ReferenceQueue$Lock)
at com.tc.object.ClientObjectManagerImpl$2.run(ClientObjectManagerImpl.java:1200)
"H2 Log Writer STATISTICS-BUFFER" daemon prio=10 tid=0x51e1f000 nid=0x2aa sleeping[0x51cba000..0x51cbb0c0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.h2.store.WriterThread.run(WriterThread.java:147)
"H2 File Lock Watchdog /home/avr/avr/live/databaseServer/statistics-127.0.1.1/statistics-buffer.lock.db" daemon prio=1
0 tid=0x51e0d400 nid=0x2a9 sleeping[0x51d0c000..0x51d0c140]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.h2.store.FileLock$1.run(FileLock.java:198)
at java.lang.Thread.run(Thread.java:619)
"com.tc.util.ToggleableReferenceManager$QueueProcessor" daemon prio=10 tid=0x51e08400 nid=0x2a7 in Object.wait() [0x51
d5c000..0x51d5d040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x575a63f0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at com.tc.util.ToggleableReferenceManager$QueueProcessor.run(ToggleableReferenceManager.java:129)
"DSO Lock Object.wait() timer" daemon prio=10 tid=0x51e03c00 nid=0x2a6 in Object.wait() [0x51dad000..0x51dadec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:567)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl.runGC(ClientLockManagerImpl.java:121)
- locked <0x575a4690> (a com.tc.object.lockmanager.impl.ClientLockManagerImpl)
at com.tc.object.lockmanager.impl.ClientLockManagerImpl$LockGCTask.run(ClientLockManagerImpl.java:631)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
"RemoteTransactionManager Flusher" daemon prio=10 tid=0x51e00800 nid=0x2a5 in Object.wait() [0x51dfe000..0x51dfef40]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.tx.RemoteTransactionManagerImpl.waitUntilRunning(RemoteTransactionManagerImpl.java:440)
at com.tc.object.tx.RemoteTransactionManagerImpl.getCompletedTransactionIDLowWaterMark(RemoteTransactionManage
rImpl.java:401)
- locked <0x575a5380> (a java.lang.Object)
at com.tc.object.tx.RemoteTransactionManagerImpl.access$100(RemoteTransactionManagerImpl.java:43)
at com.tc.object.tx.RemoteTransactionManagerImpl$RemoteTransactionManagerTimerTask.run(RemoteTransactionManage
rImpl.java:461)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
"HealthChecker" daemon prio=10 tid=0x08379800 nid=0x2a4 sleeping[0x51f5d000..0x51f5ddc0]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at java.lang.Thread.sleep(Thread.java:298)
at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:24)
at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:16)
at com.tc.net.protocol.transport.ConnectionHealthCheckerImpl$HealthCheckerMonitorThreadEngine.run(ConnectionHe
althCheckerImpl.java:183)
at java.lang.Thread.run(Thread.java:619)
"TCComm Main Selector Thread" daemon prio=10 tid=0x08373800 nid=0x2a3 runnable [0x51fae000..0x51faee40]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:184)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x57615e10> (a sun.nio.ch.Util$1)
- locked <0x57615e00> (a java.util.Collections$UnmodifiableSet)
- locked <0x57614690> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
at com.tc.net.core.CoreNIOServices.selectLoop(CoreNIOServices.java:338)
at com.tc.net.core.CoreNIOServices.run(CoreNIOServices.java:61)
"SampledCounterManager Timer" daemon prio=10 tid=0x08516400 nid=0x2a2 in Object.wait() [0x51fff000..0x520000c0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x57614098> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"startlevel job thread" daemon prio=10 tid=0x08bbe400 nid=0x2a0 in Object.wait() [0x520b5000..0x520b5fc0]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.knopflerfish.framework.Queue.removeWait(Queue.java:108)
- locked <0x574bdc98> (a org.knopflerfish.framework.Queue)
at org.knopflerfish.framework.StartLevelImpl.run(StartLevelImpl.java:170)
at java.lang.Thread.run(Thread.java:619)
"Finalizer" daemon prio=10 tid=0x080f2400 nid=0x29f in Object.wait() [0x52720000..0x52721040]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x570b38c0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x080f1c00 nid=0x29e in Object.wait() [0x52771000..0x52771ec0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x570b38e0> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x08058c00 nid=0x29a in Object.wait() [0xb7e2a000..0xb7e2b218]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x576f7408> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at com.tc.object.bytecode.ManagerImpl.objectWait0(ManagerImpl.java:423)
at com.tc.object.bytecode.ManagerUtil.objectWait0(ManagerUtil.java:454)
at de.avr.DatabaseServer.main(DatabaseServer.java:110)
- locked <0x576f7408> (a java.lang.Object)
|
|
|
Hi,
I start my terracotta clients with "-Dcom.tc.l1.cachemanager.enabled=false". Then I have started the admin console and watched the "Object Flush Rate" and I have seen that those clients still flush out some objects (not many, but sometimes I have a smal peak).
Any ideas how I can avoid that any objects are flushed?
Thanks,
Johannes
|
|
|
I have some sort of working configuration here. But I do *not* recommend to use Joda Time with Terracotta at the moment.
There are two issues that I worked around:
- Object identity: I think it is necessary to patch Joda Time because it uses "==" often. But some fields of DateTime are cached (static) and therefore different on different JVMs.
- Unsynchronized writes:
org.joda.time.tz.CachedDateTimeZone#getInfo is unsynchronized and results in com.tc.object.tx.UnlockedSharedObjectException when changed.
I worked around this issue: I initialize the caches manually *before* any objects are shared using Terracotta.
But I had to change the size of the cache to get an acceptable size...
So my solution "works" for dates between 1900 and 2100. Using dates out of this range results in a com.tc.object.tx.UnlockedSharedObjectException.
Synchronizing is not a solution (very, very slow performance).
So at the moment I do *not* recommend to use Joda classes within shared objects.
Instead I suggest to store the dates as "long millis" and add a transient field that contains the DateTime instance. The getter method should instantiate that object lazyly.
|
|
|
Stupid question: How can I turn off the client side memory manager?
Or how can I ensure that *no* objects are flushed (or at least some objects)?
|
|
|
Any news on this issue? This is quite important for me. Any ideas? Thanks.
|
|
|
Well, I tried the same - but it didn't work ;-). Does it work for you?
|
|
|
Hi,
I have a problem with a clustered application (dead lock). Now I try to get a thread dump using the admin console.
Unfortunately the terracotta server is a mile away... So I tried to create a ssh tunnel (9520:localhost:9520).
If I call "telnet localhost 9520" I get some responses (javax.management.remote.messageHandshakeBeginMessage). So I think the tunnel works as expected.
Unfortuately the admin console can't connect to the server (localhost:9520).
Any hints?
|
|
|
Some seconds ago a terracotta client crashed... Is there anything I can do against? Any ideas?
Is this related to Terracotta?
Code:
#
# An unexpected error has been detected by Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x774e6476, pid=2856, tid=3308
#
# Java VM: Java HotSpot(TM) Client VM (10.0-b19 mixed mode windows-x86)
# Problematic frame:
# C [ole32.dll+0x36476]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x28a8e800): JavaThread "AWT-EventQueue-0" [_thread_in_native, id=3308, stack(0x29c90000,0x29ce0000)]
siginfo: ExceptionCode=0xc0000005, reading address 0x00000060
Registers:
EAX=0x00000000, EBX=0x00000000, ECX=0x775d6ed8, EDX=0x775d6ed8
ESP=0x29cdf114, EBP=0x29cdf144, ESI=0x00133bd8, EDI=0x00106008
EIP=0x774e6476, EFLAGS=0x00010246
Top of Stack: (sp=0x29cdf114)
0x29cdf114: 00000000 00106008 00000000 00000000
0x29cdf124: 00000000 00133bd8 00000000 774cd6bc
0x29cdf134: 0010600c 29cdf14c 774cd7dc 00008f93
0x29cdf144: 29cdf160 774e6385 28cf6318 00000000
0x29cdf154: 775d00ae 28cf6318 00106008 29cdf180
0x29cdf164: 77503d41 28cf6318 00000000 00000000
0x29cdf174: 00000000 28cf6318 28cf6318 00000000
0x29cdf184: 6d07584a 28bebc48 00000001 00000000
Instructions: (pc=0x774e6476)
0x774e6466: f6 40 0c 80 0f 84 9f 05 00 00 8b 46 40 89 46 38
0x774e6476: 8b 48 60 89 4e 48 8b 40 64 b9 a0 61 5d 77 89 46
Stack: [0x29c90000,0x29ce0000], sp=0x29cdf114, free space=316k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [ole32.dll+0x36476]
C [ole32.dll+0x36385]
C [ole32.dll+0x53d41]
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
J java.awt.event.InvocationEvent.dispatch()V
J java.awt.EventQueue.dispatchEvent(Ljava/awt/AWTEvent;)V
J java.awt.EventDispatchThread.pumpOneEventForFilters(I)Z
J java.awt.EventDispatchThread.pumpEventsForFilter(ILjava/awt/Conditional;Ljava/awt/EventFilter;)V
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::Interpreter
v ~BufferBlob::StubRoutines (1)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x270c1400 JavaThread "Thread-206" [_thread_blocked, id=716, stack(0x34560000,0x345b0000)]
0x28e45400 JavaThread "Image Fetcher 0" daemon [_thread_blocked, id=1652, stack(0x34510000,0x34560000)]
0x28d4b800 JavaThread "Thread-204" [_thread_blocked, id=3716, stack(0x344c0000,0x34510000)]
0x28d8f000 JavaThread "Thread-203" [_thread_blocked, id=2252, stack(0x34340000,0x34390000)]
0x28c80c00 JavaThread "Thread-24" [_thread_blocked, id=1256, stack(0x2de90000,0x2dee0000)]
0x28c24800 JavaThread "Thread-23" [_thread_blocked, id=3804, stack(0x2de40000,0x2de90000)]
0x00395800 JavaThread "DestroyJavaVM" [_thread_blocked, id=3616, stack(0x00990000,0x009e0000)]
0x28b5e800 JavaThread "TimerQueue" daemon [_thread_blocked, id=3596, stack(0x2e140000,0x2e190000)]
0x27b39000 JavaThread "Thread-14" [_thread_blocked, id=2264, stack(0x29ce0000,0x29d30000)]
=>0x28a8e800 JavaThread "AWT-EventQueue-0" [_thread_in_native, id=3308, stack(0x29c90000,0x29ce0000)]
0x273c1800 JavaThread "AWT-Windows" daemon [_thread_in_native, id=3272, stack(0x29b80000,0x29bd0000)]
0x27a1c800 JavaThread "AWT-Shutdown" [_thread_blocked, id=1336, stack(0x29b30000,0x29b80000)]
0x27a1c400 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=356, stack(0x29ae0000,0x29b30000)]
0x27cdf000 JavaThread "Job_Executor2" daemon [_thread_blocked, id=3096, stack(0x299f0000,0x29a40000)]
0x289e1800 JavaThread "Job_Executor1" daemon [_thread_blocked, id=2504, stack(0x27d40000,0x27d90000)]
0x28aa3800 JavaThread "Thread-12" daemon [_thread_blocked, id=2536, stack(0x299a0000,0x299f0000)]
0x27b5e400 JavaThread "Timer-3" daemon [_thread_blocked, id=1116, stack(0x29950000,0x299a0000)]
0x270c7800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=1192, stack(0x288b0000,0x28900000)]
0x2730dc00 JavaThread "CompilerThread0" daemon [_thread_blocked, id=2352, stack(0x28810000,0x28860000)]
0x2742fc00 JavaThread "Attach Listener" daemon [_thread_blocked, id=2432, stack(0x287c0000,0x28810000)]
0x2730a800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=2752, stack(0x28770000,0x287c0000)]
0x2742e800 JavaThread "WorkerThread(client_coordination_stage,0)" daemon [_thread_blocked, id=2876, stack(0x286d0000,0x28720
000)]
0x2742e000 JavaThread "WorkerThread(lock_response_stage,0)" daemon [_thread_blocked, id=3628, stack(0x28680000,0x286d0000)]
0x26db5000 JavaThread "WorkerThread(hydrate_message_stage,0)" daemon [_thread_blocked, id=2800, stack(0x28630000,0x28680000)
]
0x26db4400 JavaThread "WorkerThread(receive_transaction_complete_stage,0)" daemon [_thread_blocked, id=584, stack(0x285e0000
,0x28630000)]
0x26db3400 JavaThread "WorkerThread(dmi_stage,0)" daemon [_thread_blocked, id=2916, stack(0x28590000,0x285e0000)]
0x270abc00 JavaThread "WorkerThread(batch_txn_ack_stage,0)" daemon [_thread_blocked, id=1008, stack(0x28540000,0x28590000)]
0x27306400 JavaThread "WorkerThread(lock_statistics_response_stage,0)" daemon [_thread_blocked, id=788, stack(0x284f0000,0x2
8540000)]
0x279b1c00 JavaThread "WorkerThread(jmxremote_tunnel_stage,0)" daemon [_thread_blocked, id=2356, stack(0x284a0000,0x284f0000
)]
0x2731c800 JavaThread "WorkerThread(receive_object_stage,0)" daemon [_thread_blocked, id=1624, stack(0x28450000,0x284a0000)]
0x27959800 JavaThread "WorkerThread(receive_root_id_stage,0)" daemon [_thread_blocked, id=3684, stack(0x28400000,0x28450000)
]
0x27264c00 JavaThread "WorkerThread(object_id_request_response_stage,0)" daemon [_thread_blocked, id=3664, stack(0x283b0000,
0x28400000)]
0x2730f400 JavaThread "WorkerThread(lock_statistics_enable_disable_stage,0)" daemon [_thread_blocked, id=3380, stack(0x28360
000,0x283b0000)]
0x27080400 JavaThread "WorkerThread(receive_transaction_stage,0)" daemon [_thread_blocked, id=3588, stack(0x28310000,0x28360
000)]
0x279c2400 JavaThread "TC Memory Monitor" daemon [_thread_blocked, id=3212, stack(0x28270000,0x282c0000)]
0x279f1800 JavaThread "Reaper" daemon [_thread_blocked, id=4080, stack(0x28220000,0x28270000)]
0x2725f800 JavaThread "H2 Log Writer STATISTICS-BUFFER" daemon [_thread_blocked, id=2868, stack(0x27ff0000,0x28040000)]
0x2725d800 JavaThread "H2 File Lock Watchdog X:\client\statistics-192.168.123.90\statistics-buffer.lock.db" daemon [_thread_
blocked, id=2828, stack(0x27fa0000,0x27ff0000)]
0x270a4c00 JavaThread "com.tc.util.ToggleableReferenceManager$QueueProcessor" daemon [_thread_blocked, id=3324, stack(0x27f5
0000,0x27fa0000)]
0x2707c800 JavaThread "DSO Lock Object.wait() timer" daemon [_thread_blocked, id=3320, stack(0x27f00000,0x27f50000)]
0x270a2800 JavaThread "RemoteTransactionManager Flusher" daemon [_thread_blocked, id=772, stack(0x27eb0000,0x27f00000)]
0x27008800 JavaThread "HealthChecker" daemon [_thread_blocked, id=3688, stack(0x27e60000,0x27eb0000)]
0x26f31400 JavaThread "TCComm Main Selector Thread" daemon [_thread_in_native, id=1508, stack(0x27e10000,0x27e60000)]
0x27b06400 JavaThread "SampledCounterManager Timer" daemon [_thread_blocked, id=3072, stack(0x27d90000,0x27de0000)]
0x27951800 JavaThread "startlevel job thread" daemon [_thread_blocked, id=2552, stack(0x27ce0000,0x27d30000)]
0x26d37800 JavaThread "Finalizer" daemon [_thread_blocked, id=1976, stack(0x26e60000,0x26eb0000)]
0x26d42400 JavaThread "Reference Handler" daemon [_thread_blocked, id=3824, stack(0x26e10000,0x26e60000)]
Other Threads:
0x26d34400 VMThread [stack: 0x26dc0000,0x26e10000] [id=1276]
0x27997c00 WatcherThread [stack: 0x28900000,0x28950000] [id=1272]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap
def new generation total 36288K, used 13758K [0x02a70000, 0x051d0000, 0x051d0000)
eden space 32256K, 38% used [0x02a70000, 0x036965e8, 0x049f0000)
from space 4032K, 32% used [0x049f0000, 0x04b392c8, 0x04de0000)
to space 4032K, 0% used [0x04de0000, 0x04de0000, 0x051d0000)
tenured generation total 483968K, used 119454K [0x051d0000, 0x22a70000, 0x22a70000)
the space 483968K, 24% used [0x051d0000, 0x0c677aa8, 0x0c677c00, 0x22a70000)
compacting perm gen total 44800K, used 44712K [0x22a70000, 0x25630000, 0x26a70000)
the space 44800K, 99% used [0x22a70000, 0x2561a0b8, 0x2561a200, 0x25630000)
No shared spaces configured.
Dynamic libraries:
0x00400000 - 0x00423000 X:\client\terra\jre\bin\java.exe
0x7c910000 - 0x7c9c7000 C:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c907000 C:\WINDOWS\system32\kernel32.dll
0x77da0000 - 0x77e4a000 C:\WINDOWS\system32\ADVAPI32.dll
0x77e50000 - 0x77ee2000 C:\WINDOWS\system32\RPCRT4.dll
0x77fc0000 - 0x77fd1000 C:\WINDOWS\system32\Secur32.dll
0x7c340000 - 0x7c396000 X:\client\terra\jre\bin\msvcr71.dll
0x6d7c0000 - 0x6da10000 X:\client\terra\jre\bin\client\jvm.dll
0x7e360000 - 0x7e3f0000 C:\WINDOWS\system32\USER32.dll
0x77ef0000 - 0x77f37000 C:\WINDOWS\system32\GDI32.dll
0x76af0000 - 0x76b1e000 C:\WINDOWS\system32\WINMM.dll
0x76330000 - 0x7634d000 C:\WINDOWS\system32\IMM32.DLL
0x6d270000 - 0x6d278000 X:\client\terra\jre\bin\hpi.dll
0x76bb0000 - 0x76bbb000 C:\WINDOWS\system32\PSAPI.DLL
0x6d770000 - 0x6d77c000 X:\client\terra\jre\bin\verify.dll
0x6d310000 - 0x6d32f000 X:\client\terra\jre\bin\java.dll
0x6d7b0000 - 0x6d7bf000 X:\client\terra\jre\bin\zip.dll
0x6d750000 - 0x6d758000 X:\client\terra\jre\bin\sunmscapi.dll
0x77a50000 - 0x77ae5000 C:\WINDOWS\system32\CRYPT32.dll
0x77be0000 - 0x77c38000 C:\WINDOWS\system32\msvcrt.dll
0x77af0000 - 0x77b02000 C:\WINDOWS\system32\MSASN1.dll
0x6d570000 - 0x6d583000 X:\client\terra\jre\bin\net.dll
0x71a10000 - 0x71a27000 C:\WINDOWS\system32\WS2_32.dll
0x71a00000 - 0x71a08000 C:\WINDOWS\system32\WS2HELP.dll
0x719b0000 - 0x719f0000 C:\WINDOWS\System32\mswsock.dll
0x76ee0000 - 0x76f07000 C:\WINDOWS\system32\DNSAPI.dll
0x76f70000 - 0x76f78000 C:\WINDOWS\System32\winrnr.dll
0x76f20000 - 0x76f4d000 C:\WINDOWS\system32\WLDAP32.dll
0x76f80000 - 0x76f86000 C:\WINDOWS\system32\rasadhlp.dll
0x6d590000 - 0x6d599000 X:\client\terra\jre\bin\nio.dll
0x66710000 - 0x66769000 C:\WINDOWS\system32\hnetcfg.dll
0x719f0000 - 0x719f8000 C:\WINDOWS\System32\wshtcpip.dll
0x27de0000 - 0x27e08000 C:\WINDOWS\system32\rsaenh.dll
0x76620000 - 0x766d5000 C:\WINDOWS\system32\USERENV.dll
0x597d0000 - 0x59824000 C:\WINDOWS\system32\netapi32.dll
0x6d4c0000 - 0x6d4c9000 X:\client\terra\jre\bin\management.dll
0x28040000 - 0x28074000 X:\client\terra\lib\sigar-x86-winnt-1.5.0.1.dll
0x7e670000 - 0x7ee91000 C:\WINDOWS\system32\SHELL32.dll
0x77f40000 - 0x77fb6000 C:\WINDOWS\system32\SHLWAPI.dll
0x74c30000 - 0x74c87000 C:\WINDOWS\system32\pdh.dll
0x76350000 - 0x7639a000 C:\WINDOWS\system32\comdlg32.dll
0x5d450000 - 0x5d4ea000 C:\WINDOWS\system32\COMCTL32.dll
0x774b0000 - 0x775ed000 C:\WINDOWS\system32\ole32.dll
0x770f0000 - 0x7717b000 C:\WINDOWS\system32\OLEAUT32.dll
0x745d0000 - 0x7460d000 C:\WINDOWS\system32\ODBC32.dll
0x66b40000 - 0x66b46000 C:\WINDOWS\system32\odbcbcp.dll
0x77bd0000 - 0x77bd8000 C:\WINDOWS\system32\VERSION.dll
0x773a0000 - 0x774a3000 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.2982_x-ww_ac
3f9c03\comctl32.dll
0x28120000 - 0x28139000 C:\WINDOWS\system32\odbcint.dll
0x6d000000 - 0x6d12e000 X:\client\terra\jre\bin\awt.dll
0x72f70000 - 0x72f96000 C:\WINDOWS\system32\WINSPOOL.DRV
0x5b0f0000 - 0x5b128000 C:\WINDOWS\system32\uxtheme.dll
0x736d0000 - 0x73719000 C:\WINDOWS\system32\ddraw.dll
0x73b30000 - 0x73b36000 C:\WINDOWS\system32\DCIMAN32.dll
0x6d210000 - 0x6d263000 X:\client\terra\jre\bin\fontmanager.dll
0x746a0000 - 0x746eb000 C:\WINDOWS\system32\MSCTF.dll
0x28880000 - 0x28895000 C:\Programme\TightVNC\VNCHooks.dll
0x29c30000 - 0x29c53000 C:\Programme\ScanSoft\OmniPageSE4.0\OpHookSE4.dll
0x75250000 - 0x7527e000 C:\WINDOWS\system32\msctfime.ime
0x6d180000 - 0x6d1a3000 X:\client\terra\jre\bin\dcpr.dll
0x6d3b0000 - 0x6d3d4000 X:\client\terra\jre\bin\jpeg.dll
0x6d150000 - 0x6d17f000 X:\client\terra\jre\bin\cmm.dll
0x6d350000 - 0x6d35d000 X:\client\terra\jre\bin\JdbcOdbc.dll
0x76f90000 - 0x7700f000 C:\WINDOWS\system32\CLBCATQ.DLL
0x77010000 - 0x770e3000 C:\WINDOWS\system32\COMRes.dll
0x33570000 - 0x335aa000 C:\WINDOWS\system32\msjtes40.dll
0x28760000 - 0x2876b000 C:\WINDOWS\system32\VBAJET32.DLL
0x33dc0000 - 0x33e22000 C:\WINDOWS\system32\expsrv.dll
0x345b0000 - 0x34889000 C:\WINDOWS\system32\xpsp2res.dll
0x4de00000 - 0x4de44000 C:\WINDOWS\system32\odbcjt32.dll
0x2e470000 - 0x2e5e0000 C:\WINDOWS\system32\msjet40.dll
0x2e5e0000 - 0x2e675000 C:\WINDOWS\system32\mswstr10.dll
0x6da70000 - 0x6da7f000 C:\WINDOWS\system32\odbcji32.dll
0x282e0000 - 0x282ed000 C:\WINDOWS\system32\msjter40.dll
0x28720000 - 0x2874d000 C:\WINDOWS\system32\MSJINT40.DLL
0x34290000 - 0x342dc000 C:\WINDOWS\system32\msrd3x40.dll
VM Arguments:
jvm_args: -Xbootclasspath/p:X:\client\terra\bin\..\bin\..\lib\dso-boot\dso-boot-hotspot_win32_160_05.jar -Dtc.install-root=X:\
client\terra\bin\..\bin\.. -Xms512M -Xmx512M
java_command: client-1.0-SNAPSHOT.jar
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME="X:\client\terra\bin\..\jre"
PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
USERNAME=G-Oliveira
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 8 Stepping 1, AuthenticAMD
--------------- S Y S T E M ---------------
OS: Windows XP Build 2600 Service Pack 2
CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 8 stepping 1, cmov, cx8, fxsr, mmx, sse, mmxext, 3dnow, 3dnow
ext
Memory: 4k page, physical 458224k(17644k free), swap 1495356k(588700k free)
vm_info: Java HotSpot(TM) Client VM (10.0-b19) for windows-x86 JRE (1.6.0_05-b13), built on Feb 22 2008 01:16:53 by "java_re"
with MS VC++ 7.1
time: Fri May 23 22:23:37 2008
elapsed time: 768 seconds
|
|
|
I use 1.6 - and yes, I am sure ;-) (checked the system property "java.version".
It is possible to connecto to the terracotta server.
But my client (started with dso-java.sh) is reported as "Unknown Application". When I selected this app, VisualVM just shows very, very basic information and tells me "Not supported for this JVM"...
So I don't really know what the problem might be.
|
|
|
Great news - I am looking forward for the prototype. Thanks for your great work with Terracotta...
|
|
|
I still use breadth first prefetching... I will try the parameters changes.
Can you tell me what an expected/good/normal Object Fault Rate might be (for a "normal" machine)?
|
|
|
Some days ago a RC1 of VisualVM (https://visualvm.dev.java.net/) has been released. And it looks really promising.
Of course I tried to monitor some clustered applications. But I only got "JVM not supported".
Is it possible or will it be possible to use VisualVM together with Terracotta?
Regards...
|
|
|
Okay, I think I improved the things a little bit.
I started the tc-server with Xmx1024M - now it is much faster...
The network is definitively not the bottle neck. When copying a file using scp I get much, much higher transfer rates.
What are "normal" or expected object fault rates?
|
|
|
|
|