Author |
Message |
|
Thanks for your answer Erik, it seems you're right
Someone file this kind of bug
https://support.zabbix.com/browse/ZBXNEXT-1274 <- vote to get this issue fixed asap
Closed thread
|
|
|
Hi folks, i can't connect Zabbix server (monitoring tool) to Terracotta.
Versions
Zabbix 2.2.3
Zabbix-Java-Gateway 2.2.3
Terracotta Server Array: 3.7.5
Zabbix and Zabbix java gateway are remote.
Zabbix connect via JMX to servers, thanks to Zabbix-Java-Gateway in whose log:
2014-06-26 03:49:09.214 [pool-1-thread-2] WARN com.zabbix.gateway.SocketProcessor - error processing request
com.zabbix.gateway.ZabbixException: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
at com.zabbix.gateway.JMXItemChecker.getValues(JMXItemChecker.java:100) ~[zabbix-java-gateway-2.2.3.jar:na]
at com.zabbix.gateway.SocketProcessor.run(SocketProcessor.java:63) ~[zabbix-java-gateway-2.2.3.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
Caused by: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369) ~[na:1.7.0_55]
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268) ~[na:1.7.0_55]
at com.zabbix.gateway.JMXItemChecker.getValues(JMXItemChecker.java:92) ~[zabbix-java-gateway-2.2.3.jar:na]
... 4 common frames omitted
Caused by: javax.naming.CommunicationException: null
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118) ~[na:1.7.0_55]
at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203) ~[na:1.7.0_55]
at javax.naming.InitialContext.lookup(InitialContext.java:411) ~[na:1.7.0_55]
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1936) ~[na:1.7.0_55]
at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1903) ~[na:1.7.0_55]
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:286) ~[na:1.7.0_55]
... 6 common frames omitted
Caused by: java.rmi.ConnectIOException: non-JRMP server at remote endpoint
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:248) ~[na:1.7.0_55]
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.7.0_55]
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:341) ~[na:1.7.0_55]
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) ~[na:1.7.0_55]
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114) ~[na:1.7.0_55]
... 11 common frames omitted
I think Terracotta is well configured because I can run jconsole without problem via:
jconsole -J-Djava.class.path="/usr/local/terracotta/terracotta-3.7.5/lib/tc.jar:/usr/lib/jvm/java-7-oracle/lib/jconsole.jar"
I tried adding tc.jar to the java gateway classpath but nothing happens.
Firewall is properly configured
Any idea?
|
|
|
hi guys,
I'm running a terracotta server 3.5.1 instance
with -xmx=768m -xms=768m
I'm getting heap space errors, I may grow up the memory, but I'm not sure because the system has more then 150MB free everytime.
I dont know what "Asking for a large amount of memory" means, why is a warn?
Should I try to avoid getting elements in cache higher than default threshold (8mb)?
Thank u
sorry about my unknown, put trace below
2011-12-14 11:21:41,013 [L2_L1:TCWorkerComm # 1_W] WARN com.tc.bytes.TCByteBufferFactory - Asking for a large amount of memory: 30352490 bytes
2011-12-14 11:25:01,218 [Statistics Logger] INFO com.terracottatech.dso - memory free : 190,814339 MB
2011-12-14 11:25:01,219 [Statistics Logger] INFO com.terracottatech.dso - memory used : 491,873161 MB
2011-12-14 11:25:01,219 [Statistics Logger] INFO com.terracottatech.dso - memory max : 682,687500 MB
2011-12-14 11:25:12,258 [WorkerThread(respond_to_request_stage, 1, 0)] WARN com.tc.net.core.TCConnection - Warning: Attempting to send a message (com.tc.net.prot
ocol.delivery.OOOProtocolMessageImpl) of size 14596487 bytes
2011-12-14 11:25:12,258 [L2_L1:TCWorkerComm # 1_W] WARN com.tc.bytes.TCByteBufferFactory - Asking for a large amount of memory: 14596519 bytes
2011-12-14 11:27:03,211 [Server Map Periodic Evictor] INFO com.tc.objectserver.impl.ServerMapEvictionStatsManager - Server Map Periodic eviction - Time taken (ms
ecs): 969, Number of segments under threshold: 14336, Number of segments over threshold: 0, Total overshoot: 0, Total number of samples requested: 0, Number of s
egments where eviction happened: 87, Total number of evicted entries: 87
2011-12-14 11:27:24,008 [L2_L1:TCWorkerComm # 1_R] WARN com.tc.bytes.TCByteBufferFactory - Asking for a large amount of memory: 14596597 bytes
2011-12-14 11:27:24,037 [WorkerThread(apply_changes_stage, 0)] WARN com.tc.object.dna.impl.BaseDNAEncodingImpl - Attempt to read a byte array of len: 14596110; t
hreshold=8000000
2011-12-14 11:28:08,124 [L2_L1:TCWorkerComm # 1_R] WARN com.tc.bytes.TCByteBufferFactory - Asking for a large amount of memory: 30352684 bytes
2011-12-14 11:28:08,301 [WorkerThread(apply_changes_stage, 0)] WARN com.tc.object.dna.impl.BaseDNAEncodingImpl - Attempt to read a byte array of len: 30352214; t
hreshold=8000000
2011-12-14 11:28:19,843 [L2_L1:TCWorkerComm # 1_R] WARN com.tc.bytes.TCByteBufferFactory - Asking for a large amount of memory: 30352684 bytes
2011-12-14 11:28:20,540 [WorkerThread(apply_changes_stage, 0)] WARN com.tc.object.dna.impl.BaseDNAEncodingImpl - Attempt to read a byte array of len: 30352214; threshold=8000000
2011-12-14 11:28:27,209 [WorkerThread(managed_object_fault_stage, 2, 0)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(managed_object_fault_stage, 2, 0),5,TC Thread Grou
p] got an uncaught exception. calling CallbackOnExitDefaultHandlers.
com.tc.objectserver.persistence.db.DBException: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:25)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.loadObjectByID(ManagedObjectPersistorImpl.java:288)
at com.tc.objectserver.impl.PersistentManagedObjectStore.getObjectByID(PersistentManagedObjectStore.java:127)
at com.tc.objectserver.handler.ManagedObjectFaultHandler.handleEvent(ManagedObjectFaultHandler.java:55)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1434)
at com.sleepycat.je.Environment.checkEnv(Environment.java:2117)
at com.sleepycat.je.Environment.beginTransactionInternal(Environment.java:1280)
at com.sleepycat.je.Environment.beginTransaction(Environment.java:1262)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newNativeTransaction(BerkeleyDBPersistenceTransactionProvider.java:30)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:23)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.flushToCompressedStorage(FastObjectIDManagerImpl.java:204)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.checkpointToCompressedStorage(FastObjectIDManagerImpl.java:283)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.access$200(FastObjectIDManagerImpl.java:31)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl$CheckpointRunner.run(FastObjectIDManagerImpl.java:331)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at com.sleepycat.je.log.LogManager.marshallIntoBuffer(LogManager.java:691)
at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:379)
at com.sleepycat.je.log.LogManager.log(LogManager.java:328)
at com.sleepycat.je.tree.LN.log(LN.java:557)
at com.sleepycat.je.tree.LN.optionalLog(LN.java:430)
at com.sleepycat.je.tree.Tree.insertNewSlot(Tree.java:2534)
at com.sleepycat.je.tree.Tree.insert(Tree.java:2467)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1209)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1799)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1756)
at com.sleepycat.je.Cursor.putNotify(Cursor.java:1689)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:1626)
at com.sleepycat.je.Database.putInternal(Database.java:1186)
at com.sleepycat.je.Database.put(Database.java:1058)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCBytesBytesDatabase.put(BerkeleyDBTCBytesBytesDatabase.java:50)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.put(BerkeleyDBTCObjectDatabase.java:37)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.insert(BerkeleyDBTCObjectDatabase.java:74)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:344)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:404)
at com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:90)
at com.tc.objectserver.impl.ObjectManagerImpl.flushAllAndCommit(ObjectManagerImpl.java:860)
at com.tc.objectserver.impl.ObjectManagerImpl.releaseAllAndCommit(ObjectManagerImpl.java:666)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.commit(ServerTransactionManagerImpl.java:389)
at com.tc.objectserver.handler.CommitTransactionChangeHandler.handleEvent(CommitTransactionChangeHandler.java:31)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
2011-12-14 11:28:27,334 [ObjectID-CompressedStorage Checkpoint] ERROR com.tc.server.TCServerMain - Thread:Thread[ObjectID-CompressedStorage Checkpoint,5,TC Thread Group] got an uncaught
exception. calling CallbackOnExitDefaultHandlers.
com.tc.objectserver.persistence.db.DBException: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:25)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.flushToCompressedStorage(FastObjectIDManagerImpl.java:204)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.checkpointToCompressedStorage(FastObjectIDManagerImpl.java:283)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.access$200(FastObjectIDManagerImpl.java:31)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl$CheckpointRunner.run(FastObjectIDManagerImpl.java:331)
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1434)
at com.sleepycat.je.Environment.checkEnv(Environment.java:2117)
at com.sleepycat.je.Environment.beginTransactionInternal(Environment.java:1280)
at com.sleepycat.je.Environment.beginTransaction(Environment.java:1262)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newNativeTransaction(BerkeleyDBPersistenceTransactionProvider.java:30)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:23)
... 4 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at com.sleepycat.je.log.LogManager.marshallIntoBuffer(LogManager.java:691)
at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:379)
at com.sleepycat.je.log.LogManager.log(LogManager.java:328)
at com.sleepycat.je.tree.LN.log(LN.java:557)
at com.sleepycat.je.tree.LN.optionalLog(LN.java:430)
at com.sleepycat.je.tree.Tree.insertNewSlot(Tree.java:2534)
at com.sleepycat.je.tree.Tree.insert(Tree.java:2467)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1209)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1799)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1756)
at com.sleepycat.je.Cursor.putNotify(Cursor.java:1689)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:1626)
at com.sleepycat.je.Database.putInternal(Database.java:1186)
at com.sleepycat.je.Database.put(Database.java:1058)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCBytesBytesDatabase.put(BerkeleyDBTCBytesBytesDatabase.java:50)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.put(BerkeleyDBTCObjectDatabase.java:37)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.insert(BerkeleyDBTCObjectDatabase.java:74)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:344)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:404)
at com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:90)
at com.tc.objectserver.impl.ObjectManagerImpl.flushAllAndCommit(ObjectManagerImpl.java:860)
at com.tc.objectserver.impl.ObjectManagerImpl.releaseAllAndCommit(ObjectManagerImpl.java:666)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.commit(ServerTransactionManagerImpl.java:389)
at com.tc.objectserver.handler.CommitTransactionChangeHandler.handleEvent(CommitTransactionChangeHandler.java:31)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
2011-12-14 11:28:27,334 [WorkerThread(commit_changes_stage, 1, 0)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(commit_changes_stage, 1, 0),5,TC Thread Group] got an un
caught exception. calling CallbackOnExitDefaultHandlers.
com.tc.objectserver.persistence.db.DBException: com.tc.objectserver.persistence.db.TCDatabaseException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:417)
at com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:90)
at com.tc.objectserver.impl.ObjectManagerImpl.flushAllAndCommit(ObjectManagerImpl.java:860)
at com.tc.objectserver.impl.ObjectManagerImpl.releaseAllAndCommit(ObjectManagerImpl.java:666)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.commit(ServerTransactionManagerImpl.java:389)
at com.tc.objectserver.handler.CommitTransactionChangeHandler.handleEvent(CommitTransactionChangeHandler.java:31)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
Caused by: com.tc.objectserver.persistence.db.TCDatabaseException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:360)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:404)
... 6 more
2011-12-14 11:28:27,211 [WorkerThread(transaction_lowwatermark_stage, 0)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(transaction_lowwatermark_stage, 0),5,TC Thread Gr
oup] got an uncaught exception. calling CallbackOnExitDefaultHandlers.
com.tc.objectserver.persistence.db.DBException: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:25)
at com.tc.objectserver.gtx.ServerGlobalTransactionManagerImpl.clearCommitedTransactionsBelowLowWaterMark(ServerGlobalTransactionManagerImpl.java:57)
at com.tc.objectserver.handler.TransactionLowWaterMarkHandler.handleEvent(TransactionLowWaterMarkHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1434)
at com.sleepycat.je.Environment.checkEnv(Environment.java:2117)
at com.sleepycat.je.Environment.beginTransactionInternal(Environment.java:1280)
at com.sleepycat.je.Environment.beginTransaction(Environment.java:1262)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newNativeTransaction(BerkeleyDBPersistenceTransactionProvider.java:30)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:23)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.flushToCompressedStorage(FastObjectIDManagerImpl.java:204)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.checkpointToCompressedStorage(FastObjectIDManagerImpl.java:283)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.access$200(FastObjectIDManagerImpl.java:31)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl$CheckpointRunner.run(FastObjectIDManagerImpl.java:331)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at com.sleepycat.je.log.LogManager.marshallIntoBuffer(LogManager.java:691)
at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:379)
at com.sleepycat.je.log.LogManager.log(LogManager.java:328)
at com.sleepycat.je.tree.LN.log(LN.java:557)
at com.sleepycat.je.tree.LN.optionalLog(LN.java:430)
at com.sleepycat.je.tree.Tree.insertNewSlot(Tree.java:2534)
at com.sleepycat.je.tree.Tree.insert(Tree.java:2467)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1209)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1799)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1756)
at com.sleepycat.je.Cursor.putNotify(Cursor.java:1689)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:1626)
at com.sleepycat.je.Database.putInternal(Database.java:1186)
at com.sleepycat.je.Database.put(Database.java:1058)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCBytesBytesDatabase.put(BerkeleyDBTCBytesBytesDatabase.java:50)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.put(BerkeleyDBTCObjectDatabase.java:37)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.insert(BerkeleyDBTCObjectDatabase.java:74)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:344)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:404)
at com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:90)
at com.tc.objectserver.impl.ObjectManagerImpl.flushAllAndCommit(ObjectManagerImpl.java:860)
at com.tc.objectserver.impl.ObjectManagerImpl.releaseAllAndCommit(ObjectManagerImpl.java:666)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.commit(ServerTransactionManagerImpl.java:389)
at com.tc.objectserver.handler.CommitTransactionChangeHandler.handleEvent(CommitTransactionChangeHandler.java:31)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
2011-12-14 11:28:27,209 [WorkerThread(managed_object_fault_stage, 1, 0)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(managed_object_fault_stage, 1, 0),5,TC Thread Grou
p] got an uncaught exception. calling CallbackOnExitDefaultHandlers.
com.tc.objectserver.persistence.db.DBException: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:25)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.loadObjectByID(ManagedObjectPersistorImpl.java:288)
at com.tc.objectserver.impl.PersistentManagedObjectStore.getObjectByID(PersistentManagedObjectStore.java:127)
at com.tc.objectserver.handler.ManagedObjectFaultHandler.handleEvent(ManagedObjectFaultHandler.java:55)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.1.7) JAVA_ERROR: Java Error occurred, recovery may not be possible.
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1434)
at com.sleepycat.je.Environment.checkEnv(Environment.java:2117)
at com.sleepycat.je.Environment.beginTransactionInternal(Environment.java:1280)
at com.sleepycat.je.Environment.beginTransaction(Environment.java:1262)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newNativeTransaction(BerkeleyDBPersistenceTransactionProvider.java:30)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBPersistenceTransactionProvider.newTransaction(BerkeleyDBPersistenceTransactionProvider.java:23)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.flushToCompressedStorage(FastObjectIDManagerImpl.java:204)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.checkpointToCompressedStorage(FastObjectIDManagerImpl.java:283)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl.access$200(FastObjectIDManagerImpl.java:31)
at com.tc.objectserver.persistence.db.FastObjectIDManagerImpl$CheckpointRunner.run(FastObjectIDManagerImpl.java:331)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
at com.sleepycat.je.log.LogManager.marshallIntoBuffer(LogManager.java:691)
at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:379)
at com.sleepycat.je.log.LogManager.log(LogManager.java:328)
at com.sleepycat.je.tree.LN.log(LN.java:557)
at com.sleepycat.je.tree.LN.optionalLog(LN.java:430)
at com.sleepycat.je.tree.Tree.insertNewSlot(Tree.java:2534)
at com.sleepycat.je.tree.Tree.insert(Tree.java:2467)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1209)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1799)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1756)
at com.sleepycat.je.Cursor.putNotify(Cursor.java:1689)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:1626)
at com.sleepycat.je.Database.putInternal(Database.java:1186)
at com.sleepycat.je.Database.put(Database.java:1058)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCBytesBytesDatabase.put(BerkeleyDBTCBytesBytesDatabase.java:50)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.put(BerkeleyDBTCObjectDatabase.java:37)
at com.tc.objectserver.storage.berkeleydb.BerkeleyDBTCObjectDatabase.insert(BerkeleyDBTCObjectDatabase.java:74)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.basicSaveObject(ManagedObjectPersistorImpl.java:344)
at com.tc.objectserver.persistence.db.ManagedObjectPersistorImpl.saveAllObjects(ManagedObjectPersistorImpl.java:404)
at com.tc.objectserver.impl.PersistentManagedObjectStore.commitAllObjects(PersistentManagedObjectStore.java:90)
at com.tc.objectserver.impl.ObjectManagerImpl.flushAllAndCommit(ObjectManagerImpl.java:860)
at com.tc.objectserver.impl.ObjectManagerImpl.releaseAllAndCommit(ObjectManagerImpl.java:666)
at com.tc.objectserver.tx.ServerTransactionManagerImpl.commit(ServerTransactionManagerImpl.java:389)
at com.tc.objectserver.handler.CommitTransactionChangeHandler.handleEvent(CommitTransactionChangeHandler.java:31)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:130)
2011-12-14 11:28:28,347 [WorkerThread(managed_object_fault_stage, 2, 0)] ERROR com.tc.logging.ThreadDumpHandler - Wed Dec 14 11:28:27 CET 2011
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode):
|
|
|
|
|