[Logo] Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register /  [Login] Login 
[Expert]
L2 exiting unexpectedly  XML
Forum Index -> Terracotta Platform
Author Message
[Post New]07/05/2012 22:14:16
    Subject: L2 exiting unexpectedly
[Up]
dorel

neo

Joined: 07/05/2012 07:57:20
Messages: 1
Offline
Hey guys,

We've been experiencing unexpected L2 exiting a couple of times a day lately with the below exception. We're using Terracotta Open Source with EHCache as Hibernate's Level 2 cache. Everything went OK for a while but a couple af weeks ago we started experiencing this. Anyone has an idea what's happening ?

We're on Ubuntu + Oracle Java 1.6.last_update + Terracotta 3.6.2, have an L2 and 6 L1 machines

I have the entire dump if it's needed


2012-07-05 04:51:23,430 [WorkerThread(transaction_lookup_stage, 0)] WARN com.tc.objectserver.api.ObjectManager - Request for non-exstent object : ObjectID=[490837] context = ObjectManagerLookupContext@72698732 : [ processed count = 0, responseContext = LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}, missing = BitSetObjectIDSet [ ] ]

com.tc.exception.TCRuntimeException: Uncaught exception in stage

at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:164)2012-07-05 04:51:23,432 [WorkerThread(transaction_lookup_stage, 0)] WARN com.tc.objectserver.tx.TransactionalObjectManagerImpl - Failed: ObjectID=[490837] actionLogical= physicalAction= isDelta=true

java.lang.AssertionError: Lookup for non-exisistent Objects : BitSetObjectIDSet [ ObjectID=[490837] ] lookup context is : LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.assertNoMissingObjects(TransactionalObjectManagerImpl.java:502)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.setResults(TransactionalObjectManagerImpl.java:474)

at com.tc.objectserver.impl.ObjectManagerImpl$ObjectManagerLookupContext.setResults(ObjectManagerImpl.java:1137)

at com.tc.objectserver.impl.ObjectManagerImpl.basicInternalLookupObjectsFor(ObjectManagerImpl.java:560)

at com.tc.objectserver.impl.ObjectManagerImpl.basicLookupObjectsFor(ObjectManagerImpl.java:512)

at com.tc.objectserver.impl.ObjectManagerImpl.lookupObjectsFor(ObjectManagerImpl.java:228)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForApplyAndAddToSink(TransactionalObjectManagerImpl.java:208)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForTransactions(TransactionalObjectManagerImpl.java:158)

at com.tc.objectserver.handler.TransactionLookupHandler.handleEvent(TransactionLookupHandler.java:18)

at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145)



Caused by: java.lang.RuntimeException: java.lang.AssertionError: Lookup for non-exisistent Objects : BitSetObjectIDSet [ ObjectID=[490837] ] lookup context is : LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForTransactions(TransactionalObjectManagerImpl.java:169)

at com.tc.objectserver.handler.TransactionLookupHandler.handleEvent(TransactionLookupHandler.java:18)

at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145)

Caused by: java.lang.AssertionError: Lookup for non-exisistent Objects : BitSetObjectIDSet [ ObjectID=[490837] ] lookup context is : LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.assertNoMissingObjects(TransactionalObjectManagerImpl.java:502)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.setResults(TransactionalObjectManagerImpl.java:474)

at com.tc.objectserver.impl.ObjectManagerImpl$ObjectManagerLookupContext.setResults(ObjectManagerImpl.java:1137)

at com.tc.objectserver.impl.ObjectManagerImpl.basicInternalLookupObjectsFor(ObjectManagerImpl.java:560)

at com.tc.objectserver.impl.ObjectManagerImpl.basicLookupObjectsFor(ObjectManagerImpl.java:512)

at com.tc.objectserver.impl.ObjectManagerImpl.lookupObjectsFor(ObjectManagerImpl.java:228)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForApplyAndAddToSink(TransactionalObjectManagerImpl.java:208)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForTransactions(TransactionalObjectManagerImpl.java:158)

... 2 more

2012-07-05 04:51:23,436 [WorkerThread(transaction_lookup_stage, 0)] ERROR com.tc.server.TCServerMain - Thread:Thread[WorkerThread(transaction_lookup_stage, 0),5,TC Thread Group] got an uncaught exception. calling CallbackOnExitDefaultHandlers.

com.tc.exception.TCRuntimeException: Uncaught exception in stage

at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:164)

Caused by: java.lang.RuntimeException: java.lang.AssertionError: Lookup for non-exisistent Objects : BitSetObjectIDSet [ ObjectID=[490837] ] lookup context is : LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForTransactions(TransactionalObjectManagerImpl.java:169)

at com.tc.objectserver.handler.TransactionLookupHandler.handleEvent(TransactionLookupHandler.java:18)

at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145)

Caused by: java.lang.AssertionError: Lookup for non-exisistent Objects : BitSetObjectIDSet [ ObjectID=[490837] ] lookup context is : LookupContext [ txnID = ServerTransactionID{ClientID[5],TransactionID=[42547]}, oids = BitSetObjectIDSet [ ObjectID=[490837] ], seqID = SequenceID=[42547], clientTxnID = TransactionID=[42547], numTxn = 1] = { pending = false, lookedupObjects.size() = 0}

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.assertNoMissingObjects(TransactionalObjectManagerImpl.java:502)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl$LookupContext.setResults(TransactionalObjectManagerImpl.java:474)

at com.tc.objectserver.impl.ObjectManagerImpl$ObjectManagerLookupContext.setResults(ObjectManagerImpl.java:1137)

at com.tc.objectserver.impl.ObjectManagerImpl.basicInternalLookupObjectsFor(ObjectManagerImpl.java:560)

at com.tc.objectserver.impl.ObjectManagerImpl.basicLookupObjectsFor(ObjectManagerImpl.java:512)

at com.tc.objectserver.impl.ObjectManagerImpl.lookupObjectsFor(ObjectManagerImpl.java:228)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForApplyAndAddToSink(TransactionalObjectManagerImpl.java:208)

at com.tc.objectserver.tx.TransactionalObjectManagerImpl.lookupObjectsForTransactions(TransactionalObjectManagerImpl.java:158)

... 2 more

2012-07-05 04:51:23,809 [WorkerThread(transaction_lookup_stage, 0)] ERROR com.tc.logging.ThreadDumpHandler - Thu Jul 05 04:51:23 UTC 2012

Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.6-b01 mixed mode):
 
[Post New]07/06/2012 13:50:12
    Subject: Re:L2 exiting unexpectedly
[Up]
ssubbiah

jedi

Joined: 05/24/2006 14:25:22
Messages: 117
Location: Saravanan Subbiah
Offline
Can u please open a jira and attach the logs there ? Its probably a better place to track this.
Saravanan Subbiah
Terracotta Engineer
[Post New]07/06/2012 13:52:18
    Subject: Re:L2 exiting unexpectedly
[Up]
klalithr

consul

Joined: 01/23/2011 10:58:07
Messages: 489
Offline
Plus, please share your hibernate version as well.
Karthik Lalithraj (Terracotta)
 
Forum Index -> Terracotta Platform
Go to:   
Powered by JForum 2.1.7 © JForum Team