We are using Terracotta 3.4.1 and the thread dump is pretty similar during the weird locking event:
"TP-Processor396" daemon prio=10 tid=0x00002aff2ba9d000 nid=0x5a7b waiting on condition [0x00002aff61e98000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002afecfb91da0> (a java.util.concurrent.Semaphore$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:286)
at com.tc.object.locks.LockStateNode$PendingLockHold.park(LockStateNode.java:179)
at com.tc.object.locks.ClientLockImpl.acquireQueued(ClientLockImpl.java:723)
at com.tc.object.locks.ClientLockImpl.acquireQueued(ClientLockImpl.java:701)
at com.tc.object.locks.ClientLockImpl.lock(ClientLockImpl.java:52)
at com.tc.object.locks.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:98)
at com.tc.object.bytecode.ManagerImpl.lock(ManagerImpl.java:747)
at com.tc.object.bytecode.ManagerUtilInternal.beginLock(ManagerUtilInternal.java:33)
at org.terracotta.locking.strategy.LongLockStrategy.beginLock(LongLockStrategy.java:16)
at org.terracotta.locking.strategy.LongLockStrategy.beginLock(LongLockStrategy.java:7)
at com.terracotta.toolkit.collections.ConcurrentDistributedMapDso.beginLock(ConcurrentDistributedMapDso.java:828)
at com.terracotta.toolkit.collections.ConcurrentDistributedMapDso.get(ConcurrentDistributedMapDso.java:195)
at com.terracotta.toolkit.collections.ConcurrentDistributedMapDsoArray.get(ConcurrentDistributedMapDsoArray.java:175)
at org.terracotta.collections.ConcurrentDistributedMap.get(ConcurrentDistributedMap.java:190)
at org.terracotta.cache.TerracottaDistributedCache.getNonExpiredEntry(TerracottaDistributedCache.java:197)
at org.terracotta.cache.TerracottaDistributedCache.getNonExpiredEntryCoherent(TerracottaDistributedCache.java:131)
at org.terracotta.cache.TerracottaDistributedCache.containsKey(TerracottaDistributedCache.java:126)
at org.terracotta.modules.ehcache.store.ClusteredStore.internalContainsKey(ClusteredStore.java:476)
at org.terracotta.modules.ehcache.store.ClusteredStore.containsKey(ClusteredStore.java:456)
at org.terracotta.modules.ehcache.store.ClusteredStore.containsKeyInMemory(ClusteredStore.java:463)
at net.sf.ehcache.Cache.searchInStoreWithStats(Cache.java:1742)
at net.sf.ehcache.Cache.get(Cache.java:1405)
at net.sf.ehcache.hibernate.regions.EhcacheTransactionalDataRegion.get(EhcacheTransactionalDataRegion.java:91)
at net.sf.ehcache.hibernate.strategy.AbstractReadWriteEhcacheAccessStrategy.get(AbstractReadWriteEhcacheAccessStrategy.java:65)
at org.hibernate.event.def.DefaultLoadEventListener.loadFromSecondLevelCache(DefaultLoadEventListener.java:524)
at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:397)
at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:165)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:223)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:126)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:906)
at org.hibernate.impl.SessionImpl.internalLoad(SessionImpl.java:874)
at org.hibernate.type.EntityType.resolveIdentifier(EntityType.java:590)
My question is, will upgrading to Terracotta 3.5.1 fix this or do we have to switch the HttpSession to HttpSessionMutexListener?
I'm not really sure what going to HttpSessionMutexListener means but the solution to upgrade Terracotta to fix this issue would obviously be attractive.
We don't see any clients timeouts in the client logs but we have the same behaviour where some weird locking is happening until the Tomcat connection pool threads are maxed out.
We use Spring MVC 2.5.1 I believe it is and tomcat 6.0.30.
Is Terracotta 3.5.1 a safe version? I seem to recall reading about people having issues with it and Terracotta 3.5.3 or 3.5.4 looking like safe versions.