Terracotta Discussion Forums

When preloading a cache instance, it takes much longer to put objects in a cache with Terracotta than standalone.

to put ~10,600 objects:
[list]w/o Terracotta: 593 millseconds
w/ Terracotta: 266973 milliseconds[/list]

code snippet:
Code:

for(final Merchant m: merchants) {
     Element element = new Element(m.getId(),m);
     merchantCache.put(element);
 }

Any ideas?

Thanks,

Darin

Terracotta 2.6-nightly-rev8060 (also tested with 2.5.2, 2.5.4 and 2.6-stable0; all comparable results)
Note: in order to make Terracotta 2.6-nightly-rev8060 work, I had to rename clustered-commons-collections-3.1 from 2.6.0-SNAPSHOT to 2.5.2.
Windows 2003 Service Pack 2, Xeon 3.00Ghz, 3.5GB of RAM
JDK 1.5.0_10
Tomcat 5.5.25
Spring 2.0.4
EHCache 1.2.4

Tomcat JVM args (using server VM):

-Xbootclasspath/p:C:\Tools\Terracotta\terracotta-2.6-nightly-rev8060\lib\dso-boot\dso-boot-hotspot_win32_150_10.jar
-Dtc.install-root=C:\Tools\Terracotta\terracotta-2.6-nightly-rev8060
-Dtc.config=10.5.8.101:9510
-Dcom.tc.session.serverid=1DA502FCD559CE673D1E08FA633D09CF
-Dcatalina.base=C:\Tools\tomcat2
-Dcatalina.home=C:\Tools\tomcat2
-Djava.endorsed.dirs=C:\Tools\tomcat2\common\endorsed
-Djava.io.tmpdir=C:\Tools\tomcat2\temp
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=8299
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-XXermSize=256M
-XX:MaxPermSize=256m
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=C:\heapdumps
-XX:+PrintHeapAtGC
-verbose:gc
-Xms1024m
-Xmx1024m
-Xss128k

Terracotta JVM args:

-Xms1408m
-Xmx1408m
-Xss128k
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9199
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-XX:MaxPermSize=256m
-Dtc.install-root=C:\Tools/Terracotta/terracotta-2.6-nightly-rev8060
-Dtc.config=../../tc-config.xml
-verbose:gc
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=C:\heapdumps
-XX:+UseParallelGC
-XX:NewRatio=2
-XX:+PrintGCDetails
-Xloggc:C:\Temp\gc_parallel.log

tc-config.xml clients section:

<clients>
<modules>
<module name="tim-ehcache-1.2.4" version="1.0.2" group-id="org.terracotta.modules" />
<module name="clustered-cglib-2.1.3" version="2.6.0-SNAPSHOT" group-id="org.terracotta.modules" />
</modules>
<logs>C:/Tools/Terracotta/client-logs/%(com.tc.session.serverid)</logs>
<statistics>C:/Tools/Terracotta/statistics/%(com.tc.session.serverid)</statistics>
<dso>
<fault-count>5000</fault-count>
<debugging>
<instrumentation-logging>
<class>false</class>
<roots>false</roots>
<locks>false</locks>
<transient-root>false</transient-root>
<distributed-methods>false</distributed-methods>
</instrumentation-logging>
<runtime-logging>
<lock-debug>false</lock-debug>
<distributed-method-debug>false</distributed-method-debug>
</runtime-logging>
<runtime-output-options>
<auto-lock-details>true</auto-lock-details>
<caller>false</caller>
<full-stack>true</full-stack>
</runtime-output-options>
</debugging>
</dso>
</clients>

Just curious, what is the timing if you change the EHCache to a ConcurrentHashMap?

In general, as you are probably aware, comparing clustering to non-clustering is apples to oranges, but I will say that your case seems excessive...it all depends on what is in Merchant I guess. If you can take a snapshot using 2.6 and attach it here it might be interesting for us to look at.

To confirm: to record a snapshot in 2.6, go to Cluster statistics recorder and click Start Recording?

Yep! And then click stop when you are done - then export the session and you will get a zip file you can post here.

Attached is the file for review. The L2 is on .101 and the only L1 you need to look at is .88.

Darin

In the FWIW category, I added in a ConcurrentHashMap as a Spring bean

Code:

<bean id="merchantMap" class="java.util.concurrent.ConcurrentHashMap" />

and the load time was 14750ms and EHCache was 246136ms. I can't really do an apples to apples because EHCache is put one by one and I did a putAll for CHM:

Code:

 Map mx = new HashMap();
 for(final Merchant m: merchants) {
     mx.put(m.getId(), m);
 }
 merchantMap.putAll(mx);

I'm going to swap out a couple more items to see how it performs in context of the rest of the application.

I see from your tc-config that you have auto-lock-details enabled with full-stacks. Can you check your client log to see if it contains bunches of stacktraces? Or just disable auto-lock-details and full-stack to see if there's any improvement.

Can you let me know about the number of threads your application is using to put the Merchant entries into your cache.

Thanks,

Just one. Here's the 2 sections of code I use for the put for 2 variations for EHCache and ConcurrentHashMap:
Code:

 merchantCache.removeAll();
 merchantCache.clearStatistics();
 for(final Merchant m: merchants) {
   Element element = new Element(m.getId(),m);
   merchantCache.put(element);
 }

Code:

 Map mx = new HashMap();
 for(final Merchant m: merchants) {
   mx.put(m.getId(), m);
 }
 merchantCacheMap.putAll(mx);

I did also try a Executors.newFixedThreadPool(25), but I didn't see much improvement.

Did you ever check into if the runtime logging you have enabled is affecting the performance? I noted previously that you have runtime logging enabled for autolocks with full stack traces. This means that anytime an autolock is taken out, debug info with a full stack trace will be emitted into the client log. That will certainly affect your performance.

I did check. The client logs did not have any stacktraces. I did turn off the logging, but there was no difference in performance.

Were you ever able to improve the performance of EhCache? Did you end up using the ConcurrentHashMap instead, or choosing an alternate approach?