Terracotta Discussion Forums

hi all,

I did a small performance test for terracotta sessions on tomcat
against tomcat clustering.

i found that tomcat clustering is having 20% overhead. and terracotta
overhead is way too much around 50%. Response time, throughput,
cpu utilization, jvm memory utilization was also higher than tomcat clustering.

Test case is simple and as follows
1. search for pet
2. add to cart
3. modify amount
4. buy pet
5. logout

This is exactly opposite of benchmark published on terracotta site
i suspect i am doing something wrong in configuration.
Attached is my tc-config.xml

I am using tomcat 5.0.28, terracotta 2.4.8 and using JPetstore as sample
application. Test client is having 120 parallel threads injecting load on
tomcat. on 2 cpu machine.
Terracotta is running second machine also 2 cpu.

Well, definitely something is not right.

I looked at your config file, you should remove this line:

Code:

 		<include>
           <class-expression>*..*</class-expression>
         </include>

That tells Terracotta to instrument the world, which can usually lead to performance problems. After removing it, you may get a few UnlockedSharedObjectExceptions, which you can fix by adding back specific include expressions for the offending class.

When the application runs without exceptions, you should try the test again.

Also, would you mind explaining in detail the architecture of the machines you are running the test on? The test on the website was run in the following manner:

1 node:
Terracotta: 1 Tomcat Server w/Terracotta Sessions on 1 machine. Terracotta Server on a separate machine (2 total)
Tomcat: 1 Tomcat Server with Tomcat Sessions (1 total)

2 nodes:
Terracotta: 2 Tomcat Server w/ Terracotta Sessions on 2 separate machines. 1 Terracotta Server on a separate machine (3 total)
Tomcat: 2 Tomcat Servers with Tomcat Sesssion on 2 separate machines. (2 total)

3 nodes:
Terracotta: 3 Tomcat Servers w/Terracotta Sessions on 3 separate machines. 1 Terracotta Server on a separate machine. (4 total)
Tomcat: 3 Tomcat Servers with Tomcat Sesssion on 3 separate machines. (3 total)

and so on...

I also just noticed this:

Code:

 <runtime-logging>
   <lock-debug>true</lock-debug>
   <wait-notify-debug>true</wait-notify-debug>
   <!-- <auto-lock-details>true</auto-lock-details> -->
 </runtime-logging>

which you should remove. Also you should double-check that you haven't enabled other additional logging in the Terracotta Server or in the Tomcat configuration.

hi,

thanks for this.. i could see noticable difference in performance
mainly transactions per second in admin console now touching
150.

Now i have to do the complete benchmark with more nodes also.

Once done i will post my number for verification.

Also i am considering impact of session size in the benchmark.

Once again thanks for the support.

I am really happy about this product.. demo is really excellent also
documentation is good. Within minutes i could run my own terracotta
enabled application.

regards,
Tushar

Glad to hear it :)

On a sidenote - 150 in the admin console still sounds far too low - I will let a field eng. step in and confirm, but I am fairly certain on reasonable hardware, that number should be in the thousands.

hi taylor,

yes load applied was less. tps i got was fairly close to the one
that i got running without terracotta.

I am using 2 cpu pentium 4 (around 2 Ghz) box.
What is the maximum tps expected?

I am still not finished with all test cases.
I am eager to see terracotta performance when session size
is changed.

regards,
Tushar

When you talk about large sessions are you talking about deep object graphs, Terracotta does quite well with large sessions - given the fine-grained change replication.

If by large you mean shallow object-graphs but content in the form of large Strings, then again the String compression feature within Terracotta should help you realize good perf/scale numbers.

Ofcourse, let us know what you find.
Cheers
Iyer.

Hi all, I'm new with terracotta, and i've stumbled upon a problem, or more-less some weird results similar to this topic. If I should make a new topic out of this let me know.
In breif: My university task was to compare Terracotta for Spring against Tomcat session clusteering (pooled mode).
I have created a simple webshop application for that purpose. Creating my own application was part of task too, if you wander why i didn't use JPetStore :-). Application overall is a Spring webapp, that uses Spring Webflow, Hibernate, and Acegi for security.
After finishing it, I've run tests with JMeter that simulate 200 concurent users with same starting points (empty DB, all unwanted processes on my machine stoppped, etc) on both clustering technologies.
Results are somewhat wrong.
For Tomcat session clustering I get the following: Throughput for 1 tomcat in cluster: 23.76/sec; for 2 tomcats: 21.46/sec; for 3 tomcats: 18.26/sec.
This is OK according to me because Tomcat session clustering doesn't provide linear scale-out.
For Terracota for Spring (WebFlow) clustering I get the following: Throughput for 1 tomcat: 16.3/sec; for 2 tomcats 11.5/sec; for 3 tomcats 9.33/sec.
This is obviously wrong because Throughput shouldn't decrease by adding more servers to the cluster, so one can conclude that linear scale-out doesn't work in my example.

I am using tomcat 5.5.15, terracotta 2.1.2 on a 1 CPU (3Ghz), machine with 1.5 GB RAM.
I know more RAM, or CPU power would be much more better, but i'm limited to this - a student computer ;-).

Note that i took several measurements and every time i got more-less same results.
I'm providing my tc-config.xml, so what am I doing wrong? :-)

Thanks in advance

If you can post your application, we may be able to analyze it in case your tc-config.xml doesn't provide some answers...

Actually, it sounds like you are running all the JVMs on that 1 3GHz computer. Is this true?

--Ari

First I would like to thank you guys for answering me in such short notice. Professional, not doubt about it.
About the Ari's question: Unfortunately, that's true. I'm running all of the app. servers and Terracotta server, Apache load balancer, and JMeter on the same machine - all JVM's are on one (my) machine. I know this is a worst case scenario, but I didn't have resources (more machines) to test on. But nevertheless, I should get linear scale-out even in such configuration, correct?
Btw, I don't think it would be a problem to provide you with the application, but probably in a day or so.
Thanks again.

If you want to test for linear scale, you should be testing with more than machine. Ideally you should be testing with one machine dedicated to each tomcat instance, terracotta server, jmeter and load balancer. If you can not do that, you should at least run terracotta server on a different machine.

When you benchmark clustering results, you will see the real comparisons when tomcat instances are on distributed nodes, otherwise results will be skewed and can not be trusted.

Hi

Thanks for all the answers. They will be my guide for future tests.
Why I do I get the feeling that I knew this will be the answer. :-) When I run those tests on more reasonable hardware I will post the results.
I'm sure they will be much, much better then previous results.

Regards

Hi everybody,

I also have some problem with Terracotta benchmark, because results don't look like as should be. I compare 2 systems: JBoss solution for clustering and Apache Tomcat with Terracotta. Run configuration is:
8 x PC with JBoss AS
vs
8 x PC with Apache Tomcat + 1 PC with Terracotta Server

+ and also in both approachs I use 1 PC with Apache as load-balancer (mod_jk) and 1 PC for Jmeter which I use to sent many requests.

I simulate 100 host which are sending request in loop and all test time is 5min.

I attach files:
servlet code I use (RepHSTer.java)
tc-config.xml
tables with result which I got. (.ods file - Open Office)

One thing is wrong...THROUGHPUT. Terracotta has approximately 90request/sek and JBoss almost 300r/sek.

What is wrong?

Just as a first observation it appears your test is doing 100% writes. Is that a realistic simulation of your application?