Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

Hi all,

we're currently evaluating Terracotta to our server cluster. We've a master - server setup where Tasks are being pushed into a ThreadPoolExecutor backed by a LinkedBlockingQueue.
I want this Queue now to be distributed over all nodes via Terracotta. Terracotta and our jetty instances are starting up fine, but the queue is not instrumented.

Since this is my first steps with terracotta, I guess I'm just missing something somewhere.

I tried to configure via tim-annotations and tc-config.xml. Both doesn't work.

Here is my setup:
The "queue" is what I want to be distributed.
Code:

package com.company.package.impl;
 
 import java.util.List;
 
 public class UserSchedulerImpl implements UserScheduler {
 	static Logger log = LoggerFactory.getLogger(UserSchedulerImpl.class);
 
 	private ThreadPoolExecutorService threadPoolExecutorService;
 	
 	private UserPreProcessor userPreProcessor;
 
 	private TimingThreadPoolExecutor userThreadPoolExecutor;
 	
 	@Root
 	private LinkedBlockingQueue<Runnable> queue;

my tc-config.xml:

Code:

<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-5.xsd">
 
   <!-- Tell DSO where the Terracotta server can be found;
        See
        - Terracotta Configuration Guide and Reference
        - About Terracotta Configuration Files
        for additional information. -->
   <servers>
     <server host="%i" name="sample">
       <dso-port>9510</dso-port>
       <jmx-port>9520</jmx-port>
       <data>terracotta/demo-server/server-data</data>
       <logs>terracotta/demo-server/server-logs</logs>
       <statistics>terracotta/demo-server/server-statistics</statistics>
     </server>
   </servers>
   <clients>
      <!--logs>%(user.dir)/logs/terracotta</logs>
      <statistics>%(user.dir)/logs/terracotta/client-statistics/%D</statistics> -->
      <modules>
        <module name="tim-annotations" version="1.5.2" />
        <module name="tim-jetty-6.1" version="2.1.3" />
      </modules>
      <dso>
           <debugging>
                   <instrumentation-logging>
                       <class>true</class>
                       <locks>true</locks>
                       <transient-root>false</transient-root>
                       <roots>true</roots>
                       <distributed-methods>false</distributed-methods>
                   </instrumentation-logging>
                   <runtime-logging>
                       <non-portable-dump>true</non-portable-dump>
                       <lock-debug>true</lock-debug>
                       <wait-notify-debug>false</wait-notify-debug>
                       <distributed-method-debug>false</distributed-method-debug>
                       <new-object-debug>true</new-object-debug>
                   </runtime-logging>
                   <runtime-output-options>
                       <auto-lock-details>true</auto-lock-details>
                       <caller>false</caller>
                       <full-stack>false</full-stack>
                   </runtime-output-options>
           </debugging>
      </dso>
    </clients>
         <application>
 
                 <dso>
                   <web-applications>
                     <web-application>ROOT</web-application>
                   </web-applications>
                         <instrumented-classes>
                                 <include>
                                         <class-expression>com.company.package.imp.UserSchedulerImpl</class-expression>
                                 </include>
                         </instrumented-classes>
                         <roots>
                                 <root>
                                         <field-name>com.company.package.imp.UserSchedulerImpl.queue</field-name>
                                 </root>
                         </roots>
                 </dso>
         </application>
 </tc:tc-config>

Jetty process:
Code:

/opt/jdk/jdk-nowplus/bin/java -javaagent:/opt/sng/spring-instrument-3.0.0.RELEASE.jar
 -Xbootclasspath/p:/opt/terracotta/current/lib/dso-boot/dso-boot-hotspot_linux_160_18.jar
 -Dtc.install-root=/opt/terracotta/current -Dtc.config=/opt/terracotta/current/tc-config.xml
 -Djetty.home=/opt/jetty-6.1.18 -Djava.io.tmpdir=/tmp -jar /opt/jetty-6.1.18/start.jar
 /opt/jetty-6.1.18/etc/jetty-logging.xml /opt/jetty-6.1.18/etc/jetty.xml

Maybe there's something obvious I got wrong. I tried lots of stuff, but I'm somehow stuck here.

Thanks a lot in advance!
Thomas

I haven't looked to see why your LBQ is not clustered.

But 2 questions for you:

1. Did you know sharing an LBQ in Terracotta is one of the slowest performing things you can do? Why didn't you use TIM-Messaging at least? It abstracts away the queue and speeds things up in clusters larger than 1 or 2 JVMs

2. Have you tried to map your problem to the Terracotta Toolkit? The Toolkit gets rid of the need for bytecode manipulation and bootjars and stuff. Ideally you would map to Ehcache which is highly tuned and can do 100k+ tps to queues 100+ tps (orders of magnitude faster). But the toolkit might be worth a try too.

Is it possible for you to retool at this point? You will be a lot happier w/ the Toolkit or, better yet, Ehcache.

--Ari

Hey Ari,

thanks for your reply. To answer your questions:

1. No didn't know that terracotta is performing slow here. We already have this queue in place and this would have been the easiest way forward. However I will definetly have a look at TIM-Messaging now. Didn't know that at all.

2. Will have a look at the toolkit as well.

I'm new to terracotta and only have a basic idea of it's capabilities. We're currently using a distributed ehcache as well (w/o terracotta) and migrating the distributed cache to terracotta might be step two.

Basically we push userIds to the queue and each user task will take around 10-60s. So a slow performing queue shouldn't even be a bottleneck. However TIM-Messaging sounds interesting. Will have a look at it.

I'm still clueless why my queue is not distributed with the current configs.

Cheers,
Thomas