[Logo] Terracotta Discussion Forums
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Remoting DMI  XML
Forum Index -> General
Author Message
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

My clustered app will be a client of an external service (via a socket connection) where only ONE node in the cluster will maintain the connection. All the other nodes will need to utilize the external service via the node connected to it.

Initially I thought I'd expose the service via a TC managed queue, but that proved too complicated. Now I'm thinking I should expose the service to the other nodes via Jini. And if I'm going to use Jini anyway, maybe I should use JavaSpaces for my data grid.

Could I use DMI for this? It seems like DMI is a good way to run methods on other nodes, but there doesn't seem to be a way to run a method on a specific node and get back its return value.

Any advice?
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

Well, please don't take this as an "official" recommendation, but you could do something like:

Code:
 public class MyAPIObject
 {
   public MyAPIObject = new MyAPIObject();
 
   private transient boolean connected = false;
 
   private MyAPIObject() { }
 
   public synchronized connect()
   {
      connected = true;
      // establish and maintain the connection - assume you stay in this method
   }
 
   public synchronized MyReturnValueFoo remoteFoo()
   {
        if (!connected) { return null; }
 
        // do foo
   }
 
   public synchronized MyReturnValueBar remoteBar()
   {
       if (!connected) { return null; }
 
       // do bar
   }
 }
 


Where remoteFoo and remoteBar are the methods to be remoted that would fire via DMI. You would need to ensure that they have read locks, not write locks.

The strategy would be for every node to start a new thread and call "connect" in that thread on the singleton object - this would ensure that should one node lose the connection or die, another will then be allowed into the synchronized method and become the new "connected" node.

When a node connects, it sets the transient boolean connected flag for true. If this flag is false, the methods that fire on a node where the flag is false will just short-circuit. On the node that is connected, these methods will then execute.

The appropriate config for this would be something like:
Code:
 <instrumented-classes>
   <include>
     <class-expression>MyAPIObject</class-expression>
     <honor-transient>true</honor-transient>
   </include>
  </instrumented-classes>
  <locks>
     <autolock>
       <method-expression>* MyAPIObject.connect(..)<method-expression>
       <lock-level>write</lock-level>
     </autolock>
     <autolock>
       <method-expression>* MyAPIObject.remote*(..)</method-expression>
       <lock-level>read</lock-level>
     </autolock>
  </locks>
   <distributed-methods>
     <method-expression>* MyAPIObject.remote*(..)</method-expression>
   </distributed-methods>
 


I think you have presented an interesting use case - I think this DMI solution may solve it - but it seems that maybe some additional features could be added to the product.

I think the use of a DynamicProxy may also help remove the boilerplate code and turn it in to a generic solution. Hung blogged about DynamicProxy a little while ago: http://unserializableone.blogspot.com/

[WWW]
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

Thanks again for the quick reply. You guys are great about that on this forum.

So, if I invoke a DMI method on node A and it is only executed on one other node B, then A will get the return value returned by the invocation on B? What TC magic makes that happen?

Also, how can a method with a non-void return type call "return;"?

Thanks again
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

Hmmm...my idea may be entirely incorrect. Let me get back to you.
[WWW]
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

Ok, so I think your original idea was correct - using a queue (actually, two queues) is the right way.

I wrote a proxy to wrap it all up. Here's the class:

Code:
 import java.util.concurrent.*;
 import java.lang.reflect.*;
 
 public class RemoteInvoker implements InvocationHandler, Runnable
 {
     private BlockingQueue<MethodArguments> callMethod = new LinkedBlockingQueue<MethodArguments>();
     private BlockingQueue<MethodReturn> callReturn = new LinkedBlockingQueue<MethodReturn>();
 
     private final Object instance;
 
     public RemoteInvoker(Object instance)
     {
         this.instance = instance;
         start();
     }
 
     private static class MethodArguments
     {
         public final Object proxy;
         public final Method method;
         public final Object[] args;
 
         public MethodArguments(Object proxy, Method method, Object[] args)
         {
             this.proxy = proxy;
             this.method = method;
             this.args = args;
         }
     }
 
     private static class MethodReturn
     {
         public final Object object;
         public final Exception exception;
 
         public MethodReturn(Object object, Exception exception)
         {
             this.object = object;
             this.exception = exception;
         }
     }
 
     private void start()
     {
         Thread t = new Thread(this);
 //        t.setDaemon(true);
         t.start();
    }
 
     public void run()
     {
         synchronized (instance) {
             System.out.println("I am servicing requests...");
 
             MethodArguments arguments;
             while (true) {
                 try {
                     arguments = callMethod.take();
                     try {
                         callReturn.put(new MethodReturn(arguments.method.invoke(instance, arguments.args), null));
                     } catch (Exception e) {
                         callReturn.put(new MethodReturn(null, e));
                     }
                 } catch (InterruptedException e) {
                     // do nothing
                 }
             }
         }
     }
 
     public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
     {
         callMethod.put(new MethodArguments(proxy, method, args));
         MethodReturn ret = callReturn.take();
 
         if (ret.exception != null) {
             throw ret.exception;
         }
 
         return ret.object;
     }
 }
 


A test interface:

Code:
 public interface TestInterface
 {
     public int count(int count);
 }
 


A test file:

Code:
 import java.lang.reflect.*;
 
 public class Test implements TestInterface
 {
     int counter = 0;
 
     public int count(int count)
     {
         System.out.println("Incrementing counter by: " + count);
 
         counter += count;
         return counter;
     }
 
     public static void main(String[] args)
     {
         Test t = new Test();
         RemoteInvoker invoker = new RemoteInvoker(t);
         TestInterface proxy = (TestInterface) Proxy.newProxyInstance(t.getClass().getClassLoader(), new Class[] { TestInterface.class } , invoker);
         System.out.println("Proxy returns: " + proxy.count(3));
     }
 }
 


And the config:

Code:
 <tc:tc-config xmlns:tc="http://www.terracotta.org/config"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">
 
   <application>
     <dso>
       <instrumented-classes>
         <include>
           <class-expression>RemoteInvoker$*</class-expression>
         </include>
         <include>
           <class-expression>Test</class-expression>
         </include>
       </instrumented-classes>
       <locks>
         <autolock>
            <method-expression>* RemoteInvoker.run(..)</method-expression>
         </autolock>
       </locks>
       <roots>
         <root>
           <field-name>RemoteInvoker.callMethod</field-name>
         </root>
         <root>
           <field-name>RemoteInvoker.callReturn</field-name>
         </root>
         <root>
           <field-name>RemoteInvoker.instance</field-name>
         </root>
       </roots>
     </dso>
   </application>
 </tc:tc-config>
 


Running this is as simple as:

Code:
 $ start-tc-server.sh &
 $ dso-java.sh Test
 I am servicing requests...
 Incrementing counter by: 3
 Proxy returns: 3
 


After starting one, start another:
Code:
 $ dso-java.sh Test
 Proxy returns: 6
 


Kill it, start another. Rinse, repeat, watch the counter increment. Then kill the first one and watch as this most recent VM takes over the role of servicing requests. Notice that the state of Test is preserved, since it was passed in to RemoteInvoker which uses the "instance" variable as the persistent state and this is marked as a root in Terracotta.

Normally, you would want the service threads to be daemons, but for the purposes of demonstration, I left them as non-daemon threads because the test code doesn't do anything so the VM would exit since it has nothing to do.

[WWW]
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

I'm going to start a project to put this source code into. There's still a bug - it's in the return queue. But it will work for two JVMs :)
[WWW]
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

Nice. That's a lot better that what I was writing.

I wonder about support for concurrent processing of remote invocations. Perhaps you could have an ExecutorService with a configurable number of threads processing the callMethod queue. I think there would be problems with the order results would be posted to the callReturn queue, though. Perhaps the MethodArguments class could have a "result" property that could be "wait"ed and then "notify"ed on.
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

Tweaked it a little to get concurrent processing of invocations. Didn't test it under TC yet. Please let me know if I screwed anything up.

Code:
package flintstone.station.enginemanager.cluster.impl;
 
 import java.lang.reflect.InvocationHandler;
 import java.lang.reflect.Method;
 import java.util.concurrent.BlockingQueue;
 import java.util.concurrent.LinkedBlockingQueue;
 import java.util.logging.Logger;
 
 public class RemoteInvoker implements InvocationHandler
 {
     static class MethodInvocation
     {
         public final Object proxy;
         public final Method method;
         public final Object[] args;
         public MethodReturn result;
 
         public MethodInvocation(Object proxy, Method method, Object[] args)
         {
             this.proxy = proxy;
             this.method = method;
             this.args = args;
         }
     }
 
     private static class MethodReturn
     {
         public final Object object;
         public final Exception exception;
 
         public MethodReturn(Object object, Exception exception)
         {
             this.object = object;
             this.exception = exception;
         }
     }
 
     static private final transient Logger logger = Logger.getLogger(QueueProcessorRunnable.class
         .getName());
     
     class QueueProcessorRunnable implements Runnable {
         public void run()
         {
             MethodInvocation invocation;
             while (true)
             {
                 try
                 {
                     invocation = callMethodQueue.take();
                     try
                     {
                         invocation.result = new MethodReturn(invocation.method.invoke(
                             instance,
                             invocation.args), null);
                     }
                     catch (Exception e)
                     {
                         invocation.result = new MethodReturn(null, e);
                     }
                     synchronized (invocation)
                     {
                         invocation.notify();
                     }
                 }
                 catch (InterruptedException e)
                 {
                     Thread.currentThread().interrupt();
                     return;
                 }
             }
         }
     }
 
     private static final int NUM_THREADS = 3;
     
     BlockingQueue<MethodInvocation> callMethodQueue = 
         new LinkedBlockingQueue<MethodInvocation>();
 
     private final Object instance;
 
     public RemoteInvoker(Object instance)
     {
         this.instance = instance;
         start();
     }
     
     private void start()
     {
         Thread thread = new Thread(new Runnable() {
         
             @Override
             public void run()
             {
                 synchronized (instance)
                 {
                     logger.info("Servicing requests...");
                     for (int i = 0; i < NUM_THREADS; i++)
                     {
                         Thread thread = new Thread(new QueueProcessorRunnable(),
                             getClass().getName()+"-queueProcessor-"+i);
                         thread.setDaemon(true);
                         thread.start();
                     }
                     Object lock = new Object();
                     synchronized (lock)
                     {
                         try
                         {
                             lock.wait();//wait forever
                         }
                         catch (InterruptedException e)
                         {
                             Thread.currentThread().interrupt();
                         }
                     }
                     logger.severe("dying");
                 }
             }
         
         });
         thread.setDaemon(true);
         thread.start();
     }
 
     public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
     {
         MethodInvocation methodInvocation = new MethodInvocation(proxy, method, args);
         callMethodQueue.put(methodInvocation);
         MethodReturn ret = null;
         synchronized (methodInvocation)
         {
             while (null == (ret = methodInvocation.result))
                 methodInvocation.wait();
         }
 
         if (ret.exception != null)
         {
             throw ret.exception;
         }
 
         return ret.object;
     }
 }


And the unit test:

Code:
package flintstone.station.enginemanager.cluster.impl;
 
 import java.lang.reflect.Proxy;
 import java.util.UUID;
 import java.util.concurrent.BlockingQueue;
 import java.util.concurrent.LinkedBlockingQueue;
 
 import net.sourceforge.groboutils.junit.v1.MultiThreadedTestRunner;
 import net.sourceforge.groboutils.junit.v1.TestRunnable;
 
 import org.apache.commons.lang.time.StopWatch;
 import org.junit.Before;
 import org.junit.Test;
 
 import static org.junit.Assert.*;
 
 import flintstone.station.enginemanager.cluster.impl.RemoteInvoker.MethodInvocation;
 
 public class RemoteInvokerTest
 {
     private TestImpl test;
     private BlockingQueue<MethodInvocation> queue;
 
     static interface TestInterface
     {
         public String echo(String in);
     }
 
     static class TestImpl implements TestInterface
     {
         volatile public int called = 0;
 
         @Override
         public String echo(String in)
         {
             try
             {
                 Thread.sleep(100);
             }
             catch (InterruptedException e)
             {
                 throw new RuntimeException(e);
             }
             called++;
             return in;
         }
     }
 
     @Before
     public void setUp() throws Exception
     {
         test = new TestImpl();
         queue = new LinkedBlockingQueue<MethodInvocation>();
     }
 
     @Test//(timeout = 30 * 1000)
     public void testGeneral() throws Throwable
     {
         StopWatch stopWatch = new StopWatch();
         class MyTestRunnable extends TestRunnable
         {
             public void runTest() throws Throwable
             {
                 RemoteInvoker invoker = new RemoteInvoker(test);
                 invoker.callMethodQueue = queue;
                 TestInterface proxy = (TestInterface) Proxy.newProxyInstance(
                     test.getClass().getClassLoader(), 
                     new Class[] { TestInterface.class }, invoker);
                 String randomString = UUID.randomUUID().toString();
                 assertEquals(randomString, proxy.echo(randomString));
             }
         }
         int numThreads = 100;
         TestRunnable[] runners = new TestRunnable[numThreads];
         for (int i = 0; i < runners.length; i++)
         {
             runners[i] = new MyTestRunnable();
         }
         MultiThreadedTestRunner mttr = new MultiThreadedTestRunner(runners);
         stopWatch.start();
         mttr.runTestRunnables();
         System.out.println("time="+stopWatch.getTime());
         assertEquals(numThreads, test.called);
     }
 
 }


Dependencies on GroboUtils, Apache Commons Lang.
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

Yes you have the idea - put the wait lock for the result into the original arguments passed in by the caller and the wait/notify should work across JVMs - that was what I was going to do to eliminate the return queue and the problem it has with ordering results.


In other words, I would write in the MethodInvocation class:
Code:
 public synchronized MethodReturn getResult() throws InterruptedException
 {
      while (result == null) { wait(); }
      return result;
 }
 
 public synchronized setResult(MethodReturn result)
 {
     this.result = result;
     notify();
 }
 
[WWW]
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

Here's a first crack at a Spring exporter and ProxyFactoryBean for the pattern. I used the term QMI (Queue Method Invocation) because I thought it was cute. The wait/notify change you mentioned is included.

Code:
package flintstone.station.enginemanager.cluster.impl;
 
 import java.lang.reflect.Method;
 import java.util.concurrent.BlockingQueue;
 import java.util.logging.Logger;
 
 import org.apache.commons.lang.Validate;
 import org.springframework.beans.factory.InitializingBean;
 import org.springframework.remoting.support.RemoteExporter;
 
 public class QmiServiceExporter extends RemoteExporter
     implements InitializingBean
 {
     static private final transient Logger logger = Logger.getLogger(QmiServiceExporter.class
         .getName());
     
     public static class QmiInvocation
     {
         public final Method method;
         public final Object[] args;
         public QmiReturn result = new QmiReturn();
 
         public QmiInvocation(Method method, Object[] args)
         {
             this.method = method;
             this.args = args;
         }
     }
 
     public static class QmiReturn
     {
         public Object result;
 
         public Object getResult() throws InterruptedException
         {
             synchronized (this)
             {
                 while (result == null) { wait(); }
             }
             return result;
         }
 
         public void setResult(Object result)
         {
             synchronized (this) {
                 this.result = result;
                 notify();
             }
         }
     }
 
     class QueueProcessorRunnable implements Runnable {
         public void run()
         {
             QmiInvocation invocation;
             while (true)
             {
                 try
                 {
                     invocation = methodQueue.take();
                     try
                     {
                         invocation.result.setResult(
                             invocation.method.invoke(
                             getService(),
                             invocation.args));
                     }
                     catch (Exception e)
                     {
                         invocation.result.setResult(e);
                     }
                     synchronized (invocation)
                     {
                         invocation.notify();
                     }
                 }
                 catch (InterruptedException e)
                 {
                     Thread.currentThread().interrupt();
                     return;
                 }
             }
         }
     }
 
     private BlockingQueue<QmiInvocation> methodQueue; 
     private int numProcessingThreads = 3;
 
     public void setNumProcessingThreads(int numProcessingThreads)
     {
         this.numProcessingThreads = numProcessingThreads;
     }
 
     @Override
     public void afterPropertiesSet() throws Exception
     {
         Validate.notNull(getService());
         Validate.notNull(methodQueue);
         start();
     }
 
     public void setMethodQueue(BlockingQueue<QmiInvocation> methodQueue)
     {
         this.methodQueue = methodQueue;
     }
 
     private void start()
     {
         Thread thread = new Thread(new Runnable() {
         
             @Override
             public void run()
             {
                 synchronized (getService())
                 {
                     logger.info("Servicing requests for "+getService());
                     for (int i = 0; i < numProcessingThreads; i++)
                     {
                         Thread thread = new Thread(new QueueProcessorRunnable(),
                             getClass().getName()+"-queueProcessor-"+i);
                         thread.setDaemon(true);
                         thread.start();
                     }
                     Object lock = new Object();
                     synchronized (lock)
                     {
                         try
                         {
                             lock.wait();//wait forever
                         }
                         catch (InterruptedException e)
                         {
                             Thread.currentThread().interrupt();
                         }
                     }
                     logger.severe("dying");
                 }
             }
         
         });
         thread.setDaemon(true);
         thread.start();
     }
 }


Code:
package flintstone.station.enginemanager.cluster.impl;
 
 import java.util.concurrent.BlockingQueue;
 
 import org.aopalliance.intercept.MethodInterceptor;
 import org.aopalliance.intercept.MethodInvocation;
 import org.springframework.aop.framework.ProxyFactory;
 import org.springframework.beans.factory.BeanClassLoaderAware;
 import org.springframework.beans.factory.FactoryBean;
 import org.springframework.remoting.support.RemoteAccessor;
 import org.springframework.util.ClassUtils;
 
 import flintstone.station.enginemanager.cluster.impl.QmiServiceExporter.QmiInvocation;
 
 public class QmiProxyFactoryBean extends RemoteAccessor
     implements FactoryBean, BeanClassLoaderAware, MethodInterceptor
 {
     private BlockingQueue<QmiInvocation> methodQueue; 
 
     private ClassLoader beanClassLoader = ClassUtils.getDefaultClassLoader();
 
     private Object serviceProxy;
 
     public void setBeanClassLoader(ClassLoader classLoader) {
         this.beanClassLoader = classLoader;
     }
 
     public void setMethodQueue(BlockingQueue<QmiInvocation> methodQueue)
     {
         this.methodQueue = methodQueue;
     }
 
     public void afterPropertiesSet() {
         if (getServiceInterface() == null) {
             throw new IllegalArgumentException("Property 'serviceInterface' is required");
         }
         if (methodQueue == null) {
             throw new IllegalArgumentException("Property 'methodQueue' is " +
             		"required");
         }
         this.serviceProxy = new ProxyFactory(getServiceInterface(), this)
             .getProxy(this.beanClassLoader);
     }
 
 
     public Object getObject() {
         return this.serviceProxy;
     }
 
     public Class<?> getObjectType() {
         return getServiceInterface();
     }
 
     public boolean isSingleton() {
         return true;
     }
 
     @Override
     public Object invoke(MethodInvocation invocation) throws Throwable
     {
         QmiInvocation qmiInvocation = new QmiInvocation(
             invocation.getMethod(), 
             invocation.getArguments());
         methodQueue.put(qmiInvocation);
         Object ret = qmiInvocation.result.getResult();
 
         if (ret instanceof Throwable)
         {
             throw (Throwable)ret;
         }
 
         return ret;
     }    
 }


Unit test:

Code:
package flintstone.station.enginemanager.cluster.impl;
 
 import java.util.ArrayList;
 import java.util.List;
 import java.util.UUID;
 import java.util.concurrent.BlockingQueue;
 import java.util.concurrent.LinkedBlockingQueue;
 
 import net.sourceforge.groboutils.junit.v1.MultiThreadedTestRunner;
 import net.sourceforge.groboutils.junit.v1.TestRunnable;
 
 import org.apache.commons.lang.math.RandomUtils;
 import org.apache.commons.lang.time.StopWatch;
 import org.junit.Before;
 import org.junit.Test;
 
 import flintstone.station.enginemanager.cluster.impl.QmiServiceExporter.QmiInvocation;
 import static org.junit.Assert.*;
 
 public class QmiTest
 {
     private TestImpl test;
     private TestImpl2 test2;
     private BlockingQueue<QmiInvocation> methodQueue1;
     private BlockingQueue<QmiInvocation> methodQueue2;
 
     static interface TestInterface
     {
         public String echo(String in);
     }
 
     static class TestImpl implements TestInterface
     {
         volatile public int called = 0;
 
         @Override
         public String echo(String in)
         {
             try
             {
                 Thread.sleep(100);
             }
             catch (InterruptedException e)
             {
                 throw new RuntimeException(e);
             }
             called++;
             return in;
         }
     }
     
     static interface TestInterface2
     {
         public int echo(int in);
     }
     
     static class TestImpl2 implements TestInterface2
     {
         volatile public int called = 0;
         
         @Override
         public int echo(int in)
         {
             try
             {
                 Thread.sleep(100);
             }
             catch (InterruptedException e)
             {
                 throw new RuntimeException(e);
             }
             called++;
             return in;
         }
     }
 
     @Before
     public void setUp() throws Exception
     {
         test = new TestImpl();
         test2 = new TestImpl2();
         methodQueue1 = new LinkedBlockingQueue<QmiInvocation>();
         methodQueue2 = new LinkedBlockingQueue<QmiInvocation>();
     }
 
     @Test//(timeout = 30 * 1000)
     public void testGeneral() throws Throwable
     {
         StopWatch stopWatch = new StopWatch();
         class MyTestRunnable extends TestRunnable
         {
             public void runTest() throws Throwable
             {
                 QmiServiceExporter exporter = new QmiServiceExporter();
                 exporter.setService(test);
                 exporter.setServiceInterface(TestInterface.class);
                 exporter.setMethodQueue(methodQueue1);
                 exporter.setNumProcessingThreads(10);
                 exporter.afterPropertiesSet();
                 
                 QmiProxyFactoryBean proxyFactory = new QmiProxyFactoryBean();
                 proxyFactory.setServiceInterface(TestInterface.class);
                 proxyFactory.setMethodQueue(methodQueue1);
                 proxyFactory.afterPropertiesSet();
                 TestInterface proxy = (TestInterface)proxyFactory.getObject();
 
                 String randomString = UUID.randomUUID().toString();
                 assertEquals(randomString, proxy.echo(randomString));
             }
         }
         class MyTestRunnable2 extends TestRunnable
         {
             public void runTest() throws Throwable
             {
                 QmiServiceExporter exporter = new QmiServiceExporter();
                 exporter.setService(test2);
                 exporter.setServiceInterface(TestInterface2.class);
                 exporter.setMethodQueue(methodQueue2);
                 exporter.setNumProcessingThreads(10);
                 exporter.afterPropertiesSet();
                 
                 QmiProxyFactoryBean proxyFactory = new QmiProxyFactoryBean();
                 proxyFactory.setServiceInterface(TestInterface2.class);
                 proxyFactory.setMethodQueue(methodQueue2);
                 proxyFactory.afterPropertiesSet();
                 TestInterface2 proxy = (TestInterface2)proxyFactory.getObject();
                 
                 int randomInt = RandomUtils.nextInt();
                 assertEquals(randomInt, proxy.echo(randomInt));
             }
         }
         int numThreads = 100;
         List<TestRunnable> runners = new ArrayList<TestRunnable>();
         for (int i = 0; i < numThreads; i++)
         {
             runners.add(new MyTestRunnable());
             runners.add(new MyTestRunnable2());
         }
         MultiThreadedTestRunner mttr = new MultiThreadedTestRunner(
             runners.toArray(new TestRunnable[0]));
         stopWatch.start();
         mttr.runTestRunnables();
         System.out.println("time="+stopWatch.getTime());
         assertEquals(numThreads, test.called);
         assertEquals(numThreads, test2.called);
     }
 
 }

mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

I'm trying to get this working with TC4Spring. I created a bean definition for the method queue:

Code:
    <bean id="imServiceMethodQueue" class="java.util.concurrent.LinkedBlockingQueue"/>
 


and added it to my tc-config.xml. I had to make some other changes to tc-config.xml to get around locking and portability problems.

Code:
<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">
 
     <application>
         <!-- See the Terracotta Spring Guide for more details of the the <spring/> 
             section of the  tc-config file -->
         <spring>
             <jee-application name="*">
                 <instrumented-classes>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiInvocation</class-expression>
                     </include>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn</class-expression>
                     </include>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$MainRunnable</class-expression>
                     </include>
                 </instrumented-classes>
 
                 <locks>
                     <autolock>
                         <method-expression>void ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn.setResult(..)</method-expression>
                     </autolock>
                     <autolock>
                         <method-expression>* ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn.getResult(..)</method-expression>
                     </autolock>
                     <autolock>
                         <method-expression>void ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$MainRunnable.run(..)</method-expression>
                     </autolock>
                 </locks>
                 
                 <application-contexts>
                     <application-context>
                         <!-- Tell DSO the location of the bean definition file for this 
                             application context -->
                         <paths>
                             <path>flintstone/station/enginemanager/spring-runlevel1.xml</path>
                             <path>flintstone/station/enginemanager/spring-runlevel2.xml</path>
                             <path>flintstone/station/enginemanager/spring-runlevel3.xml</path>
                             <path>flintstone/station/enginemanager/applicationContext.xml</path>
                             <path>flintstone/station/enginemanager/spring-persistence.xml</path>
                             <path>flintstone/station/enginemanager/spring-security.xml</path>
                             <path>flintstone/station/enginemanager/spring-shipment-processing.xml</path>
                             <path>flintstone/station/enginemanager/spring-transactional-services.xml</path>
                         </paths>
                         <!-- Here we're telling DSO about the list of Spring beans that we
                             we wish to share -->
                         <beans>
                             <bean name="imServiceMethodQueue"/>
                         </beans>
                     </application-context>
                 </application-contexts>
             </jee-application>
         </spring>
 
         <dso>
             <additional-boot-jar-classes>
                 <include>java.net.InetAddress</include>
                 <include>java.net.Inet4Address</include>
             </additional-boot-jar-classes>
         </dso>
     </application>
 </tc:tc-config>


I'm having a problem where take() returns an object with null values for all its fields. Specifically, QmiServiceExporter$QueueProcessorRunnable.run calls methodQueue.take() that returns a QmiInvocation with null values for its method, args, and result fields. This should be impossible, because method and args are final, and I changed the constructor to check for null values.

hhuynh

cherubim

Joined: 06/16/2006 11:54:06
Messages: 761
Offline

Hello,

Is there any stack trace in the client log? If there is, could you post the whole stack trace here.

Incidentally, I notice you didn't specify the lock-level for the auto-locks. By default, lock-level is "write". For methods that only do reads, you get less contention with a "read" lock:

<autolock>
<method-expression>* ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn.getResult(..)</method-expression>
<lock-level>read</lock-level>
</autolock>
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

There is no stack trace in the client log:

Code:
2007-10-10 17:37:42,030 [main] INFO com.tc.properties.TCProperties - Loading default properties from tc.properties
 2007-10-10 17:37:42,030 [main] INFO com.tc.properties.TCProperties - Loaded TCProperties : TCProperties={l2.objectmanager.maxObjectsInTxnObjGrouping=5000, l1.cachemanager.logging.enabled=false, l2.berkeleydb.je.cleaner.minAge=5, ehcache.evictor.pool.size=1, session.debug.hops=false, l2.nha.tribes.mcast.mcastAddress=228.0.0.4, l1.objectmanager.remote.maxDNALRUSize=60, session.request.bench.enabled=false, l2.transactionmanager.logging.printStats=true, l2.nha.send.timeout.millis=16000, l2.nha.tribes.orderinterceptor.enabled=false, l1.connect.versionMatchCheck.enabled=true, l2.berkeleydb.je.lock.timeout=180000000, l2.nha.tribes.mcast.mcastFrequency=500, l2.transactionmanager.logging.verbose=false, l2.berkeleydb.je.maxMemoryPercent=25, l2.transactionmanager.logging.enabled=false, l1.transactionmanager.strings.compress.enabled=true, l2.beanshell.port=9929, l2.cachemanager.logging.enabled=false, l2.cachemanager.criticalThreshold=90, l1.cachemanager.criticalThreshold=90, ehcache.global.eviction.frequency=10, session.debug.hops.interval=100, l2.nha.tribes.mcast.tcpListenHost=127.0.0.1, l2.cachemanager.enabled=true, session.request.tracking.interval=2500, tc.management.test.mbeans.enabled=false, l1.transactionmanager.maxOutstandingBatchSize=8, l1.cachemanager.sleepInterval=3000, ehcache.global.eviction.rest.timeMillis=10, session.request.tracking.threshold=15000, l2.seda.commitstage.threads=4, session.invalidator.sleep=300, l1.cachemanager.enabled=true, l1.transactionmanager.logging.enabled=false, l1.transactionmanager.maxBatchSizeInKiloBytes=128, l2.berkeleydb.je.cleaner.bytesInterval=20000000, l2.lfu.agingFactor=1, l2.objectmanager.maxObjectsToCommit=5000, l2.cachemanager.leastCount=2, memory.monitor.forcebasic=false, l1.transactionmanager.maxSleepTimeBeforeHalt=1024, session.request.tracking=false, session.debug.invalidate=false, l2.cachemanager.threshold=70, l2.lfu.recentlyAccessedIgnorePercentage=20, l1.reconnect.timeout.millis=5000, l2.cachemanager.sleepInterval=3000, ehcache.global.eviction.segments=2, l2.beanshell.enabled=false, ehcache.concurrency=1, l1.jvm.check.compatibility=true, l2.berkeleydb.je.lock.nLockTables=7, l2.objectmanager.maxTxnsInTxnObjectGrouping=500, l2.nha.tribes.mcast.mcastPort=7777, l1.transactionmanager.maxPendingBatches=32, l1.reconnect.enabled=false, l2.berkeleydb.je.cleaner.lookAheadCacheSize=32768, ehcache.global.eviction.enable=true, l2.berkeleydb.je.checkpointer.bytesInterval=100000000, l2.nha.tribes.mcast.tcpListenPort=4000, l2.objectmanager.cachePolicy=lfu, l2.lfu.debug.enabled=false, l2.cachemanager.percentageToEvict=10, l1.cachemanager.percentageToEvict=10, l2.seda.flushstage.threads=4, tc.management.mbeans.enabled=true, l2.objectmanager.fault.logging.enabled=false, l2.berkeleydb.je.cleaner.detailMaxMemoryPercentage=5, l1.transactionmanager.strings.compress.minSize=512, session.request.tracking.dump=false, l2.nha.mcast.enabled=false, l2.seda.faultstage.threads=4, tc.stage.monitor.enabled=false, l2.cachemanager.monitorOldGenOnly=true, l1.cachemanager.monitorOldGenOnly=true, l2.objectmanager.deleteBatchSize=5000, l1.cachemanager.threshold=70, tc.stage.monitor.delay=5000, session.invalidator.bench.enabled=true, l1.transactionmanager.strings.compress.logging.enabled=false, l2.nha.tribes.mcast.memberDropTime=3000, l1.cachemanager.leastCount=2, l1.max.connect.retries=-1}
 2007-10-10 17:37:42,046 [main] INFO com.tc.management.beans.logging.TCLoggingBroadcasterMBean - No resource bundle exists for MBean com.tc.management.beans.logging.TCLoggingBroadcasterMBean
 2007-10-10 17:37:42,046 [main] INFO com.terracottatech.general - New logging session started.
 2007-10-10 17:37:42,077 [main] INFO com.terracottatech.console - Terracotta 2.4.3, as of 20070906-130949 (Revision 5455 by cruise@WXPMO0 from 2.4)
 2007-10-10 17:37:42,077 [main] INFO com.terracottatech.console - Terracotta 2.4.3, as of 20070906-130949 (Revision 5455 by cruise@WXPMO0 from 2.4)
 2007-10-10 17:37:42,077 [main] INFO com.tc.logging.TCLogging - Terracotta 2.4.3, as of 20070906-130949 (Revision 5455 by cruise@WXPMO0 from 2.4)
 2007-10-10 17:37:42,405 [main] INFO com.tc.config.schema.setup.StandardXMLFileConfigurationCreator - Attempting to load configuration from the file at 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\..\..\etc\tc-config.xml'...
 2007-10-10 17:37:42,405 [main] INFO com.tc.config.schema.setup.StandardXMLFileConfigurationCreator - Successfully loaded configuration from the file at 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\..\..\etc\tc-config.xml'. Config is:
 
 <?xml version="1.0" encoding="UTF-8"?>
 <tc:tc-config xmlns:tc="http://www.terracotta.org/config"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">
 
     <application>
         <!-- See the Terracotta Spring Guide for more details of the the <spring/> 
             section of the  tc-config file -->
         <spring>
             <jee-application name="*">
                 <instrumented-classes>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiInvocation</class-expression>
                     </include>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn</class-expression>
                     </include>
                     <include>
                         <class-expression>ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$MainRunnable</class-expression>
                     </include>
                 </instrumented-classes>
 
                 <locks>
                     <autolock>
                         <method-expression>void ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn.setResult(..)</method-expression>
                     </autolock>
                     <autolock>
                         <method-expression>* ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$QmiReturn.getResult(..)</method-expression>
                     </autolock>
                     <autolock>
                         <method-expression>void ca.digitalrapids.spring.remoting.qmi.QmiServiceExporter$MainRunnable.run(..)</method-expression>
                     </autolock>
                 </locks>
                 
                 <application-contexts>
                     <application-context>
                         <!-- Tell DSO the location of the bean definition file for this 
                             application context -->
                         <paths>
                             <path>flintstone/station/enginemanager/spring-runlevel1.xml</path>
                             <path>flintstone/station/enginemanager/spring-runlevel2.xml</path>
                             <path>flintstone/station/enginemanager/spring-runlevel3.xml</path>
                             <path>flintstone/station/enginemanager/applicationContext.xml</path>
                             <path>flintstone/station/enginemanager/spring-persistence.xml</path>
                             <path>flintstone/station/enginemanager/spring-security.xml</path>
                             <path>flintstone/station/enginemanager/spring-shipment-processing.xml</path>
                             <path>flintstone/station/enginemanager/spring-transactional-services.xml</path>
                         </paths>
                         <!-- Here we're telling DSO about the list of Spring beans that we
                             we wish to share -->
                         <beans>
 <!--                            <bean name="clusterManager"/>
 -->                            <bean name="imServiceMethodQueue"/>
                         </beans>
                     </application-context>
                 </application-contexts>
             </jee-application>
         </spring>
 
         <dso>
             <additional-boot-jar-classes>
                 <include>java.net.InetAddress</include>
                 <include>java.net.Inet4Address</include>
             </additional-boot-jar-classes>
         </dso>
     </application>
 </tc:tc-config>
 
 2007-10-10 17:37:42,671 [main] INFO com.terracottatech.console - Configuration loaded from the file at 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\..\..\etc\tc-config.xml'.
 2007-10-10 17:37:42,671 [main] INFO com.terracottatech.console - Configuration loaded from the file at 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\..\..\etc\tc-config.xml'.
 2007-10-10 17:37:42,796 [main] INFO com.terracottatech.console - Log file: 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\logs-192.168.0.147\terracotta-client.log'.
 2007-10-10 17:37:42,796 [main] INFO com.terracottatech.console - Log file: 'C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1\logs-192.168.0.147\terracotta-client.log'.
 2007-10-10 17:37:42,796 [main] INFO com.tc.logging.TCLogging - ========================================================================
 All Java System Properties for this Terracotta instance:
 awt.toolkit                  : sun.awt.windows.WToolkit
 file.encoding                : Cp1252
 file.encoding.pkg            : sun.io
 file.separator               : \
 java.awt.graphicsenv         : sun.awt.Win32GraphicsEnvironment
 java.awt.printerjob          : sun.awt.windows.WPrinterJob
 java.class.path              : C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\bin;C:\kpwong\svnworkarea\Flintstone\trunk\Engine\bin;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\bin;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\activation.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-beanutils.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-codec-1.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-collections-3.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-discovery-0.2.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-io-1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-jxpath-1.2.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-lang-2.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-pool-1.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\mail.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\nachocalendar-0.23.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\swing-layout-1.0.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\easymock.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\GroboUtils-5-core.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\junit-4.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\commons-logging-1.0.4.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\opencsv-1.7.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\lib\ejb3-persistence.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\easymockclassextension.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\jmockit.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\jmockit-asm2.jar;C:\kpwong\svnworkarea\Flintstone\trunk\WWJava\bin;C:\kpwong\svnworkarea\Flintstone\trunk\DRSpring\bin;C:\kpwong\svnworkarea\Flintstone\trunk\DRSpring\lib\spring.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRSpring\testlib\spring-mock.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRSpring\lib\servlet-api-2.5-6.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\cglib-2.1.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Helpers\testlib\asm.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\bin;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\smack.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\smackx.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\smackx-debug.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\testlib\xpp3-1.1.4c.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\ant-1.6.5.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\core-3.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\jetty-6.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\jsp-2.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\jsp-api-2.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\servlet-api-2.5-6.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\jetty\jetty-util-6.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\acegi\acegi-security-1.0.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\bin;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\testlib\hsqldb.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\antlr-2.7.6.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\asm.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\asm-attrs.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\c3p0-0.9.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\cglib-2.1.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\concurrent-1.3.2.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\connector.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\dom4j-1.6.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\ehcache-1.2.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\hibernate3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\hibernate-annotations.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\hibernate-commons-annotations.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jaas.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jacc-1_0-fr.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\javassist.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jboss-cache.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jboss-common.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jboss-jmx.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jboss-system.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jdbc2_0-stdext.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jgroups-2.2.8.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\jta.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\log4j-1.2.11.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\oscache-2.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\proxool-0.8.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\DRHibernate\lib\hibernate\swarmcache-1.0rc2.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\hsqldb.jar;C:\kpwong\svnworkarea\Flintstone\trunk\Common\lib\dozer-3.4.jar;C:\Program Files\PostgreSQL\8.1\jdbc\postgresql-8.1-405.jdbc3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseClientJ\bin;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\bin;C:\kpwong\svnworkarea\Flintstone\trunk\DRHaspJ\ebin;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\antlr-2.7.6.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\asm.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\asm-attrs.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\bcprov-jdk16-136.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\c3p0-0.9.0.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\castor-1.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\cglib-2.1.3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\commons-collections-2.1.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\commons-logging-1.0.4.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\dom4j-1.6.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\ejb3-persistence.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hibernate3.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hibernate-annotations.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hibernate-commons-annotations.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hibernate-tools.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hibernate-validator.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\hsqldb.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\jdom.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\jetty-6.1.0.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\jetty-util-6.1.0.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\JSAP-2.1.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\jta.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\log4j-1.2.13.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\servlet-api-2.5.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseSystem\lib\xercesImpl.jar;C:\kpwong\svnworkarea\Flintstone\trunk\LicenseClientJ\lib\castor-1.1.1.jar
 java.class.version           : 50.0
 java.endorsed.dirs           : C:\Program Files\Java\jre1.6.0_03\lib\endorsed
 java.ext.dirs                : C:\Program Files\Java\jre1.6.0_03\lib\ext;C:\WINDOWS\Sun\Java\lib\ext
 java.home                    : C:\Program Files\Java\jre1.6.0_03
 java.io.tmpdir               : C:\DOCUME~1\kevin\LOCALS~1\Temp\
 java.library.path            : C:\Program Files\Java\jre1.6.0_03\bin;.;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Java\jre1.6.0_03\bin\client;C:\Program Files\Java\jre1.6.0_03\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Subversion\bin;C:\Program Files\Java\apache-ant-1.7.0\bin;C:\Program Files\Java\jdk1.6.0_03\bin;C:\Program Files\ZipGenius 6\;C:\Program Files\raserver\bin;R:\ThirdParty\Java\GWT\gwt-windows-1.3.3;C:\Program Files\Common Files\GTK\2.0\bin;C:\Program Files\QuickTime\QTSystem\;C:\Sun\AppServer\bin
 java.runtime.name            : Java(TM) SE Runtime Environment
 java.runtime.version         : 1.6.0_03-b05
 java.specification.name      : Java Platform API Specification
 java.specification.vendor    : Sun Microsystems Inc.
 java.specification.version   : 1.6
 java.vendor                  : Sun Microsystems Inc.
 java.vendor.url              : http://java.sun.com/
 java.vendor.url.bug          : http://java.sun.com/cgi-bin/bugreport.cgi
 java.version                 : 1.6.0_03
 java.vm.info                 : mixed mode
 java.vm.name                 : Java HotSpot(TM) Client VM
 java.vm.specification.name   : Java Virtual Machine Specification
 java.vm.specification.vendor : Sun Microsystems Inc.
 java.vm.specification.version: 1.0
 java.vm.vendor               : Sun Microsystems Inc.
 java.vm.version              : 1.6.0_03-b05
 line.separator               : 
 
 os.arch                      : x86
 os.name                      : Windows XP
 os.version                   : 5.1
 path.separator               : ;
 sun.arch.data.model          : 32
 sun.boot.class.path          : C:\Program Files\Terracotta\terracotta-2.4.3\lib\dso-boot\dso-boot-hotspot_win32_160_03.jar;C:\Program Files\Java\jre1.6.0_03\lib\resources.jar;C:\Program Files\Java\jre1.6.0_03\lib\rt.jar;C:\Program Files\Java\jre1.6.0_03\lib\sunrsasign.jar;C:\Program Files\Java\jre1.6.0_03\lib\jsse.jar;C:\Program Files\Java\jre1.6.0_03\lib\jce.jar;C:\Program Files\Java\jre1.6.0_03\lib\charsets.jar;C:\Program Files\Java\jre1.6.0_03\classes
 sun.boot.library.path        : C:\Program Files\Java\jre1.6.0_03\bin
 sun.cpu.endian               : little
 sun.cpu.isalist              : 
 sun.desktop                  : windows
 sun.io.unicode.encoding      : UnicodeLittle
 sun.java.launcher            : SUN_STANDARD
 sun.jnu.encoding             : Cp1252
 sun.management.compiler      : HotSpot Client Compiler
 sun.os.patch.level           : Service Pack 2
 tc.config                    : ../../etc/tc-config.xml
 tc.install-root              : C:\Program Files\Terracotta\terracotta-2.4.3
 user.country                 : US
 user.dir                     : C:\kpwong\svnworkarea\Flintstone\trunk\EngineManager\private\em1
 user.home                    : C:\Documents and Settings\kevin
 user.language                : en
 user.name                    : kevin
 user.timezone                : America/New_York
 user.variant                 : 
 ========================================================================
 
 2007-10-10 17:37:44,343 [L1Management JMX registration] INFO com.tc.management.L1Management - Terracotta JMX connector available at[service:jmx:terracotta://localhost]
 2007-10-10 17:37:44,405 [main] INFO com.tc.net.protocol.transport.ClientMessageTransport - ConnectionID(-1.ffffffffffffffffffffffffffffffff): Attaching new connection: com.tc.net.core.TCConnectionJDK14@22193741: connected: true, closed: false local=127.0.0.1:4478 remote=127.0.0.1:9510 connect=[Wed Oct 10 17:37:44 EDT 2007] idle=31ms
 2007-10-10 17:37:44,421 [main] INFO com.tc.management.remote.protocol.terracotta.TunnelingEventHandler - Client JMX server ready; sending notification to L2 server
 2007-10-10 17:37:44,421 [WorkerThread(client_coordination_stage,0)] INFO com.tc.object.handshakemanager.ClientHandshakeManager - ChannelID=[-1]: Unpause State[ PAUSED ]
 2007-10-10 17:37:44,530 [main] INFO com.terracottatech.dso.instrumentation - Loading aspect module [com.tc.object.config.SpringAspectModule] in loader sun.misc.Launcher$AppClassLoader@4337374
 2007-10-10 17:37:50,186 [TC Memory Monitor] INFO com.tc.runtime.TCMemoryManagerImpl - Sleep time changed to : 545
 2007-10-10 17:37:52,983 [TC Memory Monitor] INFO com.tc.runtime.TCMemoryManagerImpl - Sleep time changed to : 272
 2007-10-10 17:37:56,187 [main] ERROR com.terracottatech.dso - Refusing to instrument CGLIB generated proxy type flintstone.station.enginemanager.data.entity.ShipmentTask$$EnhancerByCGLIB$$158018bc (CGLIB terracotta plugin not installed)
 2007-10-10 17:37:56,280 [main] ERROR com.terracottatech.dso - Refusing to instrument CGLIB generated proxy type flintstone.station.enginemanager.data.entity.WatchFolderConfig$$EnhancerByCGLIB$$a2410b10 (CGLIB terracotta plugin not installed)
 2007-10-10 17:37:56,296 [main] ERROR com.terracottatech.dso - Refusing to instrument CGLIB generated proxy type flintstone.station.enginemanager.data.entity.ShipmentEvent$$EnhancerByCGLIB$$1744beb1 (CGLIB terracotta plugin not installed)
 2007-10-10 17:37:56,312 [main] ERROR com.terracottatech.dso - Refusing to instrument CGLIB generated proxy type flintstone.station.enginemanager.data.entity.ShipmentInfo$$EnhancerByCGLIB$$f6da5b25 (CGLIB terracotta plugin not installed)
 2007-10-10 17:38:44,074 [DSO Lock Object.wait() timer] INFO com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[8]: Running Lock GC...
 2007-10-10 17:39:44,084 [DSO Lock Object.wait() timer] INFO com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[8]: Running Lock GC...
 2007-10-10 17:40:44,094 [DSO Lock Object.wait() timer] INFO com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[8]: Running Lock GC...
 2007-10-10 17:40:55,749 [CommonShutDownHook - com.tc.object.msg.RequestManagedObjectResponseMessage$1@319d4b] INFO com.tc.net.protocol.TCNetworkMessage - No of times Buffers wasted = 9 Buffers wasted count = 20
 2007-10-10 17:40:55,765 [Thread-0] INFO com.tc.object.tx.RemoteTransactionManagerImpl - ChannelID=[8]: stop(): took 0 millis to complete


I didn't set the lock-level to read for QmiReturn#getResult or MainRunnable#run because the nature of both of their implementations is such that they should wait on the lock regardless of the fact that they are just reading data. The locking in those methods is primarily for thread coordination.
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

I can think of one way this can happen, that isn't entirely obvious.

It's written up as a "Gotcha" here:

http://www.terracotta.org/confluence/display/docs1/Gotchas#Gotchas-Uninstrumentedaccesstosharedobjects

Basically, if you access a public final field in a DSO from a class that is not-instrumented those fields may not be resolved, resulting them in being null. Can you explain how you are determining that the fields are null - and if it is code that is getting the null can you ensure that the code is instrumented?

The reason I guess this is the problem is because I don't see any kind of lock on the QMIReturn class. If there are accessor methods for QMIReturn, you should make sure they are read-locked - though that is not what I would suspect would be the root cause (I suspect you are accessing from non-instrumented code)

[WWW]
mrpantsu

journeyman

Joined: 10/02/2007 12:27:11
Messages: 21
Offline

That gotcha was exactly it. QmiInvocation was instrumented but some code that accessed its fields directly was not. I changed the offending code to use accessor methods, which fixed it.

Much obliged!
 
Forum Index -> General
Go to:   
Powered by JForum 2.1.7 © JForum Team