Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

After looking over some of the tomcat code, it's not immediately obvious why the context in your request is null. Two things we can try:

1) If possible, I'd like to see your tomcat config files (conf/server.xml, any context files inside your war, etc). I'm mostly wondering if some form of virtual host or something is messing up our code. If there is any sensitivity with those files (please either use the private message button or sanitize before posting)

2) If you're comfortable/able to hook up a debugger to your tomcat instance, I'd very interested to know why the request context is null. The call that should make it non-null I believe is on line 281 of org.apache.coyote.tomcat5.CoyoteAdapter.postParseRequest(), the line looks like:Code:

   request.setContext((Context) request.getMappingData().context);

http://svn.apache.org/repos/asf/tomcat/container/tags/tc5.0.x/TOMCAT_5_0_30/catalina/src/share/org/apache/coyote/tomcat5/CoyoteAdapter.java

I'll look into that Assertion error -- for some reason the context for request is null, not sure why that would be.

Do you know the exact version of tomcat you are running? If the version is unknown (or known to be older than 5.0.28), trying to reproduce this issue with 5.0.28+ would be helpful.

Not quite sure what is going on, but stepping through things in systematic way can usually shed some light....restart your browser, the terracotta server and your tomcat server(s). This should ensure you are stating with a clean slate.

You're saying that as soon as you login to the application, something on the page is indicating that you are not really logged in? The only way to trace that is to look into how the application makes that decision and trace it from there (again this where a clean slate can help)

It shouldn't matter, but how many tomcat servers are involved when you are running this test? Does using a single tomcat server make the problem go away?

With regards to using apache in front of tomcat, thing should work, but not very efficiently (as least not out of the box). I'm assuming you're talking about mod_jk? If so, the problem is that our JSESSIONID values are not currently in a format that mod_jk can use to make sticky routing decisions.

Some more info (and a workaround) for this issue:
http://forums.terracotta.org/forums/posts/list/202.page
http://jira.terracotta.org/jira/browse/CDV-233

p.s. The posts where you mention the boot jar classes that you needed to add -- was that just to let us know, or is there a question in there?

I can't think of why there would be a performance issue with this, but maybe I'm missing something. Is there something in particular that prompted that question?

I opened http://jira.terracotta.org/jira/browse/CDV-260 to track this issue

Unfortunately I don't know how to setup the multicast version of active/passive for the terracotta server (which now that I read the subject of the post, it sounds like that is what you want to run...)

In the off chance that you want to use the active/passive setup w/o multicast, you'd need your two servers to share a common reliable disk (NFS, etc). I'm assuming your two servers are on different hosts, and as such the <server><data> locations will be local and independent on each machine.

I asked someone familiar with the new multicast stuff to jump in here, but for good measure, please confirm that is what you're after. thanks

I'm not the authority on this subject, but it looks like the only control on the creation of the mbean server in the terracotta code happens in L1Management.shouldCreateMBeanServer() (at the bottom of this link): http://svn.terracotta.org/fisheye/browse/~raw,r=1164/Terracotta/dso/trunk/code/base/dso-l1/src/com/tc/management/L1Management.java

Given that code, I think the only way to force us to not create a server would be to run on 1.5+ JDK and to not define the "com.sun.management.jmxremote" system property. Those might not be acceptable constraints, but it looks like all one could do without changing the terracotta code (which if you want to do we can help you through)

I suspect there is a bug I should create for this, but I'd like to look at the tomcat code before opening it. Can you point me to the tomcat code that assumes the first mbean server MBeanServerFacotry.findMBeanServer() is the platform default?

If indeed what you're trying to share are the HttpSession objects for your web application, the other thing I can think of is that the <web-applications> section of your tc-config.xml is not correct.

The value you want to use for <web-application> is the context root of your application. From the config file attached, your application would need to reside at http://<your_web_server>/site/* (note: the "site" in that URL)

If your application resides in the root context (ie. http://<your_web_server>/*), use the value "ROOT" in tc-config.xml

hope this helps

I wonder if maybe you're having problems because of Xgl/Compiz/Looking Glass/Beryl. I don't actually know much about these, I just remember problem reports of swing apps on linux with this fancy X11 stuff.

If possible, a thread dump of the hung process might be useful. To that you can try a "killall -3 java" or "kill -3 <process id>". It would be best to attach the thread dump as a file (as opposed to directly in the post).

Looks like our code is getting confused by the version information in your tomcat installation. At least back in our 2.2.1 release, it explicitly wants to find a version number that starts with "5.0." (note the trailing dot).

The 2.3 release (and later) don't seem to be so restrictive.

I second the question on the need to modify ManagerUtil, very curious :-)

I'm guessing here, but if you're putting a modified ManagerUtil class anywhere besides the dso boot jar, a NoSuchMethodError is bound to happen. Updated the copy in tc.jar, or trying add to your own in CLASSPATH or something will not override the one in the boot jar.

Out of curiosity, what is prompting you to interrupt threads that are in terracotta code? We definitely have code to clean up in this area, but in the mean time maybe we can figure out a different approach

Good deal. I think we collectively know and agree on what the bug is here. Steve already posted the link to the bug. I've confirmed the issue in a test case here.

There is a race condition with that custom ObjectOutputStream, but you're not likely to observe it. Even though you're using reflection to read/resolve all of the fields of the objects being serialized, there is still the chance that the DSO memory manager could page out those references before they were committed into the serialization stream. Our fix for this problem will obviously need to make sure this race is dealt with.

Sounds like you've found a bug, and your analysis seems sound. I'm going to look into further.

I don't know the actual mechanics of how java serialization retrieves field values during the serialization process, but it is almost certainly not through bytecode which DSO has had a chance to enhance (thus it can see unresolved reference fields).

As a short term workaround, I would think if you did readObject() and writeObject() manually, then the field access in those methods would trigger DSO to resolve references that may have been paged out. I mostly suggest it to help give us more data about the problem, not as a "solution".

We've have similar problems with the native implementation of Object.clone().

I'll report back when I have more info, and more than likely a link to the bug report

Sounds good. I've recently come to discover that if one is using mod_jk in apache and Terracotta sessions, you won't have server affinity any more (ie. no sticky session). Although sticky sessions are not required for Terracotta sessions, it is very a good idea to keep locality of reference in check.

Given that requests are potentially jumping between servers, if you are using transients in your shared sessions objects, then an NPE could be the result of not properly initializing the values of the transient fields. If you're not using the <transient-fields> section or <honor-transient> element in your tc-config.xml, then this does not apply. The section named "Transience in Terracotta" in this doc has more information about transients:

http://www.terracotta.org/confluence/display/docs1/Concept+and+Architecture+Guide

So I fired up WLS9.2 + Terracotta 2.2.1 on win32 w/o any issues. I didn't try to do any object sharing, but the server started okay at least

The InvalidClassException is almost certainly the result of some old version of the XML namespace classes sitting around somewhere.

Can you maybe try adding this to your startup script

Code:

-XX:+TraceClassLoading

That should produce a lot of output -- you're looking to see which jar(s) javax.xml.namespace.QName is loaded from. I think it might be possible to see a line for that class name more than once if more than one classloader loads it.

Here's a dev2dev link with a similar issue:
http://forums.bea.com/bea/message.jspa?messageID=600036067&tstart=0