[Logo] Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Getting previously persisted session data on restart of server  XML
Forum Index -> Terracotta for Web Sessions Go to Page: 1, 2 Next 
Author Message
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi,
Just started on using Terracotta sessions on a clustered tomcat environment. Consider this scenario:
I have one load balancer that balances load between 2 of my tomcat servers. I have another server which runs the terracotta sessions server. Since both the tomcat machines refer to the tc server, their sessions are replicated. This happens just fine. I checked by shutting down one server which was serving and the other server took over pretty neatly. However my problem comes when the server which was shutdown starts up and joins the cluster again. At this point its session is blank and only the changes since its start up are replicated. But I would need the session of the restarted machine to get to the state it was before it shutdown. One of the web sites mentioned putting the
<mode>permanent-store</mode> in the server configuration which I did but to no avail. Could anyone please enlighten me on this. Thanks a lot.
Attached is my config file I used for both the server and the client tomcats.
 Filename tc-config-tomcat.xml [Disk] Download
 Description Server and client config file
 Filesize 3 Kbytes
 Downloaded:  459 time(s)

zeeiyer

consul

Joined: 05/24/2006 14:28:28
Messages: 493
Offline

hi Zeus

That sounds odd - even without persistence mode, just restarting one of your "client" Tomcats, should result in sessions getting faulted in from the Terracotta server - so it should behave exactly like how you'd like it to...In fact, you could shutdown your entire application server tier and resume with session-state from the time at which your "outage" occurs.

I am having some issues with your attachment - if i can't resolve those, i might send you a private message asking you to resend it and may need the log (client) files as well.

In the meanwhile, perhaps you could try your app with the configurator, just shutdown both tomcats and restart them and you should see what you expect.

Thanks
Sreeni Iyer
Terracotta.

Sreeni Iyer, Terracotta.
Not a member yet - Click here to join the Terracotta Community
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi Sreeni,
Thanks for the early reply. Can we use the configurator for managing separate tomcat machines? Will send you the config file if you need it.

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
zeeiyer

consul

Joined: 05/24/2006 14:28:28
Messages: 493
Offline

No the configurator currently only allows for Tomcats on the same host (not separate hosts).

Are you sure that there isn't some exception in the logs - look at catalina.out
and terracotta-client logs. (If you could also attach these logs) and see if you see the sessions being clustered via the Admin Console. (I have the config file, btw and it looks normal - take it you are clustering 2 applications - EdgeGateway and SessionServlet - right ?)


Thanks.

Sreeni Iyer, Terracotta.
Not a member yet - Click here to join the Terracotta Community
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi Sreeni,
I'll just list down the steps I did just clear any possibilities of having done it wrong.
1. Untarred the contents of terracotta sessions on to a linux FC 6 machine.
2. Since we are using jre 1.5.0_10, i deleted the jre folder provided with terracotta and made a symbolic link to our version on jre in the same TC_INSTALL folder.
3. Made the boot jar by using the make-boot-jar.sh script and also passing as argument the config file I had attached earlier. This created the dso-boot-hotspot_linux_150_10.jar under the dso_boot directory
4. Next I installed Tomcat 5.0.28 in two different FC 6 machines. Also copied the untarred terracotta folder onto these machines (along with the new boot jar) and added the following variables to the start of catalina.sh file of tomcat (I understand this is to run the client which is currently bundled together with the server in the installation) :
DSO_BOOT_JAR="/home/test/terracotta-2.2.1/common/lib/dso-boot/dso-boot-hotspot_linux_150_10.jar"
TC_INSTALL_DIR="/home/test/terracotta-2.2.1"
JAVA_OPTS="${JAVA_OPTS} -Xbootclasspath/p:${DSO_BOOT_JAR}"
JAVA_OPTS="${JAVA_OPTS} -Dtc.install-root=${TC_INSTALL_DIR}"
JAVA_OPTS="${JAVA_OPTS} -Dtc.config=/home/test/terracotta-2.2.1/tc-config-tomcat.xml"
JAVA_OPTS="${JAVA_OPTS} -Dwebserver.log.name=tc_server"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote"
export JAVA_OPTS
5. Added the two tomcats to a load balancer and uses IP based persistance while load balancing (i.e any request from a client is routed to the same tomcat server unless it is down).
6. Used a sample application which put data onto the servlet session using session.putValue (key, value) which is deployed in both the tomcat servers.
7. Started accesing one server and putting values onto the session of that server . Then I shutdown the server which was serving my browser to check if the session was maintained in the other server as well.

This is the procedure I followed. Please comment if there is anything that I might have missed.

Thanks ,

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi ,
Have been trying to cluster our application without solving the earlier mentioned problem, but now I get a new error in the tomcat log. Could you please help with this? We are using Hibernate with OSCache enabled for the L2 cache. It seems to be coming from there. But not sure how to set it right. The stack trace is as follows:
********************************** ERROR ***********************************
* Terracotta client shutting down due to error java.lang.AssertionError:
* com.cloudhub.hibernate.ResourceAttributeField$$EnhancerByCGLIB$$e7f45916.CGLIB$CALLBACK_0
* does not exist in map returned from __tc_getallfields. Class is
* com.cloudhub.hibernate.ResourceAttributeField$$EnhancerByCGLIB$$e7f45916.
* field Values =
* {com.cloudhub.hibernate.ResourceAttributeField.resourceType=com.cloudhub.hibernate.ResourceType@69949d,
* com.cloudhub.hibernate.ResourceAttributeField.langKey=LANGKEY,
* com.cloudhub.hibernate.ResourceAttributeField.resourceTypeIdNull=false,
* com.cloudhub.hibernate.ResourceAttributeField.id=12,
* com.cloudhub.hibernate.ResourceAttributeField.enabledStatusNull=false,
* com.cloudhub.hibernate.ResourceAttributeField.name=TAG,
* com.cloudhub.hibernate.ResourceAttributeField.resourceTypeId=1,
* com.cloudhub.hibernate.ResourceAttributeField.enabledStatus=1,
* com.cloudhub.hibernate.ResourceAttributeField.dataType=STRING,
* com.cloudhub.hibernate.ResourceAttributeField.defaultValue=LANGKEY}
****************************************************************************

java.lang.AssertionError: com.cloudhub.hibernate.ResourceAttributeField$$EnhancerByCGLIB$$e7f45916.CGLIB$CALLBACK_0 does not exist in map returned from __tc_getallfields. Class is com.cloudhub.hibernate.ResourceAttributeField$$EnhancerByCGLIB$$e7f45916. field Values = {com.cloudhub.hibernate.ResourceAttributeField.resourceType=com.cloudhub.hibernate.ResourceType@69949d, com.cloudhub.hibernate.ResourceAttributeField.langKey=LANGKEY, com.cloudhub.hibernate.ResourceAttributeField.resourceTypeIdNull=false, com.cloudhub.hibernate.ResourceAttributeField.id=12, com.cloudhub.hibernate.ResourceAttributeField.enabledStatusNull=false, com.cloudhub.hibernate.ResourceAttributeField.name=TAG, com.cloudhub.hibernate.ResourceAttributeField.resourceTypeId=1, com.cloudhub.hibernate.ResourceAttributeField.enabledStatus=1, com.cloudhub.hibernate.ResourceAttributeField.dataType=STRING, com.cloudhub.hibernate.ResourceAttributeField.defaultValue=LANGKEY}
at com.tc.object.applicator.PhysicalApplicator.dehydrate(PhysicalApplicator.java:133)
at com.tc.object.TCClassImpl.dehydrate(TCClassImpl.java:164)
at com.tc.object.TCObjectImpl.dehydrate(TCObjectImpl.java:247)
at com.tc.object.change.TCChangeBufferImpl.writeTo(TCChangeBufferImpl.java:89)
at com.tc.object.tx.TransactionBatchWriter.addTransaction(TransactionBatchWriter.java:128)
at com.tc.object.tx.TransactionSequencer.addTransactionToBatch(TransactionSequencer.java:66)
at com.tc.object.tx.TransactionSequencer.addTxnInternal(TransactionSequencer.java:98)
at com.tc.object.tx.TransactionSequencer.addTransaction(TransactionSequencer.java:76)
at com.tc.object.tx.RemoteTransactionManagerImpl.commit(RemoteTransactionManagerImpl.java:205)
at com.tc.object.tx.ClientTransactionManagerImpl.commitInternal(ClientTransactionManagerImpl.java:401)
at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:367)
at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:313)
at com.tc.object.bytecode.ManagerImpl.commitLock(ManagerImpl.java:338)
at com.tc.object.bytecode.ManagerUtil.commitLock(ManagerUtil.java:122)
at com.terracotta.session.util.Lock.commitLock(Lock.java:26)
at com.terracotta.session.util.DefaultSessionId.commitLock(DefaultSessionId.java:48)
at com.terracotta.session.TerracottaSessionManager.postprocessSession(TerracottaSessionManager.java:186)
at com.terracotta.session.TerracottaSessionManager.basicPostprocess(TerracottaSessionManager.java:151)
at com.terracotta.session.TerracottaSessionManager.postprocess(TerracottaSessionManager.java:134)
at com.tc.tomcat50.session.SessionValve50.tcInvoke(SessionValve50.java:62)
at com.tc.tomcat50.session.SessionValve50.invoke(SessionValve50.java:47)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:149)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:564)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:972)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:206)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:700)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)
Feb 28, 2007 6:08:47 PM org.apache.coyote.tomcat5.CoyoteAdapter service

Thanks,

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
zeeiyer

consul

Joined: 05/24/2006 14:28:28
Messages: 493
Offline

Hello:

1.
Wrt Problem #1, I don't see anything wrong with the steps you executed. Wouldn't hurt to be doubly sure that the 2 boxes are identical in terms of Tomcat versions, tc-config.xml. If you could do a fresh run and give us the terracotta-client logs and catalina logs if you see anything unusual (e.g. an exception stack-trace). We can do so offline as well. There are also some debug options that you can turn on (see the reference tc-config.xml that comes with the sessions kit)

2.
Wrt Problem #2, Terracotta does not support Hibernate as of now (we CAN cluster the 2nd level cache non-transactionally) - but as a product offering, its on our roadmap for summer of this year. I suppose you ran into this since your sessions somehow have a reference to a hibernate generated POJO. If that reference shouldn't be there or dosen't need to be replicated across the cluster, you should mark it as transient in the tc-config.xml file and rehydrate it on the other nodes in application-code or bean-shell (on-load element in the tc-config.xml file).

Regards
Iyer

Sreeni Iyer, Terracotta.
Not a member yet - Click here to join the Terracotta Community
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Thanks, will try out for the first problem again with fresh installations to be sure. Will also send the logs on error. In the mean time could you guide me to any documents that talk about setting these fields as transient (i am guessing it is not the Java transient keyword) and also their associative configurations ? Didnt quite get the part about rehydration. Did set all the offending variables in the exception under the <transient-fields> tag. But Now I get an IllegalStateException. The stack trace given below:
java.lang.IllegalStateException: This session is invalid
at com.terracotta.session.util.StandardSession.checkIfValid(StandardSession.java:176)
at com.terracotta.session.util.StandardSession.setAttribute(StandardSession.java:47)

Since I only started exploring terracotta, am not able to get all the lingo about the same. If you have any documentation that mentions about setting transient fields and also about rehydrating, would be good. Did look up some webs sites and a couple of pdf's provided, but were not of too much help in understanding the concept. I guess you could throw some light. Must admit the forum replies have been very responsive. Thanks to you for that!

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
zeeiyer

consul

Joined: 05/24/2006 14:28:28
Messages: 493
Offline

Hi:

There is an actionable description of "transients" and their impact at http://www.terracotta.org/confluence/display/docs1/Troubleshooting+Guide#TroubleshootingGuide-Iamgettingcom.tc.exception.TCNonPortableObjectException.Whatnext%3F

Hope that helps - also look at the reference config file under sessions/config-sample directory of your kit.

Wrt the illegal state exception, will research it and get back to you. If you could send the new tc-config and logs after your re-attempt, we would hopefully be able to pinpoint any other issues.
Regards

Sreeni Iyer, Terracotta.
Not a member yet - Click here to join the Terracotta Community
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi,
Please find attached the configuration file and the catalina.out of the server. Hope this has some clues. Will look into the link you provided. Sorry for the urgency, I need to evaluate and know if this fits our project requirements which is under a tight release schedule.

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
 Filename tc-config-tomcat.xml [Disk] Download
 Description config file
 Filesize 4 Kbytes
 Downloaded:  1335 time(s)

 Filename catalina.out [Disk] Download
 Description stack trace
 Filesize 12 Kbytes
 Downloaded:  385 time(s)

Gordon

journeyman

Joined: 12/13/2006 00:38:30
Messages: 44
Offline

Hi Zeus,

Thank you for posting your Terracotta configuration and catalina.out files. I noticed a couple of things in the Terracotta config -- the transient fields section was commented out, and there were no corresponding uses of bean shell scripting or of an on-load method to to insure the fields defined as transient were instantiated when the parent object was replicated from the first node on which it was created to any other node. Did the errors you mention regarding invalid sessions occur with this Terracotta config file, or another version?

I will also look into the invalid sessions error. Of course, sessions can be invalidated programmatically or they can time out. Is either of these a possibility in your application?

One general trouble shooting step I will mention is -- since I saw in an earlier thread that the Tomcat instances are loading their configuration file locally -- is the suggestion that you recycle your Terracotta server between any changes to the Terracotta configuration and subsequent restarts of the Tomcat servers. In fact, to keep your testing as pristine as possible, whenever you restart both Tomcat servers for a new round of testing, I would restart the Terracotta server first as well. I also would not use the permanent-store option for testing -- I would use temporary-swap-only, which assures you that nothing will be around from a previous testing session after you restart the Terracotta server. These steps may help specifically with the invalid session message, but I can not guarantee that without knowing a lot more about your application.

It is clear you are under some time pressure. Please don't hesitate to post throughout the day with any questions, test results, etc. Clustering Tomcat is an extremely straightforward use case in its own right. The variable here appears to be that we don't support Hibernate directly yet, and how do we then successfully create the correct Terracotta config file.

Regards,

Gordon
Sr. Systems Engineer
Terracotta, Inc.
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi Gordon,
Nice to hear from you. To get around the hibernate problem, I checked out to see if any hibernate persistant classes are being added to session. To overcome this problem, we are now putting clones of the persistant classes onto the session. This I believe and hope would solve the problem of having to define transient fields (which could be huge in number) . Now I repeatedly get the IllegalStateException when I login. But it works fine out of the cluster. The sessions are invalidated in the case of an old session coming during login and a new session is assigned to the user. Would that be a problem?
Did change the config files during the course. Please find attached the config file, the server log, the terracotta server logs.

PS: Have taken your suggestion about the restarts and the persistent storage of session.

Regards,
Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
 Filename tc-config-tomcat.xml [Disk] Download
 Description Config file
 Filesize 3 Kbytes
 Downloaded:  389 time(s)

 Filename catalina.out [Disk] Download
 Description Tomcat server log
 Filesize 23 Kbytes
 Downloaded:  385 time(s)

 Filename terracotta-client.log [Disk] Download
 Description terracotta client log
 Filesize 12 Kbytes
 Downloaded:  384 time(s)

Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Sorry the terracotta server log was missed due to the attachment limit. Please find it attached here.
 Filename terracotta-server.log [Disk] Download
 Description Terracotta server log
 Filesize 12 Kbytes
 Downloaded:  387 time(s)

Tim Eck

journeyman
[Avatar]
Joined: 01/25/2007 08:57:02
Messages: 47
Location: San Mateo, CA
Offline

I actually ran across a similar looking exception (IllegalStateException) just now. It was happening in the standard sessionDestroyed() session event callback. Apparently the Terracotta implementation disallows attribute access while a session is being actively invalidated (in this detail it appears we differ from the tomcat implementation -- I'm not sure who the servlet spec agrees with).

Anyway, I was wondering what the complete stack trace was when you got the IllegalStateException? If I knew that I could say whether you are experiencing the same issue.

-tim
Zeus

journeyman

Joined: 02/26/2007 22:42:53
Messages: 14
Offline

Hi Tim,
The attached catalina.out in my previous post is the complete stack trace I get when I login. We are invalidating a session when a user logs in if and only if due to cookies his earlier session is still alive. After the invalidation, a new session is created and assigned to that user. Till then, I believe there is no access of the attribute fields from the session object. Is this the same situation you could replicate? Does it have a work around or any suggestions I could try?

Thanks,

Regards,
Anoop Gopalakrishnan
Project Manager,
Device Driven
www.devicedriven.com
 
Forum Index -> Terracotta for Web Sessions Go to Page: 1, 2 Next 
Go to:   
Powered by JForum 2.1.7 © JForum Team