[Logo] Terracotta Discussion Forums
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Terracotta Clustering Problem  XML
Forum Index -> General
Author Message
sanjoykr78

journeyman

Joined: 08/14/2009 06:20:11
Messages: 31
Offline

Hi guys,

I am trying to setup of a Terracotta cluster together with Tomcat on 2 nodes (node1: 192.168.1.141 and node2: 192.168.1.120). Running terracotta on first node is OK but when I run terracotta on second node I get the below error:


2009-09-23 09:48:18,474 [WorkerThread(group_events_dispatch_stage,0)] INFO com.tc.l2.ha.L2HACoordinator - NodeID[192.168.1.120:9510] joined the cluster
2009-09-23 09:48:18,474 [WorkerThread(group_events_dispatch_stage,0)] INFO com.terracottatech.console - NodeID[192.168.1.120:9510] joined the cluster
2009-09-23 09:48:18,510 [WorkerThread(group_events_dispatch_stage,0)] ERROR com.tc.l2.state.StateManagerImpl - Recd wrong response from : NodeID[192.168.1.120:9510] : msg = L2StateMessage [ NodeID[192.168.1.120:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
2009-09-23 09:48:18,511 [WorkerThread(group_events_dispatch_stage,0)] ERROR com.tc.l2.ha.L2HACoordinator - A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.120:9510] Zapping it to allow it to join as PASSIVE standby (backup):
com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.120:9510] : msg = L2StateMessage [ NodeID[192.168.1.120:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)
2009-09-23 09:48:18,512 [WorkerThread(group_events_dispatch_stage,0)] WARN com.terracottatech.console - Requesting node to quit : NodeID : NodeID[192.168.1.120:9510] Error Type : COMMUNICATION ERROR
2009-09-23 09:48:18,512 [WorkerThread(group_events_dispatch_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:18,512 [WorkerThread(group_events_dispatch_stage,0)] WARN com.tc.net.groups.TCGroupManagerImpl - Zapping node : NodeID[192.168.1.120:9510] type = 1 reason = A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.120:9510] Zapping it to allow it to join as PASSIVE standby (backup):
Exception : com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.120:9510] : msg = L2StateMessage [ NodeID[192.168.1.120:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)

my weight = [0]
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - State[ ACTIVE-COORDINATOR ] received Zap Node request from another State[ ACTIVE-COORDINATOR ]
NodeID : NodeID[192.168.1.120:9510] Error Type : Two or more Active servers detected in the cluster Details : State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.141:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - A Terracotta server tried to join the mirror group as a second ACTIVE : My weights = 0 Other servers weights = 0
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.state.Enrollment - Two Enrollments with same weights : Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 0 ] == Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 0 ]
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - Not quiting since the other servers weight = 0 is not greater than my weight = 0
2009-09-23 09:48:18,514 [WorkerThread(receive_group_message_stage,0)] WARN com.terracottatech.console - Ignoring Quit request from NodeID[192.168.1.120:9510] since remote servers weight is not greater than local weight
2009-09-23 09:48:18,551 [WorkerThread(l2_state_message_handler_stage,0)] ERROR com.tc.l2.state.StateManagerImpl - State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.120:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE
2009-09-23 09:48:18,567 [WorkerThread(l2_state_message_handler_stage,0)] WARN com.terracottatech.console - Requesting node to quit : NodeID : NodeID[192.168.1.120:9510] Error Type : Two or more Active servers detected in the cluster
2009-09-23 09:48:18,568 [WorkerThread(l2_state_message_handler_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:18,568 [WorkerThread(l2_state_message_handler_stage,0)] WARN com.tc.net.groups.TCGroupManagerImpl - Zapping node : NodeID[192.168.1.120:9510] type = 255 reason = State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.120:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE my weight = [0]
2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - State[ ACTIVE-COORDINATOR ] received Zap Node request from another State[ ACTIVE-COORDINATOR ]
NodeID : NodeID[192.168.1.120:9510] Error Type : COMMUNICATION ERROR Details : A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.141:9510] Zapping it to allow it to join as PASSIVE standby (backup):
Exception : com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.141:9510] : msg = L2StateMessage [ NodeID[192.168.1.141:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)


2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - A Terracotta server tried to join the mirror group as a second ACTIVE : My weights = 0 Other servers weights = 0
2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.state.Enrollment - Two Enrollments with same weights : Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 0 ] == Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 0 ]
2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - Not quiting since the other servers weight = 0 is not greater than my weight = 0
2009-09-23 09:48:18,576 [WorkerThread(receive_group_message_stage,0)] WARN com.terracottatech.console - Ignoring Quit request from NodeID[192.168.1.120:9510] since remote servers weight is not greater than local weight


My tc-config.xml on both nodes:

<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">
<servers>
<server name="node01" host="192.168.1.141">
<data>/opt/terracotta/server1-data</data>
<logs>/var/log/terracotta/server-logs</logs>
<statistics>/var/log/terracotta/server-stats</statistics>
<l2-group-port>9530</l2-group-port>
</server>
<server name="node02" host="192.168.1.120">
<data>/opt/terracotta/server2-data</data>
<logs>/var/log/terracotta/server-logs</logs>
<statistics>/var/log/terracotta/server-stats</statistics>
<l2-group-port>9530</l2-group-port>
</server>
<ha>
<mode>networked-active-passive</mode>
<networked-active-passive>
<election-time>5</election-time>
</networked-active-passive>
</ha>
</servers>
<clients>
<logs>/var/log/terracotta/client-logs</logs>
<statistics>/var/log/terracotta/client-stats</statistics>
</clients>
<application>
<dso>
<instrumented-classes>
<include>
<class-expression>com.xxx.neo.action..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.branding..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.exception..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.jasper..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.jmesa..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.mail..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.model..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.neoimagemanager..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.neopaginator..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.report..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.search..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.security..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.service..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.sms..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.taglib..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.util..*</class-expression>
</include>
<include>
<class-expression>com.xxx.neo.widgets..*</class-expression>
</include>
<include>
<class-expression>org.hibernate.collection.PersistentSortedMap</class-expression>
</include>

<exclude>org.apache.coyote..*</exclude>
<exclude>org.apache.catalina..*</exclude>
<exclude>org.apache.jasper..*</exclude>
<exclude>org.apache.tomcat..*</exclude>
</instrumented-classes>
<web-applications>
<web-application>neo3</web-application>
</web-applications>
</dso>
</application>
</tc:tc-config>


Any idea ?
rsingh

journeyman

Joined: 10/22/2008 18:45:27
Messages: 43
Offline

please post the log files for both the servers.
sanjoykr78

journeyman

Joined: 08/14/2009 06:20:11
Messages: 31
Offline

Hi,
thanks for reply.

In the first post you see a server log that is from first node.

here is the another log from second node :


2009-09-23 09:48:17,626 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.tx.ServerTransactionManager - Waiting for txns to complete
2009-09-23 09:48:17,627 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.tx.ServerTransactionManager - No more txns in the system.
2009-09-23 09:48:17,627 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.tx.ResentTransactionSequencer - Making callback com.tc.objectserver.gtx.GlobalTransactionIDLowWaterMarkProvider$2@1079ff pending since in State[ ADD_RESENT ] resent txns size : 0
2009-09-23 09:48:17,627 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.handshakemanager.ServerClientHandshakeManager - Starting DSO services...
2009-09-23 09:48:17,627 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.persistence.impl.TransactionStoreImpl - shutdownAllClientsExcept() : Removing txns from DB : 0
2009-09-23 09:48:17,665 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.impl.ObjectRequestManagerRestartImpl - Processing Pending Lookups = 0
2009-09-23 09:48:17,666 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.tx.ResentTransactionSequencer - Unregistering ResentTransactionSequencer since no more resent Transactions : 0
2009-09-23 09:48:17,666 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.tx.ResentTransactionSequencer - Adding Pending resent CallBacks to TxnMgr : com.tc.objectserver.gtx.GlobalTransactionIDLowWaterMarkProvider$2@1079ff
2009-09-23 09:48:17,669 [WorkerThread(l2_state_change_stage,0)] INFO com.tc.objectserver.gtx.GlobalTransactionIDLowWaterMarkProvider - Switching GlobalTransactionID Low Water mark provider since all resent transactions are applied
2009-09-23 09:48:17,670 [WorkerThread(l2_state_change_stage,0)] INFO com.terracottatech.console - Terracotta Server instance has started up as ACTIVE node on 0:0:0:0:0:0:0:0:9510 successfully, and is now ready for work.
2009-09-23 09:48:17,734 [WorkerThread(group_discovery_stage,0)] INFO com.tc.net.protocol.transport.ConnectionHealthCheckerImpl. TCGroupManager - Health monitoring agent started for 192.168.1.141:9530
2009-09-23 09:48:17,841 [main] INFO com.tc.server.TCServer - Server started as node02
2009-09-23 09:48:17,877 [TCComm Main Selector Thread (listen 0:0:0:0:0:0:0:0:9530)] INFO com.tc.net.protocol.transport.ConnectionHealthCheckerImpl. TCGroupManager - HealthCheckCallbackPort verification PASSED for 192.168.1.141:9530(callbackport: 9530)
2009-09-23 09:48:17,877 [Update Checker] INFO com.tc.server.UpdateCheckAction - Update Checker: Checking...
2009-09-23 09:48:17,910 [WorkerThread(l2_state_message_handler_stage,0)] ERROR com.tc.l2.state.StateManagerImpl - State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.141:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE
2009-09-23 09:48:17,910 [WorkerThread(group_events_dispatch_stage,0)] INFO com.tc.l2.ha.L2HACoordinator - NodeID[192.168.1.141:9510] joined the cluster
2009-09-23 09:48:17,911 [WorkerThread(group_events_dispatch_stage,0)] INFO com.terracottatech.console - NodeID[192.168.1.141:9510] joined the cluster
2009-09-23 09:48:17,911 [WorkerThread(l2_state_message_handler_stage,0)] WARN com.terracottatech.console - Requesting node to quit : NodeID : NodeID[192.168.1.141:9510] Error Type : Two or more Active servers detected in the cluster
2009-09-23 09:48:17,912 [WorkerThread(l2_state_message_handler_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:17,912 [WorkerThread(l2_state_message_handler_stage,0)] WARN com.tc.net.groups.TCGroupManagerImpl - Zapping node : NodeID[192.168.1.141:9510] type = 255 reason = State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.141:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE my weight = [0]
2009-09-23 09:48:17,926 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - State[ ACTIVE-COORDINATOR ] received Zap Node request from another State[ ACTIVE-COORDINATOR ]
NodeID : NodeID[192.168.1.141:9510] Error Type : COMMUNICATION ERROR Details : A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.120:9510] Zapping it to allow it to join as PASSIVE standby (backup):
Exception : com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.120:9510] : msg = L2StateMessage [ NodeID[192.168.1.120:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)


2009-09-23 09:48:17,926 [WorkerThread(receive_group_message_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:17,926 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - A Terracotta server tried to join the mirror group as a second ACTIVE : My weights = 0 Other servers weights = 0
2009-09-23 09:48:17,927 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.state.Enrollment - Two Enrollments with same weights : Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 0 ] == Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 0 ]
2009-09-23 09:48:17,927 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - Not quiting since the other servers weight = 0 is not greater than my weight = 0
2009-09-23 09:48:17,927 [WorkerThread(receive_group_message_stage,0)] WARN com.terracottatech.console - Ignoring Quit request from NodeID[192.168.1.141:9510] since remote servers weight is not greater than local weight
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - State[ ACTIVE-COORDINATOR ] received Zap Node request from another State[ ACTIVE-COORDINATOR ]
NodeID : NodeID[192.168.1.141:9510] Error Type : Two or more Active servers detected in the cluster Details : State[ ACTIVE-COORDINATOR ] Received Election Won Msg : L2StateMessage [ NodeID[192.168.1.120:9510], type = ELECTION_WON_ALREADY, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]]. A Terracotta server tried to join the mirror group as a second ACTIVE
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - A Terracotta server tried to join the mirror group as a second ACTIVE : My weights = 0 Other servers weights = 0
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.state.Enrollment - Two Enrollments with same weights : Enrollment [ NodeID[192.168.1.141:9510], isNew = false, weights = 0 ] == Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 0 ]
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] WARN com.tc.l2.ha.L2HAZapNodeRequestProcessor - Not quiting since the other servers weight = 0 is not greater than my weight = 0
2009-09-23 09:48:17,976 [WorkerThread(receive_group_message_stage,0)] WARN com.terracottatech.console - Ignoring Quit request from NodeID[192.168.1.141:9510] since remote servers weight is not greater than local weight
2009-09-23 09:48:17,976 [WorkerThread(group_events_dispatch_stage,0)] ERROR com.tc.l2.state.StateManagerImpl - Recd wrong response from : NodeID[192.168.1.141:9510] : msg = L2StateMessage [ NodeID[192.168.1.141:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
2009-09-23 09:48:17,978 [WorkerThread(group_events_dispatch_stage,0)] ERROR com.tc.l2.ha.L2HACoordinator - A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.141:9510] Zapping it to allow it to join as PASSIVE standby (backup):
com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.141:9510] : msg = L2StateMessage [ NodeID[192.168.1.141:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)
2009-09-23 09:48:17,978 [WorkerThread(group_events_dispatch_stage,0)] WARN com.terracottatech.console - Requesting node to quit : NodeID : NodeID[192.168.1.141:9510] Error Type : COMMUNICATION ERROR
2009-09-23 09:48:17,979 [WorkerThread(group_events_dispatch_stage,0)] INFO com.tc.net.core.TCConnectionManager - Active connections : 0 out of 0
2009-09-23 09:48:17,979 [WorkerThread(group_events_dispatch_stage,0)] WARN com.tc.net.groups.TCGroupManagerImpl - Zapping node : NodeID[192.168.1.141:9510] type = 1 reason = A Terracotta server tried to join the mirror group as a second ACTIVE: NodeID[192.168.1.141:9510] Zapping it to allow it to join as PASSIVE standby (backup):
Exception : com.tc.net.groups.GroupException: Recd wrong response from : NodeID[192.168.1.141:9510] : msg = L2StateMessage [ NodeID[192.168.1.141:9510], type = RESULT_CONFLICT, Enrollment [ NodeID[192.168.1.120:9510], isNew = false, weights = 9223372036854775807,9223372036854775807 ]] while publishing Active State
at com.tc.l2.state.StateManagerImpl.validateResponse(StateManagerImpl.java:324)
at com.tc.l2.state.StateManagerImpl.publishActiveState(StateManagerImpl.java:316)
at com.tc.l2.ha.L2HACoordinator.nodeJoined(L2HACoordinator.java:235)
at com.tc.l2.handler.GroupEventsDispatchHandler.handleEvent(GroupEventsDispatchHandler.java:24)
at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)

my weight = [0]
2009-09-23 09:48:19,367 [Update Checker] INFO com.tc.server.UpdateCheckAction - Update Checker: No updates found
2009-09-23 09:48:19,367 [Update Checker] INFO com.tc.server.UpdateCheckAction - Update Checker: Next check at Wed Sep 30 09:48:19 BST 2009
2009-09-23 10:03:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 445554008
2009-09-23 10:03:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory used : 77947560
2009-09-23 10:03:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 10:18:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 439712888
2009-09-23 10:18:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory used : 83788680
2009-09-23 10:18:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 10:33:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory free : 436035072
2009-09-23 10:33:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory used : 87466496
2009-09-23 10:33:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 10:48:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 430924464
2009-09-23 10:48:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory used : 92577104
2009-09-23 10:48:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 10:48:17,671 [DGC-Thread] INFO com.tc.objectserver.dgc.impl.MarkAndSweepGarbageCollector - DGC[ 1 ] complete : GarbageCollectionInfo [ Iteration = 1 ] = type = full, startTime = 1253699297669 begin object count = 0 markStageTime = 1 pauseStageTime = 0 deleteStageTime = 0 elapsedTime = 1 totalMarkCycleTime = 1 candiate garabage count = 0 actual garbage count = 0 pre rescue count = 0 rescue1Time = 0 rescue 1 Count = 0 rescue2Time = 0
2009-09-23 11:03:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 425101200
2009-09-23 11:03:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory used : 98400368
2009-09-23 11:03:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 11:18:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 422527840
2009-09-23 11:18:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory used : 100973728
2009-09-23 11:18:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 11:33:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 417761584
2009-09-23 11:33:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory used : 105739984
2009-09-23 11:33:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 11:48:10,484 [Statistics Logger] INFO com.terracottatech.dso - memory free : 415188224
2009-09-23 11:48:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory used : 108313344
2009-09-23 11:48:10,485 [Statistics Logger] INFO com.terracottatech.dso - memory max : 523501568
2009-09-23 11:48:17,672 [DGC-Thread] INFO com.tc.objectserver.dgc.impl.MarkAndSweepGarbageCollector - DGC[ 2 ] complete : GarbageCollectionInfo [ Iteration = 2 ] = type = full, startTime = 1253702897672 begin object count = 0 markStageTime = 0 pauseStageTime = 0 deleteStageTime = 0 elapsedTime = 0 totalMarkCycleTime = 0 candiate garabage count = 0 actual garbage count = 0 pre rescue count = 0 rescue1Time = 0 rescue 1 Count = 0 rescue2Time = 0
mgovinda

journeyman

Joined: 10/16/2007 12:32:55
Messages: 30
Offline

Were there any network problems when both the servers were started ? Seems both servers weren't able to talk to each other when started and so each of them became ACTIVE. After a while, when the network resumed both were able to talk to each other. But, since each were running as ACTIVE with no clients connected, they couldn't determine which server should continue to run as ACTIVE and which one should die and restart as PASSIVE.

When two servers are running as ACTIVE in the cluster, you need to manually restart one, which will then join the cluster as PASSIVE.

The logs you posted doesn't show the what happened when the servers were started and during the initial election. please attach full logs of both servers ?
sanjoykr78

journeyman

Joined: 08/14/2009 06:20:11
Messages: 31
Offline

Hi,

I have managed to run both servers one as ACTIVE & other as PASSIVE. dso-port & jmx-port were missing in tc-config.xml file.

Now a new problem occurs.

I am getting a PermGem error when I start tomcat.

Other error occurs related to CGLIB integration module

This is from terracotta-client.log:

2009-09-24 10:20:23,836 [main] ERROR com.terracottatech.dso - Refusing to instrument CGLIB generated proxy type com.xxx.internal.model.ZapFile$$EnhancerByCGLIB$$39830788 (CGLIB integration module not enabled)

Do you know which tim-module I should enable.

ilevy

consul

Joined: 04/16/2008 10:26:42
Messages: 357
Offline

Doesn't look like you have the Tomcat TIM installed via tc-config.xml. See this doc:

http://www.terracotta.org/web/display/orgsite/Tomcat+Integration#TomcatIntegration-custom
 
Forum Index -> General
Go to:   
Powered by JForum 2.1.7 © JForum Team