2007-10-16 01:11:41,401 [WorkerThread(receive_transaction_stage,0)] WARN com.tc.object.gtx.ClientGlo
balTransactionManagerImpl - Server didnt send a Low water mark higher than ALLOWED_LWM_DELTA for 100
2 times. This may lead to OutOfMemoryError. Resetting count.
Does the above error mean something?
Can I make some changes to tc-config.xml to address this issue?
I do see that after some time terracotta server dies due to out of memory on the heap.
Actually this message is not right and shouldn't be a warning anyways. The message is already changed in the trunk. The server will not OOME because of this condition. Your problem is probably something totally different.
How much heap did you give the server ? How many clients connect to it and what is the transaction rate ? How long does the server run before OOMEing ?
Send us some more info on the environment along with the server logs and we will look into it.
Do you have a solution for that problem?
I faced with the same situation. There are such records in all client logs:
2014-01-09 21:39:02,862 [WorkerThread(receive_transaction_stage, 0)] WARN com.tc.object.gtx.ClientGlobalTransactionManagerImpl - Current Low water Mark = GlobalTransactionID=[1788837177] Server sent GlobalTransactionID=[1788837177]
2014-01-09 21:39:02,872 [WorkerThread(receive_transaction_stage, 0)] WARN com.tc.object.gtx.ClientGlobalTransactionManagerImpl - Server didnt send a Low water mark higher than ALLOWED_LWM_DELTA for 10002 times. applied.size() = 10161 Resetting count.
It leads to OldGen filling on the standby instance and OutOfMemory. Then the app restarts itself but the problem is relevant anyway. Also we see that OldGen consumption is increased on the primary instance and applications-clients.
How can we fix the issue? What can lead to these warnings? What information/data would you like to have to be able to help me?
What version of Terracotta are you using ? Please open a ticket at https://jira.terracotta.org. We would most probably need all the logs and a cluster dump to help you debug this problem.