[Logo] Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Terracotta 3.7.0 doesn't close its HTTP connections -- many CLOSE_WAIT sockets  XML
Forum Index -> General
Author Message
ecd

neo

Joined: 10/08/2012 12:29:43
Messages: 8
Offline

1. Start up terracotta server
Code:
 $ tar xzf terracotta-3.7.0.tar.gz 
 $ terracotta-3.7.0/bin/start-tc-server.sh

2. Make some requests to pull the config at port 9530
Code:
 $ for i in $(seq 1000); do curl -s "http://localhost:9530/config" > /dev/null; done

3. Check terracotta's open sockets
Code:
 $ lsof -i -P | grep $(pgrep java) | grep CLOSE_WAIT | wc -l
 781


It seems that Terracotta's HTTP server isn't properly closing its TCP connections. The file descriptor limit can be reached easily, causing Terracotta to die:
Code:
 2012-10-08 15:17:02,245 [WorkerThread(dso-http-bridge, 0)] ERROR com.tc.net.protocol.HttpConnectionContext - Exception thrown
 java.net.SocketException: Too many open files
 	at java.net.Socket.createImpl(Socket.java:414)
 	at java.net.Socket.getImpl(Socket.java:477)
 	at java.net.Socket.getSoTimeout(Socket.java:1050)
 	at com.tc.server.TerracottaConnector$SocketWrapper.getSoTimeout(TerracottaConnector.java:156)
 	at org.mortbay.jetty.bio.SocketConnector$Connection.<init>(SocketConnector.java:183)
 	at com.tc.server.TerracottaConnector.handleSocketFromDSO(TerracottaConnector.java:39)
 	at com.tc.server.HttpConnectionHandler.handleEvent(HttpConnectionHandler.java:35)
 	at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145)


Sometimes the terracotta server becomes unresponsive and the process crashes:
Code:
 java.lang.RuntimeException: java.lang.AssertionError: Protocol is 2
 	at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:293)
 Caused by: java.lang.AssertionError: Protocol is 2
 	at com.tc.net.protocol.ProtocolSwitch.getReadBuffers(ProtocolSwitch.java:139)
 	at com.tc.net.core.TCConnectionImpl.getReadBuffers(TCConnectionImpl.java:794)
 	at com.tc.net.core.TCConnectionImpl.doReadFromBufferInternal(TCConnectionImpl.java:433)
 	at com.tc.net.core.TCConnectionImpl.doReadFromBuffer(TCConnectionImpl.java:306)
 	at com.tc.net.core.TCConnectionImpl.doReadInternal(TCConnectionImpl.java:290)
 	at com.tc.net.core.TCConnectionImpl.doRead(TCConnectionImpl.java:266)
 	at com.tc.net.core.CoreNIOServices$CommThread.selectLoop(CoreNIOServices.java:624)
 	at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:290)


Could someone please confirm this is an issue? The CLOSE_WAIT sockets will never die without the application (terracotta) closing them.
rajoshi

seraphim

Joined: 07/04/2011 04:36:10
Messages: 1491
Offline

This seems to be a bug you can open a Community jira for this at :

https://jira.terracotta.org/jira/secure/BrowseProjects.jspa

Rakesh Joshi
Senior Consultant
Terracotta.
ecd

neo

Joined: 10/08/2012 12:29:43
Messages: 8
Offline

Ok, thanks for the response. I've reported it here: https://jira.terracotta.org/jira/browse/CDV-1631
 
Forum Index -> General
Go to:   
Powered by JForum 2.1.7 © JForum Team