Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

Hi.

Terracotta Open Source (3.5) installation contains start-tc-server.sh script and it contains the code:

for JAVA_COMMAND in \
"${JAVA_HOME}/bin/java -d64 -server -XX:MaxDirectMemorySize=64g" \
"${JAVA_HOME}/bin/java -server -XX:MaxDirectMemorySize=1g" \
"${JAVA_HOME}/bin/java -d64 -client -XX:MaxDirectMemorySize=64g" \
"${JAVA_HOME}/bin/java -client -XX:MaxDirectMemorySize=1g" \
"${JAVA_HOME}/bin/java"
do
${JAVA_COMMAND} -version > /dev/null 2>&1
if test "$?" = "0" ; then break; fi
done

This makes that only if JVM supports it, JVM is started with -XX:MaxDirectMemorySize=64g.
I wouldn't care this, but it seems, that after few hours of running terracotta process' resident memory size exceeds it's -Xmx memory setting by much more than permgen size - even by 2-3GB. Looking at pmap it looks like it is allocated out of heap (i can see one block of memory equal to current heap commited and about 20 blocks of 128MB)

The questions are:
1. Does terracotta without BigMemory need -XX:MaxDirectMemorySize setting?
2. Why -XX:MaxDirectMemorySize=64g is the default setting?
3. Is editing start-tc-server.sh the only/preferred way to change these options?

TIA

Ryszard.

I don't think you need -XX:MaxDirectMemorySize set to anything in particular when not using big memory. Artificially constraining it to something small could be problematic since java NIO socket calls need some level of direct memory available.

The setting of 64g I think was fairly arbitrary and would let people run with a pretty large big memory config without touching the script. In fact someone suggested yesterday that we should set that setting as high as it can go (like 2^63-1)

Editing the script is one way to go about altering the setting but I suspect that if you pass another -XX:MaxDirectMemorySize in JAVA_OPTS the last one in the command line will win. Of course if you want to remove mention of it all from the java invocation that would require an edit.

I don't know all that much about pmap output for a java process to make much of what you're reporting unfortunately. I'll ask around on that.

If you are trying to track down where all that RSS is coming from there are a couple of things you can try.

1. Switching on -XX:+PrintGCDetails will dump the heap details along with the stack dumps (e.g. using kill -3). This will give you the start and end addresses of the various heap regions.

2. If you're not averse to doing some reflection hackery there is a private static field in the java.nio.Bits class called reservedMemory which will tell you how much memory has been reserved for NIO direct buffers.

3. In principle if you get a heap dump of the JVM you could look at the address fields of the "top-level" DirectByteBuffer objects and find the addresses for those buffers to try and match up with the pmap output (I have never actually tried to do this).