Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

Can you please explain how it does it in parallel? Since the server runs as one JVM I don't see how the JVM can concurrently transmit to each client.

I'm a system programming beginner. I do know that a processor can only run one thread at a time. The process in this case would be the JVM. My guess was that the server would create another thread and the processor would give one thread a time slice then switch to another thread for a time slice, etc. How would the server JVM transmit with many cores/processors available?

Thanks.

Thanks for the tip.

Question:
Lets say I have four servers running in permanent-store mode, all with the same data directory and node0 is active and ten clients connected. If all clients asked for the same shared memory to read at the same time would the clients receive the data one at a time? The passive servers are not utilized in a situation where the read may take 5 seconds for each client? It seems to me passive servers are basically dead to the clients unless there is a network connection failure to the active server.

Thank you!

Thanks thats all I needed to know. I might as well post my code and what it does.
Its meant to be run with 2 as the program runtime argument.

Its interesting to think that the program could have an infinite loop if it was fast enough to do the second dmi before node 0 exited.

Output of Node 0.
Code:

 Waiting...
 Arrived:0
 String 0
 String 1
 Waiting for dmi:
 Looking For DMI calls:0
 HelloWorld
 Looking For DMI calls:2
 SLeep
 End SLeep, call dmi
 HelloWorld

Output of Node 1.
Code:

 Waiting...
 Arrived:1
 String 0
 String 1
 SLeep
 End SLeep, call dmi
 HelloWorld

Main.java
Code:

 
 package sim;
 
 import java.util.concurrent.BrokenBarrierException;
 import java.util.concurrent.CyclicBarrier;
 
 
 public class Main implements Runnable {
 
 	private final CyclicBarrier barrier;
 	final int participants;
 	private Agent[] agentlist;
 	private int DMIamount;
 
 	public Main(final CyclicBarrier barrierIN, int threads, Agent[] agentlistIN)
 	{
 		agentlist=agentlistIN;
 		barrier=barrierIN;
 		participants=threads;
 		DMIamount=0;
 	}
 	
 	public void run()
 	{
 		System.out.println("Waiting...");
 		
 		int arrival=-1;
 			try {
 				arrival=barrier.await();
 			} catch (InterruptedException e1) {
 				// TODO Auto-generated catch block
 				e1.printStackTrace();
 			} catch (BrokenBarrierException e1) {
 				// TODO Auto-generated catch block
 				e1.printStackTrace();
 			}
 		
 		System.out.println("Arrived:"+arrival);
 		
 		agentlist[arrival].put("String "+arrival);
 		
 		try {
 			barrier.await();
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		} catch (BrokenBarrierException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 		
 		System.out.println(agentlist[0].get(0));
 		System.out.println(agentlist[1].get(0));
 		
 		if(arrival==0)waithere();
 		
 		System.out.println("SLeep");
 		try {
 			Thread.currentThread();
 			Thread.sleep(1000);
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 		System.out.println("End SLeep, call dmi");
 		dmi("HelloWorld");
 	}
 	
 	public void dmi(String msg)
 	{
 		System.out.println(msg);
 		synchronized(this){
 			DMIamount++;
 		}
 	}
 	
 	public void waithere()
 	{
 		System.out.println("Waiting for dmi:");
 		
 		
 		while(DMIamount < 2){
 			
 			try {
 				Thread.currentThread().join(1000);
 			} catch (InterruptedException e) {
 				// TODO Auto-generated catch block
 				e.printStackTrace();
 			}
 			System.out.println("Looking For DMI calls:"+DMIamount);
 		}
 		
 	}
 	
 	public static void main(String[] args) throws Exception
 	{
 		int threads=Integer.parseInt(args[0]);
 		Agent [] agentlist=new Agent[threads];
 		for(int i=0;i<threads;i++) agentlist[i]=new Agent();
 		
 		Roots roots=new Roots(new Main(new CyclicBarrier(threads),
 										threads,
 										agentlist
 										));
 		
 		(roots.getMain() ).run();
 	}
 }

Roots.java
Code:

 package sim;
 
 
 
 public class Roots {
 	private static Main instance;
 	
 	public Roots(Main instanceIN) {
 		instance=instanceIN;
 	}
 	
 	public Main getMain()
 	{
 		return instance;
 	}
 	
 }

Agent.java
Code:

 package sim;
 
 import java.util.ArrayList;
 
 public class Agent {
 	
 	public ArrayList<String> inbox;
 	
 	public Agent()
 	{
 		inbox=new ArrayList<String>();
 	}
 	
 	public void put(String in)
 	{
 		synchronized(inbox){
 			inbox.add(in);
 		}
 	}
 	
 	public String get(int index)
 	{
 		synchronized(inbox){
 			return inbox.get(index);
 		}
 	}
 	
 	public void clear()
 	{
 		synchronized(inbox){
 			inbox.clear();
 		}
 	}
 }

tc-config.xml
Code:

 <?xml version="1.0" encoding="UTF-8"?>
 <con:tc-config xmlns:con="http://www.terracotta.org/config">
   <servers>
     <server host="%i" name="localhost">
       <dso-port>9510</dso-port>
       <jmx-port>9520</jmx-port>
       <data>terracotta/server-data</data>
       <logs>terracotta/server-logs</logs>
       <statistics>terracotta/cluster-statistics</statistics>
     </server>
   </servers>
   <clients>
     <logs>terracotta/client-logs/%D</logs>
     <statistics>terracotta/client-statistics/%D</statistics>
   </clients>
   <application>
     <dso>
       <distributed-methods>
           <method-expression>void sim.Main.dmi(String)</method-expression>
       </distributed-methods>
       <instrumented-classes>
         <include>
           <!--include all classes in the example package for bytecode instrumentation-->
           <class-expression>sim..*</class-expression>
         </include>
       </instrumented-classes>
       <locks>
         <autolock>
           <method-expression>void *..*(..)</method-expression>
           <lock-level>write</lock-level>
         </autolock>
       </locks>
       <roots>
         <root><field-name>sim.Roots.instance</field-name>
         </root>
       </roots>
     </dso>
   </application>
 </con:tc-config>

I am trying to make my own example of a dmi and I'm failing terribly.

Here is the code that was thrown together. One node calls the distributed method but the other node does not get the call and hangs at 'waiting for dmi'. This code is meant to be run on two nodes at the moment so run it with 2 as an argument.

Thanks for your help.

Main.java
Code:

 package sim;
 
 import java.util.concurrent.BrokenBarrierException;
 import java.util.concurrent.CyclicBarrier;
 
 
 public class Main implements Runnable {
 
 	private final CyclicBarrier barrier;
 	final int participants;
 	private int arrival = -1;
 	private Agent[] agentlist;
 	
 
 	public Main(final CyclicBarrier barrierIN, Agent[] agentlistIN, int threads)
 	{
 		agentlist=agentlistIN;
 		barrier=barrierIN;
 		participants=threads;
 	}
 	
 	public void run()
 	{
 		System.out.println("Waiting...");
 		try {
 			arrival=barrier.await();
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		} catch (BrokenBarrierException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 		System.out.println("Arrived:"+arrival);
 		
 		agentlist[arrival].put("String "+arrival);
 		
 		try {
 			barrier.await();
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		} catch (BrokenBarrierException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 		
 		System.out.println(agentlist[0].get(0));
 		System.out.println(agentlist[1].get(0));
 		
 		if(arrival==0)waithere();
 		
 		System.out.println("SLeep");
 		try {
 			Thread.currentThread();
 			Thread.sleep(1000);
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 		System.out.println("End SLeep, call dmi");
 		dmi("HelloWorld");
 	}
 	
 	public static void dmi(String msg)
 	{
 		System.out.println(msg);
 	}
 	
 	public void waithere()
 	{
 		System.out.println("Waiting for dmi:");
 		try {
 			Thread.currentThread().join();
 		} catch (InterruptedException e) {
 			// TODO Auto-generated catch block
 			e.printStackTrace();
 		}
 	}
 	
 	public static void main(String[] args) throws Exception
 	{
 		Roots roots=new Roots(new CyclicBarrier(Integer.parseInt(args[0])), Integer.parseInt(args[0]));
 		
 		new Main(roots.getBarrier(),roots.getAgents(), roots.getTheads()) 
 		.run();
 	}
 }

Roots.java
Code:

 package sim;
 
 
 import java.util.concurrent.CyclicBarrier;
 
 public class Roots {
 	private final CyclicBarrier barrier;
 	private final Agent[] agentlist;
 	private int threads;
 	
 	public Roots(CyclicBarrier barrierIN, int numOfAgents) {
 		barrier = barrierIN;
 		threads=numOfAgents;
 		agentlist=new Agent[numOfAgents];
 		synchronized(agentlist){
 		for(int i=0;i<numOfAgents;i++) agentlist[i]=new Agent();
 		}
 	}
 	
 	public CyclicBarrier getBarrier() {
 		return barrier;
 	}
 
 	public Agent[] getAgents() {
 		return agentlist;
 	}
 	
 	public int getTheads(){
 		return threads;
 	}
 	
 	public static void print()
 	{
 		System.out.println("HI");
 	}
 }

Agent.java
Code:

 package sim;
 
 import java.util.ArrayList;
 
 public class Agent {
 	
 	public ArrayList<String> inbox;
 	
 	public Agent()
 	{
 		inbox=new ArrayList<String>();
 	}
 	
 	public void put(String in)
 	{
 		synchronized(inbox){
 			inbox.add(in);
 		}
 	}
 	
 	public String get(int index)
 	{
 		synchronized(inbox){
 			return inbox.get(index);
 		}
 	}
 	
 	public void clear()
 	{
 		synchronized(inbox){
 			inbox.clear();
 		}
 	}
 }

tc-config.xml
Code:

 <?xml version="1.0" encoding="UTF-8"?>
 <con:tc-config xmlns:con="http://www.terracotta.org/config">
   <servers>
     <server host="%i" name="localhost">
       <dso-port>9510</dso-port>
       <jmx-port>9520</jmx-port>
       <data>terracotta/server-data</data>
       <logs>terracotta/server-logs</logs>
       <statistics>terracotta/cluster-statistics</statistics>
     </server>
   </servers>
   <clients>
     <logs>terracotta/client-logs/%D</logs>
     <statistics>terracotta/client-statistics/%D</statistics>
   </clients>
   <application>
     <dso>
       <distributed-methods>
           <method-expression>void sim.Main.dmi(String)</method-expression>
       </distributed-methods>
       <instrumented-classes>
         <include>
           <!--include all classes in the example package for bytecode instrumentation-->
           <class-expression>sim..*</class-expression>
         </include>
       </instrumented-classes>
       <locks>
         <autolock>
           <method-expression>void *..*(..)</method-expression>
           <lock-level>write</lock-level>
         </autolock>
       </locks>
       <roots>
         <root>
           <field-name>sim.Roots.barrier</field-name>
         </root>
         <root>
           <field-name>sim.Roots.agentlist</field-name>
         </root>
       </roots>
     </dso>
   </application>
 </con:tc-config>

Thanks for the links. I editted and played around with the dmi project and found a few questions.

What is the difference between join(millisec) and sleep(millisec) ? Also please answer the question in a comment below.

Thanks so much!

-Quincy

Code:

 import java.util.concurrent.CyclicBarrier;
 
 public class Main
 {
     public static Main instance = new Main ();
 	private final CyclicBarrier barrier = new CyclicBarrier(2);
 
     public void receive() throws InterruptedException
     {
         System.out.println("Waiting for dmi method calls...");
 
         /*
          * Is there a condition to test if a DMI is called?
          */
          for(int i=0;i<10;i++){
         	
         	System.out.println("Sleep.");
 // there can be a Hello printed inbetween Sleep and End Sleep
         	Thread.currentThread().sleep(10000);
         	System.out.println("End Sleep.");
         	Thread.currentThread().join(1000);
         }
         
     }
 
     public void send()
     {
         print("HELLO");
     }
  
     // this method gets called on the originating node from the send() method, and, 
     // since it is marked as dmi, that call is packaged up and executed on all
     // all other nodes that have this shared object resident 
     public void print(String message)
     {
         System.out.println(message);
         
     }
 
     public void run() throws Exception
     {
     	int arrival=barrier.await();
         if (arrival==0) receive();
         else send();
         
     }
 
     public static void main(String[] args) throws Exception
     {
         instance.run();
     }
 }

Thanks, this thread is helping a lot. I have one question from your response,

The evictor starts to evict clustered memory when overall heap utilization reaches a threshold which is configurable.

Where can I configure this threshold?

Next topics we kinda already discussed a bit. Distributed Garbage Collection seems pretty straightforward from the guide's description. Locks and Transactions also seem straightforward with the knowledge acquired above. Locks/Transactions just have some syntax that could easily be forgotten unless seen in the gotchas.

One important thought is to remember that terracotta does not require the same program to be run on a root. Client A running program A can use a root that client B running program B is also using. So locking and sycro methods are on objects used in certain lines of code not on the lines of code themselves. I think this is a very flexible attribute of terracotta that makes it more usable than other parallel programming concepts.

Distributed Method Invocation (DMI)
Editted: I deleted this paragraph because I want to keep the thread as correct as possible.

Am I understanding the material? Thanks for your consistent input and replies.

Quincy

Thanks again for your quick replies. On to the next topic.

==========Copy from source==========================
Virtual Heap

Terracotta virtual heap allows arbitrarily large clustered object graphs to fit within a constrained local heap size in client virtual machines. The local heap has a "window" on a clustered object graph. Portions of a clustered object graph are faulted in and paged out as needed. Virtual heap is similar in concept to virtual memory and is sometimes referred to as "Network Attached Memory."

Clustered objects are lazily loaded from the server as they are accessed by a client JVM. This happens by injecting augmented behavior around the GETFIELD bytecode instruction that checks to see if the object referred to by that field is currently instantiated on the local heap. If it isn't on the local heap yet, the client will request the object from the server, instantiate the object on the local heap, and ensure that the field refers to that newly instantiated object.

Conversely, less frequently used objects may be transparently purged from local heap by Terracotta, subject to the whims of the local JVM garbage collector. The amount of clustered object data kept in local heap is determined at runtime. Terracotta actively sets references to objects that have fallen out of the cache to null so that they may become eligible for local garbage collection. Because clustered objects may be lazily loaded, purged objects will be transparently retrieved from the server as references to them are traversed.
=========================================

Again, I'll parrot and please correct me if I'm wrong in anything.

This is a really cool idea. Though like a double edged sword there are ways it could be used for bad. In memory intensive applications the trick is to have the client hold onto the least about of huge objects as possible so there are a low amount of transfers from server to client.

Let me give an example that could result in a very slow program. Lets say we apply an operation to N amount of trees to create a new tree. One bad scenario would be where only one tree can fit on a client and the operation was the following:

Step 0: start at the root of tree 0.
Step 1: read the data of current node to determine which node in which tree to go to next.
Step 2: go to that next node. then go to step 1.

At each step the client would need to purge all its memory and reload all its memory. If each tree was very big, then the run time would be ridiculously long.

A question that I have is how exactly does terracotta decide this: "The amount of clustered object data kept in local heap is determined at runtime." Is it possible to have a client say "out of memory" if it only uses clustered memory and never adds to the clustered memory?

Thank you again for your time,
Quincy

First and foremost the reference:

http://www.terracotta.org/confluence/display/docs1/Concept+and+Architecture+Guide#ConceptandArchitectureGuide-roots
===============================================

Hello, my name is Quincy. I am doing some research in Agent Simulation. I am hoping Terracotta has the specs of concept and architecture that I want.

The purpose of me posting is to find out more of the simple specifics of what terracotta is and how it works. I will start off down the list of topics from the website in order. Just for a matter of simplicity I'll start with the first two.

======From site=======
Roots
A "root" is the top of a clustered object graph as declared in the Terracotta configuration. Object sharing in Terracotta starts with these declared roots. Any object that is reachable by reference from a Terracotta root becomes shared by Terracotta which means that it is given a cluster-wide object id and its changes are tracked by Terracotta and made available to the cluster.

Clustered Objects

When a root field is first assigned, the Java object assigned to that field and all object reachable by reference from that object become clustered objects. This means that each object is given a cluster-wide unique id and all its field data is added to the current Terracotta transaction. When the Terracotta transaction is committed, all of the object data is sent to the Terracotta server. All changes to clustered objects must be made within the context of a Terracotta transaction.
===================

I am going to parrot back what I understand. Please correct me.

All shared objects have a root.
No non-shared objects have a root.
Each client define roots.
When a client initializes roots, if the server doesn't have the root initialized then its initialized on the server. If any of the root's graph is changed on the client then the client tells the server to change the graph on the server in the same manner.
The clients do the program processing only. The servers are only dumby memory banks that provide a way of sharing data between clients.

Am I understanding so far? Thank you.