Terracotta Discussion Forums

In a previous thread Ari elaborated on the nature of distributed Ehcache via terracotta. It is clear in that case it is between 100% distributed and 100% replicated.

What about Ehcache replication via RMI and JGroups? And when the Ehcache server is in the mix as well?

I did a quick test. I have a 2-node distributed Ehcache set up using RMI. I call the Ehcache server to write to node 1, and then read from node 2. If I read immediately after write, I get a FileNotFoundException. This leads me to think that:

1. The Ehcache server only reads from local cache
2. Cached elements are replicated among nodes. So eventually all nodes will have all elements

To me, this is not very efficient, as I want the cache to be as close to 100% distributed as possible.

Greg or Ari - can I get some pointers from you guys on how to improve the situation?

Thanks,

Eric

Reading through the Ehcache Cache Server doc, maybe I should use the client hash-based routing approach when writing/reading to maximize the chance of local cache hits and avoid data replication across nodes, or the data is automatically replicated no matter what?

Eric

If you want to do a read straight after a write to any node in the Cache Server cluster, then I recommend using 1.7.0 with Terracotta for coherent clustering. You set this up in the Cache Server's ehcache.xml as per a normal ehcache.xml configuration. See http://ehcache.org/documentation/distributed_caching_with_terracotta.html

Thanks Greg. My primary concern at this moment is not really the coherence of data, as I don't have a sub-second transaction system, but rather how to maximize the amount of data I can cache when I use the RESTful cache server. The doc tells me:

"The RESTful Ehcache Server is designed to achieve massive scaling using data partitioning - all from a RESTful interface. The largest ehcache single instances run at around 20GB in memory. The largest disk stores run at 100Gb each. Add nodes together, with cache data partitioned across them, to get larger sizes. 50 nodes at 20GB gets you to 1 Terabyte."

I am very excited about the potential, but want to understand how that's achievable, if I only hit a single cache server in the grid. Want to confirm I need to build partition logic for write/read.

Yes, with consistent hashing. You can do it in the load balancer or in the Java client.

Some example Java code:

String[] cacheservers = new String[]{"cacheserver0.company.com", "cacheserver1.company.com",
"cacheserver2.company.com", "cacheserver3.company.com", "cacheserver4.company.com",
"cacheserver5.company.com"};
Object key = "123231";
int hash = Math.abs(key.hashCode());
int cacheserverIndex = hash % cacheservers.length;
String cacheserver = cacheservers[cacheserverIndex];

See the Creating Massive Caches with Load Balancers and Partitioning section in http://ehcache.org/documentation/cache_server.html

The issue seems to be resolved.Please let know in case of more information.