Terracotta Discussion Forums

my ehcache.xml has the following line.

<terracottaConfig url="localhost:9510" rejoin="true" />

This allows us to use the Terracotta Server to replicate cache between a couple tomcat nodes.

However im trying to implement a failover to local.

During startup if the TC server is not reachable, TerracottaClient just continues to retry indefinetely.

We couldn't load configuration data from the server at 'localhost:9510'; retrying. (Error: Connection refused: connect.)...
We couldn't load configuration data from the server at 'localhost:9510'; retrying. (Error: Connection refused: connect.)
We couldn't load configuration data from the server at 'localhost:9510'; retrying. (Error: Connection refused: connect.)
........

No exception that I can catch is ever thrown, so I can never get it to switch to some alternate configuration.

What is the correct way to limit the number of retries? or limit the time? And raise an exception that could be handled by my code?

There is a rather blunt way to control this aspect right now.

-Dcom.tc.tc.config.total.timeout=5000

This property limits the time (in milliseconds) that a client spends attempting to make an initial connection.

See this page:
http://terracotta.org/documentation/terracotta-server-array/high-availability

I believe you'll receive an exception when this timeout is exceeded (although the runtime type of that exception is likely an internal terracotta class). I also think there is a bug of sorts with this property in that the timeout is effectively rounded up to nearest 5 seconds.

Another workaround/hack would be start your CacheManager in a separate thread and implement your own timeout logic waiting on that thread

Thanks, this does indeed seem to raise a catchable exception.

net.sf.ehcache.CacheException: Unable to load class net.sf.ehcache.terracotta.StandaloneTerracottaClusteredInstanceFactory. Initial cause was java.lang.RuntimeException: com.tc.config.schema.setup.ConfigurationSetupException:
*******************************************************************************
Could not fetch configuration data from the server at 'localhost:9510'. Fetch attempt duration: 6 seconds.

To correct this problem specify a valid configuration location using the -f/--config command-line options.
*******************************************************************************

Havent tested it all the way yet, but catching an exception is a good sign.

P.S. I also tried the property l1.max.connect.retries=5, which appears in that same link, and would appear to do what I needed. However it had no effect. Possibly due to https://jira.terracotta.org/jira/browse/CDV-1542 ?

The l1.max.connect.retries is taken into effect only if the client can connect to the cluster...and since in your case, you can't...that's why it has no effect in this case.

From that same link:
"The property l1.max.connect.retries controls connection attempts only after the client has obtained and resolved its configuration from the server."

Thanks everyone for your help, -Dcom.tc.tc.config.total.timeout=5000, solved my problem.

I did have to do a tiny bit of customization to get it to failover nicely for hibernate as well. Some pseudocode below.

Basically try with standard config, catch an exception, set the standard config property to some secondary value (local ehcache in mycase) and try again.

Code:

 public class CustomEhCacheRegionFactory extends EhCacheRegionFactory {
    String CUSTOM_EHCACHE_CONFIGURATION_RESOURCE_NAME ="some.secondary.property.to.define.in.hibernate.cfg.xml;
 
   @Override
   public void start(Settings settings, Properties properties) throws CacheException {
 			
   try{
       super.start(settings, properties);
   }catch(Exception ex){
       properties.setProperty(NET_SF_EHCACHE_CONFIGURATION_RESOURCE_NAME, properties.getProperty(CUSTOM_EHCACHE_CONFIGURATION_RESOURCE_NAME));
        super.start(settings, properties);
   }
 }