Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

I would like to use disk persistence using Ehcache (2.5.1). I have done the following so far -

1. set diskPersistent=true on each cache I want to persist
2. enabled shutdown hook

Using eclipse, I am able to test that when I exit the process gracefully, persistence mechanism creates .index (and .data) files correctly. The next restart of the VM then loads the cache (I have configured net.sf.ehcache.store.DiskStoreBootstrapCacheLoaderFactory).

1. However, when I terminate the process, the .index files are not generatd and next restart of the VM doesn't load cache as expected. I would have thought enabling shutdown hook would take care in this situation?

2. Since #1 didn't work, I scheduled a task that flushes all caches to disk every 1 min. With this, I am able to see all .index files. Even then, when the process is terminated, the index files are corrupted and the next restart of VM throws this error -

Warning net/sf/ehcache/store/disk/DiskStorageFactory The index for data file C:\temp\ehcache\ValueClass.data is out of date, probably due to an unclean shutdown. Deleting index file C:\temp\ehcache\ValueClass.index (noclass.)

Is there anything that I could do to *guarantee* that files are always persisted correctly and cache loaded on next VM restart?

When you say "terminate the process" what exactly do you mean? Depending on how you are achieving your termination the JVM may not be running the shutdown hooks: http://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)

The disk-persistence provided by Ehcache is not designed to be fault-tolerant. It relies on the cache-manager having been shut down cleanly in order to work (in your case using a shutdown hook). The warning you are seeing on startup indicates that an unclean shutdown was detected and therefore the on-disk data is being deleted.

In short the only way you can "guarantee" persistence is by ensuring that you always shut the cache-manager down cleanly.

Hope this clarifies things for you,

Chris

By 'terminate' I mean, in Eclipse IDE menu 'Run' -> 'Terminate' operation. I am not 100% whether it terminates or aborts the JVM, but it appears that shutdown hook is not sufficient in this case.

According to your post, the only way to ensure correct disk-persistence between VM restarts is to shut cache manager down cleanly.

Is there anything that app code could do to periodically "snapshot" .index files, instead of waiting until graceful cache manager shutdown?

I believe (although I haven't checked this) that Eclipse uses the Process.destroy() call to terminate spawned processes. From what I remember on *nix platforms this maps to SIGKILL, so you're not going to get the shutdown hooks run in that situation.

In terms of snapshotting .index files, that wouldn't be enough - you'll also need to snapshot the .data files too. What you could do in principle is the following (this is just a proposal not a recommendation - I really don't think this is a good idea):

1. Suspend all of your cache operations (presumably using some kind of external locking or thread co-ordination).
2. Request a flush.
3. Copy the .index and .data files for the cache to a safe place.
4. Resume your cache operations.

On recovery you can then copy the backed up files over the dirty ones, and then you should be able to restore the cache state. You may have to be careful with file timestamps here though since the startup code will check them to try and assert whether the files are 'clean' or not.

Again I really don't think this is a good idea, your better solution is to arrange for a more graceful termination of your code.

Regards,

Chris

You should check out the new Gladstone Tech Preview available at: http://terracotta.org/gladstonetechpreview Among other new features, this preview introduces a new crash resilient store for Standalone Ehcache. This supports full fault tolerance using new fast Restart capability and continuous accessibility of previously cached data after a planned or unplanned shutdown.

It will be great if you can try it out and send feedback to pm@terracottatech.com

thanks
Terracotta Product Mgr