[Logo] Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Terracotta with Java Fork/Join  XML
Forum Index -> General
Author Message
Hamlet

neo

Joined: 08/11/2008 18:40:23
Messages: 8
Offline

I'm interested in using both Terracotta and Doug Lea's Fork/Join framework (http://jcp.org/en/jsr/detail?id=166) JSR 166.

To my sadness, I see that the Definitive Guide to Terracotta warns against using the Java 5 executor services, which the fork/join framework is written against.

I haven't looked much at the CommonJ Work Manager library, but don't relish the idea of rewriting a work-stealing and forking framework on top of it (by "don't relish" I of course mean that I'm incapable).

Does anyone have any suggestions on how to get a Fork/Join proof of concept running on top of Terracottta? My current direction is to have each JVM start up their own ExecutorService and then share the work queue somehow. Thought I'd post here before investing too much time.

Thanks in advance.
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

What page says that?

You want to have a look here:

http://www.terracotta.org/confluence/display/labs/Master+Worker

Which is the book's M/W implementation but improved since the publication of the book.

It allows for parallelization of work across nodes. That said, it currently supports only the CommonJ interface for work submission.

Sergio Bossa has been putting a lot of effort into this library, as have I. One thing that would not be too terribly hard is in fact to refactor the library to have a front-end interface of the Executor, then layer the CommonJ on top of that.

Fork/Join will be some work - it uses tail-recursion and a few other things from what I know. We had some discussions with Brian Goetz this year at Java One about it, he thinks it shouldn't be too hard - but frankly it's just been hard to get time to give it a go these days.

If none of this suits your fancy, by all means send some email to Brian to let him know there is interest in Fork/Join + Terracotta.


[WWW]
Hamlet

neo

Joined: 08/11/2008 18:40:23
Messages: 8
Offline

Page 247 "Using the java.util.concurrent Abstractions" contains a few paragraphs on why not to use them. 1) It does not seperate the master from the worker. Clustering the ExecutorService "will ot allow us to scale out the master independently of the workers but force us to run them all on every JVM". 2) ExecutorService lacks a layer of reliability and control because of its black box nature. Failing over tasks to other nodes becomes impossible.

Also, the ForkJoinExecutor does not implement java.util.concurrent.ExecutorService, it just has matching type signatures. I was mistaken, but it doesn't really matter.

I'm a beginner at Terracotta, so bear with me... from my perspective, there doesn't seem to be anything that I can reliably share/cluster within the ExecutorService. I'd like to share the ForkJoinPool/ThreadPool's task queue, but I can't reliably reach into the compiled class and configure which data structure is shared. (But maybe I should look at the source code and hopefully the synchronization policy is amenable to this).

This is all just for fun, not production code, so I'm not going to email Brian. But any advice on how you'd do this is greatly appreciated.
tgautier

seraphim

Joined: 06/05/2006 12:19:26
Messages: 1781
Offline

Have a look at the tclib framework as mentioned previously which clusters a queue and then places a ThreadPoolExecutor on the end of that clustered queue.

Yes, the book is correct on this point, one should not cluster any existing concrete implementations of ExecutorServices (such as ThreadPoolExecutor), but rather a concrete implementation of ExecutorService should be specially built for Terracotta as a front-end.
[WWW]
 
Forum Index -> General
Go to:   
Powered by JForum 2.1.7 © JForum Team