Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

Hi all,
I'm trying to run some codes from http://svn.terracotta.org/svn/forge/projects/thebook/tags/1.2/ (chapter 11 Master Worker).
I think there are some small bugs in the source code:

+In the class org.terracotta.workmanager.DefaultWorkItem, Ecllpse tells me that the compareTo() method doesn't handle WorkException. The same thing in the class org.terracotta.workmanager.routing.RoutableWorkItem.
So I wrapped these codes in try/catch blocks.
+In the class test.Master, the method run():
Code:

	public void run() {
 ...
 		m_pipeManager.clearPipes();
 ...
 	}

There is no method clearPipes() in the class PipeManager. I think the author wanted to use the method reset() instead. So I changed it to:
Code:

	public void run() {
 ...
 		m_pipeManager.reset();
 ...
 	}

A little further, in the methods testWaitForAll(), testWaitForAny_ReturnSingleItem(), testWaitForAny_ReturnCollection(), Eclipse tells me that there are non handled InterupptedExceptions. So I wrapped these codes with try/catch blocks.

+Then in the tc-config.xml (located in the directory "test"), the root points to an non existing object:
Code:

             <root>
                <field-name>org.terracotta.workmanager.routing.PipeManager.m_pipes</field-name>
             </root>

So I changed to:
Code:

        <root>
           <field-name>org.terracotta.workmanager.pipe.PipeManager.m_pipes</field-name>
         </root>

Finally, when I run test.Master, I got this exception:
Code:

Exception in thread "main" java.lang.IllegalStateException: pipe manager has been shut down
 	at org.terracotta.workmanager.pipe.PipeManager.getPipes(PipeManager.java:89)
 	at org.terracotta.workmanager.dynamic.LoadBalancingRouter.__tc_wrapped_waitForAtLeastOnePipe(LoadBalancingRouter.java:72)
 	at org.terracotta.workmanager.dynamic.LoadBalancingRouter.waitForAtLeastOnePipe(LoadBalancingRouter.java)
 	at org.terracotta.workmanager.dynamic.LoadBalancingRouter.route(LoadBalancingRouter.java:34)
 	at org.terracotta.workmanager.dynamic.DynamicWorkManager.schedule(DynamicWorkManager.java:48)
 	at org.terracotta.workmanager.dynamic.DynamicWorkManager.schedule(DynamicWorkManager.java:56)
 	at test.Master.run(Master.java:43)
 	at test.Master.main(Master.java:100)

I can't see how to fix this.
Can somebody help me to run this example?
Thanks a lot.
Chu

Hmm, sorry that's not working for you. I know there was a working example in the source code not sure why that tag is not functional.

In any case, the book M/W code has been superceded by the tc-lib framework which incorporated the code and is wrapped up into generic set of patterns.

More detail is here:

http://www.terracotta.org/confluence/display/labs/Master+Worker

Hi Gautier,
Thanks for replying me!
I confirm that the spyder example can run after being built with maven.
However its scripts don't work for me, so I have to launch it with command line (I had to add some missing librairies):
Master:
Code:

$TC_HOME/bin/dso-java.sh 
 -classpath /home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/spider/target/classes/
 :/home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/workmanager/target/classes/
 :/home/nam/Documents/terracotta-definitive-guide-1.2/foo-commonj-1.0.1/lib/commonj-twm.jar
 :/home/nam/Documents/terracotta-definitive-guide-1.2/foo-commonj-1.0.1/lib/jericho-html-2.6.jar
 :/home/nam/Documents/terracotta-definitive-guide-1.2/foo-commonj-1.0.1/lib/commons-cli-1.1.jar
  -server -Xms64m -Xmx64m -Dtc.install-root=$TC_HOME 
 -Dtc.config=/home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/spider/tc-config.xml 
 org.terracotta.workmanager.spider.StartMaster 
 --url http://www.google.com --depth 4 --external-links false -i 1:2:3:4:5

Worker:
Code:

$TC_HOME/bin/dso-java.sh
  -classpath /home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/spider/target/classes/
 :/home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/workmanager/target/classes/
 :/home/nam/Documents/terracotta-definitive-guide-1.2/foo-commonj-1.0.1/lib/commonj-twm.jar
 :/home/nam/Documents/terracotta-definitive-guide-1.2/foo-commonj-1.0.1/lib/jericho-html-2.6.jar 
 -server -Xms64m -Xmx64m -Dtc.install-root=$TC_HOME 
 -Dtc.config=/home/nam/Documents/terracotta-definitive-guide-1.2/chapter_11_masterworker/spider/tc-config.xml 
 org.terracotta.workmanager.spider.StartWorker 1

But the master and the worker don't work. They just keep waiting. I checked the worker's pipe but there was nothing in it.
Master:
Code:

Starting Spider: 
   start url 			= http://www.google.com
   max parse depth 		= 4
   follow external links 	= false
   list with routing IDs 	= [ 1 2 3 4 5 ]
 created PipeManager for Initiator: MASTER
 DefaultQueueListener has started - serving pipe: 1
 DefaultQueueListener has started - serving pipe: 2
 DefaultQueueListener has started - serving pipe: 3
 DefaultQueueListener has started - serving pipe: 4
 DefaultQueueListener has started - serving pipe: 5
 Submitting URL: http://www.google.com
 scheduled work #: 0

Code:

Worker:
 -- starting worker...
 -- kill it with ^C when finished
 created PipeManager for Initiator: WORKER
 registering a new work queue for routing ID: 1
 Starting worker with routing ID: 1

About the tc-lib framework, I've run through your site many times and can see a lot of your efforts. I've seen a lot of master/worker examples (of course, with Terracotta enabled) on the internet and I think your examples are the closest ones to a real production system.
Regards,
Chu

Hi Chu.

I think you are running the master worker library in the book incorrectly. You need to pass it just one routing id in the master.

But please, use the tclib. It has been enhanced greatly with some nice additions by Sergio Bossa, and he has added automated system tests and javadocs.

The spider demo is there too - the instructions are on the page I already gave you. I apologize, but I cannot help you out with M/W code from the book after this message - and there is no one else that can either.

You need to switch to tclib - then I can help, and Sergio can help, and many others at Terracotta can help. As I said before, tclib is very similar to the book code, but much more mature, well tested, documented, and supported.

Thanks.