Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

I have long running jobs which get randomly scheduled every day. If a job is running when i kill Tomcat what happens to its state within the scheduler? Will it restart when the scheduler is restarted or is it lost since it was already fired?

nope. it does not seem to adversely affect anything, however it is disconcerting to see. I am guessing it has to do with batch acquiring triggers not being handled properly.

here is the configuration. Yes it is inside my war so it is the same on both Jboss servers. I am using SimpleTrigger

Code:

 TriggerBuilder<Trigger> triggerBuilder = TriggerBuilder.newTrigger();
 			triggerBuilder.withIdentity( job.getKey().getName(), "MyBuilderJob.triggers" );
 			triggerBuilder.startAt( new Date( System.currentTimeMillis() + (1000 * 10) ) );
 			triggerBuilder.withSchedule( SimpleScheduleBuilder.simpleSchedule().withMisfireHandlingInstructionFireNow() );

Code:

 #============================================================================
 # Configure Main Scheduler Properties  
 #============================================================================
 
 org.quartz.scheduler.instanceName: myScheduler
 org.quartz.scheduler.instanceId: AUTO
 org.quartz.scheduler.skipUpdateCheck: true
 
 #============================================================================
 # Configure ThreadPool  
 #============================================================================
 
 org.quartz.threadPool.class: org.quartz.simpl.SimpleThreadPool
 org.quartz.threadPool.threadCount: 100
 org.quartz.threadPool.threadPriority: 5
 
 #============================================================================
 # Configure JobStore  
 #============================================================================
 
 org.quartz.jobStore.misfireThreshold: 60000
 
 org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreCMT
 org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
 org.quartz.jobStore.useProperties = false
 org.quartz.jobStore.dataSource = quartzXA
 org.quartz.jobStore.tablePrefix = QSMS_
 org.quartz.jobStore.isClustered = true
 org.quartz.jobStore.nonManagedTXDataSource = quartzNonXA
 org.quartz.jobStore.dontSetAutoCommitFalse = true
 
 org.quartz.scheduler.batchTriggerAcquisitionMaxCount = 40
 org.quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow = 5000
 org.quartz.jobStore.acquireTriggersWithinLock = true
 
 
 #============================================================================
 # Configure Datasources  
 #============================================================================
 org.quartz.dataSource.quartzXA.jndiURL=java:/comp/env/jdbc/my-xa
 org.quartz.dataSource.quartzNonXA.jndiURL=java:/comp/env/jdbc/my-nonXa
 
 
 #============================================================================
 # Configure Plugins 
 #============================================================================
 
 #org.quartz.plugin.shutdownHook.class: org.quartz.plugins.management.ShutdownHookPlugin
 #org.quartz.plugin.shutdownHook.cleanShutdown: true
 #org.quartz.plugin.triggHistory.class: org.quartz.plugins.history.LoggingJobHistoryPlugin
 
 
 org.quartz.plugin.jobInitializer.class = org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin
 org.quartz.plugin.jobInitializer.fileNames = my-quartz.xml
 org.quartz.plugin.jobInitializer.failOnFileNotFound = true
 org.quartz.plugin.jobInitializer.scanInterval = 0
 org.quartz.plugin.jobInitializer.wrapInUserTransaction = false

We have Quartz clustered on two servers with Oracle RAC on the backend. I have about 120 or so jobs scheduled at any one time basically a rolling set of jobs with one being created every 10 seconds and repeating once every 5 minutes for 20 minutes.

I am using the Batch Aquire of triggers and use the aquire triggers in lock. Periodically we get:

Code:

 [o.q.c.ErrorLogger] - An error occurred while scanning for the next triggers to fire.
 org.quartz.JobPersistenceException: Couldn't acquire next trigger: Couldn't retrieve trigger: No record found for selection of Trigger with key: 'DEFAULT.6da64b5bd2ee-894279e1-a6bd-4151-bb9d-77acf5647ade' and statement: SELECT * FROM QSMS_SIMPLE_TRIGGERS WHERE SCHED_NAME = 'xxx' AND TRIGGER_NAME = ? AND TRIGGER_GROUP = ?
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2840) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport$40.execute(JobStoreSupport.java:2746) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3811) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2742) ~[quartz-2.1.5.jar:na]
 	at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:264) ~[quartz-2.1.5.jar:na]
 Caused by: org.quartz.JobPersistenceException: Couldn't retrieve trigger: No record found for selection of Trigger with key: 'DEFAULT.6da64b5bd2ee-894279e1-a6bd-4151-bb9d-77acf5647ade' and statement: SELECT * FROM QSMS_SIMPLE_TRIGGERS WHERE SCHED_NAME = 'xxx' AND TRIGGER_NAME = ? AND TRIGGER_GROUP = ?
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1524) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2790) ~[quartz-2.1.5.jar:na]
 	... 4 common frames omitted
 Caused by: java.lang.IllegalStateException: No record found for selection of Trigger with key: 'DEFAULT.6da64b5bd2ee-894279e1-a6bd-4151-bb9d-77acf5647ade' and statement: SELECT * FROM QSMS_SIMPLE_TRIGGERS WHERE SCHED_NAME = 'xxx' AND TRIGGER_NAME = ? AND TRIGGER_GROUP = ?
 	at org.quartz.impl.jdbcjobstore.SimpleTriggerPersistenceDelegate.loadExtendedTriggerProperties(SimpleTriggerPersistenceDelegate.java:95) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTrigger(StdJDBCDelegate.java:1801) ~[quartz-2.1.5.jar:na]
 	at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1520) ~[quartz-2.1.5.jar:na]
 	... 5 common frames omitted

if there are a handful of jobs running we never see this. only when we got up to around 60 or so jobs in the sheduler.

light5 wrote:

Tomcat7 does not provide a transaction manager by default, thus you won't have UserTransaction available. Quartz can work with a standalone database local transaction instead of a global tx manager. In the quartz config file try use a JobStoreTx intead of JobStoreCMT. And change other related properties related to it.

Yeah, I did not realize Tomcat does not include a JTA provider. I will have to think of a workaround for not having JTA and using the JobStoreTx. Thanks for the reply.

I am porting my application out of Weblogic 10.3 to Tomcat 7. I have gotten everything working up to where Quartz is trying to start a transaction. I added the javax.transaction.jta dependency to my maven build but quartz still fails with a:

Code:

 2011-06-07 14:00:46.555 [ERROR] [Thread-3] [o.q.p.x.XMLSchedulingDataProcessorPlugin] - Failed to start UserTransaction for plugin: jobInitializer
 org.quartz.SchedulerException: UserTransactionHelper could not lookup/create UserTransaction.
 	at org.quartz.ee.jta.UserTransactionHelper$UserTransactionWithContext.<init>(UserTransactionHelper.java:150) ~[quartz-1.8.5.jar:na]
 	at org.quartz.ee.jta.UserTransactionHelper.lookupUserTransaction(UserTransactionHelper.java:110) ~[quartz-1.8.5.jar:na]
 	at org.quartz.plugins.SchedulerPluginWithUserTransactionSupport.startUserTransaction(SchedulerPluginWithUserTransactionSupport.java:175) [quartz-1.8.5.jar:na]
 	at org.quartz.plugins.SchedulerPluginWithUserTransactionSupport.start(SchedulerPluginWithUserTransactionSupport.java:142) [quartz-1.8.5.jar:na]
 	at org.quartz.core.QuartzScheduler.startPlugins(QuartzScheduler.java:2322) [quartz-1.8.5.jar:na]
 	at org.quartz.core.QuartzScheduler.start(QuartzScheduler.java:495) [quartz-1.8.5.jar:na]
 	at org.quartz.impl.StdScheduler.start(StdScheduler.java:143) [quartz-1.8.5.jar:na]
 	at org.quartz.ee.servlet.QuartzInitializerServlet.init(QuartzInitializerServlet.java:184) [quartz-1.8.5.jar:na]
 	at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1189) [catalina.jar:7.0.11]
 	at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1103) [catalina.jar:7.0.11]
 	at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1010) [catalina.jar:7.0.11]
 	at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4915) [catalina.jar:7.0.11]
 	at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5242) [catalina.jar:7.0.11]
 	at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5237) [catalina.jar:7.0.11]
 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [na:1.6.0_23]
 	at java.util.concurrent.FutureTask.run(FutureTask.java:138) [na:1.6.0_23]
 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_23]
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_23]
 	at java.lang.Thread.run(Thread.java:662) [na:1.6.0_23]
 Caused by: javax.naming.NamingException: Cannot create resource instance
 	at org.apache.naming.factory.TransactionFactory.getObjectInstance(TransactionFactory.java:116) ~[catalina.jar:7.0.11]
 	at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:304) ~[na:1.6.0_23]
 	at org.apache.naming.NamingContext.lookup(NamingContext.java:826) ~[catalina.jar:7.0.11]
 	at org.apache.naming.NamingContext.lookup(NamingContext.java:145) ~[catalina.jar:7.0.11]
 	at org.apache.naming.NamingContext.lookup(NamingContext.java:814) ~[catalina.jar:7.0.11]
 	at org.apache.naming.NamingContext.lookup(NamingContext.java:159) ~[catalina.jar:7.0.11]
 	at org.apache.naming.SelectorContext.lookup(SelectorContext.java:158) ~[catalina.jar:7.0.11]
 	at javax.naming.InitialContext.lookup(InitialContext.java:392) ~[na:1.6.0_23]
 	at org.quartz.ee.jta.UserTransactionHelper$UserTransactionWithContext.<init>(UserTransactionHelper.java:147) ~[quartz-1.8.5.jar:na]
 	... 18 common frames omitted

Yes, my jobs are stateful. Does that mean the next trigger time is not until after the running job finishes?

Job 123 is setup to run every minute

0:00 Job 123 starts, Node #1
0:35 Job 123 finishes, Node #1 (35 second run, everything is fine)
1:00 Job 123 starts, Node #3
2:00 Job 123 DOES NOT start on Node #1 or #2 because it is still running. this trigger time gets skipped?
2:08 Job 123 finishes, node #3 (68 second run so overlaps the trigger time)
3:00 Job 123 starts on one of the nodes. the 2:00 got skipped?

Say i have a cluster of three scheduler instances, and i have a job which fires every minute. If said job takes over a minute to run, will another instance overlap?
For example....

Job 123 is setup to run every minute

0:00 Job 123 starts, Node #1
0:35 Job 123 finishes, Node #1 (35 second run, everything is fine)
1:00 Job 123 starts, Node #3
2:00 does Job 123 start Node #1 or #2?
2:08 Job 123 finishes, node #3 (68 second run so overlaps the trigger time)

Hi. I am trying to get quartz clustering to work in Weblogic 10.3 with the Oracle Thin Driver.

I am using the

org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.oracle.weblogic.WebLogicOracleDelegate

And i have tried the
#org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.oracle.OracleDelegate

I see the scheuler start in the log, and it reads my configuration, however it appears that the database is being locked. When i kill the weblogic server, i see my job in the database, but not when the server is up.

I have changed to the Weblogic Oracle driver and that seems to work fine, however i can not change the whole project over to use the Weblogic Driver at this point as it is too risky.