Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)

Hello,
I had a look in your forum without finding anything really relate to my question so I put a new topic.

This is my configuration:
AS: JBOSS 4.2.2 GA
JAVA: 1.5.17 and 1.6.13 (the problem raise in both environments)
Quarts 1.6 with persistence (Oracle 10)
Hibernate 3

In an environment that is working I get:
....
2010-01-21 11:20:14,120 DEBUG [org.quartz.simpl.SimpleJobFactory] Producing instance of Job 's-969480259.MIU0@tp-test-sb001_MIU1@tp-test-sb001s-969480259', class=it.oneans.iemx.audit.scheduler.AuditJob
2010-01-21 11:20:14,138 INFO [it.oneans.iemx.audit.scheduler.SchedulerJob] metodo init - inizio metodo
2010-01-21 11:20:14,141 INFO [it.oneans.iemx.audit.scheduler.SchedulerJob] metodo init - fine metodo
2010-01-21 11:20:14,141 DEBUG [org.quartz.ee.jta.JTAJobRunShell] Looking up UserTransaction.
2010-01-21 11:20:14,142 DEBUG [org.quartz.ee.jta.JTAJobRunShell] Beginning UserTransaction.
2010-01-21 11:20:14,142 DEBUG [org.quartz.ee.jta.JTAJobRunShell] Calling execute on job

Unfortunately I have also two not working environments where the log stops without any warning/errors before
DEBUG [org.quartz.simpl.SimpleJobFactory] Producing instance of Job 's-969480259.MIU0@tp-test-sb001_MIU1@tp-test-sb001s-969480259', class=it.oneans.iemx.audit.scheduler.AuditJob:

...
2010-01-21 18:02:03,200 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is desired by: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:03,200 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is being obtained: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:03,204 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' given to: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:03,210 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' returned by: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:06,971 DEBUG [edu.umich.auth.cosign.CosignConfig] MonitoringThread woke up: checking config file for modification
2010-01-21 18:02:30,151 DEBUG [org.quartz.impl.jdbcjobstore.JobStoreCMT] MisfireHandler: scanning for misfires...
2010-01-21 18:02:30,152 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is desired by: QuartzScheduler_AuditSchedulerSingleton-1_MisfireHandler
2010-01-21 18:02:30,152 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is being obtained: QuartzScheduler_AuditSchedulerSingleton-1_MisfireHandler
2010-01-21 18:02:30,155 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' given to: QuartzScheduler_AuditSchedulerSingleton-1_MisfireHandler
2010-01-21 18:02:30,157 DEBUG [org.quartz.impl.jdbcjobstore.JobStoreCMT] Found 0 triggers that missed their scheduled fire-time.
2010-01-21 18:02:30,161 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' returned by: QuartzScheduler_AuditSchedulerSingleton-1_MisfireHandler
2010-01-21 18:02:30,161 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is desired by: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:30,161 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' is being obtained: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:30,165 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' given to: AuditSchedulerSingleton_QuartzSchedulerThread
2010-01-21 18:02:30,170 DEBUG [org.quartz.impl.jdbcjobstore.StdRowLockSemaphore] Lock 'TRIGGER_ACCESS' returned by: AuditSchedulerSingleton_QuartzSchedulerThread
...

I suppose the job is created and stored in the DB (I can see it in the QRTZ_FIRED_TRIGGERS table) with several different states (executing is the most common), but it is never fired (sometimes i cant see the job in the db).

I suspect the problem can be something in quarz configuration (maybe related to jboss).

Restarting the AS (also cleaning the tem/data/work and cleaning the quartz table in DB) does not produce any difference.

Any Ideas?

Best regards
stmod

Attached is a longer portion of the log and a dump of running threads

here is my auditjob.properties:

# Default Properties file for use by StdSchedulerFactory
# to create a Quartz Scheduler Instance, if a different
# properties file is not explicitly specified.
#

org.quartz.scheduler.instanceName = AuditSchedulerSingleton
org.quartz.scheduler.instanceId = 1
#org.quartz.scheduler.rmi.export = false
#org.quartz.scheduler.rmi.proxy = false

org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.threadPool.threadPriority = 5
org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true

org.quartz.jobStore.misfireThreshold = 60000

#DATASOURCE

org.quartz.dataSource.myNonTXDs.driver = oracle.jdbc.driver.OracleDriver
org.quartz.dataSource.myNonTXDs.URL = jdbc:oracle:thin:@IP:1521:MYSID
org.quartz.dataSource.myNonTXDs.user = MYUSER
org.quartz.dataSource.myNonTXDs.password = MYUSERMYPWD
org.quartz.dataSource.myNonTXDs.maxConnections = 30

org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.oracle.OracleDelegate
org.quartz.jobStore.dataSource = myDS
org.quartz.jobStore.nonManagedTXDataSource = myNonTXDs
org.quartz.jobStore.tablePrefix = QRTZ_

#JNDI

org.quartz.dataSource.myDS.jndiURL = java:iEMX-CCME-ds
#org.quartz.dataSource.myNonTXDs.jndiURL = java:iEMX-CCME-ds

org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreCMT
#org.quartz.jobStore.useProperties = false
#org.quartz.jobStore.isClustered = false
#org.quartz.jobStore.dontSetAutoCommitFalse = true
#org.quartz.jobStore.txIsolationLevelSerializable = SERIALIZABLE_ISOLATION
#org.quartz.jobStore.dontSetNonManagedTXConnectionAutoCommitFalse = DONT_TURN_OFF_AUTO_COMMIT
#org.quartz.jobStore.txIsolationLevelReadCommitted = READ_COMMITTED_ISOLATION
org.quartz.scheduler.wrapJobExecutionInUserTransaction = true
org.quartz.scheduler.userTransactionURL= UserTransaction

I'm sorry, I have no idea what your problem is... especially with the lack of any error message.

I would highly suggest looking into possible issues with datasource configurations (e.g. are the right ones XA datasources, and the other ones not XA datasources), are the transaction isolation levels properly setup, etc.

Thanks for your suggestion,

the problem is this if I manually copy the entire jboss from a machine to another it sometimes work, therefore we are starting to think to something relate to active processes on the not working machine...

I think my problem is really close to this one

http://markmail.org/message/4tlwntqvgnmv63yg#query:+page:1+mid:4tlwntqvgnmv63yg+state:results

(I don't know if forum policy does not allow to post link to other forum, in case sorry)

where the problem is with a different version of quartz, but the behaviour is really similar, nut, in any case, the forum does not provide any useful answer...

stmod

Are you by chance pointing more than one instance at the very same set of quartz database tables?

Yes, this could be true.... I will check.

In case is it a problem?

Thanks

Yes, Quartz documentation says in a number of places to never do this (unless the property isClustered=true) or it will result in erratic behavior and possible data corruption.

jhouse you were right.

The problem was that the environment where the joss was running is a clustered environment with two jboss (one supposed active and the other supposed down) using the same DB.

Using the virtual address of the cluster I always saw one jboss running, but actually the running instances of the AS were two, and, probably, this caused Quartz not working.

Leaving just one instance running the things come back to work...

thanks to all.

stmod