[Logo] Terracotta Discussion Forums (LEGACY READ-ONLY ARCHIVE)
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
[Expert]
Too many db connection retries during database restart  XML
Forum Index -> Quartz
Author Message
schmidma

neo

Joined: 02/11/2014 06:58:05
Messages: 1
Offline

Hi everyone,

in our application we observed a strange behaviour of quartz using the JDBC Jobstore during a database restart.
During this phase the quartz did several hundred reconnect attempts per second, although the db retry Interval was set to 15 s per default.

We debugged the Quartz code (version 2.2.1) and found out that
and exception is thrown in the method

protected <T> T executeInNonManagedTXLock(
String lockName,
TransactionCallback<T> txCallback, final TransactionValidator<T> txValidator) throws JobPersistenceException {
boolean transOwner = false;
Connection conn = null;
try {
if (lockName != null) {
// If we aren't using db locks, then delay getting DB connection
// until after acquiring the lock since it isn't needed.
if (getLockHandler().requiresConnection()) {
conn = getNonManagedTXConnection();
}

transOwner = getLockHandler().obtainLock(conn, lockName);
}

if (conn == null) {
conn = getNonManagedTXConnection(); <-- Exception is thrown
}

final T result = txCallback.execute(conn);
try {
commitConnection(conn);
} catch (JobPersistenceException e) {
rollbackConnection(conn);
if (txValidator == null || !retryExecuteInNonManagedTXLock(lockName, new TransactionCallback<Boolean>() {
@Override
public Boolean execute(Connection conn) throws JobPersistenceException {
return txValidator.validate(conn, result);
}




an exception is thrown during getNonManagedTXConnection() which causes to bypass the dbRetryInterval-Logic.

Is there any idea behind this behaviour or is it a gap?

In our case these caused heavy load on the database server and delay in the database server startup which is critical.

Many thanks and best regards,
Manfred



hostalp

neo

Joined: 07/11/2013 02:31:10
Messages: 8
Offline

Yes I can confirm the same. If the DB goes down while the Quartz scheduler instance is running, the QuartzSchedulerThread goes wild trying to get a new connection repeatedly (as fast as it can), easily able to fully utilize 1 CPU core by doing just that.

It's exactly the same situation as described by schmidma above, with the most important part of the stack trace of the affected thread being (Quartz 2.2.1):
Code:
         at org.quartz.utils.DBConnectionManager.getConnection(DBConnectionManager.java:108)
         at org.quartz.impl.jdbcjobstore.JobStoreCMT.getNonManagedTXConnection(JobStoreCMT.java:165)
         at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3784)
         at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2756)
         at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:272)
 
Forum Index -> Quartz
Go to:   
Powered by JForum 2.1.7 © JForum Team