Integrating Quartz Scheduler with Spring Boot and Kubernetes: A Practical Guide
Task scheduling is a fundamental requirement for many modern applications, from running batch jobs to triggering notifications. Quartz Scheduler stands out as a powerful, open-source Java library designed for robust job scheduling. When combined with the popular Spring Boot framework and deployed using Kubernetes, it offers a scalable and resilient solution for managing scheduled tasks.
This guide provides a practical walkthrough on setting up Quartz Scheduler within a Spring Boot application, configuring it for job persistence using PostgreSQL, and deploying it as a clustered service on Kubernetes. Key features covered include job scheduling, execution, persistence, and clustering.
Prerequisites
Before starting, ensure the following tools and dependencies are available:
- Java Development Kit (JDK) 17 or later
- Gradle (Groovy DSL) or Maven for dependency management
- Spring Boot 3.x
- Quartz Scheduler library
- PostgreSQL Driver (for job persistence)
- Lombok (optional, for reducing boilerplate code)
- Docker (for containerization)
- Kubernetes cluster (Minikube, Kind, or a cloud provider’s K8s service)
Project Setup
Begin by creating a standard Spring Boot application using your preferred method, such as the Spring Initializr (https://start.spring.io/). Include the necessary dependencies.
Here’s an example build.gradle file with the required dependencies:
plugins {
id 'java'
id 'org.springframework.boot' version '3.4.4' // Use a relevant Spring Boot version
id 'io.spring.dependency-management' version '1.1.7' // Use a relevant version
}
group = 'com.example.quartz'
version = '0.0.1-SNAPSHOT'
java {
toolchain {
languageVersion = JavaLanguageVersion.of(17)
}
}
configurations {
compileOnly {
extendsFrom annotationProcessor
}
}
repositories {
mavenCentral()
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-quartz'
implementation 'org.springframework.boot:spring-boot-starter-jdbc' // Needed for JDBC JobStore
compileOnly 'org.projectlombok:lombok'
runtimeOnly 'org.postgresql:postgresql'
annotationProcessor 'org.projectlombok:lombok'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
tasks.named('test') {
useJUnitPlatform()
}
Implementing Quartz Jobs
Quartz Jobs are simple Java classes that implement the org.quartz.Job interface. This interface has a single method, execute(JobExecutionContext context), which contains the logic to be performed when the job triggers.
Let’s create two sample jobs: one running every minute and another every 30 seconds.
Key Annotations:
@DisallowConcurrentExecution: Prevents multiple instances of the same job definition from running concurrently. If a job instance is already running, the next trigger will wait until the current one finishes.@PersistJobDataAfterExecution: Ensures that any changes made to the JobDataMap within theexecutemethod are persisted back to the JobStore. This is useful for maintaining state between job runs.
OneMinuteJob.java:
package com.example.quartz.scheduled;
import lombok.extern.slf4j.Slf4j;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.quartz.PersistJobDataAfterExecution;
import java.time.OffsetDateTime;
@Slf4j
@DisallowConcurrentExecution
@PersistJobDataAfterExecution
public class OneMinuteJob implements Job {
@Override
public void execute(JobExecutionContext context) throws JobExecutionException {
log.info("Executing OneMinuteJob at {}", OffsetDateTime.now().toInstant());
// Add job logic here
}
}
ThirtySecondJob.java:
package com.example.quartz.scheduled;
import lombok.extern.slf4j.Slf4j;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.quartz.PersistJobDataAfterExecution;
import java.time.OffsetDateTime;
@Slf4j
@DisallowConcurrentExecution
@PersistJobDataAfterExecution
public class ThirtySecondJob implements Job {
@Override
public void execute(JobExecutionContext context) throws JobExecutionException {
log.info("Executing ThirtySecondJob at {}", OffsetDateTime.now().toInstant());
// Add job logic here
}
}
Configuring Quartz in Spring Boot
To integrate Quartz seamlessly with Spring Boot and enable dependency injection within jobs, a custom JobFactory is needed.
AutowiringSpringBeanJobFactory.java:
This factory allows Spring to manage the creation and autowiring of job instances.
package com.example.quartz.config;
import org.quartz.spi.TriggerFiredBundle;
import org.springframework.beans.factory.config.AutowireCapableBeanFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.scheduling.quartz.SpringBeanJobFactory;
public final class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory implements ApplicationContextAware {
private transient AutowireCapableBeanFactory beanFactory;
@Override
public void setApplicationContext(final ApplicationContext context) {
beanFactory = context.getAutowireCapableBeanFactory();
}
@Override
protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception {
final Object job = super.createJobInstance(bundle);
// Autowire dependencies into the job instance
beanFactory.autowireBean(job);
return job;
}
}
SchedulerConfig.java:
This configuration class defines the JobDetails, Triggers, and the main SchedulerFactoryBean.
- JobDetail: Represents an instance of a Job. It includes properties like durability (whether the job should remain stored even if no triggers are associated with it).
- Trigger: Defines the schedule upon which a Job will be executed.
- SimpleTrigger: Used for one-time execution or repeated execution at a fixed interval with a specific start delay.
- CronTrigger: Used for scheduling based on cron expressions, offering more complex scheduling patterns (e.g., “every weekday at 2 AM”).
package com.example.quartz.config;
import com.example.quartz.scheduled.OneMinuteJob;
import com.example.quartz.scheduled.ThirtySecondJob;
import lombok.AllArgsConstructor;
import lombok.Data;
import org.quartz.Job;
import org.quartz.JobDetail;
import org.quartz.Trigger;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.quartz.QuartzDataSourceScriptDatabaseInitializer;
import org.springframework.boot.autoconfigure.quartz.QuartzProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.DependsOn;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.scheduling.quartz.CronTriggerFactoryBean;
import org.springframework.scheduling.quartz.JobDetailFactoryBean;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
import org.springframework.scheduling.quartz.SimpleTriggerFactoryBean;
import org.springframework.stereotype.Component;
import org.springframework.transaction.PlatformTransactionManager; // Import needed
import javax.sql.DataSource;
import java.util.List;
import java.util.Properties;
@Configuration
@AllArgsConstructor
public class SchedulerConfig {
@Component
@Data
@ConfigurationProperties("app.scheduling")
public static class SchedulerConfigProperties {
@Data
public static class ScheduledJobTimingConfig {
private String cron;
private long startDelayMillis; // Use long for consistency
private long fixedDelay; // Use long for consistency
}
private ScheduledJobTimingConfig oneMinuteJob;
private ScheduledJobTimingConfig thirtySecondJob;
}
private final SchedulerConfigProperties schedulerConfigProperties;
private final QuartzProperties quartzProperties;
private final DataSource dataSource; // Inject DataSource
private final PlatformTransactionManager transactionManager; // Inject TransactionManager
// --------------------- Job Details -----------------------------
@Bean("oneMinuteJobDetail") // Use distinct bean names for details
public JobDetailFactoryBean oneMinuteJob() {
return createJobDetail(OneMinuteJob.class, "One Minute Job");
}
@Bean("thirtySecondJobDetail") // Use distinct bean names for details
public JobDetailFactoryBean thirtySecondJob() {
return createJobDetail(ThirtySecondJob.class, "Thirty Second Job");
}
// --------------------- Triggers ------------------------
@Bean("thirtySecondJobTrigger")
public SimpleTriggerFactoryBean thirtySecondJobTrigger(@Qualifier("thirtySecondJobDetail") JobDetail jobDetail) {
var trigger = new SimpleTriggerFactoryBean();
trigger.setJobDetail(jobDetail);
trigger.setName("ThirtySecondTrigger"); // Unique trigger name
trigger.setGroup("DEFAULT"); // Assign to a group
trigger.setDescription(jobDetail.getDescription());
trigger.setStartDelay(schedulerConfigProperties.getThirtySecondJob().getStartDelayMillis());
trigger.setRepeatInterval(schedulerConfigProperties.getThirtySecondJob().getFixedDelay());
trigger.setRepeatCount(SimpleTriggerFactoryBean.REPEAT_INDEFINITELY); // Ensure it repeats
return trigger;
}
@Bean("oneMinuteJobTrigger")
public CronTriggerFactoryBean oneMinuteJobTrigger(@Qualifier("oneMinuteJobDetail") JobDetail jobDetail) {
return createCronTrigger(jobDetail,
schedulerConfigProperties.getOneMinuteJob().getCron(),
schedulerConfigProperties.getOneMinuteJob().getStartDelayMillis());
}
// --------------------- Scheduler Factory ------------------------
@Bean
// Ensure database schema is initialized before scheduler starts
@DependsOn("quartzDataSourceInitializer")
public SchedulerFactoryBean schedulerFactoryBean(List<Trigger> triggers, List<JobDetail> jobDetails) { // Inject triggers and jobDetails
var factory = new SchedulerFactoryBean();
var props = new Properties();
props.putAll(quartzProperties.getProperties());
factory.setQuartzProperties(props);
factory.setDataSource(dataSource); // Use injected DataSource
factory.setTransactionManager(transactionManager); // Use injected TransactionManager
factory.setJobFactory(autowiringSpringBeanJobFactory());
factory.setJobDetails(jobDetails.toArray(new JobDetail[0])); // Register JobDetails
factory.setTriggers(triggers.toArray(new Trigger[0])); // Register Triggers
factory.setAutoStartup(true); // Start scheduler automatically
factory.setOverwriteExistingJobs(true); // Overwrite existing jobs on startup
factory.setApplicationContextSchedulerContextKey("applicationContext"); // Expose Spring context
return factory;
}
// --------------------- Helper Beans ---------------------------
// Initializes Quartz schema if needed
@Bean
public QuartzDataSourceScriptDatabaseInitializer quartzDataSourceInitializer(DataSource dataSource) {
// No need to pass quartzProperties if using spring.quartz.jdbc.initialize-schema
return new QuartzDataSourceScriptDatabaseInitializer(dataSource, quartzProperties);
}
// Provides the custom job factory
@Bean
public AutowiringSpringBeanJobFactory autowiringSpringBeanJobFactory() {
return new AutowiringSpringBeanJobFactory();
}
// Helper method to create JobDetailFactoryBean
private <T extends Job> JobDetailFactoryBean createJobDetail(Class<T> jobClass, String jobName) {
JobDetailFactoryBean jobDetailFactoryBean = new JobDetailFactoryBean();
jobDetailFactoryBean.setJobClass(jobClass);
jobDetailFactoryBean.setDescription(jobName);
jobDetailFactoryBean.setDurability(true); // Make job durable
jobDetailFactoryBean.setGroup("DEFAULT"); // Assign to a group
jobDetailFactoryBean.setName(jobName.replace(" ", "")); // Unique job name
return jobDetailFactoryBean;
}
// Helper method to create CronTriggerFactoryBean
private CronTriggerFactoryBean createCronTrigger(JobDetail jobDetail, String cron, long startDelayMillis) {
CronTriggerFactoryBean cronTriggerFactoryBean = new CronTriggerFactoryBean();
cronTriggerFactoryBean.setJobDetail(jobDetail);
cronTriggerFactoryBean.setCronExpression(cron);
cronTriggerFactoryBean.setStartDelay(startDelayMillis);
cronTriggerFactoryBean.setName(jobDetail.getKey().getName() + "Trigger"); // Unique trigger name
cronTriggerFactoryBean.setGroup("DEFAULT"); // Assign to a group
cronTriggerFactoryBean.setDescription(jobDetail.getDescription());
return cronTriggerFactoryBean;
}
}
application.yml:
Configure the database connection, Quartz properties, and job schedules here.
spring:
application:
name: QuartzDemo # Application name
datasource:
# Use environment variables or Kubernetes secrets for production credentials
url: jdbc:postgresql://quartz-postgres-service:5432/postgres # K8s service name
username: postgres
password: postgres
driver-class-name: org.postgresql.Driver
jpa: # Not strictly required for Quartz JDBCJobStore, but common in Spring Boot
show-sql: false
hibernate:
ddl-auto: none # Let Quartz manage its schema
properties:
hibernate:
dialect: org.hibernate.dialect.PostgreSQLDialect
quartz:
job-store-type: jdbc # Use JDBCJobStore for persistence and clustering
jdbc:
initialize-schema: always # Create Quartz tables on startup if they don't exist (use 'never' in production after first run)
properties:
org:
quartz:
scheduler:
instanceName: quartz-demo-scheduler # Name for this scheduler instance
instanceId: AUTO # Automatically generate instance ID (required for clustering)
jobStore:
class: org.quartz.impl.jdbcjobstore.JobStoreTX # Standard JDBC JobStore
driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegate # Delegate for PostgreSQL specifics
useProperties: false # Job data stored as BLOBs
misfireThreshold: 60000 # 60 seconds threshold for misfire detection
clusterCheckinInterval: 10000 # Check in with cluster every 10 seconds
isClustered: true # Enable clustering!
tablePrefix: QRTZ_ # Default Quartz table prefix
threadPool:
class: org.quartz.simpl.SimpleThreadPool
threadCount: 10 # Number of worker threads
threadPriority: 5 # Default priority
# Custom application properties for scheduling configuration
app:
scheduling:
oneMinuteJob:
cron: "0 * * * * ?" # Standard cron expression: every minute at second 0
start-delay-millis: 10000 # 10 second delay after application start
thirtySecondJob:
fixed-delay: 30000 # Repeat every 30 seconds
start-delay-millis: 20000 # 20 second delay after application start
Note: The PostgreSQL hostname (quartz-postgres-service) refers to the Kubernetes service name we will create later. For local development, replace it with localhost or your Docker container’s address.
Persisting Job State with PostgreSQL
Using spring.quartz.job-store-type=jdbc tells Quartz to persist job and trigger information in a database. This is crucial for:
- Statefulness: Job schedules and state survive application restarts.
- Clustering: Multiple instances of the application can coordinate via the shared database, ensuring jobs are executed by only one instance at a time (load balancing) and preventing duplicate executions.
The spring.quartz.jdbc.initialize-schema=always setting automatically creates the necessary QRTZ_ tables in the configured PostgreSQL database on application startup. Caution: Use always for development/testing; set to never in production once the schema is established.
Containerization with Docker
Create a Dockerfile to package the Spring Boot application into a container image.
# Use a slim Java 17 base image
FROM openjdk:17-jdk-slim-buster
LABEL maintainer="Your Name or Team"
# Set working directory
WORKDIR /app
# Copy the executable JAR file to the container
# Assumes Gradle builds the JAR in build/libs/
COPY build/libs/QuartzDemo-*.jar /app/quartz-demo.jar
# Command to run the application
ENTRYPOINT ["java", "-jar", "/app/quartz-demo.jar"]
Build the Docker image using:
./gradlew build
docker build -t quartz-demo:0.0.1-SNAPSHOT .
Deployment to Kubernetes
Deploying to Kubernetes allows for scalability and high availability. We’ll set up PostgreSQL and the Spring Boot application.
1. Namespace (Optional but Recommended):
# k8s/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: quartz-namespace
2. PostgreSQL PersistentVolume and Claim:
This ensures PostgreSQL data persists across pod restarts. Adapt hostPath for your specific cluster storage or use a StorageClass.
# k8s/postgres-pv-pvc.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: quartz-postgres-pv
namespace: quartz-namespace
spec:
capacity:
storage: 2Gi # Adjust size as needed
accessModes:
- ReadWriteOnce # Suitable for a single PostgreSQL instance
persistentVolumeReclaimPolicy: Retain # Keep data even if PVC is deleted
storageClassName: standard # Or your cluster's default/custom storage class
hostPath: # Example for local testing (Minikube/Kind). Use appropriate storage in production.
path: "/mnt/data/quartz-postgres"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: quartz-postgres-pvc
namespace: quartz-namespace
spec:
accessModes:
- ReadWriteOnce
storageClassName: standard # Match the PV's storage class
resources:
requests:
storage: 2Gi # Request the desired storage size
3. PostgreSQL ConfigMap and Secret:
Store configuration and sensitive credentials separately. Use Secrets for passwords!
# k8s/postgres-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: quartz-postgres-config
namespace: quartz-namespace
data:
POSTGRES_DB: "postgres" # Database name
---
apiVersion: v1
kind: Secret
metadata:
name: quartz-postgres-secret
namespace: quartz-namespace
type: Opaque
data:
# Encode username and password using base64
# echo -n 'postgres' | base64
POSTGRES_USER:cG9zdGdyZXM=
POSTGRES_PASSWORD:cG9zdGdyZXM= # Replace with your actual encoded password
4. PostgreSQL Deployment:
Deploys a single instance of PostgreSQL using the official image.
# k8s/postgres-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: quartz-postgres-deployment
namespace: quartz-namespace
labels:
app: quartz-postgres
spec:
replicas: 1
selector:
matchLabels:
app: quartz-postgres
template:
metadata:
labels:
app: quartz-postgres
spec:
containers:
- name: postgres
image: postgres:14 # Use a specific PostgreSQL version
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5432
protocol: TCP
envFrom:
- configMapRef:
name: quartz-postgres-config
- secretRef:
name: quartz-postgres-secret
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: postgres-storage
subPath: postgres # Store data in a sub-directory within the volume
volumes:
- name: postgres-storage
persistentVolumeClaim:
claimName: quartz-postgres-pvc
5. PostgreSQL Service:
Exposes the PostgreSQL deployment within the cluster.
# k8s/postgres-service.yaml
apiVersion: v1
kind: Service
metadata:
name: quartz-postgres-service # This name is used in application.yml
namespace: quartz-namespace
spec:
selector:
app: quartz-postgres # Selects the PostgreSQL pods
ports:
- protocol: TCP
port: 5432 # Service port
targetPort: 5432 # Container port
type: ClusterIP # Only accessible within the cluster
6. Spring Boot Application Deployment:
Deploys the Quartz application. Setting replicas: 2 demonstrates clustering.
# k8s/quartz-app-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: quartz-demo-deployment
namespace: quartz-namespace
labels:
app: quartz-demo
spec:
replicas: 2 # Run two instances for clustering/load balancing demo
selector:
matchLabels:
app: quartz-demo
template:
metadata:
labels:
app: quartz-demo
spec:
containers:
- name: quartz-demo
image: quartz-demo:0.0.1-SNAPSHOT # Your built Docker image
imagePullPolicy: IfNotPresent # Or Always if using latest tag or frequent updates
ports:
- containerPort: 8080 # Default Spring Boot port
protocol: TCP
# Add readiness/liveness probes for production robustness
# readinessProbe:
# httpGet:
# path: /actuator/health/readiness
# port: 8080
# initialDelaySeconds: 30
# periodSeconds: 10
# livenessProbe:
# httpGet:
# path: /actuator/health/liveness
# port: 8080
# initialDelaySeconds: 60
# periodSeconds: 15
7. Spring Boot Application Service (Optional):
If the application needs to be accessed (e.g., via REST endpoints), create a Service. For purely background jobs, this might not be necessary.
# k8s/quartz-app-service.yaml
apiVersion: v1
kind: Service
metadata:
name: quartz-demo-service
namespace: quartz-namespace
spec:
selector:
app: quartz-demo # Selects the Spring Boot app pods
ports:
- port: 80 # Service port
targetPort: 8080 # Container port
protocol: TCP
type: LoadBalancer # Or NodePort/ClusterIP depending on access needs
8. Deployment Script:
A simple script to apply the configurations.
#!/bin/bash
NAMESPACE="quartz-namespace"
echo "Applying Kubernetes configurations in namespace: ${NAMESPACE}..."
# Create namespace if it doesn't exist
kubectl create namespace ${NAMESPACE} --dry-run=client -o yaml | kubectl apply -f -
# Apply configurations (order matters for dependencies like PVC)
echo "Applying PostgreSQL PV/PVC..."
kubectl apply -f k8s/postgres-pv-pvc.yaml -n ${NAMESPACE}
echo "Applying PostgreSQL ConfigMap and Secret..."
kubectl apply -f k8s/postgres-config.yaml -n ${NAMESPACE}
echo "Applying PostgreSQL Deployment..."
kubectl apply -f k8s/postgres-deployment.yaml -n ${NAMESPACE}
echo "Applying PostgreSQL Service..."
kubectl apply -f k8s/postgres-service.yaml -n ${NAMESPACE}
# Wait for PostgreSQL to be ready (optional but good practice)
echo "Waiting for PostgreSQL to be ready..."
kubectl wait --for=condition=available --timeout=300s deployment/quartz-postgres-deployment -n ${NAMESPACE}
echo "Applying Spring Boot Application Deployment..."
kubectl apply -f k8s/quartz-app-deployment.yaml -n ${NAMESPACE}
# Apply service if needed
# echo "Applying Spring Boot Application Service..."
# kubectl apply -f k8s/quartz-app-service.yaml -n ${NAMESPACE}
echo "Deployment process initiated. Check pod status with: kubectl get pods -n ${NAMESPACE}"
Run the script: bash deploy.sh
Verifying Cluster Behavior
With two replicas running and clustering enabled (isClustered: true), Quartz uses database locking (QRTZ_LOCKS table) to coordinate. Only one instance will acquire the lock to execute a specific job trigger at its scheduled time.
Check the logs of both application pods:
kubectl logs -f -n quartz-namespace -l app=quartz-demo --tail=50
Observe the output. You should see logs like:
# Pod 1 Log Snippet
... INFO --- [demo-scheduler_Worker-1] c.e.q.scheduled.ThirtySecondJob : Executing ThirtySecondJob at 2023-10-27T10:30:00.123Z
... INFO --- [demo-scheduler_Worker-5] c.e.q.scheduled.OneMinuteJob : Executing OneMinuteJob at 2023-10-27T10:31:00.456Z
# Pod 2 Log Snippet
... INFO --- [demo-scheduler_Worker-3] c.e.q.scheduled.ThirtySecondJob : Executing ThirtySecondJob at 2023-10-27T10:30:30.789Z
... INFO --- [demo-scheduler_Worker-8] c.e.q.scheduled.ThirtySecondJob : Executing ThirtySecondJob at 2023-10-27T10:31:30.987Z
Notice that the execution of each specific trigger (e.g., the ThirtySecondJob run scheduled for 10:30:00) occurs on only one of the pods. The workload is distributed across the available instances, demonstrating successful load balancing and clustered job execution.
Conclusion
Integrating Quartz Scheduler with Spring Boot provides a robust mechanism for handling scheduled tasks. By leveraging Quartz’s JDBC JobStore with PostgreSQL and deploying the application as multiple replicas on Kubernetes, a highly available, persistent, and scalable scheduling system is achieved. This setup ensures that jobs run reliably, state is maintained across restarts, and the workload is efficiently distributed in a clustered environment.
At Innovative Software Technology, we specialize in building robust, scalable backend systems using technologies like Spring Boot, Quartz Scheduler, and Kubernetes. If you need expert assistance in designing and implementing reliable task scheduling solutions, optimizing job persistence, or deploying clustered applications for high availability, our team can help. We deliver tailored solutions that leverage the power of Quartz and Kubernetes to ensure your critical background processes run efficiently and reliably, supporting your business needs. Contact Innovative Software Technology to discuss how we can enhance your application’s scheduling capabilities.