Make IT Simple: What Is Quartz

by Chuck Cavaness, author of Programming Jakarta Struts, Second Edition
09/28/2005

Quartz: Quartz is an open source job-scheduling framework written entirely in Java and designed for use in both J2SE and J2EE applications. It offers great flexibility without sacrificing simplicity. You can create simple or complex schedules for executing any job. It includes features such as database support, clustering, plugins, prebuilt jobs for EJB, JavaMail and others, support for cron-like expressions, and many more.

Have you ever needed an application to perform a task that runs daily, every other Tuesday at 11:30 p.m., or maybe only on the last day of every month? A task that can run automatically, in a hands-off mode, and, if a serious error occurs during execution, the application will be so self aware that it declares a misfire and attempts to run itself again? Are you and your team programming in Java? If you've answered yes to any of these questions, then you ought to be using the Quartz Scheduler.

Job Scheduling Made Easy

Quartz is an open source job-scheduling framework written entirely in Java. Don't let the term job scheduling frighten you. While the Quartz framework is packed with many bells and whistles, in its simplest form, it's almost scary how easy it is to use.

Simply create a Java class that implements the org.quartz.Job interface. The Job interface contains a single method:

public void execute(JobExecutionContext context)
   throws JobExecutionException;

In your Job class, add some logic to the execute() method. Once you configure the Job class and set up the schedule, Quartz will take care of the rest. When the Scheduler determines it's time to notify your Job, the Quartz framework will call the execute() method on your Job class and allow it to do its thing. You don't have to report anything back to the Scheduler or call anything special. Just perform the tasks within the Job and end. If you configure your Job to be called again at a later time, the framework will take care of calling again at the right time.

If you've used other popular open source frameworks, like Apache Struts, you'll be comfortable with the design and components in Quartz. Even though the two open source projects solve completely different problems, there are enough similarities that everyday users of open source software will feel right at home. Quartz can be used within a stand- alone J2SE application, as an RMI server, within a web application, and even within a J2EE Application Server.

The History Behind Quartz

Quartz has been around for a while, although it has started to receive a lot of attention this year. It was created by James House and originally added as a SourceForge project in the spring of 2001. Over the next several years, a number of features and releases came, but it wasn't until the project was moved to a new site and became part of the OpenSymphony family of projects, that it really started to take off and receive the attention it deserves.

House still participates in much of the development work, with several part-time developers assisting him. The Quartz development team has been able to release several new versions this year, including the 1.5 release, which is currently in a candidate release stage.

Getting Your Hands on Quartz

The Quartz project is hosted on OpenSymphony's site. On the Quartz site, you will find all of the usual suspects: JavaDocs, documentation including tutorials, CVS access, links to the user and developer forums, and, of course, the downloads.

Grab the distribution from the download link and unzip it to a local directory. The download includes a prebuilt Quartz binary that you can drop into your application. The framework requires very few third-party libraries, and those that are required, you are probably already using anyway.

From the distribution, you'll want to add the third-party libraries from the /lib/core and /lib/optional directories to your project. Most of these libraries are the standard Jakarta Commons libraries that we all know and love like Commons Logging, Commons BeantUtils, and so on.

The `quartz.properties` File

Quartz includes a configuration file called quartz.properties, which allows you to modify the runtime environment of the framework. By default, the version of this file contained within the Quartz binary is used. You should create a copy of this file and put it in your classes directory so that it can be found by the class loader. A sample quartz.propeties file is shown in Example 1.

Example 1. The quartz.properties file allows you modify the Quartz runtime:

#===============================================================
# Configure Main Scheduler Properties
#===============================================================

org.quartz.scheduler.instanceName = QuartzScheduler
org.quartz.scheduler.instanceId = AUTO

#===============================================================
# Configure ThreadPool
#===============================================================

org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount =  5

org.quartz.threadPool.threadPriority = 5

#===============================================================
# Configure JobStore
#===============================================================

org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore

Once you have the Quartz binary and the third-party libraries added to your project, and the quartz.properties in the classes directory, it's time to create some Jobs. Before we do, however, let's take a short detour and talk briefly about the Quartz architecture.

Inside the Quartz Architecture

In terms of size, Quartz is comparable to most other open source frameworks. It contains approximately 300 Java Classes and Interfaces organized into roughly 12 packages. This compares to approximately 325 Classes and Interfaces and 11 packages in Apache Struts. Although size is rarely a characteristic used to determine the quality of a framework, the point here is that there's a lot of functionality included within Quartz, and functionality and feature set is, and should be, one factor used to judge the quality of a framework, open source or not.

The Quartz Scheduler

At the heart of the Quartz framework is the Scheduler. The Scheduler is responsible for managing the runtime environment for your Quartz application. It doesn't do all of the work by itself, but depends on some very important components within the framework. Quartz is very much about threads and thread management. To ensure scalability, Quartz is based on a multithreaded architecture. When started, the framework initializes a set of "worker threads" that are used by the Scheduler to execute the scheduled Jobs. This is how Quartz is able to run many Jobs concurrently. Quartz relies on a loosely coupled set of ThreadPool management components to manage the thread environment. We'll mention it several times throughout the article, but just about everything in Quartz is either configurable or customizable. So, for example, if you wanted to "plugin" your own ThreadPool management facilities, guess what, you can!

Jobs, Jobs, and More Jobs

In Quartz lingo, a Job is simply a Java class that performs a task. This task can be anything that you can code in Java. The only requirement is that you implement the org.quartz.Job interface and throw a JobExecutionException in the case of a serious error. You saw in the beginning that the Job interface contains a single method, execute().

Once you implement the Job interface and implement the execute() method, Quartz will invoke your Job when it determines it's time for the Job to run. What you do inside the execute() method is entirely up to you. Here are a few examples of things you might want to do inside a Job:

Use JavaMail (or another Mail framework like Commons Net) to send emails
Create a remote interface and invoke a method on an EJB
Get a Hibernate Session, and query and update data in a relational database
Use OSWorkflow and invoke a workflow from the Job
Use FTP and move files around
Call an Ant build script to kick off scheduled builds

The possibilities are endless and that's what makes the framework so powerful. Quartz provides you with the mechanism to establish a very granular, repeatable schedule, and then you just create Java classes that get called to execute.

Job Management and Storage

Once Jobs have been scheduled, the Scheduler needs to remember and keep track of the Jobs and the times to execute them. It wouldn't be very useful if your Job is called 30 minutes late or even 30 seconds early. In fact, Job execution needs to be very exact and prompt about calling the execute() methods on the scheduled jobs. Job storage and management is done through a concept Quartz refers to as a JobStore.

Available JobStores

The Quartz framework provides two basic types of JobStores. The first type, which utilizes ordinary memory (RAM) to persist Scheduler information, is called RAMJobStore. This type of JobStore is the easiest to configure and get up and running. For many applications, this JobStore will be sufficient. However, since the Scheduler information is stored in the memory assigned to the JVM, when the application is stopped, all knowledge of the schedule is lost. If you need to persist schedule information between restarts, then you're going to need the second type of JobStore.

The second type of JobStore actually comes with two different implementations in the framework, but both are commonly referred to as JDBC JobStores. Both JDBC JobStores use a JDBC driver and require a backing from a relational database to persist the scheduler information. The two types differ in whether or not you want to control the database transactions or relinquish control to an application container such as BEA's WebLogic or JBoss. (This is similar to the difference between Bean Managed Transactions or Container Managed Transactions from the J2EE world.)

The two JDBC JobStores are:

JobStoreTX: used when you want to control transactions or when you're operating in a nonapplication server environment
JobStoreCMT: used when operating within an application server environment and you want the container to manage the transactions

The JDBC JobStores are designed for users who require the Scheduler to maintain scheduling information, even after a shutdown and startup.

Jobs and Triggers

Jobs are only half the equation. The designers of Quartz made a design choice to separate the Job from the schedule. A Trigger in Quartz is used to tell the Scheduler when a Job should be triggered (or fired). The framework comes with a handful of Trigger types, but the two most commonly used are the SimpleTrigger and the CronTrigger.

The SimpleTrigger is designed for when you need a simple firing schedule. Typically, if you need to fire a Job at a given time and repeat (n) number of times, possibly waiting (m) seconds between firings, then the SimpleTrigger is for you. If on the other hand, you have a much more complicated scheduling need for your Job, then the CronTrigger will probably be required.

The CronTrigger is based on Calendar-like schedules. When you need a Job executed every day at 10:30 a.m., except on Saturdays and Sundays, then a CronTrigger should be used. As the name implies, the CronTrigger is based on the Unix cron expression. As an example, the following Quartz cron expression would execute a Job at 10:15 a.m. every day, Monday through Friday.

0 15 10 ? * MON-FRI

and this expression

0 15 10 ? * 6L 2002-2005

fires at 10:15 a.m. on the last Friday of every month during the years 2002, 2003, 2004, and 2005.

You can't do this with the SimpleTrigger. Either can be used for your Jobs. Which one just depends on your schedule needs.

Scheduling a Job

So, let's take this discussion into the practical realm by looking at an example Job. Suppose that you manage a department that needs to be notified by email whenever a client stores a file on its FTP site. Our Job will FTP to a remote server and download any files found there. It will then send an email containing the number of files found and downloaded. This Job would be handy to prevent someone from having to manually performing this task during the day, not even considering during the night. We can set this Job up to check every 60 seconds and to run indefinitely, 24 hours a day, seven days a week. This is a perfect use of the Quartz framework!

The first step is to create the Job class that will perform the FTP and Email logic. The following Example shows our Quartz Job class, which implements the org.quartz.Job interface.

Example 2. A Quartz Job that downloads files from a FTP site and sends an Email

public class ScanFTPSiteJob implements Job {
  private static Log logger = LogFactory.getLog(ScanFTPSiteJob.class);

  /*
   * Called the scheduler framework at the right time
   */
  public void execute(JobExecutionContext context)
          throws JobExecutionException {

      JobDataMap jobDataMap = context.getJobDataMap();

      try {
          // Check the ftp site for files
          File[] files = JobUtil.checkForFiles(jobDataMap);

          JobUtil.sendEmail(jobDataMap, files);
      } catch (Exception ex) {
          throw new JobExecutionException(ex.getMessage());
      }
  }

We intentionally kept the ScanFTPSiteJob very simple. We created a utility class for this example called JobUtil. That's not part of Quartz, but it makes sense to build your own library of utilities that various Jobs can reuse. We could have just as easily put all of that code inside the Job class and the Quartz Scheduler would have been just as happy, but reuse doesn't stop just because we're using Quartz.

The parameters that the JobUtil.checkForFiles() and JobUtil.sendEmail() methods use come from the JobDataMap object, which is a Quartz-created object. An instance of it is created for every Job execution and it is a way to pass configuration parameters to your Job class.

The implementation of the JobUtil is not shown here, but we could very easily use the Commons Net from Jakarta to implement both the FTP and Email functionality.

Calling Your Jobs with the Scheduler

Creating the Job is the first step, but to get your Job called by the Scheduler, you need to indicate to the Scheduler how often, and at what time, your Job should be called. This is done by associating a Trigger to the Job. Since we are only interested in calling the Job every 60 seconds or so, and repeating this Scheduler forever, we are going to use a SimpleTrigger.

Jobs and Triggers are scheduled through the Quartz Scheduler interface. In order to get an instance of the Scheduler, we need to create one from the factory. The easiest way to do this is to call the static method getDefaultScheduler() on the StdSchedulerFactory class.

When using the Quartz framework, you're required to start the Scheduler by calling the start() method. The code in Example 3 follows a common pattern of most Quartz applications: create one or more Jobs, create and set up the Triggers, schedule Jobs and Triggers with the Scheduler, and start the Scheduler. (Note: the Scheduler could have been started first, it doesn't matter.)

Example 3. Quartz Jobs have to be scheduled through the Quartz Scheduler.

public class MyQuartzServer {

  public static void main(String[] args) {
      MyQuartzServer server = new MyQuartzServer();

      try {
          server.startScheduler();
      } catch (SchedulerException ex) {
          ex.printStackTrace();
      }
  }

  protected void startScheduler() throws SchedulerException {

      // Use the factory to create a Scheduler instance
      Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();

      // JobDetail holds the definition for Jobs
      JobDetail jobDetail =
new JobDetail("ScanFTPJob", Scheduler.DEFAULT_GROUP,
              ScanFTPSiteJob.class);

// Store job parameters to be used within execute()
jobDetail.getJobDataMap().put(
"FTP_HOST",
"\\home\\cavaness\\inbound");

      // Other neccessary Job parameters here


       // Create a Trigger that fires every 60 seconds
      Trigger trigger = TriggerUtils.makeSecondlyTrigger(60);
    
      // Setup the Job and Trigger with the Scheduler
      scheduler.scheduleJob(jobDetail, trigger );
    
      // Start the Scheduler running
      scheduler.start();
  }
}

Programmatic vs. Declarative Scheduling

In Example 3, we scheduled our ScanFTPSiteJob using a programmatic approach. That is, we used Java code to setup the Job and Trigger with the Scheduler. The Quartz framework also supports setting up the Job schedule declaratively in XML files. A declarative approach allows us to more quickly modify which Jobs are being executed and when.

The Quartz framework includes a "plugin" that reads an XML file containing Job and Trigger information upon startup of a Quartz application. All Jobs within the XML are added to the Scheduler, along with their associated Triggers. You still have to write the Job classes, of course, but configuring the Scheduler with those Job becomes very dynamic. Example 4 shows an example XML file that performs the same logic as the code in Example 3, but just in a

declarative fashion.

Example 4. Quartz Jobs can also be scheduled using an XML File.



  
      
          ScanFTPSiteJob
          DEFAULT
          
              A job that scans an ftp site for files
          
          ScanFTPSiteJob



          
              
                  FTP_HOST
                  \home\cavaness\inbound
              
            
              

          
      

      
          
              ScanFTPSiteJobTrigger
              DEFAULT
              ScanFTPSiteJob
              DEFAULT
              2005-09-11 6:10:00 PM
              
              -1
              60000

You should be able to match the XML elements from Example 4 with the Java code from

Example 3--they are conceptually the same. The benefit of using a declarative approach like the one shown in Example 4 is that maintenance becomes extremely simple, as changes only require a change to the XML file and a restart of the Quartz application. No source code changes, no recompilation, and no redeployment of the new build.

Stateful and Stateless Jobs

The Quartz Job examples that you have seen in this article have all been stateless. This means that for each Job execution, any changes made to the JobDataMap during the Job's execution are not maintained between executions. If you need to ability to add, change, or remove values from the JobDataMap, and have those changes seen by the Job during the next execution, a Quartz Stateful Job will be required.

If you're an experienced EJB developer, I'm sure you're wincing right now because Stateful has taken on a negative connotation. This is primarily due to the scalability issues that "Stateful EJBs" present. Quartz Stateful Jobs are implemented by the org.quartz.StatefulJob interface. The key difference between stateless and stateful Jobs is that Stateful Jobs can have only a single instance of the Job definition executing at a time. So, using our example from Example 3, we could only have one instance of the "ScanFTPJob" Job executing at a time. For most situations, this doesn't present a huge problem. If, however, you have a Job that needs to execute so often, or one that takes a long time to finish, Stateful Quartz Jobs may present a scalability problem for you.

Other Features of the Quartz Framework

The Quartz framework has a very rich feature set. In fact, there are too many features to really appreciate in just one encounter. The following list will give a taste of some of the other features contained within Quartz that we won't have time to talk about in detail here.

Listeners and Plugins

Everybody likes Listeners and Plugins. Download just about any open source framework today and you'll be sure to find support for both of these concepts.

Quartz Listeners are Java Classes that you create and that will receive callbacks from within the framework when key events take place. For example, when a Job is scheduled or unscheduled, or when a Trigger has ended and will not fire anymore, all can be setup to notify your Listener. The framework includes Listeners for the Scheduler, for Jobs, and for Triggers. You can also configure the Job and Trigger Listeners to be global or specific to a Job and/or Trigger.

Once your concrete Listener is called, you can use this knowledge to perform any logic you want within your Listener class. If, for example, you wanted to send an email each time a Job completes, you could program that into the Job. You could also use a JobListener, which enforces a looser coupling and probably makes for a better design.

Quartz Plugins are new functionalities that can be created and added to the framework without having to modify Quartz source. It's done for those developers who need to extend the Quartz framework but don't have time to submit changes to the development team and wait for a new release. If you're familiar with Struts plugins, then you understand the use of Quartz plugins completely.

Rather than Quartz providing a finite set of extension points that may not serve your needs, you have an almost open-ended extension point through the use of Plugins.

Clustering Quartz Applications

Quartz applications can be clustered, both horizontally and vertically, depending on your needs. Clustering offers the same benefits as any other type of clustering:

Scalability
High Availability
Load Balancing

Currently, Quartz supports clustering with help from a relational database and one of the JDBC JobStores. In a future version, this constraint will be lifted and clustering will possible with the RAMJobStore and will not require a database.

The Quartz Web Application

One of the needs of Quartz users that usually manifests after a few weeks or months of using the framework is the need to integrate Quartz into a GUI. There are facilities that currently exist in the framework that allow you to initialize and start Quartz using a Java servlet. Once you have access to the Scheduler instance, you can store it in the ServletContext of the web container and manage the scheduling environment through the Scheduler interface.

Fortunately, some Quartz developers have been working on a stand-alone Quartz Web Application that can be used to better manage the Quartz Scheduler environment. This GUI, built on top of several popular open source projects like Struts and Spring, supports great functionality, all wrapped up in a simple interface. A screenshot of that GUI is shown in Figure 1.

Figure 1. The Quartz Web Application allows easier management of the Quartz environment.

What's down the Road?

Quartz is a project on the move. The development team is definitely not resting on its laurels. Things are already in motion for the next major release of the framework. You can get a taste of the features and design being considered for Quartz 2.0 on OpenSymphony's wiki.

As always, everyday Quartz users are free to add suggestions for features and design ideas so that they can be considered for the core framework.

Finding Out More About Quartz

As you start to use more of the features of the Quartz framework, the User and Developer Forum becomes an extremely useful resource for answering questions and communicating with other Quartz users. The individuals that frequent the forums are always helpful and constructive, and you can also count on James House to share his knowledge and comment on your needs and requirements.

The forum is free and you're not required to have a login to search and view the forum archive. However, once you're comfortable with navigating the forum and need to post a reply to someone, you'll need to sign up for a free account.

Chuck Cavaness is a graduate from Georgia Tech with degrees in computer engineering and computer science. He has built Java-based enterprise systems in the healthcare, banking, and B2B sectors. He is also the author of two O'Reilly books, Programming Jakarta Struts and Jakarta Struts Pocket Reference.

Make IT Simple

Thursday, May 10, 2007

What Is Quartz

In This Article:

Job Scheduling Made Easy

The History Behind Quartz

Getting Your Hands on Quartz

The `quartz.properties` File

Inside the Quartz Architecture

The Quartz Scheduler

Jobs, Jobs, and More Jobs

Job Management and Storage

Available JobStores

Jobs and Triggers

Scheduling a Job

Calling Your Jobs with the Scheduler

Programmatic vs. Declarative Scheduling

Stateful and Stateless Jobs

Other Features of the Quartz Framework

Listeners and Plugins

Clustering Quartz Applications

The Quartz Web Application

What's down the Road?

Finding Out More About Quartz

No comments:

Thursday, May 10, 2007

What Is Quartz

In This Article:

Job Scheduling Made Easy

The History Behind Quartz

Getting Your Hands on Quartz

The quartz.properties File

Inside the Quartz Architecture

The Quartz Scheduler

Jobs, Jobs, and More Jobs

Job Management and Storage

Available JobStores

Jobs and Triggers

Scheduling a Job

Calling Your Jobs with the Scheduler

Programmatic vs. Declarative Scheduling

Stateful and Stateless Jobs

Other Features of the Quartz Framework

Listeners and Plugins

Clustering Quartz Applications

The Quartz Web Application

What's down the Road?

Finding Out More About Quartz

No comments:

The `quartz.properties` File