Monday, January 01, 2007

Job scheduling with Quartz-The Quartz API takes a multifaceted approach to job scheduling in Java applications

Michael Lipton, Software Engineer, IBM
Soobaek Jang (sjang@us.ibm.com), IT Architect/Integration, IBM

21 Nov 2006

Quartz is an open source project that offers an extensive set of job scheduling features. In this article, software engineer Michael Lipton and IT architect Soobaek Jang introduce the Quartz API, starting with a general overview of the framework and concluding with a series of code examples that illustrate its fundamental features. After reading this article and following the code examples, you should feel capable of incorporating the basic features of Quartz into any Java™ application.

As modern Web applications continue to grow in scope and complexity, each underlying component of the applications must similarly grow. Job scheduling is a common requirement for Java applications in modern systems, and so is a constant preoccupation for Java developers. While current scheduling technology has evolved from more primitive methods of database trigger flags and separate scheduler threads, job scheduling is still a non-trivial problem. One of the most desirable solutions to this problem is the Quartz API from OpenSymphony.

Quartz is an open source job scheduling framework that provides simple but powerful mechanisms for job scheduling in Java applications. Quartz allows developers to schedule jobs by time interval or by time of day. It implements many-to-many relationships for jobs and triggers and can associate multiple jobs with different triggers. Applications that incorporate Quartz can reuse jobs from different events and also group multiple jobs for a single event. While you can configure Quartz through a property file (in which you can specify a data source for JDBC transactions, global job and/or trigger listeners, plug-ins, thread pools, and more) it is not at all integrated with the application server's context or references. One result of this is that jobs do not have access to the Web server's internal functions; in the case of the WebSphere application server, for example, Quartz-scheduled jobs cannot interfere with the server's Dyna-cache and data sources.

This article introduces the Quartz API using a series of code examples to illustrate mechanisms such as jobs, triggers, job stores, and properties.

Getting started

To start using Quartz, you need to configure your project with the Quartz API. The procedure is as follows:

1. Download the Quartz API.
2. Extract and place the quartz-x.x.x.jar into your project folder, or put the file into your project classpath.
3. Place the jar files from the core and/or optional folder into your project folder or project classpath.
4. If using JDBCJobStore, place all JDBC jar files into your project folder or project classpath.

For your convenience, we have compiled all the necessary files, including the DB2 JDBC files, into a single zip. See the Downloads section to download the code.

Now let's look at the main components of the Quartz API.


Jobs and triggers

The two fundamental units of Quartz's scheduling package are jobs and triggers. A job is an executable task that can be scheduled, while a trigger provides a schedule for a job. While these two entities could easily have been combined, their separation in Quartz is both intentional and beneficial.

By keeping the work to be performed separate from its scheduling, Quartz allows you to change the scheduled trigger for a job without losing the job itself, or the context around it. Also, any singular job can have many triggers associated with it.


Example 1: Jobs

You can make a Java class executable by implementing the org.quartz.job interface. An example of a Quartz job is given in Listing 1. This class overriddes the execute(JobExecutionContext context) method with a very simple output statement. The method can contain any code we might wish to execute. (All the code samples are based on Quartz 1.5.2, the stable release at the time of this writing.)

Listing 1. SimpleQuartzJob.java

package com.ibm.developerworks.quartz;

import java.util.Date;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

public class SimpleQuartzJob implements Job {

public SimpleQuartzJob() {
}

public void execute(JobExecutionContext context) throws JobExecutionException {
System.out.println("In SimpleQuartzJob - executing its JOB at "
+ new Date() + " by " + context.getTrigger().getName());
}
}


Notice that the execute method takes a JobExecutionContext object as an argument. This object provides the runtime context around the job instance. Specifically, it gives access to the scheduler and trigger, which collaborated to initiate execution of the job as well as the job's JobDetail object. Quartz separates the execution and the surrounding state of a job by placing the state in a JobDetail object and having the JobDetail constructor initiate an instance of a job. The JobDetail object stores the job's listeners, group, data map, description, and other properties of the job.


Example 2: Simple triggers

A trigger develops a schedule for job execution. Quartz offers a few different trigger options of varying complexity. The SimpleTrigger in Listing 2 introduces the fundamentals of triggers:

Listing 2. SimpleTriggerRunner.java

public void task() throws SchedulerException
{
// Initiate a Schedule Factory
SchedulerFactory schedulerFactory = new StdSchedulerFactory();
// Retrieve a scheduler from schedule factory
Scheduler scheduler = schedulerFactory.getScheduler();

// current time
long ctime = System.currentTimeMillis();

// Initiate JobDetail with job name, job group, and executable job class
JobDetail jobDetail =
new JobDetail("jobDetail-s1", "jobDetailGroup-s1", SimpleQuartzJob.class);
// Initiate SimpleTrigger with its name and group name
SimpleTrigger simpleTrigger =
new SimpleTrigger("simpleTrigger", "triggerGroup-s1");
// set its start up time
simpleTrigger.setStartTime(new Date(ctime));
// set the interval, how often the job should run (10 seconds here)
simpleTrigger.setRepeatInterval(10000);
// set the number of execution of this job, set to 10 times.
// It will run 10 time and exhaust.
simpleTrigger.setRepeatCount(100);
// set the ending time of this job.
// We set it for 60 seconds from its startup time here
// Even if we set its repeat count to 10,
// this will stop its process after 6 repeats as it gets it endtime by then.
//simpleTrigger.setEndTime(new Date(ctime + 60000L));
// set priority of trigger. If not set, the default is 5
//simpleTrigger.setPriority(10);
// schedule a job with JobDetail and Trigger
scheduler.scheduleJob(jobDetail, simpleTrigger);

// start the scheduler
scheduler.start();
}



Listing 2 starts by instantiating a SchedulerFactory and getting the scheduler. As we discussed earlier, the JobDetail object is created by taking the Job as an argument to its constructor. As implied by its name, the SimpleTrigger instance is quite primitive. After creating the object, we set a few basic properties scheduling the job for execution immediately and then to repeat every 10 seconds until the job had been executed 100 times.

There are a number of other ways to manipulate a SimpleTrigger. In addition to a specified number of repeats and a specified repeat interval, you may schedule jobs to execute at a specific calendar time, given a maximum time of execution, or given a priority, which we discuss below. The maximum time of execution overrides a specified number of repeats, thus ensuring that a job does not run past the maximum time.


Example 3: Cron triggers

A CronTrigger allows for more specific scheduling than a SimpleTrigger and is still not very complex. Based on cron expressions, CronTriggers allow for calendar-like repeat intervals rather than uniform repeat intervals -- a major improvement over SimpleTriggers.

Cron expressions consist of the following seven fields:

* Seconds
* Minutes
* Hours
* Day-of-month
* Month
* Day-of-week
* Year (optional field)

Special characters

Cron triggers utilize a series of special characters, as follows:

* The backslash (/) character denotes value increments. For example, "5/15" in the seconds field means every 15 seconds starting at the fifth second.

* The question (?) character and the letter-L (L) character are permitted only in the day-of-month and day-of-week fields. The question character indicates that the field should hold no specific value. Therefore, if you specify the day-of-month, you can insert a "?" in the day-of-week field to indicate that the day-of-week value is inconsequential. The letter-L character is short for last. Placed in the day-of-month field, this schedules execution for the last day of the month. In the day-of-week field, an "L" is equivalent to a "7" if placed by itself or means the last instance of the day-of-week in the month. So "0L" would schedule execution for the last Sunday of the month.

* The letter-W (W) character in the day-of-month field schedules execution on the weekday nearest to the value specified. Placing "1W" in the day-of month field schedules execution for the weekday nearest the first of the month.

* The pound (#) character specifies a particular instance of a weekday for a given month. Placing "MON#2" in the day-of-week field schedules a task on the second Monday of the month.

* The asterisk (*) character is a wildcard character and indicates that every possible value can be taken for that specific field.

All of these definitions may seem daunting, but cron expressions become simple after just a few minutes of practice.

Listing 3 shows an example of a CronTrigger. Notice that the instantiation of the SchedulerFactory, Scheduler, and JobDetail are identical to that found in the SimpleTrigger example. In this case, we have only changed the trigger. The cron expression we have specified here ("0/5 * * * * ?") schedules the task for execution every 5 seconds.

Listing 3. CronTriggerRunner.java

public void task() throws SchedulerException
{
// Initiate a Schedule Factory
SchedulerFactory schedulerFactory = new StdSchedulerFactory();
// Retrieve a scheduler from schedule factory
Scheduler scheduler = schedulerFactory.getScheduler();

// current time
long ctime = System.currentTimeMillis();

// Initiate JobDetail with job name, job group, and executable job class
JobDetail jobDetail =
new JobDetail("jobDetail2", "jobDetailGroup2", SimpleQuartzJob.class);
// Initiate CronTrigger with its name and group name
CronTrigger cronTrigger = new CronTrigger("cronTrigger", "triggerGroup2");
try {
// setup CronExpression
CronExpression cexp = new CronExpression("0/5 * * * * ?");
// Assign the CronExpression to CronTrigger
cronTrigger.setCronExpression(cexp);
} catch (Exception e) {
e.printStackTrace();
}
// schedule a job with JobDetail and Trigger
scheduler.scheduleJob(jobDetail, cronTrigger);

// start the scheduler
scheduler.start();
}


Advanced Quartz

You can access a vast amount of functionality using only jobs and triggers as detailed above. Quartz is a comprehensive and flexible scheduling package, however, offering much more functionality to those who choose to explore it. The next sections discuss some of the advanced features of Quartz.

Job stores

Quartz offers two different means by which to store the data associated with jobs and triggers in memory or a database. The former, an instance of the RAMJobStore class, is the default setting. This job store is the easiest to use and offers the best performance because all data is stored in memory. This method's major deficiency is lack of data persistence. Because the data is stored in RAM, all information will be lost upon an application or system crash.

To remedy this problem, Quartz offers the JDBCJobStore. As the name infers, this job store places all data in a database through JDBC. The trade-off for data persistence is a lower level of performance, as well as a higher level of complexity.

Setting up JDBCJobStore

You have seen a RAMJobStore instance at work in the previous examples. Because it is the default job store, it is clear that no additional setup is required to use it. Using the JDBCJobStore requires some initialization, however.

Setting up the JDBCJobStore for use in your applications requires two steps: First you must create the database tables to be used by the job store. The JDBCJobStore is compatible with all major databases, and Quartz offers a series of table-creation SQL scripts that ease the setup process. You will find table-creation SQL scripts in the "docs/dbTables" directory of the Quartz distribution. Second, you must define some properties, shown in Table 1:

Table 1. JDBCJobStore properties
Property name Value
org.quartz.jobStore.class org.quartz.impl.jdbcjobstore.JobStoreTX (or JobStoreCMT)
org.quartz.jobStore.tablePrefix QRTZ_ (optional, customizable)
org.quartz.jobStore.driverDelegateClass org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dataSource qzDS (customizable)
org.quartz.dataSource.qzDS.driver com.ibm.db2.jcc.DB2Driver (could be any other database driver)
org.quartz.dataSource.qzDS.url jdbc:db2://localhost:50000/QZ_SMPL (customizable)
org.quartz.dataSource.qzDS.user db2inst1 (place userid for your own db)
org.quartz.dataSource.qzDS.password pass4dbadmin (place your own password for user)
org.quartz.dataSource.qzDS.maxConnections 30

Listing 4 illustrates the data persistence offered by the JDBCJobStore. As in previous examples, we started by initializing the SchedulerFactory and the Scheduler. Next, rather than initialize a job and trigger, we fetched the list of trigger group names and then, for each trigger group name, the list of trigger names. Note that each existing job should be rescheduled using the Scheduler.reschedule() method. Simply reinitializing a job that was terminated in a previous application run does not accurately load the trigger's properties.

Listing 4. JDBCJobStoreRunner.java

public void task() throws SchedulerException
{
// Initiate a Schedule Factory
SchedulerFactory schedulerFactory = new StdSchedulerFactory();
// Retrieve a scheduler from schedule factory
Scheduler scheduler = schedulerFactory.getScheduler();

String[] triggerGroups;
String[] triggers;

triggerGroups = scheduler.getTriggerGroupNames();
for (int i = 0; i < triggerGroups.length; i++) {
triggers = scheduler.getTriggerNames(triggerGroups[i]);
for (int j = 0; j < triggers.length; j++) {
Trigger tg = scheduler.getTrigger(triggers[j], triggerGroups[i]);

if (tg instanceof SimpleTrigger && tg.getName().equals("simpleTrigger")) {
((SimpleTrigger)tg).setRepeatCount(100);
// reschedule the job
scheduler.rescheduleJob(triggers[j], triggerGroups[i], tg);
// unschedule the job
//scheduler.unscheduleJob(triggersInGroup[j], triggerGroups[i]);
}
}
}

// start the scheduler
scheduler.start();
}


Running JDBCJobStore

When we run our example for the first time, the trigger is initialized in the database. Figure 1 shows the database after the trigger has been initialized but before the trigger has been fired. Therefore the REPEAT_COUNT is set to 100, based on the setRepeatCount() method in Listing 4 and TIMES_TRIGGERED is 0. After letting the application run for a while, it is stopped.

Figure 1. Data in the database using JDBCJobStore (before run)
Before run with JDBCJobStore

Figure 2 shows the database after the application has been stopped. In this figure, TIMES_TRIGGERED is set to 19, which denotes the number of times that the job was run.

Figure 2. The same data after 19 iterations
After first 19 iterations

When we start the application again, the REPEAT_COUNT is updated. This is apparent in Figure 3. Here we see that REPEAT_COUNT is updated to 81, so the new REPEAT_COUNT is equal to the previous REPEAT_COUNT value minus the previous TIMES_TRIGGERED value. Furthermore, we see that in Figure 3 the new TIMES_TRIGGERED value is 7, indicating that the job has been triggered seven more times since the application was restarted.

Figure 3. Data after the second run of 7 iterations
After the second run for 7 iterations

After stopping the application again, the REPEAT_COUNT value is again updated. This is shown in Figure 4, where the application has been stopped and not yet restarted. Again, the REPEAT_COUNT value is updated by subtracting the previous TIMES_TRIGGERED value from the previous REPEAT_COUNT value.

Figure 4. Initial data before running the trigger again
Initial data before running the trigger again

Using properties

As you have seen with the JDBCJobStore, you can use a number of properties to fine-tune the behavior of Quartz. You should specify these properties in the quartz.properties file. See Resources for a listing of configurable properties. Listing 5 shows a sample of properties used for the JDBCJobStore example:

Listing 5. quartz.properties

org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.threadPool.threadPriority = 5
org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true

# Using RAMJobStore
## if using RAMJobStore, please be sure that you comment out the following
## - org.quartz.jobStore.tablePrefix,
## - org.quartz.jobStore.driverDelegateClass,
## - org.quartz.jobStore.dataSource
#org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore

# Using JobStoreTX
## Be sure to run the appropriate script(under docs/dbTables) first to create tables
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX

# Configuring JDBCJobStore with the Table Prefix
org.quartz.jobStore.tablePrefix = QRTZ_

# Using DriverDelegate
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate

# Using datasource
org.quartz.jobStore.dataSource = qzDS

# Define the datasource to use
org.quartz.dataSource.qzDS.driver = com.ibm.db2.jcc.DB2Driver
org.quartz.dataSource.qzDS.URL = jdbc:db2://localhost:50000/dbname
org.quartz.dataSource.qzDS.user = dbuserid
org.quartz.dataSource.qzDS.password = password
org.quartz.dataSource.qzDS.maxConnections = 30

In conclusion

The Quartz job scheduling framework offers the best of both worlds: an API that is both comprehensively powerful and easy to use. Quartz can be used for simple job triggering as well as complex JDBC persistent job storage and execution. OpenSymphony has successfully filled a void in the open source universe by making the otherwise tedious chore of job scheduling trivial for developers.

Double-checked locking and the Singleton pattern-A comprehensive look at this broken programming idiom

Peter Haggar (haggar@us.ibm.com), Senior Software Engineer, IBM

01 May 2002

All programming languages have their share of idioms. Many are useful to know and use, and programmers spend valuable time creating, learning, and implementing them. The problem is that some idioms are later proven not to be all that they were purported, or to simply not work as described. The Java programming language contains several useful programming idioms. It also contains some that further study has shown should not be used. Double-checked locking is one such idiom that should never be used. In this article, Peter Haggar examines the roots of the double-checked locking idiom, why it was developed, and why it doesn't work.

Editor's note: This article refers to the Java Memory Model before it was revised for Java 5.0; statements about memory ordering may no longer be correct. However, the double-checked locking idiom is still broken under the new memory model.

The Singleton creation pattern is a common programming idiom. When used with multiple threads, you must use some type of synchronization. In an effort to create more efficient code, Java programmers created the double-checked locking idiom to be used with the Singleton creation pattern to limit how much code is synchronized. However, due to some little-known details of the Java memory model, this double-checked locking idiom is not guaranteed to work. Instead of failing consistently, it will fail sporadically. In addition, the reasons for its failure are not obvious and involve intimate details of the Java memory model. These facts make a code failure due to double-checked locking very difficult to track down. In the remainder of this article, we'll examine the double-checked locking idiom in detail to understand just where it breaks down.

Singleton creation idiom

To understand where the double-checked locking idiom originated, you must understand the common singleton creation idiom, which is illustrated in Listing 1:


Listing 1. Singleton creation idiom

import java.util.*;
class Singleton
{
private static Singleton instance;
private Vector v;
private boolean inUse;

private Singleton()
{
v = new Vector();
v.addElement(new Object());
inUse = true;
}

public static Singleton getInstance()
{
if (instance == null) //1
instance = new Singleton(); //2
return instance; //3
}
}


The design of this class ensures that only one Singleton object is ever created. The constructor is declared private and the getInstance() method creates only one object. This implementation is fine for a single-threaded program. However, when multiple threads are introduced, you must protect the getInstance() method through synchronization. If the getInstance() method is not protected, it is possible to return two different instances of the Singleton object. Consider two threads calling the getInstance() method concurrently and the following sequence of events:

1. Thread 1 calls the getInstance() method and determines that instance is null at //1.

2. Thread 1 enters the if block, but is preempted by thread 2 before executing the line at //2.

3. Thread 2 calls the getInstance() method and determines that instance is null at //1.

4. Thread 2 enters the if block and creates a new Singleton object and assigns the variable instance to this new object at //2.

5. Thread 2 returns the Singleton object reference at //3.

6. Thread 2 is preempted by thread 1.

7. Thread 1 starts where it left off and executes line //2 which results in another Singleton object being created.

8. Thread 1 returns this object at //3.

The result is that the getInstance() method created two Singleton objects when it was supposed to create only one. This problem is corrected by synchronizing the getInstance() method to allow only one thread to execute the code at a time, as shown in Listing 2:


Listing 2. Thread-safe getInstance() method

public static synchronized Singleton getInstance()
{
if (instance == null) //1
instance = new Singleton(); //2
return instance; //3
}


The code in Listing 2 works fine for multithreaded access to the getInstance() method. However, when you analyze it you realize that synchronization is required only for the first invocation of the method. Subsequent invocations do not require synchronization because the first invocation is the only invocation that executes the code at //2, which is the only line that requires synchronization. All other invocations determine that instance is non-null and return it. Multiple threads can safely execute concurrently on all invocations except the first. However, because the method is synchronized, you pay the cost of synchronization for every invocation of the method, even though it is only required on the first invocation.

In an effort to make this method more efficient, an idiom called double-checked locking was created. The idea is to avoid the costly synchronization for all invocations of the method except the first. The cost of synchronization differs from JVM to JVM. In the early days, the cost could be quite high. As more advanced JVMs have emerged, the cost of synchronization has decreased, but there is still a performance penalty for entering and leaving a synchronized method or block. Regardless of the advancements in JVM technology, programmers never want to waste processing time unnecessarily.

Because only line //2 in Listing 2 requires synchronization, we could just wrap it in a synchronized block, as shown in Listing 3:


Listing 3. The getInstance() method

public static Singleton getInstance()
{
if (instance == null)
{
synchronized(Singleton.class) {
instance = new Singleton();
}
}
return instance;
}


The code in Listing 3 exhibits the same problem as demonstrated with multiple threads and Listing 1. Two threads can get inside of the if statement concurrently when instance is null. Then, one thread enters the synchronized block to initialize instance, while the other is blocked. When the first thread exits the synchronized block, the waiting thread enters and creates another Singleton object. Note that when the second thread enters the synchronized block, it does not check to see if instance is non-null.


Back to top


Double-checked locking

To fix the problem in Listing 3, we need a second check of instance. Thus, the name "double-checked locking." Applying the double-checked locking idiom to Listing 3 results in Listing 4.


Listing 4. Double-checked locking example

public static Singleton getInstance()
{
if (instance == null)
{
synchronized(Singleton.class) { //1
if (instance == null) //2
instance = new Singleton(); //3
}
}
return instance;
}


The theory behind double-checked locking is that the second check at //2 makes it impossible for two different Singleton objects to be created as occurred in Listing 3. Consider the following sequence of events:

1. Thread 1 enters the getInstance() method.

2. Thread 1 enters the synchronized block at //1 because instance is null.

3. Thread 1 is preempted by thread 2.

4. Thread 2 enters the getInstance() method.

5. Thread 2 attempts to acquire the lock at //1 because instance is still null. However, because thread 1 holds the lock, thread 2 blocks at //1.

6. Thread 2 is preempted by thread 1.

7. Thread 1 executes and because instance is still null at //2, creates a Singleton object and assigns its reference to instance.

8. Thread 1 exits the synchronized block and returns instance from the getInstance() method.

9. Thread 1 is preempted by thread 2.

10. Thread 2 acquires the lock at //1 and checks to see if instance is null.

11. Because instance is non-null, a second Singleton object is not created and the one created by thread 1 is returned.

The theory behind double-checked locking is perfect. Unfortunately, reality is entirely different. The problem with double-checked locking is that there is no guarantee it will work on single or multi-processor machines.

The issue of the failure of double-checked locking is not due to implementation bugs in JVMs but to the current Java platform memory model. The memory model allows what is known as "out-of-order writes" and is a prime reason why this idiom fails.


Back to top


Out-of-order writes

To illustrate the problem, you need to re-examine line //3 from Listing 4 above. This line of code creates a Singleton object and initializes the variable instance to refer to this object. The problem with this line of code is that the variable instance can become non-null before the body of the Singleton constructor executes.

Huh? That statement might be contradictory to everything you thought possible, but it is, in fact, the case. Before explaining how this happens, accept this fact for a moment while examining how this breaks the double-checked locking idiom. Consider the following sequence of events with the code in Listing 4:

1. Thread 1 enters the getInstance() method.

2. Thread 1 enters the synchronized block at //1 because instance is null.

3. Thread 1 proceeds to //3 and makes instance non-null, but before the constructor executes.

4. Thread 1 is preempted by thread 2.

5. Thread 2 checks to see if instance is null. Because it is not, thread 2 returns the instance reference to a fully constructed, but partially initialized, Singleton object.

6. Thread 2 is preempted by thread 1.

7. Thread 1 completes the initialization of the Singleton object by running its constructor and returns a reference to it.

This sequence of events results in a period of time where thread 2 returned an object whose constructor had not executed.

To show how this occurs, consider the following pseudo code for the line: instance =new Singleton();

mem = allocate(); //Allocate memory for Singleton object.
instance = mem; //Note that instance is now non-null, but
//has not been initialized.
ctorSingleton(instance); //Invoke constructor for Singleton passing
//instance.


This pseudo code is not only possible, but is exactly what happens on some JIT compilers. The order of execution is perceived to be out of order, but is allowed to happen given the current memory model. The fact that JIT compilers do just this makes the issues of double-checked locking more than simply an academic exercise.

To demonstrate this, consider the code in Listing 5. It contains a stripped-down version of the getInstance() method. I've removed the "double-checkedness" to ease our review of the assembly code produced (Listing 6). We are interested only in seeing how the line instance=new Singleton(); is compiled by the JIT compiler. In addition, I've provided a simple constructor to make it clear when the constructor is run in the assembly code.


Listing 5. Singleton class to demonstrate out-of-order writes

class Singleton
{
private static Singleton instance;
private boolean inUse;
private int val;

private Singleton()
{
inUse = true;
val = 5;
}
public static Singleton getInstance()
{
if (instance == null)
instance = new Singleton();
return instance;
}
}


Listing 6 contains the assembly code produced by the Sun JDK 1.2.1 JIT compiler for the body of the getInstance() method from Listing 5.


Listing 6. Assembly code produced from code in Listing 5

;asm code generated for getInstance
054D20B0 mov eax,[049388C8] ;load instance ref
054D20B5 test eax,eax ;test for null
054D20B7 jne 054D20D7
054D20B9 mov eax,14C0988h
054D20BE call 503EF8F0 ;allocate memory
054D20C3 mov [049388C8],eax ;store pointer in
;instance ref. instance
;non-null and ctor
;has not run
054D20C8 mov ecx,dword ptr [eax]
054D20CA mov dword ptr [ecx],1 ;inline ctor - inUse=true;
054D20D0 mov dword ptr [ecx+4],5 ;inline ctor - val=5;
054D20D7 mov ebx,dword ptr ds:[49388C8h]
054D20DD jmp 054D20B0


Note: To reference the lines of assembly code in the following explanation, I refer to the last two values of the instruction address because they all begin with 054D20. For example, B5 refers to test eax,eax.

The assembly code is produced by running a test program that calls the getInstance() method in an infinite loop. While the program runs, run the Microsoft Visual C++ debugger and attach it to the Java process representing the test program. Then, break the execution and find the assembly code representing the infinite loop.

The first two lines of assembly code at B0 and B5 load the instance reference from memory location 049388C8 into eax and test for null. This corresponds to the first line of the getInstance() method in Listing 5. The first time this method is called, instance is null and the code proceeds to B9 . The code at BE allocates the memory from the heap for the Singleton object and stores a pointer to that memory in eax. The next line, C3, takes the pointer in eax and stores it back into the instance reference at memory location 049388C8. As a result, instance is now non-null and refers to a valid Singleton object. However, the constructor for this object has not run yet, which is precisely the situation that breaks double-checked locking. Then at line C8, the instance pointer is dereferenced and stored in ecx. Lines CA and D0 represent the inline constructor storing the values true and 5 into the Singleton object. If this code is interrupted by another thread after executing line C3 but before completing the constructor, double-checked locking fails.

Not all JIT compilers generate the code as above. Some generate code such that instance becomes non-null only after the constructor executes. Both the IBM SDK for Java technology, version 1.3 and the Sun JDK 1.3 produce code such as this. However, this does not mean you should use double-checked locking in these instances. There are other reasons it could fail. In addition, you do not always know which JVMs your code will run on, and the JIT compiler could always change to generate code that breaks this idiom.


Back to top


Double-checked locking: Take two

Given that the current double-checked locking code does not work, I've put together another version of the code, shown in Listing 7, to try to prevent the out-of-order write problem you just saw.


Listing 7. Attempting to solve the out-of-order write problem

public static Singleton getInstance()
{
if (instance == null)
{
synchronized(Singleton.class) { //1
Singleton inst = instance; //2
if (inst == null)
{
synchronized(Singleton.class) { //3
inst = new Singleton(); //4
}
instance = inst; //5
}
}
}
return instance;
}


Looking at the code in Listing 7 you should realize that things are getting a little ridiculous. Remember, double-checked locking was created as a way to avoid synchronizing the simple three-line getInstance() method. The code in Listing 7 has gotten out of hand. In addition, the code does not fix the problem. Careful examination reveals why.

This code is trying to avoid the out-of-order write problem. It tries to do this by introducing the local variable inst and a second synchronized block. The theory works as follows:

1. Thread 1 enters the getInstance() method.

2. Because instance is null, thread 1 enters the first synchronized block at //1.

3. The local variable inst gets the value of instance, which is null at //2.

4. Because inst is null, thread 1 enters the second synchronized block at //3.

5. Thread 1 then begins to execute the code at //4, making inst non-null but before the constructor for Singleton executes. (This is the out-of-order write problem we just saw.)

6. Thread 1 is preempted by Thread 2.

7. Thread 2 enters the getInstance() method.

8. Because instance is null, thread 2 attempts to enter the first synchronized block at //1. Because thread 1 currently holds this lock, thread 2 blocks.

9. Thread 1 then completes its execution of //4.

10. Thread 1 then assigns a fully constructed Singleton object to the variable instance at //5 and exits both synchronized blocks.

11. Thread 1 returns instance.

12. Thread 2 then executes and assigns instance to inst at //2.

13. Thread 2 sees that instance is non-null, and returns it.

The key line here is //5. This line is supposed to ensure that instance will only ever be null or refer to a fully constructed Singleton object. The problem occurs where theory and reality run orthogonal to one another.

The code in Listing 7 doesn't work because of the current definition of the memory model. The Java Language Specification (JLS) demands that code within a synchronized block not be moved out of a synchronized block. However, it does not say that code not in a synchronized block cannot be moved into a synchronized block.

A JIT compiler would see an optimization opportunity here. This optimization would remove the code at //4 and the code at //5, combine it and generate the code shown in Listing 8:


Listing 8. Optimized code from Listing 7

public static Singleton getInstance()
{
if (instance == null)
{
synchronized(Singleton.class) { //1
Singleton inst = instance; //2
if (inst == null)
{
synchronized(Singleton.class) { //3
//inst = new Singleton(); //4
instance = new Singleton();
}
//instance = inst; //5
}
}
}
return instance;
}


If this optimization takes place, you have the same out-of-order write problem we discussed earlier.


Back to top


volatile anyone?

Another idea is to use the keyword volatile for the variables inst and instance. According to the JLS (see Resources), variables declared volatile are supposed to be sequentially consistent, and therefore, not reordered. But two problems occur with trying to use volatile to fix the problem with double-checked locking:

* The problem here is not with sequential consistency. Code is being moved, not reordered.

* Many JVMs do not implement volatile correctly regarding sequential consistency anyway.

The second point is worth expanding upon. Consider the code in Listing 9:


Listing 9. Sequential consistency with volatile

class test
{
private volatile boolean stop = false;
private volatile int num = 0;

public void foo()
{
num = 100; //This can happen second
stop = true; //This can happen first
//...
}

public void bar()
{
if (stop)
num += num; //num can == 0!
}
//...
}


According to the JLS, because stop and num are declared volatile, they should be sequentially consistent. This means that if stop is ever true, num must have been set to 100. However, because many JVMs do not implement the sequential consistency feature of volatile, you cannot count on this behavior. Therefore, if thread 1 called foo and thread 2 called bar concurrently, thread 1 might set stop to true before num is set to 100. This could lead thread 2 to see stop as true, but num still set to 0. There are additional problems with volatile and the atomicity of 64-bit variables, but this is beyond the scope of this article. See Resources for more information on this topic.


Back to top


The solution

The bottom line is that double-checked locking, in whatever form, should not be used because you cannot guarantee that it will work on any JVM implementation. JSR-133 is addressing issues regarding the memory model, however, double-checked locking will not be supported by the new memory model. Therefore, you have two options:

* Accept the synchronization of a getInstance() method as shown in Listing 2.

* Forgo synchronization and use a static field.

Option 2 is shown in Listing 10:


Listing 10. Singleton implementation with static field

class Singleton
{
private Vector v;
private boolean inUse;
private static Singleton instance = new Singleton();

private Singleton()
{
v = new Vector();
inUse = true;
//...
}

public static Singleton getInstance()
{
return instance;
}
}


The code in Listing 10 does not use synchronization and ensures that the Singleton object is not created until a call is made to the static getInstance() method. This is a good alternative if your objective is to eliminate synchronization.


Back to top


String is not immutable

You might wonder about the String class given the issue of out-of-order writes and a reference becoming non-null prior to the constructor executing. Consider the following code:

private String str;
//...
str = new String("hello");


The String class is supposed to be immutable. However, given the out-of-order write problem we discussed previously, could that cause a problem here? The answer is it could. Consider two threads with access to the String str. One thread could see the str reference refer to a String object in which the constructor has not run. In fact, Listing 11 contains code that shows this occurring. Note that this code breaks only with older JVMs that I tested. Both the IBM 1.3 and Sun 1.3 JVMs produce immutable Strings as expected.


Listing 11. Example of a Mutable String

class StringCreator extends Thread
{
MutableString ms;
public StringCreator(MutableString muts)
{
ms = muts;
}
public void run()
{
while(true)
ms.str = new String("hello"); //1
}
}
class StringReader extends Thread
{
MutableString ms;
public StringReader(MutableString muts)
{
ms = muts;
}
public void run()
{
while(true)
{
if (!(ms.str.equals("hello"))) //2
{
System.out.println("String is not immutable!");
break;
}
}
}
}
class MutableString
{
public String str; //3
public static void main(String args[])
{
MutableString ms = new MutableString(); //4
new StringCreator(ms).start(); //5
new StringReader(ms).start(); //6
}
}


This code creates a MutableString class at //4 that contains a String reference shared by two threads at //3. Two objects are created, StringCreator and StringReader, on two separate threads at lines //5 and //6, passing a reference to the MutableString object. The StringCreator class enters an infinite loop and creates String objects with the value "hello" at //1. The StringReader also enters an infinite loop and checks to see if the current String object has the value "hello" at //2. If it doesn't, the StringReader thread prints out a message and stops. If the String class is immutable, you should never see any output from this program. The only way for StringReader to see the str reference to be anything other than a String object with "hello" as its value is if the problem of out-of-order writes occurs.

Running this code on old JVMs like Sun JDK 1.2.1 results in the out-of-order write problem, and thus, a non-immutable String.


Back to top


Summary

In an effort to avoid costly synchronization in singletons, programmers, quite ingeniously, invented the double-checked locking idiom. Unfortunately, it was not until this idiom was in fairly wide use that it became apparent that it is not a safe programming construct due to the current memory model. Work is underway to redefine areas of the memory model that are weak. However, even under the newly proposed memory model, double-checked locking will not work. The best solution to this problem is to accept synchronization or use a static field.


Resources

* In his book Practical Java Programming Language Guide (Addison-Wesley, 2000), Peter Haggar covers a multitude of Java programming topics including an entire chapter on multithreading issues and programming techniques.

* The Java Language Specification, Second Edition by Bill Joy, et. al. (Addison-Wesley, 2000) is the definitive technical reference for the Java programming language.

* The Java Virtual Machine Specification, Second Edition by Tim Lindholm and Frank Yellin (Addison-Wesley, 1999) is the definitive document on Java compilers and runtime environments.

* Visit Bill Pugh's Java Memory Model Web site for a wealth of information on this important topic.

* For more information on volatile and 64-bit variables, see Peter Haggar's article "Does Java Guarantee Thread Safety?" in the June 2002 issue of Dr. Dobb's Journal .

* JSR-133 deals with the revision to Java platform's memory model and thread specification.

* Java software consultant Brian Goetz examines when to use synchronization in "Threading lightly: Synchronization is not the enemy" (developerWorks, July 2001).

* In "Threading lightly: Sometimes it's best not to share" (developerWorks, October 2001), Brian Goetz examines ThreadLocal and offers tips for exploiting its power.

* In "Writing multithreaded Java applications" (developerWorks, February 2001), Alex Roetter introduces the Java Thread API, outlines issues involved in multithreading, and offers solutions to common problems.

* Allen Holub proposes significant changes and additions to the Java language in "If I were king: A proposal for fixing the Java programming language's threading problems" (developerWorks, October 2000).

* Find other Java technology resources on the developerWorks Java technology zone.



About the author



Peter Haggar is a Senior Software Engineer with IBM in Research Triangle Park, North Carolina, and the author of the book Practical Java Programming Language Guide published by Addison-Wesley. In addition, he has published numerous articles on Java programming. He has a broad range of programming experience, having worked on development tools, class libraries, and operating systems. At IBM, Peter works on emerging Internet technology and is currently focused on high-performance Web services. Peter is a frequent technical speaker on Java technology at numerous industry conferences. He has worked for IBM for more than 14 years and received a B.S. in Computer Science from Clarkson University. Contact Peter at haggar@us.ibm.com.

Computers 'could store entire life by 2026' by Nic Fleming, Science Correspondent

  • Audio: You'll be able to record your entire life on video, says Nic Fleming
  • A device the size of a sugar cube will be able to record and store high resolution video footage of every second of a human life within two decades, experts said yesterday.

    Researchers said governments and societies must urgently debate the implications of the huge increases in computing power and the growing mass of information being collected on individuals.


    Some fear that the advent of "human black boxes" combined with the extension of medical, financial and other digital records will lead to loss of privacy and a dramatic expansion of the nanny state.

    Others highlight positive advances in medicine, education, crime prevention and the way history will be recorded.

    Leading computer scientists, psychologists and neuroscientists gathered to debate these issues at Memories for Life, a conference held at the British Library yesterday.

    Prof Nigel Shadbolt, president of the British Computer Society and professor of artificial intelligence at the University of Southampton, said: "In 20 years' time it will be possible to record high quality digital video of an entire lifetime of human memories. It's not a question of whether it will happen; it's already happening."

    A lap top available in the High Street can hold some 80 gigabytes (GB) of information. One hour of high resolution video footage requires 12GB.

    Since the year 2000, computing processing power has been doubling approximately every 18 months - a phenomenon known as Moore’s Law.

    Prof Shadbolt has calculated that it would take 5.5 petabytes (PT) to record every awake second of a person’s life in high resolution video.

    One PT equals one million GB. Experts expect the increase in computing power to lead to advances in “ehealth” with doctors having access to information from devices that monitor physiological data such as heart rate and blood sugar levels.

    Retailers want to get more information on their customers’ habits than they already have from their loyalty cards. The technological advances will also have a dramatic impact on the writing of biographies and history, with authors and historians able to gain vastly more information on key figures.

    Cliff Lynch, director of the US think tank Coalition for Networked Information, said the changes would allow the preservation of much more detailed memories, but could lead to a dramatic extension of state interference.

    “We will be able to replicate and pass on so much more information. In future you are going to have a much more elaborate picture for more and more people.

    “Biographers and other kinds of scholars who want to understand what someone was thinking are going to be based with an embarrassment of riches.

    “There is a certain tendency towards a technological nanny state. Imagine having a personal companion that wines at you three times a day, telling you that you are eating the wrong things and that you spent more than you earned today and you’ll never be able to retire.

    “Imagine we could end up with smart refrigerator that tells you ‘you’ve already had your beer for the day, you can’t have another one’.

    “I don’t think people would want a world like that, but the scary thing is it might be foisted on them.”

    Prof Wendy Hall, of the University of Southampton, said: “Technology can play a vital role in memory, for example by providing an artificial aid to help those with memory disorders or enabling communities to create and preserve their collective experiences.

    “However, we must also consider the social, ethical and legal issues associated with technology development and how increased access to knowledge will affect our society in open, inter-disciplinary forums.”


    Sunday, December 31, 2006

    Physics promises wireless power by Jonathan Fildes

    The tangle of cables and plugs needed to recharge today's electronic gadgets could soon be a thing of the past.

    US researchers have outlined a relatively simple system that could deliver power to devices such as laptop computers or MP3 players without wires.

    The concept exploits century-old physics and could work over distances of many metres, the researchers said.

    Although the team has not built and tested a system, computer models and mathematics suggest it will work.

    "There are so many autonomous devices such as cell phones and laptops that have emerged in the last few years," said Assistant Professor Marin Soljacic from the Massachusetts Institute of Technology and one of the researchers behind the work.

    "We started thinking, 'it would be really convenient if you didn't have to recharge these things'.

    "And because we're physicists we asked, 'what kind of physical phenomenon can we use to do this wireless energy transfer?'."

    The answer the team came up with was "resonance", a phenomenon that causes an object to vibrate when energy of a certain frequency is applied.


    This would work in a room let's say but you could adapt it to work in a factory
    Marin Soljacic

    "When you have two resonant objects of the same frequency they tend to couple very strongly," Professor Soljacic told the BBC News website.

    Resonance can be seen in musical instruments for example.

    "When you play a tune on one, then another instrument with the same acoustic resonance will pick up that tune, it will visibly vibrate," he said.

    Instead of using acoustic vibrations, the team's system exploits the resonance of electromagnetic waves. Electromagnetic radiation includes radio waves, infrared and X-rays.

    Typically, systems that use electromagnetic radiation, such as radio antennas, are not suitable for the efficient transfer of energy because they scatter energy in all directions, wasting large amounts of it into free space.

    To overcome this problem, the team investigated a special class of "non-radiative" objects with so-called "long-lived resonances".

    When energy is applied to these objects it remains bound to them, rather than escaping to space. "Tails" of energy, which can be many metres long, flicker over the surface.

    "If you bring another resonant object with the same frequency close enough to these tails then it turns out that the energy can tunnel from one object to another," said Professor Soljacic.

    Hence, a simple copper antenna designed to have long-lived resonance could transfer energy to a laptop with its own antenna resonating at the same frequency. The computer would be truly wireless.

    Any energy not diverted into a gadget or appliance is simply reabsorbed.

    The systems that the team have described would be able to transfer energy over three to five metres.

    "This would work in a room let's say but you could adapt it to work in a factory," he said.

    "You could also scale it down to the microscopic or nanoscopic world."

    Old technology

    The team from MIT is not the first group to suggest wireless energy transfer.

    Nineteenth-century physicist and engineer Nikola Tesla experimented with long-range wireless energy transfer, but his most ambitious attempt - the 29m high aerial known as Wardenclyffe Tower, in New York - failed when he ran out of money.



    Others have worked on highly directional mechanisms of energy transfer such as lasers.

    However, these require an uninterrupted line of sight, and are therefore not good for powering objects around the home.

    A UK company called Splashpower has also designed wireless recharging pads onto which gadget lovers can directly place their phones and MP3 players to recharge them.

    The pads use electromagnetic induction to charge devices, the same process used to charge electric toothbrushes.

    One of the co-founders of Splashpower, James Hay, said the MIT work was "clearly at an early stage" but "interesting for the future".

    "Consumers desire a simple universal solution that frees them from the hassles of plug-in chargers and adaptors," he said.

    "Wireless power technology has the potential to deliver on all of these needs."

    However, Mr Hay said that transferring the power was only part of the solution.

    "There are a number of other aspects that need to be addressed to ensure efficient conversion of power to a form useful to input to devices."

    Professor Soljacic will present the work at the American Institute of Physics Industrial Physics Forum in San Francisco on 14 November.

    The work was done in collaboration with his colleagues Aristeidis Karalis and John Joannopoulos.

    HOW WIRELESS POWER COULD WORK
    1) Power from mains to antenna, which is made of copper
    2) Antenna resonates at a frequency of 6.4MHz, emitting electromagnetic waves
    3) 'Tails' of energy from antenna 'tunnel' up to 5m (16.4ft)
    4) Electricity picked up by laptop's antenna, which must also be resonating at 6.4MHz. Energy used to re-charge device
    5) Energy not transferred to laptop re-absorbed by source antenna. People/other objects not affected as not resonating at 6.4MHz


    (track back URL : http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/6129460.stm)

    Wednesday, October 25, 2006

    How to Check Your Website with Multiple Browsers on a Single Machine (Cross-Browser Compatibility Checking)

    How to Check Your Website with Multiple Browsers on a Single Machine (Cross-Browser Compatibility Checking)
    by Christopher Heng, thesitewizard.com

    We all know the importance of checking our web pages with multiple browsers, especially when we are designing a new layout for a website. The number of extant browsers we need to check with are enormous: Internet Explorer (IE) 6, IE 5.5, IE 5.0, Netscape 7.X (ie, Mozilla 1.0.X), Netscape 6.X (or Mozilla 0.9.X), Mozilla 1.3.X (and above), Opera 7, Opera 6/5, Netscape 4.X, IE 4.X and so on. And then there are the different platforms: Windows, Macintosh (Mac), Linux, etc. The problem for most people is that multiple versions of certain browsers cannot co-exist with each other, the most notable example of this is IE for Windows. Unless you are privileged to have multiple computers, this presents a certain difficulty for the average webmaster. This article suggests some ways for you to run multiple versions of multiple browsers on one computer.
    Note that this article is written primarily from the point of view of a person using Windows (the majority of people reading this article), although it does address the issue of Mac browsers and Linux browsers as well.
    Mozilla and Netscape
    It's possible for Netscape 4.X, Netscape 6.X (or Mozilla 0.9.X), Netscape 7 (Mozilla 1.0.X), Mozilla 1.1.X, Mozilla 1.2.X, Mozilla 1.3.X (and so on) to all co-exist on the same machine.
    1. Netscape 4.X
    You should have one version of the Netscape 4.X series installed. Take your pick - they are all approximately the same in their level of support for the Cascading Style Sheets (CSS) standards. Some people prefer to use one of the older versions on the basis that if their page renders correctly on that version, it should theoretically render correctly on the later 4.X versions. My approach is to simply install the latest in this series: I suspect that fixes made in the later versions are mainly security fixes. In any case, my testing with this browser is restricted to making sure that people using the browser can read and navigate thesitewizard.com and thefreecountry.com with the browser. I don't spend any time at all making the site look good for the browser, since the number of people using this browser is decreasing all the time. (It's the law of diminishing returns.)
    Netscape makes older versions of its browsers available from http://wp.netscape.com/download/archive.html
    2. Netscape 6.X, Netscape 7.X, Mozilla 1.X
    If you did not already know, Netscape 6/7 (and later) and Mozilla use the same Gecko rendering engine. As such, if you have Netscape 6.X, you are in effect using the rendering engine of a beta version of Mozilla (one of the 0.9.X series); if you use Netscape 7, you are using the Mozilla 1.0.X engine; and if you use Netscape 7.1 you're using the same engine as Mozilla 1.4. The point is that you don't have to install, say, Mozilla 1.0.X if you're using Netscape 7, and so on. My personal preference is to have one of each of the major releases of Netscape installed (or their Mozilla equivalent), as well as the latest released version of Mozilla. At the time I write this, this works out to be Netscape 6.X, Netscape 7 and Mozilla 1.4 (same as Netscape 7.1).
    It is easy to make these versions of Netscape/Mozilla co-exist with each other. Install them into separate directories and create a different profile for each browser you install. (For non-Netscape/Mozilla users, this browser allows you to create different profiles so that you can store different settings for different situations.) To create a different profile, simply start up the Mozilla or Netscape Profile Manager, and answer the questions given by the wizard. Be sure you do this before you start configuring each browser, or the settings you make in one browser may bleed over to the other, and possibly confuse the other version.
    Once you've finished creating profiles, you will want to create shortcuts (Windows terminology) to run the different versions of the browser. This makes life easier for you: you can simply click the appropriate icon for the different versions, and it will load using the correct profile. To specify which profile the browser is to load, put the profile name after the "-P" option.
    For example, if you have created a profile named "netscape6", your command for running (say) Netscape with that profile may look like:

    "C:\Program Files\Netscape\Netscape 6\netscp6.exe" -P netscape6

    Similarly, your command to run Mozilla 1.4 with a profile called "mozilla" may look like:

    "C:\Program Files\mozilla.org\Mozilla\mozilla.exe" -P mozilla

    And so on.
    The latest version of Netscape can be obtained from http://channels.netscape.com/ns/browsers/download.jsp
    Mozilla can be obtained from http://www.mozilla.org
    Opera for Windows/Linux
    Opera is the third most used browser used on thesitewizard.com and thefreecountry.com, and is particularly popular among the seasoned webmaster community (which probably explains why so many of my visitors use it).
    Opera 6 and 7 can co-exist on one computer. If you want to install both versions on Windows, simply put them in separate directories. No extra steps are required. I don't know if this also works with earlier versions of Opera, since I don't bother to check my site with those.
    As for Linux, at the time I write this, I only have version 6 of Opera installed, so I have no idea if two versions can co-exist on that platform.
    You can obtain Opera for any of its supported operating systems from Opera's website. To get older versions, just navigate to the downloads page and click the "Opera archives" link.
    Internet Explorer 5.0, 5.5, 6.0 for Windows
    In all my sites, at the time of this writing, IE users comprise the majority of visitors, with the bulk of them using IE 6.0.
    My experience in coding the recent new designs for thesitewizard.com and thefreecountry.com, both of which depend heavily upon Cascading Style Sheets for layout, is that IE 6 is a very different animal from IE 5.X. Contrary to what you may expect, what works in IE 5.5 does not necessarily work in IE 6. In fact, if my limited experience with coding my sites is anything to go by, what works for IE 6 is more likely to work for IE 5.5 than vice versa. As a result, if you can only install one version of IE, and your site uses CSS, a case can be made that it is better to install IE 6 than 5.X.
    (It seems that IE 6 has a number of bugs in its CSS box model, causing sites that work in Mozilla/Netscape, Opera and IE 5.X to break under IE 6. That is not to say that IE 5 does not have bugs. All the more reason to install multiple versions.)
    Unlike Opera, Mozilla and Netscape, you can only install one version of IE in a single installation of Windows. The bulk of IE's code does not get installed into its own subdirectory (or folder) but into Windows' system directory, so even if you somehow successfully install different versions of IE, they will all wind up using the code for the latest installed version (assuming they don't crash).
    1. Running Two or Three Versions of IE on One Machine
    The simplest way to run two versions of IE on a single machine is to install two versions of Windows in that machine. That is, install Windows 95/98/ME (ie, either Windows 95 or 98 or 98 Second Edition or ME) onto one partition on your hard drive, and install Windows 2000 or XP in another partition in a dual-boot configuration.
    In plain English, this means that you need to partition your hard disk into (at least) two partitions. First install Windows 95/98/98SE/ME into drive C:. When you finish that, run the Windows 2000 or Windows XP setup program from within the first version of Windows you installed. Select the "Clean install" option and not the "Upgrade" option when you're asked. You will be presented with a window later where you can click the "Advanced options" button. In the window that is displayed, check the box labelled "I want to choose the installation partition during setup". Later in the installation process, when you are asked, choose to install Windows 2000 (or XP) in the second partition. Windows 2000 or XP will then (automatically) install a menu that appears when you start up your machine, allowing you to choose the version of Windows you want to boot.
    It is apparently also possible to install three versions of Windows (and hence three versions of IE) into one machine without resorting to third party software. You'll need to have at least three partitions on your hard disk to play with. Install Windows 95/98/98SE/ME into C: and use the above procedure to install Windows 2000 into the second partition. Finally, use the same procedure to install Windows XP into the third partition. I have not tried "triple booting" three Windows versions before, but one of thesitewizard.com's visitors has assured me that it works. With three versions of Windows to play with, you should be able to use three versions of IE on the same machine.
    Note that Windows 95 installs IE 3 by default, Windows 98 installs IE 4, Windows 98 Second Edition installs IE 5.0, Windows 2000 installs IE 5.0, Windows ME installs IE 5.5 and Windows XP installs IE 6.0, so if you might want to plan carefully which Windows version you install so that you get the IE version you need for testing. As far as I know, you can't easily (if at all) "downgrade" a version of IE (although you can upgrade it), so if you need something as old as IE 4, you might need to install Windows 98 (or even Windows 95 and upgrade IE to 4.0).
    2. Running More than Three Versions of Internet Explorer
    If you really need to run more than three versions of IE and can't afford to (or don't want to) use another machine, you'll need to install either an emulator or use one of those software that creates a virtual machine (sometimes called a virtualizer).
    Loosely speaking, an emulator allows you to run another copy of Windows within your existing version of Windows or Linux (or FreeBSD or whatever). The emulator pretends to be a new computer, and Windows gets installed into a small space on your hard disk which the emulator uses to mimic an entire drive.
    You can find a list of free PC emulators and virtualizers listed on thefreecountry.com's Free PC (x86) Emulators and Virtual Machines at http://www.thefreecountry.com/emulators/pc.shtml
    Some of the emulators listed there, such as Bochs, allow you to emulate a PC machine on which you can install Windows or Linux or any other PC operating system. Bochs itself may be installed on Windows, Linux, Mac, and possibly other systems and machines as well. If you are a Mac user, this is one way to run the Windows version of IE without having to buy a PC.
    Be warned though that full emulators like Bochs are extremely slow. You may be able to tolerate it though, if you are only using it to test your web pages once in a blue moon.
    If you already have a PC, another solution is to run a program that can create virtual machines. While emulators can pretend to be a completely different machine (such as a Mac pretending to be a PC), virtual machines merely sets up a space within your existing machine and runs a new copy of the operating system (such as Windows) within that space. The new copy of (say) Windows thinks that it is the only one running (even if you are running it within (say) yet another copy of Windows. Since virtual machines do not have to emulate a completely different machine, they tend to be slightly faster than full emulators.
    Free virtual machines can be found on the same page as the emulators on Free PC (x86) Emulators and Virtualizers at http://www.thefreecountry.com/emulators/pc.shtml
    Once you have obtained an emulator or a virtualizer, you simply install a new copy of Windows in each "machine" you set up. For each copy of Windows you set up, install a different version of IE, and you're done.
    Testing Mac Browsers
    As you might expect, the easiest way to test your page on Mac browsers (like Safari, Camino, Mozilla, Opera, etc) is to actually own a Mac.
    However, work is under way in the open source software community to create a PowerPC Mac emulator for x86 machines (ie, PCs) that will run Mac OS X. Once this project is complete, you should be able to simply download the emulator, buy a copy of Mac OS X, install them on your PC, and you have a working Macintosh in a window of your Linux or Windows machine that you can use to test your site under Mac browsers. Don't expect your emulated Macintosh to perform at the same speed as a real Mac though. For the purpose of occasionally testing a website or two, though, the emulator will probably suffice if you have a healthy dose of patience.
    You can also run Mac emulators for the older 68k Macs (like Quadra, Performa, etc). These emulators, however, require you to have a real Mac around, since you need to copy the ROM from one of those Macs before the emulator will work. In any case, for the purpose of testing web pages, the 68k Macintosh emulators are not very useful, since they can only emulate the older Macs, which only run browsers like IE 4 and Netscape 4. If your site is like mine, you will find that people using such browsers are few and far between (if at all).
    Anyway, if you're curious, you can find free Mac emulators listed on thefreecountry.com's Free 68k and PowerPC Macintosh Emulators at http://www.thefreecountry.com/emulators/macintosh.shtml
    Testing Linux Browsers
    One of the easiest ways to test your site to see how it appears under Linux is to run Linux from a CDROM. There are numerous Linux "live" CDROMs around. These allow you to simply boot your machine from the CDROM directly into Linux without having to install anything onto your hard disk. One of the best-known "live" Linux systems around is Knoppix, which you can obtain from http://www.knopper.net/knoppix/index-en.html. Essentially, all you have to do is to download the ISO (which is just an image of the CDROM), burn it to your CDR, put it in your CDROM drive, and restart your computer. Knoppix is free.
    Alternatively, if you prefer to install Linux on your hard disk, it can be set up so that it co-exists with Windows. Make sure you have space for a new partition on your hard disk, install it and you're done.
    The default browser that comes on many Linux distributions is Mozilla. However, you will find that even though Mozilla basically renders your page identically for all platforms, the fonts available under Linux are different from those available on Windows. If you don't code your font tags or CSS font-families in a cross-platform compatible way, your site may wind up being rendered with an ugly font. For example, many sites simply specify "Arial" or "Impact" or some such Windows font for their site. Since these fonts are not available by default under non-Windows systems, your site will be rendered using either the default font on those browsers or some other font that the browser thinks matches the type of font you've specified. If you don't want to bother to run Linux to test, be sure that you at least:
    Test your pages under Mozilla for your platform.
    Specify generally available alternative fonts for your web pages. For example, don't just say,

    font-family: Arial ;

    in your style sheet, say

    font-family: Arial, Helvetica, sans-serif ;

    instead.
    There are a number of other browsers available on the Linux platform. You may have seen them in your web logs. They include, for example, Konqueror, Opera for Linux and Mozilla spin-offs.
    Conclusion
    It's a good idea to test your site with multiple versions of multiple browsers, particularly if you plan to do anything fancy with style sheets on your site. This doesn't mean that you have to support all browsers - for example, the pages on thesitewizard.com do not work well (if at all) under IE 4 and Netscape 3 (and earlier). However, when you are able to test your pages this way, you can at least reduce the number of problems your pages have with the different browsers. The tips in this article allow you to test with multiple browsers even if you have only one machine.
    Copyright 2003-2004 by Christopher Heng. All rights reserved.Get more free tips and articles like this, on web design, promotion, revenue and scripting, from http://www.thesitewizard.com/.
    This article can be found at http://www.thesitewizard.com/webdesign/multiplebrowsers.shtml
    thesitewizard™ RSS Site Feed
    Do you find this article useful? You can learn of new articles and scripts that are published on thesitewizard.com by subscribing to the RSS feed. Simply point your RSS feed reader or a browser that supports RSS feeds at http://www.thesitewizard.com/thesitewizard.xml. You can read more about how to subscribe to RSS site feeds from our RSS FAQ.
    Please Do Not Reprint This Article
    This article is copyrighted. Please do not reproduce this article in whole or part, in any form, without obtaining my written permission.
    Related Pages
    Appearance, Usability and Search Engine Visibility in Web DesignHTML and CSS Validation: Should You Validate Your Web Page?Designing for Browser and Platform CompatibilityDisabling the Image Toolbar in IE 6 for Your WebsiteIs Your Site Ready for the Average User?Two Common Web Design MythsWhich Web Host Would You Recommend? (FAQ)How to Register Your Own Domain NameFree HTML Editors and WYSIWYG Web EditorsFree FTP Clients
    New Articles / Pages
    Frequently Asked Questions (FAQ) about the Feedback Form WizardNvu Tutorial 5: How to Add a Feedback Form to Your WebsiteHow to Upload a File to Your Website Using the FileZilla FTP ClientNvu Tutorial 4: How to Create a Multiple Column LayoutNvu Tutorial 1: How to Design and Publish Your Website with Nvu (free WYSIWYG web editor)Appearance, Usability and Search Engine Visibility in Web DesignHTML and CSS Validation: Should You Validate Your Web Page?How to Check Your Website with Multiple Browsers on a Single Machine (Cross-Browser Compatibility Checking)Your Website's Spelling and the Search EnginesWhich Web Host Would You Recommend? (FAQ)
    How to Link to This Page
    To link to this page from your website, simply cut and paste the following code to your web page.
    How to Check Your Website with Multiple Browsers on a Single Machine (Cross-Browser Compatibility Checking)

    It will appear on your page as

    Java Book Reviews

    This is the place to find Java book reviews covering all aspects of Java programming, from introductions to specialised topics to J2EE to primers on object oriented development. Also covered are topics dear to the hearts of Java developers, including books on development tools such as Eclipse, XML and web services from a Java perspective to general programming topics and software methodologies.
    If you don't find what you're looking for then take a look also at the TechBookReport programming book reviews, the XML book reviews, the web book reviews or software methodologies.



    Ant: The Definitive Guide by Steve Holzner
    Apache Derby - Off To The Races by Zikopoulous, Scott and Baklarz
    Apache Jakarta Commons: Reusable Java Components by Will Iverson
    Applied Evolutionary Algorithms in Java by Robert Ghanea-Hercock
    Beginning Java 2 (JDK 1.4 Edition) by Ivor Horton
    Better, Faster, Lighter Java by Bruce Tate and Justin Gehtland
    Beyond Java by Bruce Tate
    Code Complete by Steve McConnell
    Code Quality by Diomidis Spinellis
    Code Reading by Diomidis Spinellis
    Cryptography In The Database by Kevin Kenan
    Data Crunching by Greg Wilson
    Data Structures With Java by John R. Hubbard and Anita Huray
    Design Patterns Explained by A. Shalloway and J. Trott
    Eclipse - Building Commercial-Quality Plugins by Eric Clayberg and Dan Rubel
    Eclipse 3 Live by Bill Dudney
    Eclipse 3.0 Kick Start by Carlos Valcarcel
    Eclipse by Steve Holzner
    Eclipse Cookbook by Steve Holzner
    Effective Java by Joshua Bloch
    The Elements of Java Style
    Embedded Ethernet and Internet Complete by Jan Axelson
    FindBugs
    From Java To C# - A Developer's Guide by Heng Ngee Mok
    Google Hacks by Tara Calishain and Rael Dornfest
    Google Hacks, 2 by Tara Calishain and Rael Dornfest
    Google, Amazon, and Beyond by Tom Myers and Alexander Nakhimovsky
    Guide to J2EE: Enterprise Java by John Hunt and Chris Loftus
    Hardcore Java by Robert Simmons, Jr
    Head First Design Patterns by Eric and Elisabeth Freeman
    Head First Java by Kathy Sierra and Bert Bates
    Head First Java, 2e by Kathy Sierra and Bert Bates
    Holub On Patterns by Allen Holub
    IBM Rational ClearCase, Ant and CruiseControl by Kevin Lee
    Integrated Solutions With DB2 by Rob Cutlip and John Medicke
    Java 6 Platform Revealed by John Zukowski
    Java Application Development On Linux by Carl Albing and Michael Schwarz
    Java Concurrency In Practice by Brian Goetz
    Java Cookbook by Ian Darwin
    Java Data Objects by David Jordan and Craig Russell
    Java Developers Almanac 1.4 by Patrick Chan
    Java EE and .NET Interoperability by Marina Fisher, Ray Lai, Sonu Sharma and Laurence Moroney
    Java Examples In A Nutshell by David Flanagan
    Java Garage by Eben Hewitt
    Java How To Program (5e) by Deitel and Deitel
    Java I/O by Elliotte Rusty Hardold
    Java In A Nutshell by David Flanagan
    Java Puzzlers by Joshua Bloch and Neal gafter
    Java Regular Expressions by Mehran Habibi
    JCreator 3.5
    JCreator Pro 2.5
    JDiskReport 1.2
    jEdit 4.2
    Just Java 2 by Peter Van Der Linden
    Learning Jakarta Stuts 1.2 by Stephan Wiesner
    Learning Java, 3e by Patrick Niemeyer and Jonathan Knudsen
    Learning UML 2.0 by Russ Miles and Kim Hamilton
    Murach's Java Servlets and JSP by Andrea Steelman and Joel Murach
    NetBeans IDE Field Guide by Patrick Keegan et al
    NetBeans IDE Field Guide, 2e by Patrick Keegan et al
    The No Fluff Just Stuff 2006 Anthology by Neal Ford
    The Object-Oriented Thought Process by Matt Weisfeld
    Objects First With Java by David J Barnes and Michael Kolling
    POJOs In Action by Chris Richardson
    Processing XML with Java by Elliotte Rusty Harold
    Quartz Job Scheduling Framework by Chuck Cavaness
    Refactoring: Improving the Design of Existing Code by Martin Fowler et al
    Regular Expression Pocket Reference by Tony Stubblebine
    RSSOwl 1.1.3
    Spring: A Developer's Notebook by Bruce Tate and Justin Gehtland
    SQuirreL SQL Client v1.1
    Swing Hacks by Joshua Marinacci and Chris Adamson
    Teach Yourself J2EE In 21 Days by Bond, Law et al
    Teach Yourself Java 2 In 24 Hours by Rogers Cadenhead
    Technical Java by Grant Palmer
    Test-Driven Devlopment: A J2EE Example by Thomas Hammell
    Thinking In Java (4e) by Bruce Eckel
    Thinking In Java by Bruce Eckel
    Wicked Cool Java by Brian D. Eubanks
    XML Hacks by Michael Fitzgerald
    XML Primer Plus by Nicholas Chase
    XSLT Cookbook by Sal Mangano

    Tuesday, October 17, 2006

    Brian Slesinsky's Weblog

    Wrap Strings in Classes to Increase Security

    Forgetting to validate user input is an increasingly common security hole in web applications. Here's an elegant way to fix a hole and make your code more understandable at the same time.
    Code that reads HTTP request parameters often looks something like this:

    String productId = request.getParameter("id");
    doSomething(productId);

    The problem here is that we haven't validated the id parameter, and this string came from the network, so it could contain just about anything. The security holes caused by this can be somewhat surprising. For example, suppose we go on to do a redirect:

    response.sendRedirect("https://www.example.com/page2?id=" + productId);

    See the bug yet?
    Suppose id starts with a newline, such as "\nsomething". The full HTTP response will look something like this:
    HTTP/1.x 302 Found
    Location: http://www.example.com/page2?id=
    something

    The attacker can put anything they like into the HTTP response, or even add multiple HTTP responses. (This is known as HTTP response splitting.)
    It's relatively easy to spot the problem in a simple example like this because there's so little code. The solution is also easy to state: always validate any string that comes from the network.
    However, there are often many method calls between the code that reads the request and the code that prints the response. Maybe we've crossed several layers as well. As the months go by and the code gets changed by multiple developers, it's easy to forget which methods have responsibility for doing input checking and drop the ball. It's very easy to assume that a variable named "productId" actually contains something that at least looks like a productId, but that only works if every request handler did its job.
    Software developers have come up with many ways to fix this problem. Some of them are pretty cumbersome, like writing warnings everywhere saying who has responsibility for what, or using Hungarian notation. Some languages (such as Perl and Ruby) use taint-checking to report a runtime error when the program attempts to use untrusted input.
    In Java, the approach I prefer is to introduce a new class:

    public class ProductId {
    private static final Pattern regexp = ...

    private final String id;

    ProductId(String allegedId) {
    if (!regexp.matcher(allegedId).matches()) {
    throw new IllegalArgumentException("invalid id: "+allegedId);
    }
    this.id = allegedId;
    }

    public String toString() {
    return id;
    }

    ...
    }


    At first glance this class doesn't seem to do very much. Wouldn't a checkProductId() method be sufficient?
    The beauty of introducing a class is that it gives verified product id's a type. Any API that doesn't verify input itself can take arguments of type ProductId and be confident that the caller will do the checking, because the only way to get a ProductId type is to call the constructor:

    ProductId productId = new ProductId(request.getParameter("id"));
    doSomething(productId);


    In an application written this way, the convention is that bare Strings are unverified and domain-specific types are verified (at least so far as syntax is concerned). This convention is enforced by the type-checker. When J. Random Coder comes along and writes yet another HTTP request handler, he or she will find that every relevant API takes a ProductId, and will probably complain about the silly programmers who wrote this API and went totally overboard with their do-nothing classes when a simple String would work fine, not even realizing that calling this silly constructor adds a security check.
    Comments, code review, or unit testing can't give you the same guarantee because they all assume diligence on the part of the programmer. (Not that there's anything wrong with diligence, but there is a limited supply that we'd rather spend in other ways.) Taint-checking would do it (assuming reasonable test coverage) but we don't have that in Java. What we do have is a pretty good type system, so we might as well get it to do some work for us.
    As an added bonus, when you introduce specific types for things, your code gets easier to read. For one thing, naming variables or methods "productId" becomes silly, because then you have:

    // Do you think this code uses a product id?
    ProductId productId = foo.getProductId();


    Now that we have a type for it, we don't have to keep repeating to ourselves that this String is a product id, and so is this String, and this other method returns a String that is also a product id, and the third String argument in this method call is a product id, and so on and so forth. Those sort of naming schemes are for languages with weak type systems, not Java. We can use names for higher-level concepts:

    ProductId itemToDisplay = foo.getSelection();

    And if that's not enough reason to do it, your favorite IDE will become smarter. When it does method completion on a product id, no longer will it give you irrelevant String methods like toLowerCase() or replaceFirst(), things that nobody should want to do with a product id anyway. (Or if they really want to, they can call toString() first.) Now it will give you intelligent suggestions like getShard() or isISBN(), because ProductId becomes the logical place to put these methods instead of hiding them in some utility class somewhere. And when you get usages on a product id, the IDE will no longer show you every place a String is used (which is everywhere, which is a useless search result). Instead you'll see code that actually has something to do with product id's.