Move to create less clumsy robots

The race to create more human-like robots stepped up a gear this week as scientists in Spain set about building an artificial cerebellum.

The end-game of the two-year project is to implant the man-made cerebellum in a robot to make movements and interaction with humans more natural.
The cerebellum is the part of the brain that controls motor functions.
Researchers hope that the work might also yield clues to treat cognitive diseases such as Parkinson’s.
The research, being undertaken at the Department of Architecture and Computing Technology at the University of Granada, is part of a wider European project dubbed Sensopac.

Greater subtlety
Sensopac brings together electronic engineers, physicists and neuroscientists from a range of universities including Edinburgh, Israel and Paris with groups such as the German Aerospace Centre. It has 6.5m euros of funding from the European Commission.
Its target is to incorporate the cerebellum into a robot designed by the German Aerospace Centre in two years’ time.
The work at the University of Granada is concentrating on the design of microchips that incorporate a full neuronal system, emulating the way the cerebellum interacts with the human nervous system.
Implanting the man-made cerebellum in a robot would allow it to manipulate and interact with other objects with far greater subtlety than industrial robots can currently manage, said researcher Professor Eduardo Ros Vidal, who is co-ordinating work at the University of Granada.
“Although robots are increasingly more important to our society and have more advanced technology, they cannot yet do certain tasks like those carried out by mammals,” he said.
“We have been talking about humanoids for years but we do not yet see them on the street or use the unlimited possibilities they offer us,” he added.
One use of such robots would be as home-helps for disabled people.
Interacting robots

The next stage of the Sensopac project is to develop an artificial skin for robots, making them look more human-like as well as being information-sensitive in the same way as human skin is.
This system is being developed by the German Aerospace Centre in collaboration with other research groups. The ambitious project is just one of many attempts to create more human-like robots. Another European research project - dubbed Feelix Growing - has been given 2.3m euros to develop robots that can learn from humans and respond socially and emotionally.
The medical community is making huge strides in the use of man-made parts for failures in the human brain. Last year US scientists implanted a sensor in a paralysed man’s brain that has enabled him to control objects by using his thoughts alone. The fast pace of current robotics research has prompted deeper questions about how androids would be integrated into human society. Some have called for a code of ethics for robots while others question how humans will cope in the face of machine intelligence.

Gadgets 'threaten energy savings'

The growing popularity of hi-tech devices, such as flat-screen TVs and digital radios, threaten to undermine efforts to save energy, a report says.UK consumers spend £12bn a year on electronics, much of which is less efficient than older technology, a study by the Energy Saving Trust found.

By 2020, the gadgets will account for about 45% of electricity used in UK households, the organisation projected.It said flat-screen TVs and digital radios were among the worst offenders.

See how the energy costs of different TV types compare
Paula Owen, author of the report called The Ampere Strikes Back, said household appliances currently consumed about a third of an average home's electricity.But she warned this was likely to increase as a result of people buying more energy-intensive devices.

"Your old-fashioned, bulky cathode ray tube TV on average consumed about 100 watts of electricity when it was switched on," Dr Owen explained."What we are seeing now is a trend for much bigger flat-screened TVs. On average, we are seeing a three-fold increase in the energy needed to power these TVs.

"Pretty much in every other sector [such as fridges and washing machines], we find that as the technology moves on, the products get more and more efficient."Consumer electronics does not work like that."

'Radio ga-ga'
The equivalent of 14 power stations will be needed just to power consumer electronic devices by 2020, the report warned.By that time televisions on standby will consume 1.4% of all domestic electricity, it predicted.

Digital radios were also singled out by the report as being energy intensive."Traditional analogue radios consume about two watts when they are switched on," Dr Owen said.

"We've looked at digital radios and the average consumption of these is eight watts."

She added that listening to the radio via digital TVs or set-top boxes had an average consumption of more than 100 watts.Recent research by the communications watchdog Ofcom said that more than 80% of UK homes now had digital TV.

More people are buying digital TVs or set-top boxes because by the end of 2012 the analogue TV signal will no longer be available in the UK.But not all new technology was criticised by the report.

"Mobile phones and their chargers are one area where we have seen an improvement," Dr Owen said.A few years ago, she said, the current being drawn by chargers that were plugged in but not actually attached to a phone was about three to five watts.

"We have done some testing on the newest mobile phones and chargers you can buy today and reassuringly we could see that 'no-load' consumption had fallen below one watt."

But she added that the sheer volume of mobiles being used, about 63 million in the UK, meant that a huge amount of energy was still being wasted if people were not unplugging their chargers when they were not being used.

The report called for governments, manufacturers and retailers to do more to promote energy efficient devices, but also said consumers had a role to play."The simple message to people is switch things off when you have finished using them," urged Dr Owen.

Spring on Code organization for large projects

Code organization for large projects.

I discovered this 88 minute presentation on dzone.com today. Not a lot of people have time the time to sit through a presentation this long, so I have decided to write about the presentation. The majority of my comments below are simply paraphrasing what the presenter says, and I take no credit for his thoughts or words. I simply found them to be enlightening and want to share them.

Before I start on the presentation, I want to go over some of the terms that are used, as they are the principals that are under discussion.

Cohesion: A measure of how well the lines of source code within a module work together to provide a specific piece of functionality. That is, how well a module of code focuses on one single task or function.

Low Coupling: Refers to a relationship in which one module interacts with another module through a stable interface and does not need to be concerned with the other module's internal implementation. The benefit of low coupling is that when a change in one module is required, it will not require a change in the implementation of another module.

Dependency: A state where one object uses a functionality of another object.
Java Package: A set of Java classes organized for convenience in the same directory.
Now then, the complete presentation includes video and a slide show and can be found at http://www.infoq.com/presentations/code-organization-large-projects.

Summary
Juergen Hoeller shares his experiences working on large projects (including his role as chief architect of the Spring Framework) to provide general guidelines on Packaging and package interdependencies, Layering and module decomposition, Evolving a large code base. Juergen will also discuss how tools can play a role in enforcing architectural soundness.

Bio
Juergen Hoeller has been the most active Spring developer since the open source project began from Rod's Interface21 framework back in February 2003. Juergen and Rod together continue to provide the direction for Spring.

About the conference
The Spring Experience conference is hosted by Interface21 and NoFluffJustStuff Java Symposiums


Why worry about code organization?
Many organization and or programmers don't spend time thinking about code organization, it simply evolves without thought.

Personally, I usually come up with some sort of rough package structure before I start coding, but I don’t document it, and I don’t think about modules. My packages are usually based upon the layers of my multi-tiered design.

For example, my packages typically look like:

com.mycompany.myapp.dao
dao interfaces

com.mycompany.myapp.dao.hibernate
hibernate implementations of the dao interfaces

com.mycompany.myapp.services
service class interfaces and implementations

com.mycompany.myapp.web.springmvc
or com.mycompany.myapp.web.struts
for my web controllers

com.mycompany.myapp.model
for all my model classes

I share my package structure with you so that you can be reassured that others do it this way if this is how you structure your packages. If you don’t structure your packages this way, I am interested in how you do it and why.

There isn't a lot of literature on package dependency and interdependency best practices.

I have to agree with Juergen. Given all the books that I have read, the blogs that I subscribe to and the classes that I have taken, no one has ever spoken on best practices when it comes to package organization. I may have read about package interdependency without realizing what I was reading at the time.

Code needs to be logically organization. Code bases needs to be able to evolve based on its original structure and, even years later, based on completely new requirements.Refactoring and agile development are fine, but how do you preserve backwards compatibility once the code is released? What do you do if you already have people using your published code?

How should modules evolve? Separate modules might need to interact in a later revision, despite the original design not having intended it, thereby introducing new interdependencies at the module or package level.

How well does the code base allow for repackaging into more fine-grained modules if the need arises, while preserving the API?

I like that question. I think it’s fascinating how the Spring Framework can be divided up into a dozen packages or one single package, allowing the users to include only those packages that they need to use for their application. For example, you can use spring-full for the entire spring framework, or just spring-ldap, spring-aop, or spring-core, if you have more specific needs. That fact that the project can be divided into so many modules is a testament to the care and thought that they have put into their package design and code maintainability.

Package Interdependencies
Most public code bases are not good examples of good package design. For example the JDK libraries and many open source projects such as Hibernate. They contain bad practices such as cyclical package dependencies.

Later in the presentation, Juergen goes on to demonstrate how java.lang depends on java.util and how java.util depends on java.lang. For example the String class. He also shows that, on average, any given class in Hibernate depends on two-thirds of the entire Hibernate library.
As a counter example, Juergen states that “Spring does not contain any package circles.”

Typical scenario: Package B depends on package A according to the initial architectural design. New code in package A could use some code in Package B. Since you don't want to duplicate code, you call the code from package A to package B. Now you have package B using package A using package B.
Read as “Now you have package B dependent on Package A dependent on Package B”

Central rule: Packages should have one way dependencies between each other - at most. That is B can call A, but A should not call B.

Why are one-way dependencies between packages so important? Why are circles so undesirable?

Typically, circles are not deliberate, they usually sneek in. Over time, as more programmers touch the code, the original intent or design of the code is lost. Circles can be an indication of code deterioration and indicate to the state or maintenance of the code base.Consequences of cyclical dependencies include limited reuse of packages.

Cyclical dependencies create dependencies so that you can not extract that package for use with another project or some other code base. The packages becomes inseparable. Neither package can be compiled without the other package. So that in the event that you needed to extract some code for another project, you would also need to bring along some other package that has nothing to do with the new code base.

Avoid circular dependencies between packages. This is easier said than done, and many times you don't even realize that you have done it. To remove these circular dependencies, creative refactoring is usually required, which creates backwards compatibility challenges. Regardless, always avoid code duplication.

How do you avoid circular dependencies while at the same time avoid code duplication? Usually this means taking the time to analyze your package architecture to determine the best place for this new code. For example, at first thought you might want to put the code in some deeper package, but after some careful consideration, you determine that the code really needs to be placed in a shallower package.

Module Decomposition and Layering
Modules are conceptual in java. They are conceptual boundaries within your code base. Generally modules are a collection of specific packages that collaborate and/or are conceptually related. They might live in separate source directories but do not have to. Some modules might consist of a single package only, while others may consist of many packages. Identifying modules can be challenging.

Typically, modules are driven by deployment needs as much as conceptual boundaries. For example part of the code might be used by one application deployed on one server, while another part of the code might be used be a second application. Others created modules based on multi-tier separation. However, this is unnatural since it does not match conceptual boundaries.

So, the package structure that I use, which I described above would be considered a package structure based on multi-tier separation which Juergen states is “unnatural.” Sometimes modules are isolated by specific dependencies, such as JDK 1.5 or Hibernate. However, this is not a conceptual boundary, so it is a rather incorrect use of module separation. Sometimes modules are created to keep jar file size down. Again, this is an incorrect usage.

Desirable characteristics of modules include low coupling to other modules, and high cohesion within the module.

Modules are conceptual units as much as a source management and deployment unit. Modules should allow for individual usage or a distinct role within a larger system. For example, a new developer on the team, or a new user of your code should be able to review a module and understand the reasoning or the usage of that module - thus it is a complete, implemented concept with clear boundaries.

This sounds ideal. I know it would be great if when I was assigned to a new project I could pull it down and see distinct modules, and know what each of their responsibilities were, and knew that they did not overlap.

Modules should not have circular dependencies. Therefore module 1 -> module 2 -> module 1 is an undesirable situation.

Layering is essentially a logical view on the package structure. Higher layers build on lower layers. That is, higher level packages depend on lower level packages, not the other way around.

I seem to follow this rule of thumb with my DAO packages, being com.mycompany.myapp.dao and com.mycompany.myapp.dao.hibernate.

The module structure might have a straightforward mapping into layers. However this is not strictly necessary since modules might be a vertical slice. Modules are often driven by deployment considerations more than layering. That is, modules are usually vertical while layers are usually horizontal.

Establish natural conceptual boundaries in your code base. It does not mater where your code resides. It could be a single shared source root, or one source root per module. It does help if the source code structure mirrors the conceptual structure. This creates a natural package naming system and makes navigation easier.

Evolving a Large Code Base
The hardest challenge is evolving the code as well as the architecture over time without letting the code deteriorate and without compromising on architectural quality. This becomes exponentially harder with growing size of the overall code base. Consider the situation where you have many developers involved and no single point of architectural management or enforcement at the fine-grained artifact or module level.

You are going to see inconsistencies across the code, duplicated code, inconsistent naming, and most likely a decrease in loose coupling and high cohesion.

What is the trade-off between backwards compatibility and architectural quality? Strict 100% backwards compatibility might not allow for sustaining the architectural quality level. Nevertheless, there is always a better solution than compromising on architectural quality. For example a creative internal refactoring that allows to preserve compatibility as well as well-defined package dependencies.

Certain packages, for example the public API or core packages, can almost never be changed. The code base must revolve around these packages. One solution, if you must change the public API or core, is deprecation but even this is sometimes not possible. Some changes can only be done by breaking backwards compatibility. What do you do in that case? Break backwards compatibility or reduce the integrity of the architecture? The Spring development team is willing to accept a small breakage of backwards compatibility as long as the end result is a strong architecture.

Case Study: the Evolution of Spring

Background of Spring core:

Origins back to early 2001.
First public release was in 2002
First public source release was mid 2003
1.0 release in 2004
2.0 release in 2006, largely compatible with 1.2.

The spring project has faced many code evolution challenges including a broad public API used by applications; sophisticated SPI used by advanced applications as well as sister products and third-party frameworks; and new requirements addressed in every release, often implying some refactoring.

A tough situation that would not have been possible if not for their straightforward architecture and their strict adherence to the rules: accept small breakages of backwards compatibility as long as the end result is a strong architecture, and right from the start, no circles allowed at package level, not even as a temporary measure.

How has the Spring code base survived in its original shape for 3.5 years?

Strict architecture management and loosely coupled packages with well-defined interdependencies.Right from the start, no circles allowed at package level, not even as a temporary measure.

When introducing new code or functionality, great care is given in determining where the new code will go.

Challenges
Ever changing third party libraries.
Hibernate 2.1 -> 3.0 -> 3.1 -> 3.2
Resulted in the creation of the package hibernate3
Quartz 1.3 -> 1.4 -> 1.5 -> 1.6
What do you do in case of incompatible API changes in such libraries? The goal is always to try to maintain backwards compatibility, so in many cases, Spring developers were required to use runtime reflection to determine which library is in use and to maintain this compatibility. With reflection Spring can check to see if a certain method exists, and if it does, it is called, if it does not, it is skipped.

Tools for Architectural Analysis
How do we make sure that our architecture remains sound and avoids architectural violations? The best thing to do is to always keep these restrictions in mind, but even this can only get you so far.

Manual analysis only gets you so far, it is like manual testing vs. automated, repeatable testing. The only way to guarantee this is to use tools. Since 2003, Spring has been using JDepend before every public release. They have also recently introduced the SonarJ tool.

JDepend traverses Java class file directories and generates design quality metrics for each Java package. JDepend allows you to automatically measure the quality of a design in terms of its extensibility, reusability, and maintainability to manage package dependencies effectively.

JDepend is an open source tool that has been around since 2001. It is typically used as a command line tool. It can generate an analysis report which, among many other things, includes package dependency cycles.

SonarJ is an innovative solution that helps you to manage and monitor the logical architecture and the technical quality of your Java projects.

SonarJ is a commercial tool, is gui-driven, and is highly customizable. SonarJ also allows for on-the-fly analysis.

The presentation ends with a demonstration using SonarJ to analyze Spring.

I was about to go check out JDepend, when I realized that the IDE that I use, IntelliJ, contains cyclic dependency and backwards compatibility tests via the Analyze menu. Fellow IntelliJ users should see http://www.jetbrains.com/idea/features/code_analysis.html#Dependencies_Analysis and http://blogs.jetbrains.com/idea/2006/04/analyzing-code-dependencies-part-i/

50 Common Web Design Mistakes

This started out as a “Top Ten Newbie Web Mistakes” for my beginning web design students but it quickly became obvious that I couldn’t limit it to only ten. I was finally able to edit it down to 50. But I suspect, as soon as the comments start, it will begin growing again.

And, yes, I’ve made them all at one time or another.

Page Titles
1. Untitled documents: “Untitled Document” is the default title for pages created in Dreamweaver and other web design programs. Too many people forget to change it (see My Favorite Google Search).

2. Same title for all pages: The title is important. Help the world know which of your pages they want to see.

3. Non-descriptive titles: The page title is the headline for your link in search results (not to mention an important factor in determining those results). Instead of “Jim’s Page” try something like “Cartoons and Illustrations by Jim” or better yet “Political Cartoons.”

Meta Tags
4. Duplicate meta information on all pages: The keywords and description meta tags in the head of your page help search engines categorize your page. If you duplicate the tags across all pages of your site, they will look alike to searchers. Customize the keywords and descriptions for each page or don’t use them at all.

Site Structure
5. No index.html (or equivalent) in the root directory: By default the index.html (or an equivalent such as index.htm, index.php, default.htm, etc.) is displayed when you visit http://www.yourdomain.com. If you don’t include it, visitors will get an error message or be required to type out the full URI including the file name.

6. Disorganized file structure: How you organize your site files won’t affect what the site looks like but lack of organization can make your life hell down the road when you’re trying to update or redesign the site. Use directories (folders) to help organize your pages and images.

7. Uploading non-web files: Accidentally uploading a few native Photoshop files can eat up your disk space quickly (not to mention take forever). Store your resource files (Photoshop images, Word files, etc.) in a separate folder outside your local web folder.

Pages
8. “Under Construction” pages: If a page isn’t ready to post, don’t post it. If you can’t help yourself, remember “Under Construction” is supposed to be a temporary condition and, after a month or, so it starts to seem permanent.

9. Frames: There are good reasons why you might want to use frames but there are no good reasons to actually use them.

10. Horizontal scrolling: The least common denominator for monitor width is currently 800 pixels. You also need to leave space for scroll bars, page margins etc. so 760 pixels is a good standard width for your web pages. Wider may be acceptable depending upon your target audience but be careful!

11. Worthless content: If you don’t have anything to say, then don’t say it.

12. Out-of date content: If your content is no longer timely, delete it and, if you’re going to include a copyright notice, update it each January.

13. Overly long pages: Contrary to popular belief, there’s nothing wrong with long web pages (that’s what the scroll bar is for) if the content warrants it. But, if it can be done logically, it’s usually a good idea to have several shorter pages than one very long one. If you do have very long pages, provide additional navigation to make it easy for readers to move within or off the page (such as a simplified menu of text links at the bottom of the page).

14. Unnecessarily short pages: In an effort to make the content “fit” the design, designers often resort to a series of short pages when one long one would be more user-friendly.

15. “Orphan” pages: Pages you forget to provide links to don’t exist as far as the rest of the world is concerned.

16. “Alien” pages: Pages that completely ignore the look and feel of the rest of your website leave users feeling like they’ve been suddenly transported to a website far, far away.

Navigation, navigation, who’s got the navigation?
17. Pages without navigation: If you don’t offer them an option, visitors are more likely to close your page than to hit the browser’s “Back” button.

18. Broken links: ‘Nuff said!

19. “Hidden” links: Make links easily identifiable by using a contrasting color, underlining them, using “button” images or altering the rollover state.

20. “False” links: Underlined text and rollover images scream link. Use them cautiously.

21. Menus that move: Establish consistent “navigation zones” and stick with them.

22. Inconsistent navigation: Once the user learns how to use your site’s navigation, don’t change it on him.

Headlines
23. Restyling text instead of using heading tags: h1 is not the same as big paragraph text.

24. Using heading tags for design: Headings are structural elements and should be used to define the purpose of the text they enclose. Don’t use them just because you want big bold text.

Body Text
25. Using images for text: Text in images can’t be read by search engines or screen readers.

26. Justified type: It’s hard enough to make justified text look presentable on a static printed page. On a dynamic web page it’s nearly impossible.

27. Using br br instead of p: It will make formatting difficult.

28. Using br to control line breaks within paragraphs: Let the browser determine where your lines break within paragraphs. If you force the issue you may get strange results as not all browsers size type exactly the same.

29. Typos and grammatical errors: use your spell checker and check out 10 flagrant grammar mistakes that make you look stupid.

30. Type too small: Really, 9 point type on a printed page isn’t comfortable for most people. On screen it’s unreadable for anyone over 30. Except for the “small type” you’re trying to make unreadable 12 points (or even 14) should be your minimum.

31. Too little contrast between text and background: It’s really hard to read!

32. Using non-breaking spaces to align type: For tabular data use tables. To position type as a design element use CSS styles.

33. “Ransom” note styling: Using too many fonts, too many styles, too many weights, too many sizes and too many colors is simply too much.

Images
34. Images without the alt attribute: Search engines, screen readers, and that little text box that sometimes pops up when your mouse is over an image all use the alt attribute. You should too.

35. Jigsaw puzzle graphics: Don’t slice images more than necessary. Each slice requires an extra call to the server.

36. Resizing images in the browser: Size your images in your image editing program before placing them on your pages. Images that are blown up in the browser lose quality and images that are reduced in the browser increase the loading time of a page. For example, a 1 inch by 1 inch image loads 4 times quicker than a 2 inch by 2 inch image, even if they are displayed at the same size.

37. Improper image format: JPEGs are best for photos and continuous tone images. GIFs are best for images with large areas of flat color. Also, transparent GIFs are prone to “ghosting” if used incorrectly.

38. Use of transparent PNGs without Explorer fix: PNGs offer true transparency but it doesn’t work properly in Explorer 6 without a javascript fix.

Animations
39. Gratuitous Flash: No matter how fantastic your Flash splash page is, nobody really wants to watch it more than once. If you must “Flash” yourself, at least provide a “skip animation” link.

40. Non-stop animations: Let your animation cycle a few times and then stop it before it gets overly annoying. 41. Two many animations: More than one animation on a page is just annoying.

42. Use of the blink tag: Thankfully some browsers ignore it.

Philosophy
43 “Sheet of paper” pages: Your screen is not eight-and-a-half inches wide, things don’t necessarily stay where you put them and, when you get to the bottom, you can scroll. Take advantage of the design possibilities those attributes and others offer.

44 Confusing content and design: HTML tags (p, h1, h2, etc.) are structural elements and, by themselves, say nothing about how your page should look (see for example…). Organize your content using HTML and create the design of your pages with CSS (and maybe a table if absolutely necessary).

Miscellaneous
45. No contact information: The purpose of a website is communication. Right? Make sure people have a way to contact you if they’re interested in your work, product or service.

46. Reliance on email links: E-mail links only work if the user has an email program available and correctly configured. And they don’t work with G-mail and similar services. So people in libraries or school labs can’t use them. Most ISPs offer a form processing script that can convert the contents of a form to an email and send it to you. Use it.

47. Failure to respond to contacts: Sure you can ignore spam but, when a legitimate visitor takes the time to contact you, a prompt reply is just good manners.

48. Auto-play sounds: Unexpected sounds are annoying especially in an office or classroom.

49. Opening too many windows: Cluttering up someone’s screen with a new window every time they click on a link is just bad manners.

50. Failure to check for cross-browser inconsistencies: Your site should work on Macs and Windows, in Explorer, Firefox and Safari. If it doesn’t you’ll drive visitors away.

All Things Bloggable by Brian and Stephanie ReindelThe top ten questions every programmer should ask on a job interview

In the spirit (or snare) of capitalism, we often believe that the job interview is an opportunity to prove our worth to potential employers. As much as that is true to some degree, every interview should be a time set aside not only to answer questions, but to ask them as well. In order to increase the probability that you will find a rewarding position, you have to step beyond the subjects of salary, vacation, and benefits, and put companies in the spotlight.

These ten leading questions will help you to better understand the position and company to which you are applying. Leading or clarifying questions are the key to a successful interview. They require the interviewer to become expressive, and they alleviate the need to interpret how much truth is behind a curt yes or no answer. You may feel awkward asking for such detail, but you will find in time that any company uncomfortable answering probably has something to hide.

How many hours are in a typical work week?
Skip the pleasantries regarding business hours. You can find this information on a Web site, or in the job posting. You want to know how many hours are billed for the average employee in a work week. Is it truly 40, or do most employees hover around 45-50 hours? Is this overtime, or is comp (flex) time offered? What about summer hours? Get the whole scoop, and you will discover that the standard nine-to-five work day is a thing of the past, especially for Web developers. How much it varies will differ from organization to organization.

What are the expectations regarding travel?
Almost all companies will require some travel, either for training, conferences, or for client meetings. The amount of travel is usually specified as a percentage, but this is rarely an accurate representation of the actual time spent on the road or in the air. Be sure to ask approximately how often, and for what duration, you will need to travel annually. Employers may be hoping you gloss over these details, since above average travel is a great barometer for a bump in pay scale.

On average, how long does an employee remain with your company?
Avoid questions about turnover, unless the interviewer uses the terminology. It has negative connotations, and you may receive an arbitrary response, like, “turnover is low”. This does not help you to gage employee satisfaction, so you want to get a more detailed response. In our industry, if there is a small staple of employees who hover around five to ten years, then that is a good sign. If the company is young, or a startup, then asking this question may not be a benefit to you.

What kind of on-going training can I expect to receive?
For quite some time I was afraid to ask this question. I was fearful employers would speculate that I must need training because I was not qualified. I have since learned that employers want employees who demand a more diverse set of skills. The best way to get this training, is for a company to pay for it for you. You may be allowed to attend training during normal business hours, or you may be required to take evening courses for college credit. Also, be sure to ask about mentoring programs, and whether or not certifications are a covered expense.

Can you give me an overview of the development process?
This question is better directed toward an individual in technical services. Human resources may have a basic understanding, but to paint a more accurate picture of the development process, it will require an answer from your immediate boss or a peer worker. Ask how well this process is documented, and if there are supplemental training materials and example deliverables available. Will you be required to learn while on a project, or will your first few days be spent reading about operating procedures? It will almost always be the former, but if you have written guidelines that map to what you are doing, ramp up time can be significantly decreased.

Who is your largest client, and what can you tell me about the work you do for them?
This is an excellent question for determining stability, and there are two schools of thought. If the client’s name is spread all over the employer’s Web site, as in case studies, and almost every programmer has their hand in development, then that could be a bad omen. By placing all the eggs in one basket, the employer is risking a major blow to revenue and possible layoffs if the client seeks an alternate vendor. On the other hand, an ongoing relationship with a stable brand could mean a more stable position for you. If you are someone who craves variety, you should also ask if your work will be focused primarily around this client.

Can you discuss any growing pains experienced by the company in the last two years?
With this question in particular, the intention is not to get an in depth answer, but to discern the level of honesty with which the answer is given. If clarification is necessary, inquire about any difficulties with employee retention, client satisfaction, or project overages. You want to get the employer to disclose information that is not necessarily sensitive or private, but that does affect their day-to-day operations. If they are uncomfortable or unwilling to answer, then that may be a sign they are facing internal struggles.

How is a typical performance review handled?
Be sure that you understand all the details surrounding performance reviews. This is one area that can quickly become a source of contention if there is miscommunication. How often do formal reviews take place? Is there a peer review process? Are there quarterly or semi-annual informal reviews? Can you receive a promotion or a raise without a review? When you start, will you need to set aside goals to accomplish for your first performance review? Who makes the final decision regarding a raise or a promotion? These are all important questions.

What types of social activities are organized throughout the year?
Even if you are not interested in attending social activities sponsored by the employer, this question will help you to better understand how much they appreciate your work. Even if it is only an annual holiday party, or summer picnic, it demonstrates a willingness to help you relax. Some companies will even sponsor competitive sports, or a league team made up of employees. All of this is an indicator that employee satisfaction and retention is given a high priority.

Can you tell me if I have the experience you need?
This question should always be the last question you ask, and it serves a dual purpose. You want to know how interested the employer is, and generally speaking, you want to know what expertise you are lacking. If you find that several companies are concerned about skills that you do not possess, then you should take formal training, or work toward a certification. Even if an employer will not discuss where you stand comparatively, they are usually happy to provide you with some constructive criticism. If you are called back for a second interview, use this to your advantage. Politely illustrate why you are still the perfect candidate, even if you lack some skills. Be sure to communicate your desire to learn.

JavaSpecialists Issue 146 - The Secrets of Concurrency (Part 1)

Welcome to the 146th issue of The Java(tm) Specialists' Newsletter. I am on my way to TheServerSide Java Symposium in Barcelona, where on Friday I will be presenting a new talk entitled "The Secrets of Concurrency". After many days of feverishly trying to come up with an outline for my talk, we ended up praying for inspiration. The writer's block disappeared immediately!

As mentioned in the previous newsletter, from July 2007, I will be offering Java code reviews for your team to get an expert's viewpoint of your Java system.

Secrets of Concurrency (Part 1)

Have you ever used the synchronized keyword? Are you absolutely sure that your code is correct? Here are ten laws that will help you obey the rules and write correct code, which I will explain over the next few newsletters:

1. The Law of the Ritalin Child
2. The Law of the Distracted Spearfisherman
3. The Law of the Overstocked Haberdashery
4. The Law of South African Crime
5. The Law of the Leaked Memo
6. The Law of the Corrupt Politician
7. The Law of the Micromanager
8. The Law of Greek Driving
9. The Law of Sudden Riches
10. The Law of the Uneaten Spinach

Introduction

After watching myself and those around me for 35 years, I have come to the conclusion that we are exceedingly dim-witted. We walk through life only half-awake. I have gotten in my car to drive somewhere, then arrived at a completely different place to where I wanted to go to. Lost in thought, then lost on the road.

There are some people amongst us who are wide awake. They will hear a concept once and will immediately understand it and forever remember it. These are the exceptions. If like me, you struggle to understand concepts quickly or to remember things for a long time, you are not alone.

This is what makes conferences challenging. I know that I risk exposing my own lack of intelligence by saying this, but most of the talks I have attended have ended up going completely over my head. The first quarter of the talk would normally be alright, but suddenly I would be lost, with no way of recovery. I have great admiration for those that sit through an entire talk without opening up their laptops.

Just to illustrate the point, here is a picture taken of me during the "Enterprise Java Tech Day" in Athens. The caption on the Java Champion reads "Heinz Kabutz busy preparing for his talk ..." Had the picture been taken from behind, the caption would have had to be "Heinz Kabutz busy playing backgammon on his laptop ..." In fairness, the speaker was talking in Greek, which I could not follow anyway.

To make my talk as simple as possible to understand, I have summarised the Secrets of Consulting in ten easily remembered laws. Here is a challenge. Try to forget the name "The Law of the Ritalin Child". Let me know if you manage.

Law 1: The Law of the Ritalin Child

Instead of suppressing interruptions, deal with the cause.

In some societies, it is considered abnormal for a testosterone powered boy to be naughty and to interrupt all the time, given the chance. Instead of dealing with the cause, the children (especially boys) are diagnosed as having ADHD and are then given Ritalin to calm them down.

My own son caused InterruptedExceptions continuously in class when he was five. In our case, the teacher caught the exception and notified us about the problem. We could deal with the interrupted status and since then the problem has gone away completely.

(Disclaimer: I am sure there are cases where ADHD is the correct diagnosis and where Ritalin has brought about wonderful results. I am not a child psychologist, just a father.)

How often have you seen code like this? I have even seen Sun Java Evangelists doing this in their talks!

try {
 Thread.sleep(1000); // 1 second

} catch(InterruptedException ex) {
 // ignore - won't happen
}
 

There are two questions we will try to answer:

1. What does InterruptedException mean?
2. How should we handle it?

The first question has a simple and a difficult answer. The simple answer is that the thread was interrupted by another thread.

A more difficult answer is to examine what happens and then see how we can use this in our coding. The thread is interrupted when another thread calls its interrupt() method. The first thing that happens is that the interrupted status of the thread is set to true. This interrupted status is important when we look at methods that put a thread into a WAITING or a TIMED_WAITING state. Examples of these methods are: wait(), Thread.sleep(), BlockingQueue.get(), Semaphore.acquire() and Thread.join().

If the thread is currently in either the WAITING or TIMED_WAITING states, it immediately causes an InterruptedException and returns from the method that caused the waiting state. If the thread is not in a waiting state, then only the interrupted status is set, nothing else. However, if later on the thread calls a method that would change the state into WAITING or TIMED_WAITING, the InterruptedException is immediately caused and the method returns.

Note that attempting to lock on a monitor with synchronized puts the thread in BLOCKED state, not in WAITING nor TIMED_WAITING. Interrupting a thread that is blocked will do nothing except set the interrupted status to true. You cannot stop a thread from being blocked by interrupting it. Calling the stop() method similarly has no effect when a thread is blocked. We will deal with this in a later law.

The interrupted status is nowadays commonly used to indicate when a thread should be shut down. The problem with the Thread.stop() method was that it would cause an asynchronous exception at any point of your thread's execution code. The InterruptedException is thrown at well defined places. Compare it to firing employees when they are idle, rather than when they are in the middle of important work.

Note that the interrupted status is set to false when an InterruptedException is caused or when the Thread.interrupted() method is explicitely called. Thus, when we catch an InterruptedException, we need to remember that the thread is now not interrupted anymore! In order to have orderly shutdown of the thread, we should keep the thread set to "interrupted".

What should we do when we call code that may cause an InterruptedException? Don't immediately reach for the Ritalin! Typically there are two answers to that question:

1. Rethrow the InterruptedException from your method. This is usually the easiest and best approach. It is used by the new java.util.concurrent.* package, which explains why we are now constantly coming into contact with this exception.
2. Catch it, set interrupted status, return. If you are running in a loop that calls code which may cause the exception, you should set the status back to being interrupted. For example:

while (!Thread.currentThread().isInterrupted()) {
 // do something
 try {
   TimeUnit.SECONDS.sleep(1000);
 } catch (InterruptedException e) {
   Thread.currentThread().interrupt();
   break;
 }
}

Sri Lanka under fire over Internet censorship

COLOMBO (AFP) - Media rights groups attacked Sri Lanka's government Thursday for blocking domestic access to a website favouring the Tamil Tiger rebels and for saying it would like hackers to disable the site.

ADVERTISEMENT

Reporters Without Borders (RSF) said Colombo should immediately unblock the Tamilnet.com website.

"Sri Lanka's Internet service providers have been blocking access to the website on the government's orders since June 15," RSF said. "The government must put a stop to this censorship and restore access to the site at once."

A local rights group, the Free Media Movement (FMM), also criticised government spokesman Keheliya Rambukwella over comments in which he said he would "love" to hire hackers to pull down Tamilnet.

The FMM said Rambukwellas statement was "tantamount to government sanctioned cyber-terrorism against websites that do not toe its line."

"The FMM seeks urgent clarification from the government as to whether Minister Rambukwellas comments are indicative of official government policy to shutdown, disrupt or censor content and websites on the Internet."

But Sri Lanka's Media Minister Anura Yapa insisted his ministry had nothing to do with preventing users of Sri Lanka Telecom, the country's main Internet service provider, accessing Tamilnet.

"It is unreasonable to level charges against the government," Yapa told reporters here. "We have nothing to do with this."

Military spokesman Prasad Samarasinghe said the security forces had not ordered the blocking of Tamilnet either.

"Security forces have not asked the Tamilnet to be blocked," Samarasinghe said.

Despite the denials, Sri Lanka Telecom's Internet service help desk told callers that the "government has asked to block Tamilnet."

"You can access any other site, but you can't access Tamilnet," callers are being told.

The government owns just under 50 percent of Sri Lanka Telecom, which is run by NTT of Japan.

A Colombo-based editor of Tamilnet, Dharmaratnam Sivaram, was shot dead by unidentified gunmen in April 2005. The killing remains unresolved.

Some Internet service providers, who have their main offices abroad, still allow access to the website.

Tamilnet is an influential source of Tamil views on the island's separatist conflict, which has claimed more than 60,000 lives in a 35-year campaign by rebels for a separate homeland for minority Tamils.

Sri Lanka's youth programmers battle for Microsoft recognition

June 19, 2007 (LBO) – Sri Lanka's young software developers lack resources but not dedication or talent, says former winners of a students technology competition promoted by the software giant, Microsoft.


The Imagine Cup encourages young people to apply their imagination, passion and creativity to develop technological solutions to real world issues, say its organizers.

This year a team from the University of Moratuwa was chosen as the winner from six finalists this year.

The team developed a system to monitor and record a child's progress, health and performance.

A team from the University of Moratuwa also won last year in the Sri Lanka round with a project titled 'Panchayudha' which was designed to organize clinic-related activities of expectant women and mothers with infants.

'Panchayudha' is a gold charm worn by infants around their neck.

"If an international organisation is ready give us help on improving our resources, Sri Lankans will do much better than any European or East Asian country," Chathurika Sandarenu from the Faculty of Information Technology at the University of Moratuwa said.

"The only problem is the lack of resources and access. It is not the knowledge or lack of dedication."

Resource Constraints

In countries like the United States, developers can tap into venture capital funding who make huge lose large volumes of money as many projects inevitably fail, but also make profits in the end from the few that give exponential profits.

But in Sri Lanka financial markets are not developed due to decades of restrictions and government control and a risk-averse culture.

The country also lacks basic infrastructure for widespread use of computers.

According to Sri Lanka's ministry of education, only 76 percent of schools and temple based schools have electricity, 26.2 percent have telecommunication facilities and less than ten percent have internet facilities.

Internet penetration in the island is lower than one percent. Most internet connectivity is concentrated in the western province and other major towns.

Out of the one million personal computers, only 15 percent have internet connections and only three percent have broadband connections.

To overcome low internet penetration among rural youth, the government's ICT body ICTA has launched 'Nanasalas' or IT kiosks all over the country especially in rural areas.

Helping Hand

Last year Microsoft helped teachers of rural schools to buy computers at concessionary rates to raise awareness among the academic staff since education ministry surveys showed that only a third of the teachers out of 200,000 are computer literate.

Imagine Cup is another Microsoft initiative to recognise skilled students and showcase them to the IT and software companies world wide.

"Last year's winning three teams immediately got jobs so that kind of opportunity is there and they can continue their applications," Wellington Perera of Microsoft told LBO.

"These students are getting recognition. The IT companies get to know who the skilled persons are,"

Six teamed qualified as finalists in the local round of which three teams were from the University of Moratuwa, Sri Lanka's top engineering institution.

The team from private education institute, Informatics in Colombo presented an internet encyclopaedia which can be updated by users through mobile devices.

It also contained features to pin point a user's position in any part of the island. Other projects included distant learning systems and child care projects.

Perera says the six finalists were chosen out of 265 contestants from all parts of the island.

"I visited about eight universities and institutes sending the message to Peradeniya, Ruhuna and several rural universities. There were about 50 registrations from those areas also," Perera says.

"I think that is a good start for us and next year I am going to do more awareness raising in rural universities about imagine cup."


Begun in 2003, the Imagine Cup finals have travelled from Spain, to Brazil, Japan and to India. In 2006-2007 will be held in Seoul, Korea.

Spring Framework Introduction & Training

It's strange that the first post in this blog is about the Spring Framework, because I haven't been working with Spring for about a year...

Recently I've given a one-day training for a group of young but powerful developers. It was a training of the Spring Framework 2.0 for Java.

There are many great presentations of Spring all over the Internet, but I couldn't find one that covers all issues as I wanted (I guess that it's a matter of taste...).
So I prepared a long Power Point presentation and some exercises for a hands-on training session. The presentation includes content from the reference manual of Spring, from a few publicly available presentations, from relevant Wikipedia articles and other resources. All resources are listed in the presentation.

I thought that the presentation and exercise might be useful for developers that want to learn about the Spring framework or want to start using it. It also might be useful for experienced Spring-based developers that need to train other developers and look for a technical presentation and an exercise.

So here you can freely download the presentation (which also includes the description of the exercises), a skeleton for easily starting to implement the exercise and an implemented solution for the exercise.

• Presentation:
  
  • PowerPoint File
• Excercise:
  • Skeleton ZIP File
    
  • Solution ZIP File
    

The presentation is very technical in its nature. It includes a general description of the Spring Framework and the relevant technologies (AOP - Aspect Oriented Programming, IoC - Inversion of Control, etc.), but it's main focus is on real-life usage of Spring that makes the developer's life easier.

The presentation includes the following sections:



Like any presentation, the slides alone are not enough. A person that have hands-on experience with Spring is needed for giving a training based on it.

I worked intensively with the Spring Framework in a previous start-up company I was part of. After a previous experience with J2EE (before it was renamed to JEE), which involved the BEA WebLogic Application Server, we wanted to have a much lighter and flexible environment while enjoying the concepts of IoC and AOP. We've built our very demanding applications with the help of the Spring Framework. Actually, before using Spring (and before Spring had publicly known stable versions), we have found ourselves developing frameworks that tried doing quite the same as what Spring does. Those frameworks worked nicely, but it's always better to use a popular and strong open source product instead of developing your own...

Spring allowed us to concentrate on the business logic code only and easily transform our applications between being light standalone applications, unit-tests based on mock-ups, J2EE application server (JBoss) hosted services, multi-tier architectural systems and more. Spring also made our code (& life) simpler while using Spring's integration with Hibernate for OR/M, JDBC for direct SQL, JMS queues, remote HTTP web services, Quartz scheduling, Transaction Management, JUnit tests and others.

Spring, of course, is not perfect and it's hard to tell whether it's still the best option now that EJB3 and other new technologies are available. But I enjoyed preparing and giving this training session because I believe that the Spring Framework is very useful for any Java developer and that bringing AOP and IoC from academic publications into easy-to-use every day programming allows having better written and maintainable projects.

My Favorite Firefox Extensions

Firefox is a great Web browser. It offers good Web standards support and is open source. As a Web developer who regularly creates templates/themes based on Photoshop designs, Web standards compliance is a very important aspect and one of the reasons I hate Internet Explorer. As a person who believes in the spirit of open source, it is also important that Firefox is open source software.

The speed of Web page rendering is not the biggest strength of Firefox, there are other browsers that render pages more quickly, but the most important aspect, that makes Firefox incomparably useful are the free extensions available.

Cool Firefox Extensions

Below I list the ones I am using for making my life as a Web developer easier and my browsing experience better.

Firebug

Firebug is an amazing extensions I can't imagine living without Firebug when developing Web applications. Firebug enables you to edit, debug, and monitor CSS, HTML, and JavaScript live in any Web page. I can't list all of the cool features Firebug comes with, but the ones that are most important to me:

• Displaying which CSS rules actually apply to the currently selected HTML element and changing the values to immediately see the effects.
• Debugging JavaScript, setting breakpoints, and testing code before adding it to a live site.
• Monitoring load times of the executed HTTP requests and their order of execution.

If you are a serious Web developer, Firebug is a must have extension.

Web Developer

The Web Developer extension was one of the first I used to facilitate my work. It adds lots of convenience features in form of a toolbar. The following Web Developer Toolbar features are most useful to me:

• Quick links to validation services for the currently displayed page.
• Resizing the browser window to customizable screen sizes.
• Disabling JavaScript, images, and CSS to see what a page looks like in a text browser or for robots.
• Disabling referrer information in HTTP requests.
• Disabling and clearing session cookies.
• Displaying various information for the currently displayed page, such as DIV order, ID and class details, and many more.

The Web Developer Toolbar is an extension I definitely would not want to miss.

ColorZilla

I use the ColorZilla extensions for retrieving the hex code of the color value I point my mouse cursor over on the page currently displayed. This add-on offers more features, but I only use the one mentioned, which I find very useful.

Link Counter

The Link Counter extensions allows you to copy all links as a plain text or unordered HTML list of the URLs displayed on a page. You can choose all links, internal or external links only. Very useful for writing posts on .edu-Domains.

del.icio.us Bookmarks

I use del.icio.us to manage and access my bookmarks from any computer that is connected to the Internet. The del.icio.us Bookmarks extension makes it easy to add and search my bookmarks without actually visiting the del.icio.us Website.

Download Statusbar

With the Download Statusbar I won't see the default download pop-up window that comes with Firefox. It offers more features that I don't use, mainly because I have not checked them out so far.

HTML Validator

The HTML Validator extensions tells whether the page currently displayed validates as HTML or XHTML when taking a look at the page source.

MeasureIt

MeasureIt adds a ruler to get the pixel width and height of any element or region on a Web page.

Save As Image

The Save As Image extension enables you to save a page, frame, or part of either as an image. You can use this extension to create snapshots of the full page from top to bottom.

SearchStatus

Last but not least I use the SearchStatus extension to retrieve SEO related information. SearchStatus displays the Google PageRank, Alexa rank (also affects it) and Compete ranking. SearchStatus comes with a keyword density analyzer, offers nofollow link highlighting, display of backward and related links, Alexa info, and much more.

Programming and Managing : Can you do both?

I have been getting quite a few emails recently asking me to post about certain topics. So today I start answering them.

Brendan Barber asks “I am a software development manager for a global recruitment firm. I was wondering if you have any blogs about how to juggle team leadership / management with programming tasks. I am finding it increasingly difficult to perform both the technical and the management aspects together. The nature of programming tends to require large blocks of focused time whereas management is sometimes the opposite especially when dealing with operational issues that have mostly have a lifespan of minutes. an it be done, should it be done, and if so how have you approached this?”

It is fair to say that trying to program and manage is not for everyone. In fact a large percentage of programmers are not suitable to management roles and most rightly see management as something which would require them to give up all morals and learn to speak a different language (e.g. say ‘pro active’ every other word). This is partly due to the fact that management (on the whole) positions get filled by people who understand social structures and a lot of programmers (especially some of the most talented ones) do not fall into this bracket.

But given that you are a programmer who now has management responsibilities how do you handle both?

As a starting point you firstly have to realize that as a programmer and as a manager you are never going to be as productive a programmer as you would be if you were dedicated to just programming. This seems obvious but I am not just talking about splitting your time. If you have a 50/50 split between programming and management that 50% does not equal 50% programming output. If you are just starting out trying to manage together you will be lucky to be getting 20% of your normal output.

What I hope in this article is to show you that it is possible to do both and to improve your efficiency of both and that the roles can be very complimentary.

Learn to refocus

Programming effectively is all about focus. If you are not ‘in the zone’ then your productivity tends to go through the floor because without focus it is easy for your mind to wander and get distracted by almost anything (email, RSS, YouTube, Dilbert, etc). For most programmers focus can take time to get back so each time you get distracted you lose time having to refocus.

For a programmer who also wants to manage the ability to refocus immediately and pickup were you last left off is absolutely fundamental. If you imagine a typical refocus time of 15 minutes and you get distracted by management issues 10 times in one day then you have immediately lost nearly 3 hours of work.

Improving your ability to refocus on your work is something that takes time and is not something easy to teach. But below I have listed a few tips on actions you can take to make it easier to manage.

Set out a strict daily timetable of when you try and program and when you perform your management tasks, by doing this you have a clear idea of what you should be doing at any point of the day and stops you switching tasks each time you get interrupted.
Use note taking software to keep track of your current thoughts. If you are thinking through a typical programming problem start writing them down. If you get distracted you can more easily understand what you were last doing.
If you are a typical programmer that spends the whole day with headphones on you have to change this habit fast. Although you can remove the headphones as required this means you are not learning to cope with distraction.
Make sure you have auto-save enabled in whatever development environment you have. If you have to bring up other ‘management’ tools at the same time you are likely to have your dev environment crash on you at some point. (Microsoft Project has been my main culprit in the past.)
If you have a meeting booked in an hour’s time try to fill that time with management tasks, programming is best done in longer blocks of time therefore always perform your management tasks during times of the day that you know your time is going to be split up most and programming tasks during those that will be split up least.
Become disconnected

Programmers tend to be extremely passionate about the work they do. In a dedicated programmer this is exactly what you want. As a manager you cannot view your own work in the same way. Managers need to make broader decisions than a typical programmer and this means taking in information from a range of sources and it is counterproductive to view your own work personally. Not only will it likely skew the decision but also be seen by your staff as favoritism to yourself.

In a similar vein it is often the case that programmer likes to think they are better than everyone else (you grow out of it as you get older) but as a manager you have to be objective about your own abilities and about those who work under you. Understanding your strengths as a programmer can help you focus on what tasks are best applied to yourself and to the rest of the team.

As a programmer/manager you must always take the following into account when assigning yourself tasks.
According to your schedule do any of your tasks put yourself in a position that you can delay other team members.
How difficult would it be for another team member to take over this task if you find you have less programming time because of other management responsibilities.
Are you creating any other dependencies that could be alleviated by not doing the work yourself
Are you taking the task because it’s the most interesting? (not a good idea)
It may seem that the list stops you from doing any work but in reality you cannot always remove yourself from the critical path and it may not always be the right thing to do. The aim as with any other part of management to create contingences for every situation so that when things go wrong (they will) that you have already thought through what the course of action should be.

The good news

The great news is that programmers can make some of the best managers. Here is my list of reasons why you would want to do both.

You have the ability to understand what your technical team members are telling you.
When scheduling tasks you can tell if your team members are telling you porky pies about how long something is going to take.
Building a schedule is still a nasty task but at least it may resemble reality (for at least a few days)
You can confuse other managers by talking technical (my personal favorite)
You tend to get respect from your team members rather than abuse.
I would like to hear from everyone on their experiences of either becoming a manager or your observations on the difference between technical and non-technical managers.

Management Programming

Technology for Country Folk

Comparison of OpenDocument and Office Open XML formats

Overview
The OpenDocument format was originally defined by StarDivision (later acquired by Sun Microsystems) for their StarOffice product and was brought to OASIS by Sun and IBM who wanted it ratified as a standard. OpenDocument was approved as an ISO and IEC International Standard in May 2006, designated ISO/IEC 26300. It has been a published ISO/IEC standard since November 2006.

Office Open XML is defined by Microsoft and was approved as a standard by Ecma International in December 2006[1], designated ECMA 376.[2] Control of the Ecma standard will rest with Ecma International. It has been submitted to ISO/IEC for adoption under the ISO/IEC JTC 1 process.

The OpenDocument format is the native format of both OpenOffice.org 2.0 and KDE KOffice 1.5, and is targeted as a native format for multiple applications. Office Open XML is the native format for Microsoft Office 2007. A compatible plug-in has been released for some earlier editions of the Microsoft Office suite as well. At least three different OSS plug-ins for Microsoft Office [1] [2] [3] are being developed that will add support for opening and saving files in the OpenDocument format.


[edit] Advantages of OpenDocument over Office Open XML formats
Alex Hudson, J. David Eisenberg, Bruce D'Arcus and Daniel Carrera of the OpenDocument Fellowship wrote an article published by the online journal Groklaw that argues OpenDocument has several technical advantages over Office Open XML (Hudson, 2005[3]). The article examined some problems based on the original draft of the Office Open XML standard (which has since been superseded), and claims the following differences:

OpenDocument uses a mixed content model,[4] whereas the Office Open XML format does not. "Non-mixed documents usually represent structured data; mixed documents are usually used to represent narrative. MS XML uses the non-mixed model to represent narrative (word processing). "This sort of mismatch leads to awkward markup [...] The mixed-content model makes more sense, and is closer to what a developer will be familiar to."
OpenDocument is similar to XHTML, while MS XML is not. OpenDocument uses mixed content and marks styles in a similar way. "This makes it easier to transform data accurately between OpenDocument and XHTML, and also simplifies the reuse of existing skills."
OpenDocument gives better separation of style and content. "Both formats give you some separation, and neither format gives you perfect separation. But OpenDocument goes much further in that direction."
OpenDocument hyperlink URLs are embedded in the main file, whereas in Office Open XML the URL is placed in a separate file.
OpenDocument reuses existing standards whenever possible. It uses parts of SVG for drawings, MathML for equations, XLink for linking, Dublin Core for metadata, etc. "This makes the format infinitely more transparent to someone familiar with XML technologies. It also allows you to reuse existing tools that understand these standards", whereas "MS XML re-invents the wheel" (date format incompatible with ISO 8601, language specification incompatible with ISO 639). ODF's use of SVG is limited to some attributes and SVG content as such is not supported, even though this is widely touted.
Since the article written by the OpenDocument Fellowship, Office Open XML specification now incorporates the Dublin Core metadata as well. However, OpenDocument still claims the advantage of using a mixed model similar to XHTML, as well as separation of content from presentation. Perhaps most importantly, OpenDocument continues to reuse more existing standards wherever possible (such as SVG, MathML, and so on), instead of recreating their own unique format, simplifying implementation and interoperability (as well as reusing significant work from each of those pre-existing standards).


Advantages of Office Open XML formats over OpenDocument
Proponents of the Office Open XML format have addressed some of the criticism due to comparisons with OpenDocument, and offered their own criticisms of the OpenDocument format. Much of this criticism has been offered by Brian Jones, a Microsoft program manager for Microsoft Office who works on the XML functionality and file formats in the Office product.

Microsoft has stated that a design goal for its formats was 100% compatibility with the existing base of documents and formatting used by its customers. In particular, Jones states that OpenDocument is not able to capture all the information potentially held by a binary Office file, whereas Office Open XML should be able to do that.[5]
Microsoft also states that the OpenDocument format lacks support for the complete set of functionality in Microsoft Office applications (such as VBA and OLE support, support for highlighting,[6] international numbering,[7], tables in presentations[8] and other features), so any converter that saved information from an Office file (either binary or OpenXML) into an OpenDocument format would potentially be lossy. Counter arguments raised on GrokDoc on this matter claim that such features are allowed as part of the OpenDocument format as namespace extensions therefore negating this argument.
Microsoft Excel has a well-known formula language that has been defined in its entirety in the new XML formats, whereas the OpenDocument TC has at this stage published the ODF OpenFormula specification as a draft which is expected to be incorporated in ODF v1.2.
It has been suggested that Office Open XML supports several non-western languages better than ODF - specifically, that it has better Arabicization and Internationalization.[9][10] These issues have probably arisen due to the comments of the Egypt ISO member as part of the OpenDocument ISO standardisation process. The OpenDocument TC has addressed these comments, though, and states "OpenDocument v1.0 has BiDi support, as well as support for text orientations, directions, numeric digits presentations and calendars [...] The TC intents to add a non-normative appendix which explains these features to a future version of the OpenDocument specification". Text for this appendix is available [11] and explains where the TC suggest this support is derived.
The OpenXML spreadsheet format appears to be much faster than the ODF spreadsheet format. George Ou at ZDNet has tested both XML formats in their native applications OpenOffice.org 2.0 and Microsoft Office 2003. Office Open XML takes a distinctly different approach to the storage of spreadsheet data to OpenDocument, and implements several optimizations (such as sparsely populating worksheets, and sharing strings). Microsoft's Brian Jones had added some information[12] on this subject as well. Note also that the native proprietary Excel XLS binary format appears to be much faster than both XML implementations.[13]
All external references, such as hyperlinks or linked files, reside in a single relationships XML file contained in the document archive. This allows for easy access to all external references in the document. This makes it much easier to do link fix-up if you are moving files from one server to another. Or if you want to remove all external references for security reasons, you just edit the relationships.[14].

Shortcomings of OpenDocument
OpenDocument has no macro language specification. Java applets are described in the specification, but a macro language is not compulsory for compliance with the ODF standard.
The specification is incomplete : no syntax description of spreadsheet formulas (this is due in ODF v1.2). No description of passwords hashing yet (password hashes in an XML format would seem to be pointless, since the password can be bypassed by using a text editor to read or alter the content).
No native support of tables in presentations (this is due in ODF v1.2)
ODF 1.1 has no digital signature, (this is due in ODF v1.2)

Shortcomings of Office Open XML
The specification is incomplete : some parts are referencing the (not publicly specified) behaviour of other software, like "autoSpaceLikeWord95", without further explanation. This may be an issue for applications like archiving, where some content may become inaccessible or some clarity lost in future due to content being locked up in undocumented proprietary formats or behaviours. It may also be an issue for cross vendor interoperability and interchange of data, since only Microsoft or Microsoft licensed applications can access/process the content in these undocumented formats in OOXML with full clarity. These tags are deprecated so implementations should not create new documents containing these tags. The tags can only be applied in documents created or mutated in the original application the tags is referring to, which are converted to OOXML. There are no proposals to remedy the incomplete parts of the OOXML specification.
In SpreadsheetML, a markup language for spreadsheets used in Office Open XML one of the two numeric formats used for storing dates interpretes the number 60 as 1900-02-29 as if year 1900 were a leap year. Any implementation of this date1900 format needs to skip the number 60 when interpreting the numeric datevalue. This issue originated from Lotus 1-2-3, and was preserved by Microsoft Excel for backwards compatibility.
The specification for OOXML has a covenant not to sue which states that all patent claims from Microsoft needed to implement the spec can be used freely however the patent grant does not include future updates/versions. This means Microsoft can influence the development of future revisions or fixes by Ecma of this standard, similar to what Sun can do for ODF, so that future versions will be kept compatible with prior MS versions and will support MS Office featureset. Not granting such use of patent claims for future versions would make it unlikely that a new version would be used as any current versions will then still be available free for use.

[edit] Cross-platform interoperability
Microsoft Office 2007 for Windows uses Office Open XML as its native file format. Microsoft Office 2008 for Mac OS X, scheduled for release in late summer 2007, will also use Office Open XML as its native file format.[15] An ODF converter plugin for Microsoft Office XP/2003/2007 for Windows allows one to open and save OpenDocument word processing (.odt) files.
Corel has indicated that the WordPerfect Office X3 suite will include support for OpenDocument Format as well as Office Open XML by mid-2007.[16]
Gnumeric has included support for OpenDocument spreadsheet and preliminary support for Microsoft Office Open XML spreadsheet format since version 1.7.
IBM announced that Lotus Notes will use OpenDocument as the native format for its office productivity editors in the next release, due in 2007. IBM Workplace 2.6 already supports OpenDocument format.
Google Docs and Spreadsheets supports OpenDocument word processing and spreadsheet formats.
AbiWord 2.4 supports OpenDocument word processing format.
Scribus 1.3.3, a multi-platform, open source, page layout application, supports import of OpenDocument word processing files.
OpenDocument Format is currently supported in several office suites and individual applications[17], including as the native file format for KOffice 1.5, OpenOffice.org 2.0 and StarOffice 8. Support for OpenDocument was implemented independently, first in the KOffice 1.4 suite[18] and later in OpenOffice.org 2.0. Office suites which natively support OpenDocument Format are available on Windows, Mac OS X, Linux, BSD, Solaris, and Symbian OS.

Interoperability testing
The ODF Test Suite is a publicly available interoperability test suite developed by Intel and the University of Central Florida. Automated results are available for interoperability testing of KOffice and OpenOffice.org.

As of January 2007, no publicly available interoperability test suite exists for Office Open XML format. Since no currently released office suites provide native support for the format, it is not known to what extent documents saved in the Office Open XML format will be properly formatted in other office suites.


Example XML comparisons
First an example of the mixed vs non mixed examples as provided in the groklaw comparison of the two formats. Non-mixed documents usually represent structured data; mixed documents are usually used to represent narrative. MS XML uses the non-mixed model to represent narrative (word processing).

Non-Mixed (Open XML)

<w:p>
<w:r><w:t>This is a </w:t></w:r>
<w:r><w:rPr><w:b /></w:rPr><w:t>very basic</w:t></w:r>
<w:r><w:t> document </w:t></w:r>
<w:r><w:rPr><w:i /></w:rPr><w:t>with some</w:t></w:r>
<w:r><w:t> formatting, and a </w:t></w:r><w:hyperlink w:rel="rId4" w:history="1">
<w:r><w:rPr><w:rStyle w:val="Hyperlink" /></w:rPr><w:t>hyperlink</w:t></w:r>
</w:hyperlink>
</w:p>

Mixed (ODF):

<text:p text:style-name="Standard">
  This is a
  <text:span text:style-name="T1">very basic</text:span>
  document
  <text:span text:style-name="T2"> with some </text:span>
  formatting, and a
  <text:a xlink:type="simple" xlink:href="http://example.com">hyperlink</text:a>
</text:p>

Secondly an example (provided by Brian Jones weblog) to support Microsoft's choice for smaller tagging. For this example, the top example is using SpreadsheetML from the Office Open XML format. The second example is using the OpenDocument format.

Short tag example (Open XML):

<row><c><v>1</v></c><c><v>2</v></c><c><v>3</v></c></row>
<row><c><v>4</v></c><c><v>5</v></c><c><v>6</v></c></row>

Long tag example (ODF):

<table:table-row table:style-name="ro1">
<table:table-cell office:value-type="float" office:value="1">
 <text:p>1</text:p>
</table:table-cell>
<table:table-cell office:value-type="float" office:value="2">
 <text:p>2</text:p>
</table:table-cell>
<table:table-cell office:value-type="float" office:value="3">
 <text:p>3</text:p>
</table:table-cell>
</table:table-row>
<table:table-row table:style-name="ro1">
<table:table-cell office:value-type="float" office:value="4">
 <text:p>4</text:p>
</table:table-cell>
<table:table-cell office:value-type="float" office:value="5">
 <text:p>5</text:p>
</table:table-cell>
<table:table-cell office:value-type="float" office:value="6">
 <text:p>6</text:p>
</table:table-cell>
</table:table-row>

In the second example, it is important to note that the size of the document is only marginally impacted by the length of its tags, because OpenDocument files are usually compressed. However, according to Brian Jones, the length of tags does impact compression and parse time when manipulating big documents. A non-mixed content (such as in OOXML) is likely to be more compact than a mixed one.[citation needed]

Also noted that, in the second example, ODF holds extra two attributes about the value in each cell, attributes office:value-type and office:value, for cell's type and cell's value. Cell's type can be one of "float", "currency", "percentage", "date", or "time"[19]. These attributes explicitly describe the textual representation kept in element. These information are not captured by the OOXML tags shown in the example.

Example of ODF spreadsheet value vs its textual representation, a cell stored "45.6%":

Java problem -- Too many open files

Written by Mike Noel

Thursday, 23 September 2004

Recently I was working on a Java project. The program was intended to run for a long time (weeks at a time) and process incoming requests. I started the overnight test to see how it would do receiving requests over a 12 hour period. After five hours the program crashed with this message: java.io.IOException: Too many open files So I had to dig in and figure out what was causing this. First of all, the number of open files allowed is an Operating System dependent setting. This is a limited resource that is shared by all processes on the machine. The standard programming practice is to make sure to close all files that the program opens. On Unix systems this applies to socket connections as well. I inspected my code and looked for all open statements. I made sure that each open statement had the accompanying close. After looking over my code I was certain that everything was closed. But I was still seeing the accumulation of open files as messages were handled. Usually in these type of situations I turn to Google to see if someone else has dealt with this problem too. After a bit of searching I found the clue. My program uses the Runtime.getProcess() method to get a process and then exec() an external script. What I didn't know was that the API automatically opens three streams (stdout, stderr, stdin) each time the getProcess() is called. It is the responsibility of the caller to close those streams when done. I added the code to close those and everything was fine. Of course, after doing this I read the documentation for the Runtime class. Sure enough, it mentions the three streams... It's amazing what a little bit of knowledge can do.

AJAX: The veins of Web 2.0

In Homer's Iliad "Ajax the great" is a legendary Greek warrior that the Trojans feared. AJAX is therefore a fitting name for a technology that could prove as unstoppable in conquering the web as Ajax was in fighting the Trojans.

Perhaps it is still too early to give Ajax more prominence than DHTML and Java, that have been around for much longer. But AJAX is a development platform that is already gaining traction on some of the most popular sites on the web.

Web 2.0 and AJAX are partners. Web 2.0 is about making the web more intelligent and interactive, building communities and encouraging collaboration. It doesn't prescribe how this vision is to be achieved, nor is it a development suite. It's simply a vision that speaks of a new way of thinking about the web.

AJAX is a collection of development tools that work together to make an individual web page more intelligent and interactive. In some sense AJAX fulfils the Web 2.0 culture of moving away from static web pages with fixed content towards a more dynamic and interactive experience. It doesn't quite fulfil every single Web 2.0 dream, but that may even be the reason why AJAX has become so popular so rapidly – it's a real technology with achievable implementation rather than an abstract description of a utopian vision.

The principle of making web pages more dynamic and interactive is not a new one. So what's so great about AJAX?

The early web was basically a collection of online documents that were interconnected through hypertext links. The vision was quite revolutionary at the time, because it connected chunks of isolated information with its interlinked content based around a common standard – HTML – and this increased the informational value of the entire network.

Even now, as in the early days of the web, web pages are often created offline and simply uploaded to the web server. Changes to the page requires manual editing offline and for the new version to be uploaded and copied over the old one. This was fine for posting an essay online, but it lacks interactivity and is also cumbersome to manage. The web had far greater potential in terms of information and content delivery than merely displaying static pages.

Technology evolved through the last decade and a half, to include more dynamism and interactivity on web pages using JavaScript, ActiveX, and XML, and more intelligent back-ends, such as SQL databases, CGI and ASP. This lead to the other kind of ASPs (Application Service Providers) such as Hotmail or Yahoo Notepad, as well as other interactive features like online forums and web-based chat.

HTML was never developed with real-time interactivity in mind, but it was still the foundation of the Internet and it became a major drawback. Any change to a web page – such as entering and saving information in a form or viewing the next image in a slideshow – typically required the web server to compose a new page in HTML and download that to the client.

In the late 1990s DHTML was touted as the first real answer to the demand for interactive pages. The beauty of it was that it made possible, for example, the client to display a new image on a web page without having to contact the server and request a whole new page. With DHTML however, web pages only appeared to be dynamic. All the content related to a page would be pre-loaded, and the page couldn't get additional information from the server without needing to refresh. Interactive web pages required information to be sent from the server to the client without needing to refresh the page.

The difference between AJAX and DHTML is subtle, but the implications are profound. AJAX uses many of the same tricks as DHTML, such as JavaScript and CSS, but the key difference is that AJAX can communicate with the server, and request new information, without needing to reload the page.

This seemingly trivial feature makes all the difference. No longer is a page limited by the information that can be loaded into the background. Instead it can have the full wealth of the information on the server at its fingertips, and yet present a solid and consistent front end to the user.

Without AJAX, pages like Google Earth would need to refresh each time you panned or zoomed. You can interact with the map using the intuitive tools you'd expect, such as dragging it around or zooming in and out. And all this takes place without the page needing to reload. An example of a site that does not use this technology is the Cricinfo.com – 'Live scorecard' feature which has to reload web pages constantly as new information is added.

AJAX uses a range of tools to bring web pages to life. At the core is XHTML (Extensible HTML), which is a fusion of XML and HTML and is ratified by the World Wide Web Consortium or W3C, who are the overseers of HTML and other web standards. XHTML brings the power of XML to web pages, with its ability to manage information and interact with servers in ways that HTML can't. XHTML works along with CSS to actually render the page.

The next piece of the puzzle is DOM (Document Object Model), which is a way of managing and manipulating the information in an HTML or XHTML document.

The final, and critical component, is XMLHttpRequest, which is a clumsy name for a very neat piece of technology. XMLHttpRequest is an API that works with a range of scripting languages, which in the case of AJAX is usually JavaScript. XMLHttpRequest can communicate directly with the server and request and manipulate data, and then pass that through to the page being viewed via JavaScript.

Another way to visualise AJAX in action is to picture the AJAX engine sitting in between the web and the page you're viewing. Instead of your page requesting information from the server directly, and constantly reloading, the AJAX engine does all the talking, and leaves your web page alone, with AJAX only updating the elements on the page that it needs to.

It's not that AJAX will completely transform the web, but it will further the evolution of the web from being a series of interlinked documents to being an information-based application platform. Online email, calendars, maps, stock market graphs, world clocks and image viewers are just some of the rudimentary applications of AJAX that are around today, with more sweeping implementations on the horizon.

AJAX doesn't do everything, nor does it purport to, but it has become a cornerstone of Web 2.0 technology which is without a doubt the single most significant development in web technology since the turn of the millennium.