What's happening with XML standards?

Microsoft's spate of announcements about support for XML in its Office product are the latest sign that XML is rapidly...

Microsoft's spate of announcements about support for XML in its Office product are the latest sign that XML is rapidly penetrating every corner of the IT space, even corners which, until recently, were completely proprietary.

When it started, XML was small and simple, and many people believe that's why it was so successful. However, XML isn't standing still and, despite the misgivings of some of its original creators, the standards are getting bigger and more complex all the time.

Although the base XML specification has remained unchanged for nearly five years, a vast superstructure of related standards has been built on top of these foundations. This article tries to give an overview of the landscape, and the specifications you actually need to take notice of.

A layered model
Think of the whole structure as having four layers: the XML core; the basic XML toolkit; schema and query; XML Applications.

Layer 1 is the XML Core. The most important part of the XML core is the XML syntax specification itself, which defines the "angle bracket" syntax of XML documents as well as the rather arcane, but still widely used, specifications for Document Type Descriptions.

Also in this layer is the XML Namespaces specification, which describes how different organisations using XML can avoid clashes between their different vocabularies, together with the XML Infoset, which is not so much a standard in its own right as a way of modelling the information that XML documents contain.

Coming up soon at this layer are XML 1.1 and XML Namespaces 1.1. As the numbering suggests, these will be fairly small changes to the existing specifications, and the new features are unlikely to affect many users. It has been said that XML 1.1 is designed to meet the needs of two neglected minorities: Users of the Ethiopic writing system, and users of IBM mainframes.

The changes do affect companies that produce XML parsers. However, there is some risk that during the transition period an enterprise might find itself receiving XML 1.1 documents from one of its business partners before it has upgraded to use an XML 1.1 parser, perhaps even before an XML 1.1 parser is available for a particular computing platform.

It could be argued that the people who would benefit from the changes are much fewer in number than those who would have to cope with the disruption.

But given that the specifications have been stable for five years, and that the Worldwide Web Consortium (W3C) is trying hard to live up to its name and embrace the needs of all users, it is hard to complain.

The basic XML toolkit
Layer 2 is the "basic XML toolkit". These are classified as DOM, SAX, XPath, and XSLT. These have all been around for a few years and are likely to be familiar to most XML programmers.

There are many implementations available, covering all popular platforms, and there's a good level of interoperability between them, though DOM in particular suffers from having multiple variants and modules within the specification, plus many vendor extensions: Microsoft users are often unaware of when they are using standard DOM features and when they are using Microsoft additions.

XPath and XSLT have been undergoing a major upgrade to create XPath 2.0 and XSLT 2.0. Although the 1.0 products have been successful, their capability is limited, and users have been waiting a long time for facilities such as the ability to group data by common values, as well as regular expression support.

One reason the standards have taken a long time to develop is the influence of XML Schema and XQuery.

The W3C working groups made the controversial decision to couple the next versions of XSLT and XPath closely to XML Schemas. Because this changes the type system that forms the internal backbone of any programming language, this has required some major re-engineering and it has taken some time for all the details to be agreed. With luck, these specs will be finalised during 2003.

SAX is interesting as a rare example of a standard that was produced by individuals working together without any formal standards organisation to support them. Although this worked well to produce the initial versions of the spec, there are signs that subsequent upgrades may prove more difficult using this model. However, it could help to prevent the accumulation of complexity that all the other specifications suffer from.

Schema and Query
I sometimes refer to Layer 3 as "XML information management", because it represents the emergence of XML as a key technology in the way organisations manage their information asset, as distinct from its original role as a syntax for marking up an informal document.

The two main components at this layer are XML Schema, which describes the structure of the information, and XQuery, the query language for accessing XML data. XML Schema has been a standard since May 2001, but it is a highly complex specification and, as yet, there are relatively few implementations, and these are incomplete.

Equally complex, and closely related to XML Schema, is XQuery, the proposed specification for querying XML databases, which is designed to play the same role in the XML world as SQL enjoys in the world of relational data.

XQuery has been a long time gestating, perhaps because of the sheer number of companies involved in defining it. Although this demonstrates a high level of industry commitment to the specification, designing a language in a large committee is not always a recipe for success.

XML Schema and XQuery have attracted a fair amount of controversy. They are highly complex standards, difficult to understand and difficult to implement, and not very accessible to the open-source community that has done so much to make XML layers 1 and 2 usable, affordable and, therefore, successful.

XML Schema has even caused something of a rebellion in the form of a competitor, Relax NG, which many people feel achieves the same goals with a vastly simpler architecture. Yet in big organisations, the W3C specifications appear to have widespread support, and there is every sign they will be widely adopted in time.

XML applications
Layer 4 is a bundle that could be called "XML applications". Of course, one man's "application" is another man's "infrastructure" - it all depends whether you look at it from above or below.

Into this area goes all the higher-level specifications that make use of XML, such as the standards that underpin Web Services (Soap, UDDI, WSDL), and the XSL Formatting Objects specification for high-quality XML-based publishing.

Then there are the myriad specialist XML vocabularies covering every conceivable area from financial information, business reporting, and news transmission, to specialist vocabularies such as MusicXML for music publishing, and HumanML which describes "human characteristics including physical, cultural, social, kinesic, psychological, and intentional features".

Other specifications
There are other specifications that do not fit neatly into this structure, for example the hyperlinking standards XPointer and XLink. One of the reasons these specs have not been as widely adopted as some of the others is that they are often regarded as architectural misfits.

XPointer, which allows one document to contain references to the content of another, has been rewritten several times, and perhaps the latest version will attract wider support than its predecessors.

XLink, which builds on XPointer to define a model for hyperlinking between documents, still suffers from the fact that no one quite knows how it fits in with the other specifications in the XML family.

The XML family is growing from the original XML 1.0 specification, which was about 40 pages long, to a vast library of documents, many of which run to hundreds of pages.

Layers 1 and 2 in the structure are proving pretty stable and will continue to be the baseline for most XML programmers. The new 2.0 versions of XPath and XSLT will probably catch on because they contain many facilities that are desperately needed to counterbalance the support for XML Schema, which quite a few existing users would rather do without.

The new specifications in Layer 3 - XML Schema and XQuery - are likely to be increasingly important as XML becomes central to the IT strategy and architecture of large enterprises. But as the enthusiasm for Microsoft's recent announcements demonstrates, layer 4 - the applications - is where all the real business benefits come from.

Read more on Business applications