Why XQuery?

Introduction

XQuery is a powerful, concise, and highly optimized language that is designed for processing XML data — that is, it can process any data that can be expressed as XML. XQuery allows applications to query relational data together with structured data, such as native XML. Traditionally, enterprise applications have relied on SQL queries to retrieve and update data in relational databases, but given the continuing rise in prominence of XML, this is already changing. Even before the W3C XQuery specification reaches Recommendation status, XQuery is grabbing the attention of many developers, CTOs, and development managers.

This article provides information on XQuery's background, its benefits, and its pivotal role in data integration. Next, it provides an overview of XQuery's syntax and shows examples of what an XQuery query looks like. Finally, we describe how you can implement XQuery and mention some caveats to be aware of in XQuery 1.0.

XQuery: Its History

XQuery originally evolved from another XML query language named Quilt. To understand the origin of XQuery, we asked Jonathan Robie, the XML Program Manager at DataDirect Technologies and XQuery specification co-author, to explain what triggered the effort to create a language that could query XML. Jonathan provided the following response:

"Back in 1998, those of us who had bought into the XML vision assumed that many kinds of information would be exchanged or stored as XML, and that this would involve massive amounts of data. We also realized that a query language that was based purely on the structure of XML could be used for anything that XML could represent. Even better, the semi-structured community taught us that the data did not need to be physically represented as XML, as long as we knew an XML representation and were willing to write some middleware. For instance, an XML query
could think of relational data as XML data, and middleware could translate the query to SQL, turning results
into XML for the XML query. So, we already had a universal representation for data, XML, and we wanted a universal query language for this data. By May 2000, we had developed Quilt, the predecessor of XQuery."

Unlike extensions to the SQL language that give some provision to work with XML, XQuery is designed as a separate language optimized to work with XML data. Now that we've established XQuery as a powerful XML processor, let's take a look at why its arrival is so important.

XQuery's Role in Data Integration

XML is now considered the de facto standard for exchanging data. The rapid growth of XML and increase of hierarchical messages present a fresh set of challenges, particularly to developers who have historically built their business processes around databases. Because relational database systems are well-established and reliable, they are unlikely to disappear any time soon. The growth of XML is forcing modern business applications to function seamlessly with both relational and XML data.

XQuery levels the data integration playing field by providing a single interface that lets developers access multiple data sources under a unifying data model. Products such as DataDirect XQuery™ from DataDirect Technologies deliver Java components that allow the developer to present and exchange relational data as XML and to process relational and XML data together.

Before XQuery, integration of relational and XML data forced architects and developers to choose from one of the following complex strategies:

  • Shredding (de-composing XML into relational tables) XML data into individual relational database table columns. This process "flattens" the built-in data hierarchy and often loses internal relationships. The original XML document itself also is lost, although it can, in some cases, be reproduced from the shredded data.
  • Storing the XML data as unstructured data in a relational database using the Clob data type. Clob columns can store an XML document in its entirety, preserving both the document and its internal relationships.
  • Storing the XML data as a structured XML document in a relational database. This option provides a tightly coupled relationship between XML and relational data within the context of a relational database system. However, support for the co-existence of structured XML and relational data in today's relational database systems varies considerably, which makes it difficult to implement a portable and consistent data integration strategy.

Because XQuery is about seamless data integration, the middle-tier is the logical sweet spot for developers and architects to establish the solution to their most complex data integration challenges.

XQuery: Anatomy

We've introduced XQuery, described its objectives, and explored how it will play a critical role in tackling complex data integration challenges. Now, let's look at some of the details of XQuery, starting with an overview of the data model that XQuery uses.

XQuery Data Model

XQuery uses an XML data model that can represent XML documents, sequences, or atomic typed values (such as integers or strings). For example, an XML document is represented as a tree of nodes, including a document node, element nodes, attribute nodes, comment nodes, and text nodes. Both the input and the output of any query are represented in this data model.

XQuery Data Types

The primitive data types in XQuery are the same as the data types for XML Schema. They include floating point numbers, Booleans, strings, dates, and many other data types you've encountered in other languages. In addition, XQuery allows you to extend these types using a variety of methods, providing maximum flexibility to the developer.

XQuery Expressions, Operators, and Functions

We will now present an overview of the functional language components of XQuery. For the purposes of this overview, we're using the sample data from the XQuery Use cases.

Path Expressions
All path expressions are based on the same syntax as XPath 2.0. XPath has long been the established standard for specifying paths in an XML document.

This example searches the books.xml document and returns every chapter node's title child value.

document("books.xml")//chapter/title

Element Constructors
Using simple constructs, you can create a new XML structure for an XQuery result.

This example, used in conjunction with a more complete XQuery expression, generates a new element <thesis> for each value for title.

<thesis>
   {$b/title}
</thesis>

FLWOR Expressions
The FLWOR expression, pronounced "flower," is the most powerful of XQuery expressions and is, in many ways, similar to SELECT-FROM-WHERE used in SQL. Most XQuery queries are built around the FLWOR expression. The name FLWOR is an acronym, representing the first letter of the clauses that can occur in a FLWOR expression: FOR, LET, WHERE, ORDER BY, and RETURN.

In this example, we perform a simple search that returns all books published by Addison-Wesley after 1991, inclusive of each book year and title.

<bib>
 {
  for $b in doc("http://bstore1.example.com/bib.xml")/bib/book
  where $b/publisher = "Addison-Wesley" and $b/@year > 1991
  return
    <book year="{$b/@year}">
     {$b/title}
    </book>
 }
</bib>

In this example, the FLWOR expression consists of the following clauses:

  • The FOR clause binds the $b variable to a sequence of book title values.
  • The WHERE clause specifies conditions that filter the book values to only those books published by Addison-Wesley after 1991.
  • The RETURN clause builds the result of the FLWOR expression based on the conditions specified in the WHERE clause and using the structure specified in the RETURN clause. In this case, the result looks like this:
<bib>
   <book year="1994">
      <title>TCP/IP Illustrated</title>
   </book>
   <book year="1992">
      <title>Advanced Programming in the Unix environment</title>
   </book>
</bib>

Operators and Functions
As you would expect, XQuery provides a similar range of operators and functions that you have encountered in other languages. In addition, XQuery provides functions that allow queries to leverage the XML document structure.

In this example, we use the function min() to determine the minimum price for each book. We generate the <minprice> element, with the book title as its attribute.

<results>
  {
    let $doc := doc("prices.xml")
    for $t in distinct-values($doc//book/title)
    let $p := $doc//book[title = $t]/price
    return
      <minprice title="{$t}">
        <price>{min($p)}</price>
      </minprice>
  }
</results>

There is much more to XQuery including conditional and quantified expressions that allow you to compose more powerful queries to fully leverage the potential of XQuery. We provide more information about these in the XQuery Basics Primer.

Using XQuery with Your Java Applications

Having gained an understanding of XQuery and what a basic XQuery query looks like, we now look at implementations that allow you to use XQuery with your applications today.

The XQuery API for Java (XQJ) is the standard Java API for using XQuery with Java applications. XQJ functions much like the JDBC API by channeling your XQuery query to the data source and returning the XML result so that it can be used by any Java application. We explore more on how this works in the XQJ Primer.

Middle-tier components such as DataDirect XQuery™ are built on XQJ and allow any Java application to perform XQuery queries on XML or relational data, or a combination of the two. Information about other implementations can be found on the W3C XQuery Web page.

XQuery Gotchas

XQuery is quickly approaching the final approval of the W3C standard organization. All the signs point to a Recommendation status in the not too distant future. There are, however, some caveats to be aware of in the XQuery 1.0 release:

Summary

The official W3C XQuery Recommendation is almost done! Significant interest in using XQuery for data integration is gathering steam, signaling a healthy adoption of the technology when it does reach Recommendation status. To learn more about XQuery, our XQuery Basics and XQJ Primers provide the basics you need to succeed with XQuery, and ultimately, your data integration challenges. Our XQuery Tutorial and XQJ Tutorial also provide more in-depth knowledge about these technologies.