Home    Articles    Essential XQuery - The XML Query Language

Essential XQuery - The XML Query Language Monday, February 2, 2004

Summary
XQuery or XML Query Language is a W3C specification (http://www.w3.org/TR/xquery) designed to provide a flexible and standardized way of searching through (semi-structured) data that is either physically stored as XML or virtualized as XML (such as XML Views over relational data). To give you an analogy, SQL is used to query relational data, XQuery is a standard language for querying XML data. In addition, XQuery also allows general processing of XML (such as creating nodes). Currently in the last call working draft status, the W3C XQuery specification will hopefully get the recommendation status this year. This article is an introduction to XQuery language. It is based on November 2003 working draft. (Click here to see the changes made to 23^rd July 2004 XQuery 1.0 Working Draft.)

(23 printed pages)

• This article assumes that you have some familiarity with XML 1.0, XPath 1.0, XML Namespaces, XML Schema, and XSLT. If you want to get a quick 15 minutes refresher on XPath, see my article on PerfectXML.com, XPath for .NET Developers; this article contains some introductory text on XPath.
• The examples presented in this article might not work with Microsoft SQL Server "Yukon" Beta 1 XQuery implementation, which is based on subset of XQuery drafts in range of December 20, 2001 to November 15, 2002.
• Download the latest version of Saxon from saxon.sourceforge.net to try out the examples. I am using version 7.8 (using its command line interface) in this article.

The XQuery effort at W3C is lead by the XML Query Working Group (http://www.w3.org/XML/Query). XQuery borrows a lot of features from XPath; actually, XQuery 1.0 is said to be an extension to XPath 2.0. Both, XPath 2.0 and XQuery 1.0 make use of the same data model (the abstract, parsed, logical structure of an XML document). XPath 2.0 and XQuery 1.0 are very closely related. The XML query working group works closely with XPath editors (part of XSL working group), and have published specifications together.

XQuery 1.0 is a strongly-typed language. The XQuery type system is based on XML Schema W3C recommendation (http://www.w3.org/TR/2001/REC-xmlschema-0-20010502/). XQuery 1.0 supports XML Schema simple types (such as string, integer, dateTime, etc.), types from imported schemas (such as USAddress), and XML document structure types (such as document, element, attribute, node, processing instruction, comment, etc.).

There are total twelve documents published by XML Query Working Group (some in conjunction with XSL working group). These documents together define the XQuery 1.0 and XPath 2.0 languages. When writing XQuery queries, I often refer to specifications marked as bold in the following table:

You write XQuery queries to efficiently extract data from XML data/documents. These queries can be executed via XQuery processor implementations (like Yukon, SAXON 7.8, Qexo, X-Hive/DB and so on). You can execute these queries on command prompt, or from within your code. In this article, we'll execute all the queries on the command prompt using SAXON 7.8. SQL Server Yukon allows executing XQuery queries using SQL Workbench tool; it also includes XQuery Designer allowing you to graphically create and execute XQuery expressions. In addition, you can write .NET code to execute XQuery expressions.

The concept of querying XML documents is not new. Several past research papers, proposals, and specifications have influenced the design of XQuery. These include XQL, OQL, XML-QL, Lorel, Quilt and XPath 1.0 The last two entries in the previous list (Quilt and XPath 1.0) have the greatest impact on how XQuery looks today. If you are familiar with XPath concepts (expressions, location steps, XPath functions, and so on), XQuery would seem a natural extension to that with looping, sorting, conditional, and modularization constructs added in order to meet the requirements of a query language. XPath now can be treated as a core subset of XQuery.

Instead of digging deep into XQuery concepts here, the way I want to proceed is to list some facts that you should remember, and then show you some XQuery examples, followed by discussion on XQuery theoretical concepts and study the terminology. With this, here is a list of some important facts about XQuery that you should remember:

• XQuery 1.0 queries are not written using XML syntax (XSLT is written using XML syntax, for instance). Like XPath, Expressions form the basic building block of XQuery queries. Another W3C proposal, XQueryX, is in progress that makes use of XML syntax for XQuery queries.
• XQuery is a case-sensitive language. All the XQuery keywords are in lower-case.
• XQuery is a functional language comprised of several kinds of expressions that return values. Every query is an expression to be evaluated. These expressions can be nested and composed with full generality. We'll talk about this more, and see examples of this in following sections.
• XQuery is a strongly typed language with a type system based on W3C XML Schema (XSD) specification. Strong typing provides two primary benefits: static compilation for early error detection, and query optimization by devising efficient execution plan (as the engine knows beforehand about the object types). We'll see examples of this in following sections.
• XQuery 1.0 as such does not specify any mechanism to perform updates using XQuery queries. However, almost all the XQuery implementations have custom extension to allow updates using XQuery queries.
• XQuery allows locating nodes (like XPath), constructing and re-structuring XML (like XSLT, but XQuery has a different syntax), sorting, filtering, and so on. One of the most commonly used XQuery expressions is the FLWOR (for-let-where-order by-return) expressions, pronounced "flower expressions".
• XQuery makes some of the XPath axes as optional. In other words, it is optional for XQuery processor to support ancestor, ancestor-or-self, following, following-sibling, preceding, and preceding-sibling axes. If it does support these axes (in addition to other XPath axes), it is said to support Full Axis Feature.
• XQuery offers the ability to do JOINs across documents.
• XQuery offers a comprehensive set of functions and operators that you can use in your queries.

The next two sections are provided here to give you an idea of how XQuery queries look like. Don't worry too much about syntax at this point, just look at the queries, understand the generated output, and we'll look at the details in the subsequent sections.

The series of examples in the next section are based on the following two sample XML documents: emp.xml (contains employee data) and timeoff.xml (contains absence requests data):

<?xml version="1.0" encoding="utf-8" 
         standalone="yes" ?>
<employees>

  <emp id="1">
    <name>Kelly Firkins</name>
    <email>kelly</email>
    <ssn>111-11-1111</ssn>
    <managedEmail>dean</managedEmail>
    <dateOfJoining>1994-11-16</dateOfJoining>
  </emp>

  <emp id="2">
    <name>Dean Parker</name>
    <email>dean</email>
    <ssn>222-22-2222</ssn>
    <managedEmail>mark</managedEmail>
    <dateOfJoining>1992-12-24</dateOfJoining>
  </emp>

  <emp id="3">
    <name>Kim Fell</name>
    <email>kim</email>
    <ssn>333-33-3333</ssn>
    <managedEmail>mark</managedEmail>
    <dateOfJoining>1993-08-19</dateOfJoining>
  </emp>

  <emp id="4">
    <name>Katie Parise</name>
    <email>katie</email>
    <ssn>444-44-4444</ssn>
    <managedEmail>kim</managedEmail>
    <dateOfJoining>1996-07-11</dateOfJoining>
  </emp>

  <emp id="5">
    <name>Mark Robinson</name>
    <email>mark</email>
    <ssn>555-55-5555</ssn>
    <managedEmail></managedEmail>
    <dateOfJoining>1990-01-01</dateOfJoining>
  </emp>

</employees>

<?xml version="1.0" encoding="utf-8" 
         standalone="yes" ?>
<timeoff>

  <request id="T1" empID_="2" 
    startDate="2003-01-02" endDate="2003-01-10"
    hours="40" status="approved" absenceCode="vacation" />

  <request id="T2" empID_="4" 
    startDate="2003-01-10" endDate="2003-02-10" 
    hours="180" status="approved" absenceCode="vacation" />

  <request id="T3" empID_="1" 
    startDate="2003-01-15" endDate="2003-01-31" 
    hours="80" status="declined" absenceCode="vacation" />

  <request id="T4" empID_="1" 
    startDate="2003-02-01" endDate="2003-02-01"
    hours="8" status="approved" absenceCode="sick" />

  <request id="T5" empID_="3" 
    startDate="2003-02-12" endDate="2003-02-12" 
    hours="8" status="approved" absenceCode="sick" />

  <request id="T6" empID_="1" 
    startDate="2003-03-01" endDate="2003-03-05" hours="30" 
    status="approved" absenceCode="vacation" />

  <request id="T7" empID_="4" 
    startDate="2003-03-12" endDate="2003-03-15" hours="10" 
    status="approved" absenceCode="unpaid" />
    
</timeoff>

In this section, I'll use the above two sample XML documents and show you 15 different queries. As mentioned before, I'll be using SAXON 7.8 to run these sample queries. I'll assume that all the queries and data files (above two XML files) reside in the same folder.

doc("emp.xml")/employees/emp[managerEmail='mark']

java net.sf.saxon.Query 1.xq

<?xml version="1.0" encoding="UTF-8"?>
<emp id="2">
      <name>Dean Parker</name>
      <email>dean</email>
      <ssn>222-22-2222</ssn>
      <managerEmail>mark</managerEmail>
      <dateOfJoining>1992-12-24</dateOfJoining>
   </emp>
<?xml version="1.0" encoding="UTF-8"?>
<emp id="3">
      <name>Kim Fell</name>
      <email>kim</email>
      <ssn>333-33-3333</ssn>
      <managerEmail>mark</managerEmail>
      <dateOfJoining>1993-08-19</dateOfJoining>
</emp>

Error on line 1 of file:/C:/temp/emp.xml:
Error reported by XML parser: Document root element is missing.
Warning: org.xml.sax.SAXParseException: Document root element is missing.
Error on line 1 Failed to load document emp.xml

doc("emp.xml")/employees/emp[managerEmail='' or count(managerEmail) < 1]

java net.sf.saxon.Query 2.xq

<?xml version="1.0" encoding="UTF-8"?>
<emp id="5">
      <name>Mark Robinson</name>
      <email>mark</email>
      <ssn>555-55-5555</ssn>
      <managerEmail/>
      <dateOfJoining>1990-01-01</dateOfJoining>
</emp>

doc("emp.xml")/employees/emp[upper-case(substring(name, 1, 1)) = 'K']

java net.sf.saxon.Query 3.xq

<?xml version="1.0" encoding="UTF-8"?>
<emp id="1">
      <name>Kelly Firkins</name>
      <email>kelly</email>
      <ssn>111-11-1111</ssn>
      <managerEmail>dean</managerEmail>
      <dateOfJoining>1994-11-16</dateOfJoining>
</emp>
<?xml version="1.0" encoding="UTF-8"?>
<emp id="3">
      <name>Kim Fell</name>
      <email>kim</email>
      <ssn>333-33-3333</ssn>
      <managerEmail>mark</managerEmail>
      <dateOfJoining>1993-08-19</dateOfJoining>
</emp>
<?xml version="1.0" encoding="UTF-8"?>
<emp id="4">
      <name>Katie Parise</name>
      <email>katie</email>
      <ssn>444-44-4444</ssn>
      <managerEmail>kim</managerEmail>
      <dateOfJoining>1996-07-11</dateOfJoining>
</emp>

data(doc("emp.xml")/employees/emp[name='Kim Fell']/dateOfJoining)

1993-08-19

sum(doc("timeoff.xml")/timeoff/request[@empID_=4 and 
	@status="approved"]/@hours)

190

doc("timeoff.xml")/timeoff/request[@absenceCode="unpaid" and 
	@status="approved"]

<?xml version="1.0" encoding="UTF-8"?>
<request id="T7" empID_="4" startDate="2003-03-12" 
      endDate="2003-03-15" hours="10" status="approved" 
      absenceCode="unpaid"/>

for $req in doc("timeoff.xml")/timeoff/request
order by xs:double($req/@hours) descending
return $req

<?xml version="1.0" encoding="UTF-8"?>
<request id="T2" empID_="4" startDate="2003-01-10" endDate="2003-02-10" 
          hours="180" status="approved" absenceCode="vacation"/>
<?xml version="1.0" encoding="UTF-8"?>
<request id="T3" empID_="1" startDate="2003-01-15" endDate="2003-01-31"
          hours="80" status="declined" absenceCode="vacation"/>
<?xml version="1.0" encoding="UTF-8"?>
<request id="T1" empID_="2" startDate="2003-01-02" endDate="2003-01-10"
          hours="40" status="approved" absenceCode="vacation"/>

for $emp in doc("emp.xml")/employees/emp, 
    $req in doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id]
where $req/@status = 'declined'
return data($emp/name)

Kelly Firkins

for $emp in doc("emp.xml")/employees/emp, 
    $req in doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id 
         and @status = 'declined']
return data($emp/name)

<Report>
{
   for $emp in doc("emp.xml")/employees/emp
   let $req := doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id and 
	@status="approved"]
   order by $emp/name ascending
   return
      <timeoffDetails 
            empID="{$emp/@id}" 
            empName="{$emp/name}"
            totalHours="{sum($req/@hours)}"
            />
}
</Report>

<?xml version="1.0" encoding="UTF-8"?>
<Report>
   <timeoffDetails empName="Dean Parker" empID="2" totalHours="40"/>
   <timeoffDetails empName="Katie Parise" empID="4" totalHours="190"/>
   <timeoffDetails empName="Kelly Firkins" empID="1" totalHours="118"/>
   <timeoffDetails empName="Kim Fell" empID="3" totalHours="8"/>
   <timeoffDetails empName="Mark Robinson" empID="5" totalHours="0"/>
</Report>

declare function local:count-of-requests() 
   as item()*
{
	for $emp in doc("emp.xml")/employees/emp
	let $req := doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id]
	return count($req)
};

for $emp in doc("emp.xml")/employees/emp
let $req := doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id]
where count($req) =  max(local:count-of-requests())
return data($emp/name)

Kelly Firkins

for $emp in doc("emp.xml")/employees/emp
let $req := doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id]
return
	<requestsByEmp>
		<emp name="{$emp/name}">
		{
			$req
		}
		</emp>
	</requestsByEmp>

<?xml version="1.0" encoding="UTF-8"?>
<requestsByEmp>
   <emp name="Kelly Firkins">
      <request id="T3" empID_="1" startDate="2003-01-15" 
                  endDate="2003-01-31" hours="80" status="declined" 
                  absenceCode="vacation"/>
      <request id="T4" empID_="1" startDate="2003-02-01" 
                  endDate="2003-02-01" hours="8" status="approved" 
                  absenceCode="sick"/>
      <request id="T6" empID_="1" startDate="2003-03-01" 
                  endDate="2003-03-05" hours="30" status="approved" 
                  absenceCode="vacation"/>
   </emp>
</requestsByEmp>

<overlappingReqs>
{
	for $req1 in doc("timeoff.xml")/timeoff/request

	let $req2 := doc("timeoff.xml")/timeoff/request[@id != $req1/@id 
                and ((xs:date($req1/@startDate) >= xs:date(@startDate) 
                and xs:date($req1/@startDate) <= xs:date(@endDate)) 
                or ((xs:date($req1/@endDate) >= xs:date(@startDate) 
                and xs:date($req1/@endDate) <= xs:date(@endDate))))]
	where count($req2) > 0
	return $req1
}
</overlappingReqs>

<?xml version="1.0" encoding="UTF-8"?>
<overlappingReqs>
   <request id="T1" empID_="2" startDate="2003-01-02" 
               endDate="2003-01-10" hours="40" status="approved" 
               absenceCode="vacation"/>
   <request id="T2" empID_="4" startDate="2003-01-10" 
               endDate="2003-02-10" hours="180" status="approved
               " absenceCode="vacation"/>
   <request id="T3" empID_="1" startDate="2003-01-15" 
               endDate="2003-01-31" hours="80" status="declined" 
               absenceCode="vacation"/>
   <request id="T4" empID_="1" startDate="2003-02-01" 
               endDate="2003-02-01" hours="8" status="approved" 
               absenceCode="sick"/>
</overlappingReqs>

for $req in doc("timeoff.xml")/timeoff/request[
	@status= "approved" and xs:date(@startDate)
	>= 
	xs:date("2003-01-02")  and
	xs:date(@endDate) < = xs:date("2003-01-10") ]

let $emp := doc("emp.xml")/employees/emp[@id = $req/@empID_]

return 
	element onVacation
	{
		attribute employee {$emp/name},
		attribute from {$req/@startDate},
		attribute to {$req/@endDate}
	}

<?xml version="1.0" encoding="UTF-8"?>
<onVacation employee="Dean Parker" from="2003-01-02" to="2003-01-10"/>

for $req in doc("timeoff.xml")/timeoff/request[
	@status="approved" and
	xs:date(@startDate) <= current-date()
	and xs:date(@endDate) >= current-date() ]

let $emp := doc("emp.xml")/employees/emp[@id = $req/@empID_]

return 
	element onVacationToday
	{
		attribute employee {$emp/name}
	}

<?xml version="1.0" encoding="UTF-8"?>
<onVacationToday employee="Katie Parise"/>

for $req in doc("timeoff.xml")/timeoff/request[@status="approved"]
return 
	<request>
	{
		$req/@id
	}
	{
		(:  Check if the request starts before or on 15th :)
	
		if(get-day-from-date(xs:date($req/@startDate)) <= 15) then
			"TimeOff starts in first 15 days of month."
		else "TimeOff starts in second half of the month."
	}
	</request>

<?xml version="1.0" encoding="UTF-8"?>
<request id="T1">TimeOff starts in first 15 days of month.</request>
<?xml version="1.0" encoding="UTF-8"?>
<request id="T2">TimeOff starts in first 15 days of month.</request>

The above examples hopefully gave you some idea on how XQuery expressions look like and what they can be used for. Let's now have a closer look at some of the core XQuery concepts and terminology.

XQuery processes XML documents as a logical tree of nodes, such as elements, attributes, and so on. This logical structure of XML document on which XQuery (and XPath 2.0) is based is termed as Data Model. In addition to nodes, the data model may also consist of XSD simple types, such as strings, booleans, integers, dates, and so on. These XSD 'simple types' single values are termed as Atomic Values. A single node or an atomic value is known as an Item, and series of such items is known as Sequence.

XQuery expression evaluation always results in a sequence. In other words, the result of XQuery expression is zero or more nodes or atomic values. Single item is essentially a sequence with length one. These are termed as Singleton Sequences. XQuery does not permit nesting of sequences. That means, you can not have a sequence within a sequence. An empty sequence does not contain any item and is represented as ().

• Sequence containing three atomic values: (10, "Darshan", 30)
• Sequence containing one node and one atomic value: (<Person />, 1)
• Sequence containing 21 atomic values (10 through 30), a node, and an atomic value: (10 to 30, <Person />, 20)
• Singleton Sequence: (205)

Node can be one of seven pre-defined categories: document, element, attribute, text, namespace, processing instruction, or comment. As per the XQuery data model, each node in the tree has unique identity.

The other important term you should be familiar is Document Order. As mentioned earlier, XQuery data model is defined in terms of abstract, logical tree of nodes. The in-order, depth-first traversal of such tree results in nodes in Document Order. In general, XQuery expressions result in sequences based on document order. The only way to change this ordering is to use the order by clause inside a FLWOR expression. There are many XQuery concepts that are based on document order. For example, many axis steps (such as preceding, following, following-sibling, etc.) return results based on document order, the << and >> sequence operators (discussed later) are based on document order, and so on.

In the above 15 examples, you have already seen one of the ways XQuery input source is defined - via the doc function. The other input source function is the collection function. Note that all these built-in XQuery 1.0/XPath 2.0 functions are defined in http://www.w3.org/2003/11/xpath-functions namespace, which by default has the prefix of fn:. That means you could call doc() function as fn:doc(), collection function as fn:collection, without specifically declaring the namespace prefix. The other predefined namespaces prefixes include:

• xs (http://www.w3.org/2001/XMLSchema): XML Schema
• xsi (http://www.w3.org/2001/XMLSchema-instance): XML Schema Instance
• xdt (http://www.w3.org/2003/11/xpath-datatypes): XPath data types
• local (http://www.w3.org/2003/11/xquery-local-functions): Local functions
• xml (http://www.w3.org/XML/1998/namespace) - XML

Note that the earlier drafts of the XQuery specification had third option for providing query input source: the input() function. This function allowed binding to a sequence of nodes provided by implementation-specific source. This function no longer exists in the current draft. It has been removed, and you should now use the doc() function with specific URI.

As mentioned before, Expressions are the XQuery building blocks. Each query is an expression or nested expressions composed with full generality. XQuery specification defines following categorization of expressions:

for $var in (10, 20, 30, 40, 50)
return $var

for $var in (10, 20, 30, 40, 50)
let $var := ('x', 'y', 'z')
return $var

(: This is an example of (: nested :) comments :)

("Darshan", "Singh", "PerfectXML", "YukonXML")      (: sequence containing 4 atomic values :)
(<agent id="1" score="94"/>, <agent id="2" score="97"/>)    (: sequence containing 2 nodes :)

(20 to 100)		(: integers 20 through 100 :)

(98 to 90)
("a" to "z")

fn:boolean(()), 
fn:boolean(""), 
fn:boolean(0),
fn:boolean(xs:double("NaN")),
fn:boolean(0),
1 and 0,
"" or fn:boolean(())
(: ,("") cast as xs:boolean :)

In addition to querying XML data, XQuery allows create XML nodes. There are two ways in which you can generate XML using XQuery:

• Direct: Using this approach, you write XML directly, and use curly braces { } to enclose expressions that need to be evaluated. See examples 9 and 15 above.
• Computed: With this approach, you use keywords such as, element, attribute, document, text, processing-instruction, comment, or namespace to create the respective node. See examples 13 and 14 above.

You can write declare xmlspace in the prolog section (discussed later) in the query to indicate if you wish to preserve or strip the whitespace while using the Constructors.

FLWOR (for-let-where-order by-return), pronounced "flower", is the core XQuery expression that allows looping, variable binding, sorting, filtering and returning the results.

The for and let clauses allow binding variables to expressions. The for clause evaluates the bound expression and iterates for each item in the evaluated sequence (except those which were filtered out because of where clause). The let clause on the other hand binds a variable to the entire result of an expression at once (without iteration).

let $var1 := (1, 2, 3)
return <ints>{$var1}</ints>

<ints>1 2 3</ints>

for $var1 in (1, 2, 3)
return <ints>{$var1}</ints>

<ints>1</ints>
<ints>2</ints>
<ints>3</ints>

for $var1 in (1, 2, 3), $var2 in (10, 20, 30)
return <ints>{$var1, $var2}</ints>

<ints>1 10</ints>
<ints>1 20</ints>
<ints>1 30</ints>
<ints>2 10</ints>
<ints>2 20</ints>
<ints>2 30</ints>
<ints>3 10</ints>
<ints>3 20</ints>
<ints>3 30</ints>

The for clause supports the notion of Positional Variable using the at clause. Positional variable is always of type xs:integer and as the bound variable iterates over items in the sequence, the value of the positional variable is incremented by 1 (like a counter), it starts with 1.

for $var1 at $count1 in (1, 2, 3), 
    $var2 at $count2 in (10, 20, 30)
return <ints c1="{$count1}" c2="{$count2}">{$var1, $var2}</ints>

See examples 7 through 15 above for FLWOR expression samples.

If the order of results is not important, you can specifically call fn:unordered function, and possibly get some performance benefits and the XQuery implementation might do some optimizations in such cases.

Try out following example:

fn:unordered
(
<MoreThanTwo>
{
	for $emp in doc("emp.xml")/employees/emp
	let $req := doc("timeoff.xml")/timeoff/request[@empID_ = $emp/@id]
	where count($req) > 2
	return
		$emp
}
</MoreThanTwo>
)

<?xml version="1.0" encoding="UTF-8"?>
<MoreThanTwo>
   <emp id="1">
      <name>Kelly Firkins</name>
      <email>kelly</email>
      <ssn>111-11-1111</ssn>
      <managedEmail>dean</managedEmail>
      <dateOfJoining>1994-11-16</dateOfJoining>
   </emp>
</MoreThanTwo>

You can use if, then, else keywords in your query and have expressions evaluated conditionally. The final example in the above 15 examples illustrates the conditional expressions.

Note that the "else" expression is mandatory in the if..then..else expression. If you do not really want to return anything in the else block, just return a pair of parentheses as an empty sequence.

If you need to find out if any (existential quantification) or every (universal quantification) item in a sequence satisfies a condition, you can use Quantified Expressions, some or every, in such scenarios.

<Report>
{
	for $t in doc("timeoff.xml")/timeoff
	where some $req in $t/request
		satisfies ($req/@hours <= 4)
	return
		"There are some half-day requests."
}

{
	for $t in doc("timeoff.xml")/timeoff
	where every $req in $t/request
		satisfies ($req/@hours >= 8)
	return
		"All requests are atleast for a day."
}
</Report>

Note that the above results can be obtained by many other ways in which query can be written; the above is just an example to illustrate quantified expressions.

Let's first understand what SequenceType really means. As mentioned before, XQuery is a strongly typed language. It supports some built-in types (element node, attribute node, etc.), XSD basic types (xs:integer, xs:string, etc.), and types from imported XSD schema (how to import XSD is discussed later). Values in XQuery are generally sequences of such types. This is given a generic name SquenceType. In other words, any value in XQuery can be said to be of sequence type.

Following tables lists some of the examples of sequence type indicators:

XQuery allows you to check if a value is of some specified sequence type, or do various things based on the type of the value, or cast the value from one type to the other. These facilities are provided via instance of boolean operator, typeswitch, cast, castable, and treat expressions.

Refer to XQuery specification SequenceType Syntax and Expressions on Sequence Types sections for complete details on this.

XQuery allows importing XSD schema and validating elements (using the validate expression) against that schema. Refer to XQuery specification for more details on this. As per the SAXON 7.8 XQuery Conformance notes, it does not support the validate expression.

The XQuery 1.0 and XPath 2.0 Functions and Operators document is a result of joint efforts of XSL and XQuery working groups. This document is a catalog of built-in functions and operators that are available via XPath 2.0, XQuery 1.0, and XSLT 2.0. For a complete listing of all these functions and operators, refer to the above link. In this article, I am listing few functions that I think will be used commonly. The description for the functions is taken mostly from the W3C Specification document.

To create easy-to-use re-usable queries or to encapsulate complex queries, you can create you own functions. You can declare your own function by providing the name of the function, parameters, and the data type of the result. These functions are termed as user-defined functions.

In addition to this, if you want to do something outside the query environment, and if the XQuery implementation you are using supports it, there is a concept of external functions.

The function declaration tells if the function is local user-defined or is an external function. XQuery allows recursive (function calling itself) and mutually recursive (two functions calling each other) functions. However, the current version does not allow function overloading for user-defined functions.

See example 10 above for an example of writing and calling a user-defined function.

XQuery allows putting functions in library modules, so that they can be shared and imported by any query. XQuery defines module as a "fragment of XQuery that can independently undergo the analysis phase". So far we have seen just the query body in the XQuery examples, there is more to it, and we'll study that in the next section.

In real life, an XQuery query will be quite complex than what we have seen in above 15 examples. It would be a good design to modularize to re-use and maintenance. An XQuery then would be composed of one main module and zero or more library modules.

All the above examples just contained the query body. In addition to this, an XQuery module can include things like module declaration and prolog

If a module includes prolog followed by query body, it is said to be main module. A query can have exactly one main module.

If a module includes module declaration followed by a prolog, it is called a library module. A library module can not be executed independently. It is created to provide services (in the form of functions and variable declarations) to the module that is including it. It is not allowed to have a module containing both, a module declaration and a query body.

I have used words and phrases such as prolog and module declaration without defining them. Let's look at what I mean by these terms.

Prolog    ::=    
  ((NamespaceDecl
    XMLSpaceDecl
    DefaultNamespaceDecl
    DefaultCollationDecl
    BaseURIDecl
    SchemaImport
    ModuleImport
    VarDecl
    ValidationDecl
    FunctionDecl) Separator)*
    
    Separator    ::=    ";"

The things that can go as part of prolog include:

• Namespace declaration: Defines the namespace prefix to namespace URI association.
• Xmlspace declaration: Controls the whitespace handling
• Default Namespace declaration: Defines namespace for unprefixed elements or functions.
• Default Collation declaration: Defines name of the collation to be used by functions (such as fn:compare) and operators (such as gt).
• Base URI declaration: Defines base URI property used for resolving relative URIs within a module.
• Schema Import: Imports the element, attribute, and type definitions from a named schema.
• Module Import: Imports the function declarations and variable declarations from the Prolog of a library module.
• Variable declaration: Declares a typed variable, and optionally assigns a value to it.
• Validation declaration: Establishes a default validation mode for the query.
• Function Declaration: Declare user-defined or external functions.

Library modules begin by a Module declaration. Followed by the module declaration, a library module consists of Prolog. The module declaration establishes an identity for the module, and it consists of the words module and namespace, followed by namespace prefix and a URI as a string. Library modules may import other library modules using the Module Import in its prolog section.

Refer to XQuery W3C document and/or resources at the bottom of this article for more information on prolog, and creating and using library modules.

Before I conclude this article, I would like to list some bullet points that compare and contrast XQuery with XSLT.

• XSLT is primarily a language for describing XML transformation; XQuery is primarily a language to query XML data and documents.
• XSLT uses XML-based syntax; XQuery 1.0 does not use XML-based syntax.
• XPath is at the core for both, XSLT and XQuery.
• XSLT 1.0 turned W3C recommendation on November 16, 1999. XQuery 1.0 (as of Jan 28, 2004) is in Last Call Working Draft status. Many tools, APIs, and vendors have excellent support for XSLT. XQuery support is introduced by many vendors/toolkits; it is been rapidly improved and made complete.
• XSLT and XQuery, both, provide means to construct new nodes.
• XSLT being a rule-based language scores higher when it comes to handling unpredictable structures.
• XQuery 1.0 has a concept of user-defined functions, which can be modeled in XSLT 1.0 as named templates. XSLT 2.0 matches the XQuery 1.0 functionality by allowing writing and calling user-defined functions.
• XQuery 1.0 is strongly typed language, XSLT 1.0 is not.
• XQuery provides FLWOR expression for looping, sorting, filtering; XSLT 1.0's xsl:for-each instruction (and XSLT 2.0's for expression) allows to do the same.
• XQuery does not support all the XPath axes; XSLT does.

If your data is natively stored as XML or you are working with virtual XML views or XML documents, and if you need to query them, you can use XQuery as the flexible and standard solution and write XQuery expressions that loop, filter, sort, generate new nodes, and so on, to produce the results that you want. Hopefully, this article gave you some idea of as to what XQuery is, how the XQuery expressions look like, what various XQuery-related terms mean, and how it fits with other XML standards such as XPath and XSLT.

Following is a list of some XQuery tools/products/implementations that you might find interesting:

• SAXON (http://saxon.sourceforge.net/)
• X-Hive/DB (http://www.x-hive.com)
• XQuantum™ XML Database Server (http://www.cogneticsystems.com/)
• BumbleBee (http://www.xquery.com/bumblebee/)
• XQEngine (http://xqengine.sourceforge.net/)
• XStreamDB™ (http://www.bluestream.com)
• JSR-000225 XQuery API for JavaTM (XQJ) (http://jcp.org/aboutJava/communityprocess/edr/jsr225/index.html)
• DataDirect XQuery™ (http://www.datadirect.com/techzone/xml/topics/xquerysupport/index.ssp

• XQuery Tutorial
• Mark Logic xq:zone: where xquery gets to work
• http://www.xquery.com
• XQuery: A Guided Tour - Chapter 1 from the book XQuery from the Experts
• Liquid Data: XQuery-Based Enterprise Information Integration

If you are interested in learning more about XQuery and/or using it in your projects, I would recommend the book XQuery from the Experts. Write some more queries and try out using SAXON. See how XQuery is implemented in Yukon. Go through XQuery help sections in Yukon Books Online.