XML.com: XQuery, the Server Language

XQuery, the Server Language
by Kurt Cagle | Pages: 1, 2, 3

The eXist database defines a number of these namespaces out of the box. From the standpoint of servlet development, perhaps the most important namespaces are as follows:

request: provides access to information sent from the client. Functions include get-cookie-names, get-cookie-value, get-data, get-header, get-header-names, get-method, get-parameter, get-parameter-names, get-server-name, get-uploaded-file, get-uploaded-file-name, and get-url.
response: lets the developer control the stream of data being sent back to the client. Functions include redirect-to, set-cookie, set-header, and stream-binary.
session: provides control over the user's HTTP sesssion. Functions include create, encode-url, get-attribute, get-attribute-names, get-id, invalidate, set-attribute, and set-current-user.
transform: lets the developer transform an XML node using XSLT from within the xquery. Functions include transform and stream-transform.
update: The update commands (distinct from a namespace) let you perform live updates of the data in the eXist XML database, either at the granular level of changing a value in the database or at the level of inserting or removing whole documents. This addresses one of the big shortcomings of XQuery, in that it provides for an effective read-write solution that can be invoked from within an XQuery.

Other extensions can be compiled in by rebuilding the Java JAR (a shell or batch script automates this process) for doing such things as writing SQL queries and updates designed to work with any JDBC compliant SQL database, such as Oracle, mySQL, Postgres, or SQL Server. This capability is especially important because it provides a bridge between the SQL and XML worlds, letting you perform complex queries (or updates) on your SQL database then passing this information to the XQuery to be additionally processed, filtered, sorted, or transformed.

Additionally, other extensions give access to a full range of math functions (including the oh-so-useful math:random function), let you send mail through an SMTP server, retrieve (and to a certain extent modify) images (which can also be stored in the database, by the way), and other functions that provide functionality more associated with a full bore server-side scripting language than an XML query language.

This article serves as a very basic introduction to XQuery as a server language. I will be addressing this topic in more detail in subsequent articles in this series, examining some of the more sophisticated capabilities and the gotchas inherent in working with XQuery and eXist, and showing what explosive power you can release when you combine eXist or other rest based XQuery engines with XForms and Ajax.

My prediction is that REST based XML databases like eXist will seriously challenge the existing raft of server languages, from ASP to Ruby, within the next couple of years. Right now, it's something of a closed secret among a few developers, but the power, sophistication and ease of use inherent in working with the XML as if it were a natural part of the server landscape can only be understood by trying it.

Share your experience in our forums.
(* You must be a member of XML.com to use this feature.)
Comment on this Article

Titles Only

Newest First

Why not an XML notation ?
2007-06-09 01:39:23 Alain_COUTHURES [Reply]

I still think there should be an XML notation for XQuery.

Unlike XPath, there are XML fragments embedded within XQuery programs and it would be nice to use any XML editor or any other XML tool to manipulate XQuery programs.

It would also be more easy to generate XQuery programs from XSL transformations.

Alain COUTHURES
http://www.agencexml.com
- Why not an XML notation ?
  2007-06-13 23:49:34 JorisGraaumans [Reply]
  
  Actually, there is one: http://www.w3.org/TR/xqueryx/
  - Why not an XML notation ?
    2007-06-16 01:14:16 Alain_COUTHURES [Reply]
    
    Thanks for this information.
    
    The transformation from XQueryX to XQuery is a big one and, from my point of view, XQueryX documents are quite hard to read for any standard human. It appears XQueryX has been designed to generate XQuery from it without any difficulties...
    
    Anyway, it's good to find a W3C Recommendation !

Thanks
2007-06-07 23:44:56 Lars Huttar [Reply]

Thanks for a helpful article. I'd love to see people start using Cocoon+eXist+XQuery to the point where you don't have to do a lot of hunting to figure out how to set it up.

Minor point of clarification... you write, "XQuery adds a number of features--command and control structures (such as for expressions), the ability to create intermediate date variables (the let keyword), conditional handling (if/then/else), and the like to the XPath 2.0 language."

I thought XPath 2.0 already had if/then/else? Oh, I guess you're saying XQuery adds them outside of XPath expressions. Fair enough.

Thanks again for continuing to help us get a leg up on XQuery.

Lars
- Thanks
  2007-06-08 01:14:09 Kurt Cagle [Reply]
  
  Lars,
  
  You're right - the if/then/else (and for each) structures are part of XPath 2, which is incorporated into XQuery. Since most people are not familiar with the new XPath 2 recommendation, however, the statement seemed reasonable from an explanatory standpoint even if it wasn't strictly correct.

thanks Kurt...
2007-06-07 20:44:25 itod47 [Reply]

...for doing this article. I'm also very excited about XQuery in this capacity. The potential for simplification is massive!

No middle tier, no objects, no sql, no orm. No mismatch between the data in the database, the markup on the page, and the objects required to convert between the two.

Rather, XML documents in the database converted to XHTML documents for serving via a highly customized tool called XQuery. It's data all the way down, with no clumsy, heavy stages in between.

XQuery on the server won't be for everyone... lots of people will be too used to existing practice to try it out, even on new development...

I think the commenter who complains about 'starting over from scratch' is really missing the point... this isn't about starting from scratch, it's about a whole new paradigm with lots of advantages over the old. You're truly starting out ahead.

What's missing now is the 'Rails' or 'Struts' or 'Django' of the XQuery world... a web framework that reduces drudgery and duplication of effort with reusable XQuery functions and modules. Once that is available, XQuery on the server should see significant adoption, as the benefits of a data-centric approach will be extremely attractive compared to object-centric approaches of old.

Security concerns
2007-06-07 08:02:16 john_judy [Reply]

What sort of input validation/clean-up functionality does this have? I assume the example in this article was just a simple demonstration, but it leaves a gaping hole for someone to include an XSS attack. Are there namespaces, etc. that can handle this sort of issue?
- Security concerns
  2007-06-08 01:11:52 Kurt Cagle [Reply]
  
  The point you raise is valid, though again you're also talking here in this brief article example code meant to showcase a principle, not protect against intrusion.
  
  In a more complete working implementation, security actually happens at a number of levels:
  
  1) Authentication - you can create and maintain authenticated sessions with something like eXist (and most XML databases that I'm aware of.
  2) XML validation - I've shown above the very simple example of query string parameters being passed, but realistically, XQuery really begins to come into its own once you start combining it with an XML generator (such as XForms) on the client. As you can validate such content, this holds a great deal of promise in reducing or even eliminating many types of client/server loopholes.
  3) Modularization (Web Services) - simply put, you declare XQuery functions and do your processing there, possibly with proxies to process incoming requests and filter out potentially bad ones. I'll get into that in a subsequent article.
  
  There are other techniques you can use too, but in general, most of the assumptions about security with SQL databases also carry over into XML ones.

XQuery
2007-06-06 14:48:09 jkreisa [Reply]

Kurt,

We completely agree with your assesment of XQuery and the potential future for the language.
Back when we initially conceived of our product, MarkLogic Server, we selected XQuery as the fundamental access technology and all interaction with our XML Content Server is via XQuery. In fact we also extended XQuery in a couple of key areas including adding full text search and transactional update support.

You mentioned in your article that you will be continuing your evaluation of XQuery. I encourage you to download a free copy of MarkLogic Server as part of your evaluation. You can grab it from our developer web site,
http://developer.marklogic.com/

Or if you would like to chat more about it I would love to have a conversation on the topic.

Please feel free to contact me at john.kreisa@marklogic.com

Regards,

John
- MarkLogic
  2007-06-08 01:17:42 Kurt Cagle [Reply]
  
  Hi, John!
  
  I'll take you up on that offer. MarkLogic is a solid and professional XML database, and I don't want to belittle it even inadvertently. Doing a feature on MarkLogic and other XML databases in the near future is looking more and more like something I need to schedule.
  
  -- Kurt

I predicted that too ... 8 years ago
2007-06-06 14:43:26 mchampion [Reply]

This was pretty much the vision I was following when I jumped on the XML database bandwagon in 1999. Look at some of the stuff at http://www.softwareag.com/xml/library/default.htmthat I contributed to in that timeframe. Was I a visionary who just got the timeframe wrong, or are we both more fundamentally wrong about the market appeal of XML/XQuery for mainstream apps?

I'm afraid I lean toward the latter point of view. You've offered some compelling reasons for why XQuery offers "power, sophistication, and ease of use" for those who are willing to start over with an XML database and XQuery. The trouble is, very few people in the real world have that luxury. Furthermore, for the reasons Joel Spolsky notes in the famous Things You Should Never Do post http://www.joelonsoftware.com/articles/fog0000000069.html , people who do have the luxury of starting from scratch generally fail horribly.

The power of ASP and Rails (not to mention LINQ) is that they support a more incremental approach -- Leverage those SQL databases that everyone has and everyone knows how to use at at least a basic level, and easily generate web front ends without having to learn much about a radically different approach. They allow XML as an interchange format, but don't force an entire app to be reconceptualized in terms of an XML data model. XSLT or other declarative / transformational approaches can be used to reshape data or generate HTML, but the mainstream approaches let developers use good ol' imperative code to do the nasty bit twiddling that is always needed in real apps. XQuery (at least in the current standard) doesn't offer that option if it is at the center rather than the periphery.

So, older and wiser than in 1999, I see XQuery as a great *query* language for XML stores, but I'm highly skeptical that it is the next big server-side programming language.
- I predicted that too ... 8 years ago
  2007-06-08 01:28:17 Kurt Cagle [Reply]
  
  Mike,
  
  I don't think that what Mark Birbeck (and increasingly myself) has been calling xH - XQuery/XSLT/XHTML+XForms, etc - is going to seriously challenge ASP.NET or PHP for supremacy any time soon, but that's due at least as much to the fact that most server-side languages (Ruby not so much) are imperative rather than declarative, and as such are closer to the C++/Java/C# paradigm that many developers are used to.
  
  Frankly, that's fine. However, one of the key aspects that continues to confound developers is the realistic workings of data binding. Microsoft's approach is to bind at the control level, and to work with a wide variety of data formats to do so. The xH model, on the other hand, assumes an extant and independent business data layer of which the component is only a reflection.
  
  XML is increasingly becoming the lifeblood of most enterprises, and this is one area where I see the xH technologies gaining significantly over other approaches, especially when combined with a cogent schema mapping technology. XML databases play a big part in that.
  
  It's a solution, one that may not necessarily be perfect for every occasion or developer, but invaluable for those developers who are already halfway into the XML space themselves.
- I predicted that too ... 8 years ago
  2007-06-07 01:42:39 BryanRasmussen [Reply]
  
  I predict that four years from now. I win!!
  
  Okay, but seriously most of the various front end server side languages are not too important in the scheme of things. The backing languages and how much access they have to libraries etc. are the important things.
  
  The use of SQL is perhaps not that important either, every SQL database I have to use has ways of getting XML out of them now. At any rate the Xquery as server side language is fine if extended to go against non-xml databases.
  
  Hmm, how about support of OLAP mixed with Xquery? Mondrian and Exist together?