This was pretty much the vision I was following when I jumped on the XML database bandwagon in 1999. Look at some of the stuff at http://www.softwareag.com/xml/library/default.htmthat I contributed to in that timeframe. Was I a visionary who just got the timeframe wrong, or are we both more fundamentally wrong about the market appeal of XML/XQuery for mainstream apps?
I'm afraid I lean toward the latter point of view. You've offered some compelling reasons for why XQuery offers "power, sophistication, and ease of use" for those who are willing to start over with an XML database and XQuery. The trouble is, very few people in the real world have that luxury. Furthermore, for the reasons Joel Spolsky notes in the famous Things You Should Never Do post http://www.joelonsoftware.com/articles/fog0000000069.html , people who do have the luxury of starting from scratch generally fail horribly.
The power of ASP and Rails (not to mention LINQ) is that they support a more incremental approach -- Leverage those SQL databases that everyone has and everyone knows how to use at at least a basic level, and easily generate web front ends without having to learn much about a radically different approach. They allow XML as an interchange format, but don't force an entire app to be reconceptualized in terms of an XML data model. XSLT or other declarative / transformational approaches can be used to reshape data or generate HTML, but the mainstream approaches let developers use good ol' imperative code to do the nasty bit twiddling that is always needed in real apps. XQuery (at least in the current standard) doesn't offer that option if it is at the center rather than the periphery.
So, older and wiser than in 1999, I see XQuery as a great *query* language for XML stores, but I'm highly skeptical that it is the next big server-side programming language.
I don't think that what Mark Birbeck (and increasingly myself) has been calling xH - XQuery/XSLT/XHTML+XForms, etc - is going to seriously challenge ASP.NET or PHP for supremacy any time soon, but that's due at least as much to the fact that most server-side languages (Ruby not so much) are imperative rather than declarative, and as such are closer to the C++/Java/C# paradigm that many developers are used to.
Frankly, that's fine. However, one of the key aspects that continues to confound developers is the realistic workings of data binding. Microsoft's approach is to bind at the control level, and to work with a wide variety of data formats to do so. The xH model, on the other hand, assumes an extant and independent business data layer of which the component is only a reflection.
XML is increasingly becoming the lifeblood of most enterprises, and this is one area where I see the xH technologies gaining significantly over other approaches, especially when combined with a cogent schema mapping technology. XML databases play a big part in that.
It's a solution, one that may not necessarily be perfect for every occasion or developer, but invaluable for those developers who are already halfway into the XML space themselves.
Okay, but seriously most of the various front end server side languages are not too important in the scheme of things. The backing languages and how much access they have to libraries etc. are the important things.
The use of SQL is perhaps not that important either, every SQL database I have to use has ways of getting XML out of them now. At any rate the Xquery as server side language is fine if extended to go against non-xml databases.
Hmm, how about support of OLAP mixed with Xquery? Mondrian and Exist together?