When to Use Get?
April 24, 2002
When To Use Get?
Dipping into the W3C TAG discussions once more, the XML-Deviant reports on the ongoing debate over the correct way to use the HTTP GET method.
The Issue With GET
In his recent XML-Deviant column ("TAG Watch") Kendall Clark examined the current issue list of the W3C TAG, which includes " whenToUseGet-7", the resolution of which will help clarify when to use HTTP GET rather than POST. According to the HTTP specification GET is said to have two characteristics: it is safe and idempotent. POST, however, is neither idempotent or safe.
A digression into the subtleties of defining these terms could fill a column in itself and has certainly been the topic of a number of mailing list discussions. Safe, for example, should be considered in terms of user accountability rather than side-effects. Retrieving a web page using GET might cause a hit counter to be updated -- clearly a side-effect -- but because the user did not specifically request this change they cannot be held accountable for it's occurrence. An idempotent operation is one that can be carried out repeatedly with the same results. This might lead one to believe that retrieving a dynamic web page is not an idempotent operation -- the page may alter at each request -- yet the page is consistently returning the current state of the requested resource so can be said to have the same result.
In resolving this issue the TAG aims to lend its support to the HTTP 1.1 specification, thereby ensuring that web developers are correctly working within the constraints of the Web's architecture. Dan Connolly is currently assigned the job of writing up the TAG recommendations, a draft of which, when circulated publicly this week, sparked another debate.
Emotive Reactions
The disagreement has come from within the TAG itself. David Orchard posted a lengthy response to the findings. Orchard was concerned with how these impact the work of the various Web Services working groups; specifically, that the TAG's findings might imply that using SOAP with HTTP POST is in violation of the Web's architecture. Regular XML-Deviant readers will recall that the correct use of the HTTP protocol has been the subject of several recent "REST vs SOAP/RPC" debates, e.g. see "REST and the Real World". Indeed the issue has been debated, in various guises for some time.
In his "call to arms", Orchard explained that he considered it
inappropriate to create issue resolution wording that could result in suggestions that safe web services must use a new HTTP GET binding, that the XMLP/WSDL/Web Services Architecture Groups should change probably significant sections of their wording, to delay publication of SOAP 1.2 (in order to do more work on a GET binding), or other charter/wording/schedule changes.
Orchard believed that liaising between the various W3C working groups was a better solution than the TAG simply issuing an edict that might cause other groups to have to revisit existing work. Orchard commented that "some members of the TAG seemed quite interested in changing the web services architecture instead of trying to documenting and achieve consensus on this issue." Orchard concluded his posting by expressing his disappointment that the issue was even under debate.
I find it disappointing that we are debating this issue. I would rather being doing what I consider more productive forward-looking work, such as web service security use-cases/requirements/charter or closing SOAP 1.2 issues, than debating a technical decision that has shipped in many thousands of units and does not have any consensus in the real world. But there are such extremes of opinion and desires for "correcting behavior" rather than documenting consensus that I and others will be forced to spend time on this issue.
These comments will definitely raise a few eyebrows. Offering an installed base as reason not to make changes to draft W3C specifications certainly runs counter to standard W3C disclaimers that such documents "may be updated, replaced, or obsoleted by other documents at any time." Granted, SOAP was in existence before it was submitted to the W3C, but that should have only limited influence over decisions. Orchard's dissent is also the first sign that individual Working Groups might publicly take exception to TAG recommendations.
Mark Baker, an advocate of the REST approach, strongly disagreed with Orchard, in particular his assertion that the machine-to-machine communications inherent in web services are a new problem whose solution isn't within the scope of the REST architecture. Baker maintained that the suggestion that REST is unsuitable for such applications "could not be further from the truth". Baker's later argument that Web architecture "is not a matter for consensus gathering" will certainly find agreement among those developers who already frustrated with the design by committee approach.
At the end of his equally lengthy rebuttal, Baker explained that he
would be absolutely shocked, disappointed, and upset if the TAG were not able to agree on one of the single most important architectural principles on the Web. As an AC rep, I would actively pursue having the TAG disbanded if consensus could not be reached on such a fundamental issue; I would rather have no TAG, than a TAG that could not represent Web architecture within and outside the W3C, as the latter could do more damage by silencing the voices of the architects of the Web on the TAG: Tim Berners-Lee, Dan Connolly, and Roy Fielding.
Also contributing to the discussion Fielding explained that the issue over correct use of HTTP effects not only SOAP but also many CGI applications:
The fact of the matter is that most CGI scripts are not HTTP compliant... The point is to describe to developers the ways in which an interface can be better implemented on the Web. REST is not the easiest way to provide an interface on the Web. In fact, doing it right requires fairly sophisticated software. CGI makes life easy for programmers. REST-like operation makes life easy for users and content maintainers.
However Fielding also noted that "in order for SOAP-ng to succeed as a Web protocol, it needs to start behaving like it is part of the Web."
Caching and Linking
Tim Bray was also in favor of the TAG recommendations to use GET, and he cited his own experience in which ignoring these principles incurred two significant costs:
- many useful implementation tricks in the areas of caching, proxying, and so on, are ruled out because I didn't use GET
- I can't bookmark the operation or put it in a web page without resorting to a form.
Bray argued that SOAP-based web services will also have to suffer these consequences as a result of binding to HTTP POST rather than GET.
Mark Nottingham countered Bray's first point about caching by citing difficulties in picking out efficient cache keys for RESTful requests.
SOAP's extensibility (Header) and targeting (actor) mechanisms, along with a well-defined processing model are a godsend, compared to the heuristics, SHOULDs and implementation nightmares of HTTP caching. There are some cases where it would be possible to use unaugmented HTTP caching with a Web Service, but they are vastly in the minority, and are IMO relatively uninteresting.
Caching problems were also seen as a non-issue by Don Box, who believed that such difficulties would be solved by SOAP-aware infrastructure:
I'm sorry, but if I must chose between expressing most of my app in terms of HTTP GET vs. having the switch/router/gateway/cache fabric get smarter, I'll take the latter any day.
I think SOAP-focused network infrastructure is inevitable and frankly a good thing.
Not all of the REST supporters saw caching as a major factor either. Paul Prescod considered it to be "one of the least interesting of the advantages of HTTP GET" and also argued that bookmarking is a limited use case of linking in general:
I'm talking about XInclude, XLink, XPath, RSS, XHTML, XSLT, RDF, weblogs and Topic Maps. These are *powerful* information combination and relation tools. But they cannot be used with information hidden behind SOAP endpoints. I would say that any service that denies its users the use of these tools is somewhat broken.
|
|
|
Also in XML-Deviant |
|
Having highlighted two benefits of conformance to web architecture could bring, the TAG debate has since moved on to suggestions about how the issue might be put to bed. David Orchard seems to be the lone voice on the TAG protesting the findings, but there is obviously a strong swell of support from the web services community. Liaisons between the groups will certainly be a contributing factor toward finding a resolution, while at a technical level a range of options -- from a HTTP GET binding for SOAP to revisiting the entire SOAP specification -- have been suggested.
Looking Backward and Forward
It's interesting to consider how we've ended up in this situation, and the obstacles to be faced in moving forward.
During the www-tag discussion there were some comments made about the seemingly widespread misuse of the HTTP protocol. Keith Moore suggested that RPC is both easier to understand and is the most familiar to developers; similar arguments have been made in several recent debates. While Scott Cantor observed that "most of these developers don't know they're using designs based on REST, let alone what REST means." This seems reasonable and is no doubt a problem because the Web's architecture has been left largely undefined for so long.
A few draft documents on the W3C site don't constitute a formal statement of these principles. If they did then the TAG wouldn't be investing so much energy in producing a fuller articulation. While the REST architectural principles may have been known to the cognoscenti for some time, the clearest statement of these objectives outside the relevant specifications is Roy Fielding's oft-quoted dissertation, which, while highly readable, isn't really designed as a quick reference for web application development. So from this perspective the W3C has only itself to blame.
Implementation issues with browsers and intermediaries (proxies, caches, etc) are the other source of problems. The most commonly cited example is restrictions on the length of URLs. The suggestion is that some misuses of GET and POST have been workarounds for these problems, enabling web applications to work as intended. If true, then this would mean that browsers have not only been instrumental in causing a large amount of bad HTML because of permissive parsing policies but could also have undermined the foundations of the Web's architecture.
Nobody expects a sudden revolution in the way that web applications are designed. Even the TAG members seem certain that applications will continue to work against the architectural principles. They just want to make sure that developers are clear about the real costs of doing so. Leading by example seems to be the only pragmatic option, which further underlines the need for a resolution to the issue of the Web Services issues.
The prospect of resistance to change from the various Web Service vendors among the W3C membership will be the immediate obstacle to surmount. A far larger one will be convincing developers with hard-won experience that they need to see the error of their ways and clean up their act. And that's an unenviable task, given that "conformance to the Web's architecture" rarely outranks "make it work" on anyone's priority list. But it is one that in many people's opinion is required if the W3C, as it likes to boast, is really going to "lead the Web to it's full potential".