Menu

XMLPULL: A Response

September 25, 2002

Aleksander Slominski and Stefan Haustein

Editor's note: this article is a response from the creators of the XMLPULL API to the recent article published on XML.com, The XmlPull API, by Elliotte Rusty Harold.

Introduction

We are very happy to see that XML pull parsing is gradually gaining more attention. We are also happy to see an article about XmlPull API published on XML.com, as it will expose this approach to a wide audience of Java and XML developers.

The author of the article described some API design choices as severe design flaws. We would like to defend those choices and address the concerns raised in the article. The following is a list of concerns that the article pointed out:

  1. There is no implementation that supports validation.
  2. The API is not modeling XML correctly.
  3. The API is not sufficiently object oriented.

There are no implementations supporting validation

The first point, the lack of a validating parser, is simple to address. It was of course a significant disadvantage, and we resolved it by implementing a validating XmlPull parser based on Xerces 2. Before we go more into on that, we would like to address points 2 and 3.

XmlPull does not model XML correctly

The article said,

... With respect to XML, XMLPULL does not support namespaces by default and does not read or report well-formedness errors in the internal DTD subset. The namespace flaw can be fixed by setting the appropriate feature, and in theory the internal DTD subset problem can be as well. But the existing parsers don't support this. Furthermore, the defaults are exactly backwards from what they should be for both; and while there might rarely be justification for turning off namespace processing, turning off processing of the internal DTD subset is simply not allowed by the XML specification. A parser that does not read the internal DTD subset is not an XML parser...

The first complaint can be divided into the following arguments. First, the XmlPull API does not support namespaces by default. Support for documents without namespaces was added for situations where users want to read plain XML 1.0. The default value of "false" for the namespace option was just chosen for consistency with other features, which are also switched off by default. Thus, the user does not need to be aware of a lot of different default settings, only that all features are switched off by default. However, support for namespaces is required by the XmlPull API, therefore turning it on is always supported by XmlPull parser implementations.

Second, there's the matter of allowing implementations that do not read or report well-formedness errors in the internal DTD subset. It is true that such implementations are not fully XML-compliant. This compromise was chosen to allow implementation of limited XML parsers for J2ME devices without exceeding the strict memory constraints of those devices. None of the W3C XML standards (RDF, XML Schema, SOAP, SVG) rely on this feature, which was unfortunately inherited from SGML.

For applications relying on this capability, we have defined required features that can be used to instruct the XmlPull parser to return only parsers supporting this feature. The same holds for requesting validation -- if the appropriate feature is set successfully, it is guaranteed that the parser implementation will do validation as described in the XML 1.0 specification.

It seems to us that the author confuses limitations of particular implementations, mainly aimed at constrained environments, with API design. With the availability of the validating parser this possible source of irritation should be resolved.

The API is not object oriented

Again, the article said,

The object problems are less fundamentally wrong but still extremely troubling. XMLPULL has far too few classes. The prevalence of switch statements and stacks of if-else-if blocks just to test the return type of the nextToken() method is a classic symptom of failure to take advantage of polymorphism. Another hint that something is seriously wrong here is the number of state-dependent methods that only work when the parser is positioned on a particular kind of token. Still another clue is the use of int type constants instead of a class hierarchy. The next(), nextTag(), and nextToken() methods should all return instances of a common Token superclass. Many methods in XmlPullParser could be moved into this class. The whole API smells of procedural code and so doesn't fit very well into object-oriented Java designs.

The motivation for the insufficiently object oriented design of the API may be a bit difficult to understand without a lot of experience with pull parser design and larger applications, so we fully understand that the author feels a bit uncomfortable with this approach. At first glance, it may seem cleaner to have event objects and polymorphism instead of placing all the access methods in one relatively large interface. However, while this would make the XmlPull API "look" somewhat nicer, a separate event object would also introduce several issues.

Actually, both predecessors of the common XmlPull API, KXml1 and XPP2, had separate event objects, exactly as demanded by the author. However, the experience gained with those APIs finally led us to decide against separate event objects in the XmlPull API.

The problem is that the different event object types do not have much in common. Polymorphism makes a lot of sense where an identical set of methods can be applied to a set of different objects. A good example is AWT omponents, which have common properties and methods such as the position, size, and a paint() method. The only thing in common between XML start tags and XML text events is that they are observed when parsing an XML documents: they do not share a single common property.

When using separate event objects, there are several additional design options. For example, methods like getAttributeValue() make sense only for start tags, so it seems natural to place them only there. However, when it comes to actual access, one will require an instanceof check and a type cast:

if( parser.getEvent() instanceOf StartTag ) {

   StartTag st = (StartTag) parser.getEvent ();

   // access tag via st

} else ...

While the overhead does not seem very large here, please keep in mind that in many cases there is not much done with the event. Often access is as simple as a name check. So the overhead adds up to a large percentage of the actual lines of code required for parsing.

Alternatively, we could also have used different access methods depending on the event type, avoiding the type cast. This would look like the following:

if( parser.getEventType() == parser.START_TAG ) {

  StartTag st = parser.getStartTag ();

  // access tag via st

} else ...

Obviously, while in this case neither an instanceof check nor a type cast is required, this approach would add a lot of methods to the parser, and it would no longer be significantly smaller than the integrated interface that is now used in the XmlPull API.

Another option may be to add the access methods of all event types to the event base class. However, in that case the event object would be nearly as big as the integrated interface of the XmlPull API, and the API readability advantage would be lost.

Finally, event objects are not free. Creating lots of objects that are used just for extracting some information to build an application dependent structure may create significant overhead. While this overhead may be reduced by reusing event objects, reusing event objects is extremely dangerous since an object given to the user will change in the background without further notice. By contrast, in the XmlPull API it is obvious that the return values of query methods like getEventType(), getText() and getName() will be different after a call to one of the next() methods that advance the parser to the next event.

This corresponds to another complaint, namely that the API is too J2ME-oriented. We see it as an advantage to have an API that can be used both in J2ME devices and in J2SE and J2EE environments, promoting code reuse and a flatter learning curve for client-server applications

Additional Remarks

In addition to resolving the general misunderstandings of the XmlPull API, we would like to point out some minor possible improvements to the article.

In the XmlPull API, there are two general options for iterating XML events. The preferred way is the method next(), which makes typical parsing as simple as possible by reporting START_TAG, END_TAG, TEXT, and END_DOCUMENT events only. Legacy or special purpose events such as COMMENT and PROCESSING_INSTRUCTION are silently ignored.

We think that it is a bit misleading to present the lower-level nextToken() method first. It is designed for advanced use of the API and exposes every possible detail about the input document.

The way the example of an XHTML outliner is written does not take full advantage of the XmlPull API. Instead of using a boolean state variable that can fail for nested headers (as the author notices), it is better to let the parsing code mirror the structure of XHTML. We think that the ability to minimize use of state flags (sometimes even whole state machines) is one of main advantages of XML pull parsing and should be properly exposed.

The updated example code is both easier to write and easier to maintain if it is split to reflect two distinctive functions. First, finding header elements

int event = parser.next();

while ( (event = parser.next()) != XmlPullParser.END_DOCUMENT) {

    if (event == XmlPullParser.START_TAG) {

        if (isHeader(parser.getName())) {

            printHeaderText(parser);

        }

    }

}

and, second, code that prints the text content of XHTML headers:

private static void printHeaderText(XmlPullParser parser)

    throws XmlPullParserException, IOException

{

    int level = 1;

    while( level > 0 ) {

        int evenType = parser.next();

        if (evenType == XmlPullParser.TEXT) {

            System.out.print(parser.getText());

        } else if (evenType == XmlPullParser.END_TAG) {

            --level;

        } else if (evenType == XmlPullParser.START_TAG) {

            ++level;

        }

    }

}

The full code sample is available for review.

It is important to note that namespace support is required and, although it is off by default, it can always be changed by calling:

factory.setNamespaceAware(true);

Support for validation in API

We have now provided an implementation of the XmlPull API called XNI2XmlPull, which is based on Xerces 2 and provides full support for XML validation.

In the case of example given in the article when XNI2XmlPull is used and validation is requested:

   factory.setValidating(true);

One will get an output similar to this when XHTML input is invalid:

org.xmlpull.v1.XmlPullParserException: could not parse:

:::::2:94:The content of element type "h1" must match "(a|br|

span|bdo|map|object|img|tt|i|b|big|small|em|strong|dfn|code|q|

samp|kbd|var|cite|abbr|acronym|sub|sup|input|select|textarea|

label|button|ins|del|script)". caused by: :::::2:94:The content

of element type "h1" must match "(a|br|span|bdo|map|object|img|

tt|i|b|big|small|em|strong|dfn|code|q|samp|kbd|var|cite|abbr|

acronym|sub|sup|input|select|textarea|label|button|ins|del|script)".

	at org.xmlpull.v1.xni2xmlpull1.X2Iterator.nextImpl(X2Iterator.java:763)

	at org.xmlpull.v1.xni2xmlpull1.X2Iterator.peekNextState(X2Iterator.java:784)

	at org.xmlpull.v1.xni2xmlpull1.X2Parser.nextImpl(X2Parser.java:1079)

	at org.xmlpull.v1.xni2xmlpull1.X2Parser.next(X2Parser.java:915)

	at XHTMLOutliner.printHeaderText(XHTMLOutliner.java:81)

	at XHTMLOutliner.main(XHTMLOutliner.java:56)

Conclusions

We have addressed the main concern of the author by providing an implementation of XmlPull API that supports validation. We hope that we have successfully addressed the other concerns as well.

The XmlPull API can be used now (it has been available for 8 months), and it is proven in practice. We are committed to incremental updates but also open to major changes if they are necessary. Future plans for XmlPull are to provide a serializer, more XML tests, and support for other languages (C++, PHP and so on) but we are open to user participation and new suggestions.

We are both working in the JSR 173 expert group that will probably generate a more object-oriented API without losing essential features of the XmlPull API. However, it is more important that the API is easy to use when writing code to parse XML than to enforce a particular paradigm. Since the API will be used as a building block for higher level APIs (for e.g., SOAP), it must be very efficient and should facilitate implementations with very small memory footprints.