XP and XML
by Eric van der Vlist
|
Pages: 1, 2
Several tools are already including (or developing) this kind of feature. In the world of XML editors and IDEs, there is the Topologi Collaborative Markup Editor, with its SOAP based peer to peer functionality; also, XML Cooktop and its Jabber based peer to peer features. The Eclipse open source IDE project lead by IBM, particularly its koi infrastructure, have a goal to "serve as a test bed for collaborative Eclipse Workbench plug-in functionality," using XML-RPC.
These independent efforts, using different XML protocols, suggest we have all the building blocks needed to implement a "pilot mode" in our editors which would let you see what your partner is typing and leave you the option of leaving an instant message to give your feedback.
On-site Customer
On-site Customer suffers from the same problem as pair programming; getting the customer on-site is nice, but getting her or him physically on-site is too restrictive in many cases, leading to a lack of on-site customer, which is usually solved in XP project by a Customer Proxy, i.e. a member of the development team with a privileged contact to the customer and acting as its representative.
After what we've said about virtual pair programming, an option which seems interesting to explore is the "virtual on-site customer", i.e. a real customer working remotely. The tools to use should be more or less the same as those needed for virtual pair programming.
Because an important part of the role of the customer is to answer questions, in this role collaborative editors are not necessary and may be replaced by a traditional text, audio, or video conference system. The text option should be considered seriously since even though the exchange is poorer than with audio or video, the exchange can more easily be archived and indexed.
Other parts of the role of customer, such as editing user stories and acceptance tests, would, however, gain some advantage from collaborative editors similar to those used by virtual pair programming.
Coding Standards
Last but not least, formalizing coding standards as XML would give pretty much the same benefits as we've seen with the glossary of the system metaphor: it would enable the publication of the standards on different media. Also, especially in a context where coding information is available as XML, it would give the possibility of automatically checking if the rules are followed in the project.
Summary
I think that it's quite obvious that if XML can justify itself on some of these practices, the killer application of XML would be in the interaction and interoperability between the practices. It is well known and documented that the twelve practices are very interdependent and that the benefit of using them together is greater than the sum of the benefits of using them separately. Generally speaking, XML's main benefit is in interoperability between distinct yet dependent applications, and this is exactly how it would best be used here.
XP for XML
Let's look at the other side of the coin and see what problems are specific to XML applications when following an XP methodology. I think that we have both conceptual issues, dealing with "Simple Design" with a side effect in "Refactoring," and more practical issues dealing with "Testing".
Simple Design
To put it bluntly, XML itself is anything but simply designed. Its complexity is due to two main reasons:
- The XML infoset, the "meaningful" information found in a XML document, has been defined independently of any programming language. Its structure has no natural fit in any of the built-in types of the programming languages, leading to APIs which are not as simple as they could be. The consequence is that for simple things, such as parsing a simple configuration file, it is simpler (not much simpler if you're familiar with XML APIs, but still simpler) to use more trivial mechanisms than to rely on XML.
- XML has given birth to a bunch of related specifications, and both the number and the complexity of some of them do not qualify as simple design.
Following the Simple Design practice, XP programmers must choose the simplest possible design to meet the current set of requirements and that's certainly a brake on using XML. The answer could be to continue to work toward simpler APIs -- less generic but better integrated in the various programming languages or even to add built-in types to match the XML infoset. Also there ought to be more utility packages "hiding" XML from the programmers when performing some specific tasks.
Such utilities could be for parsing a XML configuration file, but there are other domains where the flexibility and complexity of XML would be better used; for instance, the persistence layer which is needed by most XP projects. Simple tools such as JSX that provide transparent serialization and class evolution features could have a role to play.
Refactoring
Refactoring can rapidly become painful if your system is using a wide range of XML technologies. Let's say we have developed a XML application with a writer and a reader in Java to serialize and deserialize information as XML, an XSLT transformation to present the information on a web site, a schema (Relax NG, DTD, or W3C XML Schema) to validate its structure, and another one using Schematron to check some "business rules", and finally a XForms form as a UI. The impact of simply changing the name of an attribute or adding a new element will obviously be higher than if the application had been fully implemented in Java and things grouped in a single class.
The additional complexity comes from the fact that several languages need to be mastered. There isn't much we can do about this. Complexity also comes from the fact that it's easier to refactor a system where things that belong together are kept together, instead of being spread out in several documents. This second source of complexity can be compensated for by grouping code by function rather that grouping it by language. This was one of the motivations in my proposal to embed transformations and validations (xvif); and it would be easy too with our "eXtreme Literate Programming" framework.
Testing
Why should unit and conformance tests be more difficult in XML? Two reasons:
- Application-dependent syntactical variations are allowed in XML serializations, which make even the simplest tests much more difficult.
- New languages have been defined (XSLT, W3C XML Schema, Relax NG, Schematron,...) creating needs to new unit test systems.
The first approach that comes to mind when testing that a XML
fragment sent back by a function or method is what's expected is
to test the serialization of this fragment as a string against
the expected one. If I am expecting an empty element "foo" with
two attributes "bar1" and "bar2", we would thus test that the
string is <foo bar1="1" bar2="2"/>. The
problem is that most applications will consider this to be
equivalent to <foo bar2="2" bar1="1"/> or
<foo bar1='1' bar2='2'/> or <foo
bar2="2" bar1="1" />. Depending on the "writer" used
to serialize you XML, you can get one or the other.
These equivalences are defined by the XML 1.0 recommendation and
this means that string comparison isn't portable: if we use them
we have a risk of detecting false failures if we change or
upgrade the "writer" used to serialize our result. Also note
that if we were writing a XML pretty printer or a
canonicalization library, we would not consider these variations
as equivalent, and that many more equivalences are usual in XML
vocabulary. The value of namespace prefixes, the relative order
of children elements, and whitespace variability are examples of
such variations which are sometimes considered equivalent and
sometimes not, depending on the application. Still worse, for a
given application, these variations can be significant only at
some locations in a document: for instance, in RSS 1.0, as for
any RDF application, the relative order of children elements is
not significant, except within specific elements such as
rdf:Seq.
There are two solutions: you can either define a "canonical" format (i.e., a "normal" form) for your application and perform string comparison on the canonical form or do an application specific comparison on the XML fragments interpreted as DOM trees, SAX events, or whatever representation best fits your needs. Note that in both cases, the canonicalization or the comparison must be adapted to the vocabulary which is used.
That's one of the issues I have found when I developed my Python implementation of "Regular Fragmentations" and the research I did at that time (July 2002) has shown that in the many "XML diff" libraries that are available, their focus is on identifying differences between versions of a document for editing, and that although they can often recreate a version from the differences and the other version, the criteria to determine if two fragments are equivalent cannot be customized.
I have thus developed a very simple comparison library implementing the features I needed at that time, which could serve as a base for a more complete library.
The second difficulty is in defining unit tests for XML specific languages such as XSLT. Since XSLT is a programming language, and since XSLT transformations are modular and may become complex, it is necessary to be able to define unit tests for XSLT templates. That's the purpose of my XSLTUnit framework.
It's interesting to note that these two issues are linked and that XSLTUnit assertions need to compare XML fragments. The current version of XSLTUnit doesn't support the customization of these tests, but that would be a useful enhancement.
To a lesser extent, there is also a need for unit testing with other languages: XQuery obviously, but even schema languages. Schemas are usually more complex than the instance document, and it might be worth being able to test that a Relax NG pattern, a W3C XML Schema complex type, or a Schematron rule does not go wild, and actually does what its author is expecting.
Summary
The issues we've seen in this second part do not seem unsolvable. XP can be used right now to implement XML applications and it appears that this is a matter of mindset and tool availability more than any real incompatibility between XML and XP. (It would definitely be nice if XML developers, designers and architects had more of a "XP mindset," and were eager to follow the "Simple Design" practice.)
Conclusion and Acknowledgments
Although the reluctance of the XP community toward XML noted last year by Leigh Dodds is persisting, there is a huge opportunity to make XML a technology of choice for implementing better tools for XP, and the XML community still has lots to learn from the XP community. I'd like to thank Michel Duperrier for giving me the basic idea behind this article, Nicolas Chauvat and Laurent Bossavit for confirming some of my early conclusions, and Laurent Bossavit (again) for pointing me to JSX and its potential.
Do you agree with Eric's conclusions? Do you have relevant experience of XP and XML? Share your comments in our forum.
(* You must be a member of XML.com to use this feature.)
Comment on this Article
| Titles Only | Titles Only | Newest First |
- XP and XML are not incompatible; but XML drifted
2003-04-17 15:26:56 Andy Glew [Reply]
I'm an XP advocate:
* Pair Programming: I'm not currently working paired, in fact I'm currently not really working on a team at all, although I did work paired for about 18 months.
* Test First: really, really, helps
* Refactoring: really, really, helps
etc.
I found it had to go to XP because my job title is "Computer Architect" (as in hardware), and architecture is BDUF, Big Design WAAAYYY Up Front". But I realized that XP is what I have done in some situations where I have been most successful.
---
But, though I'm an XP advocate, I'm also serious about XML. In fact, in many ways I was doing XML, at least self-describing data, before XML.
E.g. I wrote PerlSQL (google that if you care);
I would have written it to use SQL, if I had known about XML at the time).
---
I'm interested in XML as a way to solve, once and for all, problems that I have repeatedly written ad-hoc custom programs to solve. I'm interested in XML because it may allow me to leverage XML tools in ways that will help me. I.e. I am interested in XML because it may be a non-local optimization: it may not be the fastest way to solve the problem I have today (which, interestingly enough, is a test database for a tool that I am applying XP on), but I think that it may be worth a few extra days to use XML, and make subsequent tools easier.
(5 years ago I would have used PerlSQL for today's problem; I could still do so today,
but I am willing to give XML a go.)
---
XML is a horrid, clunky, verbose data format.
My database tools will continue to support
CSV, PerlLikeRecords, and LISP-like S-expressions.
But XML will join the gang,
and I will be able to convert into XML as I desire,
and back again.
XML programming tools are, well, even worse.
DOM vs. SAX-push vs. pull interfaces...
OK, maybe XML programming tools are okay for Java and scripting languages; about half of the time I can get along with scripting languages, and I would *love* to be able to use Java, but for the other half of the time Java, Perl, Python, Ruby, etc. are just too damned slow, and I have to use C++. (And in the computer architecture / hardware design community I am considered radical for using C++ rather than C.) And usable C++ libraries for XML are scarce --- I hope that something better than DOM and SAX are available.
(I am also hoping to bait people into telling me about them... send email.)
---
The XML community has a definite bias towards
what seems to me to be fairly "lightweight" programming - webservers, data interchange, etc.
Important problems, just not at the same level as I have to work at.
And, because of this, XML tools are somewhat limited.
By the way, the eXtreme Programming community is also heavily biased towards commercial programming, developing of websites, etc.
I'm not quite sure how this relates to an earlier poster's comment about software production vs. software applications, but I suspect it does.
And I am in still another domain of software (and hardware) enginnering.
---
One comment:
I arrived at XML from the point of view of self describing data. I had great hopes for XML in that regard in the early days.
But XML has spent the last few years concentrating on schemas, DTDs, etc.;
and in most databases changing a schema is a heavyweight action. The exact opposite of the
sort of agility that XP espouses; the exact opposite of my world, where the "schema" changes every time I run an experiment, and realize what new question to ask. (I asked Jim Gray and Andreas Reuter what relational schema could be used for my experimental data, and the consensus was fully, completely, normalized. Schemas don't help much there... Maybe overlapping partial schemas might. But, everything must be lightweight.)
So, it may just be that XML took a turn that took it away from the self describing data world for a while.
---
Another quick comment: maybe XP would look at XML
more kindly if there was any Wiki based on XML.
Wikis are at the heart of the XP movement.
In fact, Wikis are really a move away from HTML, and away from XML, towards even lighterweight and more human friendly data.
---
Overall XML and XP are not necessarily incompatible.
You just gota see that you need it.
- XP and product development
2003-03-16 17:00:17 Naren Chawla [Reply]
XP has several good ideas. However, I am not sure if XP is a good fit in software product development environment (versus software appliction development). In other words, I could see it fit nicely in building "inventory management system" versus building a "database engine".
The reasons -
1. On-site customer - who is the customer for a software product ? Typically, the product is shipped to multiple customers. When potential customers are asked for feedback - 50% will prefer feature "X" and rest won't. In other words, customer feedback is more like suggestion as oppossed to "requirements".
Product Manager can fill the role of "on-site customer". However, in reality for most innovative products, the requirements are generated more dynamically through several brainstorming session involving multiple people (senior technical leaders, engineering management and product management).
In other words, there is no one person who can act as an "on-site customer"
2. Pair programming - Most accomplished software designers work in a zone where I feel the presence of another person may not necessarily be ideal. And as mentioned in the article, telecommuting is a norm nowadays.
3. Continuous integration - for any serious product running the entire regression suite takes more than 2-3 hours. Trying to do this cycle, 2-3 times a day is non-productive use of time. It might be more useful towards the end of the release.
I would like to hear from people who have adopted XP practices in software product development.
- To a man with a hammer...
2003-03-09 22:27:27 William Pietri [Reply]
I'm an XP practitioner and an enthusiastic user of XML. Despite that, Eric var der Vlist's article mainly strikes me as a puzzlement. There are a number of good ideas, but perhaps because of his lack of experience with XP, many of them are solutions looking for problems.
Certainly, I regularly use XML-related solutions in my XP work. One current project generates XML UIs which are fed to XSLT, outputting HTML or VoiceXML depending on how the user interacts with the system. Another uses XML configuration files and saves all persistent data in XML files, eschewing a traditional database. So I'm not shy about using XML when it's the right tool for the job.
And I'm also not reluctant to use technological aids to improve the development process. For example, I use IntelliJ's IDEA, an IDE with tightly integrated refactoring tools (and a good XML editor, too).
But one of the fundamental insights of XP is that simplicity is a virtue. XP advocates keeping track of stories via simple index cards not because we don't know how to create fancy tools to manage stories, but because we've found that fancy tools are much more expensive and much less effective than a stack of cards. We choose in-person communication because it has a higher bandwidth, lower latency, and better user interface than any computer-mediated tool.
XP advocates solving today's problems today, and leaving tomorrow's problems for tomorrow. Anybody interested in trying XP should learn from that. The first step is to try XP as other people do; only once actual problem present themselves should you start work on the solutions.
There's nothing wrong with reengineering one's bicycle, but one should learn how to ride it first.
