Storing XML in Relational Databases
by Igor Dayen
|
Pages: 1, 2, 3, 4
Sybase Adaptive Server
SQL to XML Mapping
Sybase employs an XML document type, ResultSet, to describe both XML document metadata (element name, type, size, etc.) and actual row data. Here is excerpt from a hypothetical FxTradeSet.xml:
<?xml version="1.0"?>
<!DOCTYPE ResultSet SYSTEM "ResultSet.dtd">
<ResultSet>
<ResultSetMetaData>
<ColumnMetaData
...
getColumnLabel="CURRENCY1"
getColumnName="CURRENCY1"
getColumnType="12"
... />
...
</ResultSetMetaData>
<ResultSetData>
<Row>
<Column name="CURRENCY1">GBP</Column>
...
</Row>
</ResultSetData>
</ResultSet>
|
The ResultSet DTD does not appear to permit definition of the nested elements.
Extracting XML from the database
The Java class ResultSetXml has a constructor which
takes an SQL query as an argument, thereafter the
getXmlLText method extracts an XML document from result
set:
jcs.xml.resultset.ResultSetXml rsx = new jcs.xml.resultset.ResultSetXml
("Select * from FxTrade", <other parameters>);
FileUtil.string2File ("FxTradeSet.xml", rsx.getXmlText());
|
Storing XML in the database
The ResultSetXml class constructor can also take an
XML document as an argument. Then the method toSqlScript
generates sequences of SQL statements for insert/update into a
specified table from a result set.
String xmlString = FileUtil.file2string ("FxTradeSet.xml");
jcs.xml.resultset.ResultSetXml rsx = new jcs.xml.resultset.ResultSetXml
(xmlString);
String sqlString = rsx.toSqlScript ("FxTrade", <other parameters>)
|
Summary
The extraction and storage of XML documents are symmetrical in nature. Storing does not seem to permit modifying more than one table. Extraction transforms the results of an SQL query into a document with a flat structure.
Vendor Comparison
|
Vendor |
Mapping rules |
Single table / Multiple tables |
Means of transformation |
Symmetrical extraction / storing |
|
Oracle |
Implicitly; by constructing object-relational data model |
Multiple |
Designated Java classes |
Symmetrical, if XML document and object-relational model match |
|
IBM |
Data Access Definition file |
Multiple |
Designated stored procedures |
Symmetrical |
|
Microsoft |
SQL extension; row set function |
Multiple for extraction; Single for storing |
By using SQL construct FOR XML and row set OPENXML |
Asymmetrical |
|
Sybase |
Result Set DTD |
Single; query may encompass multiple tables |
By using Java classes |
Symmetrical |
Common features between vendors:
- XML persistence is achieved on an ad hoc basis, i.e., there's no general facility for storing arbitrary XML documents); special mapping is required if the XML document uses a new grammar;
- storage often requires data preprocessing, such as number/date reformatting with respect to the locale being used; XSLT can be utilized to perform XML data massaging.
An Alternative Strategy
Can the task of XML document storage in a database be split into stages? First, store XML in generic database structures without applying any mapping rules; and, second, applying a framework for subsequent data processing? The advantage of this approach would be to have a universal acquisition agent for virtually any XML document. The remainder of this article proposes just such a solution.
In this strategy, XML documents are saved as normalized tree structures -- raw tree data -- for further application processing. The framework for further data processing is then a set of database objects (stored procedures), similar to DOM APIs, for navigating data by employing traditional SQL processing.
There are advantages to using a framework approach as opposed to full automation: the population of database tables is normally controlled by application logic distributed between "pure" application code, transaction processing, database layer, and stored procedures, not to mention that some tables may reside in remote databases.
Additionally, in a heterogeneous database environment, having a unified tool to perform the same task for different databases has obvious advantages.
The author's solution, ObjectCentric Solutions x-Persistence Utility, implements the strategy discussed here.
Raw tree structures are defined by means of the following tables.
1) Tree Node Definition CREATE TABLE NodeDef ( nodeID udd_id NOT NULL, // unique node id dimID udd_id NOT NULL, // tree category: XML name udd_name NOT NULL, // node name value udd_paramvalue NULL, // node value value_type udd_type NULL, // value type power udd_power NULL, // node tree level isRoot udd_logical NOT NULL, // top node flag descriptor udd_descriptor NULL, // DOM Node type color udd_color NULL // informal data ) 2) Parent-Child relationships between nodes CREATE TABLE NodeLinks ( parentID udd_id NOT NULL, // parent node childID udd_id NOT NULL // child node ) |
Example usage
Storing an XML document in a database is a matter of invoking the XML2SQL program:
XML2SQL <XML document filename>
Extracting an XML document from a database as a tree structure is implemented using the following stored procedures:
get_tree_roots<tree category> -- extract all document root ids for a given forestget_tree<root id> -- extract all nodes for a given root (document)get_children<node, level> -- extract all children of a certain level for a given node
Implementation details:
- Current platform coverage: Sybase, MS SQL Server.
- Platforms being evaluated: Oracle, DB2, and PostgreSQL.
- The utility is built on top of the Xerces XML Parser.
- Database installation involves the addition of just a few tables, and stored procedures to navigate the database tables as tree structures.
The x-Persistence Utility is currently used as the core of an XML-based data acquisition agent in Open Finance Laboratory, a customizable integration platform for a range of financial applications such as portfolio management, asset-liability management, risk management, etc. Open Finance Laboratory adopted FpML as a descriptive language for Interest Rate Derivatives products. x-Persistence Utility is currently used as a FpML/FRA data acquisition agent.
References
- Oracle XML-SQL Utility, http://otn.oracle.com/tech/xml/oracle_xsu
- IBM DB2 XML Extender, www.ibm.com/software/data/db2/extenders/xmlext
- XML Perspective. In control with FOR XML Explicit. SQL Server Magazine, http://msdn.mcrosoft.com/library/periodic/period01/xmlExplicit.htm
- Writing XML Using OPENXML
- OPENXML
- Technology Preview: x-Persistence Utility, ObjectCentric Solutions, Inc., 2001, www.objcentric.com
Involved in similar projects, or have comments, clarifications or questions about this article? Leave feedback using our forum.
(* You must be a member of XML.com to use this feature.)
Comment on this Article
| Titles Only | Titles Only | Newest First |
- SportsFeed Xml Processor
2008-01-22 02:18:47 twostepted [Reply]
You can use SportsFeed Xml Processor to automate transforming and importing remote or local xml data into many different databases including Sql Server, Mysql, Oracle, SQLite, PostgreSql and others. It is a cross platform service application with scheduling and monitoring features which can work with any type of xml. There is a users group at http://groups.google.com/group/sportsfeed-users/
- Please rewrite and update this article!
2006-06-14 16:42:42 craign1 [Reply]
This article was written in 2001 and it is horribly out of date. Most of the techniques here are elsewhere referred to as 'shredding' XML into a relational model. DB2 and probably the other good SQL systems now have XML datatypes for columns, support XPATH to get elements or branches of elements, and support SQL to actually access rows within the XML document. An actual comparison of these features would be helpful in the rewritten article. 'shredding' is an obsolete, low performance, labor intensive practice that should now be deprecated.
- Importing a Data Array from XML to DB
2005-11-10 18:25:41 bdrudolph [Reply]
I have a data collection box that outputs the data gathered in an XML format similar to the following:
<data>
<temp meas_id="Inlet" timestamp="11/10/05 19:00">100 150 200 250 300 350 400 450 500
</temp>
<temp meas_id="Outlet" timestamp="11/10/05 19:00">1000 1500 2000 2500 3000 3500 4000 4500 5000
</temp
</data>
As you can see the data resides at the element level but the attributes define the data. I am fairly new to XML and tranformations and can not figure out how import the data. I am using MS Access which I realize isn't a particulary powerful DB but it is the only DB at my disposal. My manager will not provide funding for software development thus a lowly Process Engineer is trying to do the development. Any input, comments, or help is greatly appreciated in advance.
Brad
- Importing a Data Array from XML to DB
2005-11-10 18:28:03 bdrudolph [Reply]
I am going to reply to myself as I thought of additional information to provide. I have been able to import either the element data or the attribute data but not both together. Do I need to take a multi step approach to tearing the XML apart and then restructuring it? Again thanks in advance for any advice.
Brad
- Importing a Data Array from XML to DB
2005-12-04 11:14:16 Igor Dayen [Reply]
One thing you may consider is to use XSLT to transform attributes into relevant element-based representation.
- Storing .xml files into database
2007-08-21 04:01:35 santoshlocuz [Reply]
hello dear,
i created tree by using dom and 3xml files, i didnt used database interaction for retriving the data. i just simply uses my xml files .but i need to store these xml files in MY SQl database.? how?
- Importing a Data Array from XML to DB
2007-08-21 04:00:59 santoshlocuz [Reply]
hello dear,
i created tree by using dom and 3xml files, i didnt used database interaction for retriving the data. i just simply uses my xml files .but i need to store these xml files in MY SQl database.? how?
- Importing a Data Array from XML to DB
2007-08-21 19:26:35 Igor Dayen [Reply]
You may consider:
(a) using CLOBS if processing XML within a database is not significant; database is used as a storage and XML processing is delegated to application code;
(b) using stored procedures as it suggested in http://rpbouman.blogspot.com/2006/03/importing-xml-data-into-mysql-using.html.
MySQL team is considering to add XMLType support in future, see:
http://forums.mysql.com/read.php?44,155525,157883#msg-157883
Thank you.
- Importing a Data Array from XML to DB
- Storing .xml files into database
- Importing a Data Array from XML to DB
- Importing a Data Array from XML to DB
- HELP! newbie needing to do something tricky...
2005-04-08 15:08:16 RacerZack [Reply]
Wow, what a cool site. I'm new to working with XML (although I am familiar with relational databases). I currently need to import a hierarchical XML file into MS Access or SQL Server. My problem is that child nodes in the XML document do not contain any key data from the parent. How do I get this data imported into a relational database and make sense of it?
Example:
<rootNote>
<Customer>
<Name>Zack</Name
<CustomerID>123</CustomerID>
<Order>
<OrderID>888</OrderID>
<PartNumber>176</PartNumber>
</Order>
<Order>
<OrderID>355</OrderID>
<PartNumber>999</PartNumber>
</Order>
</Customer>
</rootNode>
This XML will give me 2 tables in Access. One table, [Customer] contains 2 fields: [Name] and [CustomerID]. The other table, [Order], also contains 2 fields: [OrderID] and [PartNumber]. Once inside of Access, I have nothing to connect the data in the [Order] table with the data in the [Customer] table.
Within the hierarchical XML, the relationship is obvious from the fact that <Order> nodes are children to <Customer>.
Is there ANY solution for this? I've had an open issue with Microsoft on this for a week now and it pretty much looks like the answer from them is "can't be done." Grrr.
The only solution I can see is to include parent key data (such as <CustomerID>) at each child node. But that would suck. My example is little, but the real XML file I have is huge and has *multiple* levels of hierarchy.
Thanks!
- HELP! newbie needing to do something tricky...
2005-04-09 09:55:03 Igor Dayen [Reply]
If you are using MS SQL Server 2000 / 2005 you may consider using OPENXML for storing XML in a single,
flat table; and thereafter transform the latter one into two tables.
- HELP! newbie needing to do something tricky...
- how to measure performance of tow diff xml databases
2005-01-03 08:43:46 amolee [Reply]
am gona compare between differnt databases in storing and querying XML documents, and really need some help .
what i need is 2 know how to measure the performance for all those databases?? do i have built an application that is well suited for native XML database and used it to benchmark the two types of databases?? and does any body have bulit that application be4 so i can use directly and save more time...
- how to measure performance of tow diff xml databases
2005-02-28 19:34:40 Igor Dayen [Reply]
With respect to relational databases, you may be interested in reading article "Which Storage XML?"
by Sean Dillon, Oracle Magazine, March/April 2005.
Article discusses performance implications while using BLOB vs. XMLType as storage data type.
- how to measure performance of tow diff xml databases
2005-01-03 20:27:21 Igor Dayen [Reply]
If you'd like to compare various SQL databases, you may consider having same comparison baseline: same database object(s) definition, same XML schema(s), same set of input XML files (and different schema-database tables mapping xmls - if any). The client code would rely on database packages/APIs. [Note: with x-Persistent Utility
your client code will remain unchanged]. As for native XML database as a benchmark one - it could be, probably, a separate task to identify the database that would suite your performance criteria(s).
- how to measure performance of tow diff xml databases
2005-02-27 14:40:36 amolee [Reply]
Well, Im a student doing a project to compare between (Oracle-Tamino) databases in storing and querying XML documents, I already generated the xml documents and loaded them to the databases ( I used XMark benchmark) BUT now am facing problem with measuring query response time .
what i need is 2 know how u measure the performance for those databases?? Is there a tool I can use to measure the performance?? or can I use se timing on function from Oracle and _duration=on function in Tamino???
Can I relay on those functions to measure the query time response ???
- how to measure performance of tow diff xml databases
- how to measure performance of tow diff xml databases
- XML document to be decomposed into a collection of xml subdocuments, using XSLT ?
2003-08-26 09:36:11 punam verma [Reply]
Thanks this article is very informative .
on page 1 of this article point 2 uve mentioned -'2. ...the XML document can be decomposed into a collection of "flat" subdocuments, using XSLT.'
I have an Invoice Xml file it needs to be split into Customer,InvoiceDetails, Line Item xml files.
Question is how do i perform this thru xslt . I need 3 separate files in 1 step .
Also pls could u give me some links where I could find solutions to queries similar to mine .
thanks in advance .
- XML document to be decomposed into a collection of xml subdocuments, using XSLT ?
2003-08-28 15:40:23 punam verma [Reply]
This can be done by
For further reading/example visit http://www.w3.org/TR/xslt11/#multiple-output
- XML document to be decomposed into a collection of xml subdocuments, using XSLT ?
2003-08-26 19:42:06 Igor Dayen [Reply]
Thank you for feedback.
You may condider creating separate stylesheets
for Customer,InvoiceDetails, etc. to be applied
to the input document Invoice.xml. The useful reference which would be in line with the subject
of relational databases is "Building Oracle XML Applications" by Steve Muench from O'Reilly. Actually, chapter 7 on XSLT is available on web, see http://www.oreilly.com/catalog/orxmlapp/chapter/ch07.html#9820. I also would recommend "XSLT Quickly" by Bob DuCharme from Manning, see section 3.6 "Copying Elements to Result Tree". To extract Customer element from Invoice you may consider something like:
<xsl:template match="Customer">
<xsl:copy-of select=".">
</xsl:template>
Regards,
Igor Dayen
- XML document to be decomposed into a collection of xml subdocuments, using XSLT ?
- Storing XML in Relational Databases
2003-08-21 06:07:40 Barry Schaeffer [Reply]
Thanks for the information.
I am puzzled, however, by the lack of differentiation between XML that the DB vendors target (flat or message related) and that encountered in document content applications. Oracle states, for example, in its developers guide, "complex XML is difficult to map into a relational model" after which it suggests that any such structures should be handled via CLOBs.
This differentiation occurs in other vendors' approaches as well and, if ignored by prospective users, can seriously midlead the direction of XML storage. Indeed, using a RDBMS for anything more than, as Oracle suggests, "using XML to port data into structures already stored in a relational model" (they use a PO as their example) they face a set of potentially serious problems, not the least of which are performance degrades as XML trees become more complex, visibility degrades as valuable XML content is stored in LOBs and cost increases as the user must create software to support import and access, again as the DB vendors illustrate in simple form but which is much more complex in actual practice.
An interesting sidenote in your piece might have been some notice of the fact that the software industry is moving toward a native approach to XML, represented by a number of native XML repositories. We use Tamino in much of our work and find the ability to import, manage and query complex XML without mapping it to another model to be quite liberating. There are pros and cons on Tamino itself and on the Software AG approach, but it is representative of an approach that is specifically designed to handle the type of deeply nested, richly tagged XML that we often find in legal, technical and scientific content.
There is, I believe, a bright line between interchange use of XML that the RDB vendors seek to support, and content modeling use of XML that they want no part of (the BLOB approach to storage is nearly 15 years old.)
Thanks again for the analysis.
Regards,
Barry Schaeffer
CEO
X.Systems, Inc.
- Storing XML in Relational Databases
2003-08-23 08:17:52 Igor Dayen [Reply]
Thanks for feedback and suggestions very much.
I agree that the market of native XML databases has been evolved
substantially for the past couple of years; you may be interested in
checking Ronald Bourret's product directory
http://www.rpbourret.com/xml/XMLDatabaseProds.htm for details.
As it is stated in the reference above, Tamino's powerful import facilities allow
"individual XML documents to be composed of data from multiple, heterogeneous sources,
such as the native XML data store, relational databases, and the file system".
It is also worth to note that XML support in RDBMS incorporates now some
features attributed to native XML databases. Oracle 9i, for example, introduced
XMLType, which allows applying XPath expressions and indexes towards XML documents.
Regards,
Igor Dayen,
ObjectCentric Solutions Inc.
- Storing XML in Relational Databases
2004-08-23 23:38:02 query [Reply]
I need to convert Access table into XML format for onward submission to Middleware. Appreciate your urgent help.
- Storing XML in Relational Databases
2004-08-24 19:09:57 Igor Dayen [Reply]
Here are a couple of options you may consider:
(a) export from Access into flat file
with further conversion into XML using
custom-made parsing solution or
using a vendor tool
(you may check for CredibleXML, see
http://www.crediblexml.com/);
(b) export from Access into MS SQL Server
and generate XML out of SQL Server table with
use of FOR XML syntax.
Igor Dayen
- Storing XML in Relational Databases
- Storing XML in Relational Databases
- Storing XML in Relational Databases
- how to transfer data from xml document to database tables?
2003-03-25 09:38:43 manas mallik [Reply]
i am doing my project in these lines so i need to know how to transfer data from xml document to database tables taking into account i already have the required tables created.
- how to transfer data from xml document to oracle database tables?
2004-08-20 05:14:40 Deepthi9 [Reply]
i have a problem importing xml data from XML to orcale DataBase. I want the full code for conversion.
- how to transfer data from xml document to database tables?
2003-07-14 19:58:21 Igor Dayen [Reply]
You may consider using xPersistence Utility
(www.objcentric.com).
XML data is stored in persistent DOM tables
(NodeDef, DimensionDef, others).
Out of these tables you may copy data into
your tables using regular SQL.
Here are brief "HOW-TO" instructions:
A. Installation instructions
============================
Prerequisites:
download and install
OLAP-support General-Purpose Database Objects
(see xdb\readme.txt)
(i) add jar-file xdbEval.jar to the class path,
see samples:
setclasspathEval.bat [Evaluation copy],
setclasspath.bat [Commercial version]
(ii) Xerces jar-files are already included into download;
alternatively, download Xerces XML parser:
http://xml.apache.org/dist/xerces-j/
(iii) add xerces.jar to the %classpath%
(iv) add xercesSamples.jar to the %classpath% [for evaluation copy only]
B. Use cases
============
0. Install database properties in user.home:
Default settings for Windows:
Documents and Settings\user name\ocsdb.ini:
dbname: xdb
user:sa (default)
pwd: (default)
driver: sun.jdbc.odbc.JdbcOdbcDriver (default)
url: jdbc:odbc:LocalServer (default)
1. x-Persistence Utility employs
OLAP-support General-Purpose Database Objects.
Default dimension for storing XML documents is "xml".
2. To store XML document in the database:
cd XPersistence
cd samples
java com.objcentric.xdb.XPersistence -c personal.xml >nul
3. To get root node IDs for the stored XML document(s):
get_tree_roots <dimension>
4. To get stored XML nodes:
get_tree <nodeID>
- how to transfer data from xml document to oracle database tables?
- Problem: Large Tables mapped to XML Documents result in large XML Documents. Any solutions for this problem?
2002-11-21 05:46:12 stefan fischerfeier [Reply]
I've selected all rows of a very large db table and generated an XML document with Oracle XSU. The result was a large XML document. If you have 300.000 rows with 4 columns you will get an XML document with several megs, if you store it on your hard drive. It will take several minutes to read that file again and work with it.
Are there any solutions for this problem?
- How to store XML data into relational database
2002-09-18 04:05:12 cvinoth Chidambara [Reply]
I need to know how to store the XML data into
relational database.We are using Sybase12.5 database and is there any jar files needed to parse the XML data and store that data into Sybase database..
Thanks & Regards,
Vinoth
- How to store XML data into relational database
2006-10-13 07:34:06 pvaidya [Reply]
Hi vinoth,
I am searching for the same information. Please let me know what you gathered.
Thank you,
Pvaidya
- How to store XML data into relational database
2003-07-14 13:56:18 dave Mack [Reply]
- How to store XML data into relational database
2003-09-29 00:51:29 thandar win [Reply]
i want to know how to import xml documents to relational databases.
- How to store XML data into relational database
- How to store XML data into relational database
- Please state version of Oracle
2001-07-24 13:25:32 Mark Drake [Reply]
Your review of Oracle fails to make any mention of the native XML functionality in Oracle9i. Please can you explain why you choose to ommit this, or clearly state which version of the database you were reviewing.
- Please state version of Oracle
2001-07-25 06:32:21 Igor Dayen [Reply]
Reviewed database version is Oracle8i. Thanks for comments.
- Please state version of Oracle
- Do you have any rough perofrmance comparation among these DBs ?
2001-07-12 16:34:24 jun wang [Reply]
Hi, If you have some points aobut perofrmance comparation among these DBs processing XML solution, that would be really appreciated.
BTW, If you can put a bit more hint about your Universial XML framework how to improve performance, that would be great.
- Do you have any rough perofrmance comparation among these DBs ?
2001-07-18 21:14:05 Igor Dayen [Reply]
At the current stage unfortunately I can not provide you
with vendors' comparison statistics under the
same operating environment (and on the
same set of data).
Any collaboration on the testing would
be appreciated.
Here is statistics on storing XML with
sample "personal.xml" included with
Xerces 1_4_1 for Java.
Here is statistics (in milliseconds):
</personnel>
Statistics: ms = 1232
Hardware/Sotware:
OS Name Microsoft Windows 2000 Professional
Version 5.0.2195 Service Pack 1 Build 2195
OS Manufacturer Microsoft Corporation
System Name OBJECTCENTRIC
x86 Family 6 Model 8 Stepping 6 GenuineIntel ~1000 Mhz
Total Physical Memory 261,424 KB
MS SQL Server 7.0
- Do you have any rough perofrmance comparation among these DBs ?
- obvious problem with this approach
2001-07-03 10:58:16 K Vainstein [Reply]
try doing a 20-deep self-join on a large table. could be solved, but result'd be quite bletcherous.
- obvious problem with this approach
2001-07-18 20:53:23 Igor Dayen [Reply]
To store XML documents in database utility does not employ SQL with self-joins.
- obvious problem with this approach
- XML/SQL components
2001-06-28 13:51:52 Cory Casanave [Reply]
We have faced similar requirements in Component-X (www.enterprise-component.com) and have implemented full SQL/XML mapping - the approach is to use a superset of the "domain" xml document with attributes to select actions (query, insert, modify...). This works with nested as well as simple tables. This structure is much more user friendly than an SQL query like thing. The output is also SQL. We would like feedback of the way queries are formed in this component.
- What about XML Extensibility?
2001-06-25 12:33:53 Vittorio Viarengo [Reply]
This is a good overview of current support for XML in the major relational database systems. It highlights how still relatively painful it is to store and manage arbitrary XML data into relational databases. That's because relational databases are optimized to deal with data that has a very well defined schema that does not change very frequently (the schema that is).
One of the main advantage of XML is extensibility and it seems to me that relational databases have a long way to go to really support this.
I think that when people embrace XML for real, they'll want to have a XPath and XSLT processor that works right on the data without 3 layer of translations, plus the ability to extend their XML data structures at will.
As an example, Oracle's limitation in dealing with XML attribute seems to me like a huge limitation that most XML programmers could just not deal with (what about those industry standards being defined out there? Should they all of the sudden drop attributes from their modeling techniques????)
So, if the XML documents needs to be generated from existing tables, relational databases provide a pretty good solution, when it comes to dealing with more "unstructured" or "semi-structured" XML data or data that comes from outside the firewall to begin with, then I think people should look into more XML-native solutions such as the ones provided by eXcelon, Software AG and the other smaller startups (Ipedo, Neocore...). For applications where XML plays a central role in the architecture, I think that these alternatives provide some serious productivity and flexibility advantages.
Regards
Vittorio
http://www.stylusstudio.com
- What about XML Extensibility?
2001-06-26 08:53:18 jack smith [Reply]
I agree with you. A Hybrid/ alternative solution may be the nest approach. I also am hopeful the coming adoption of XML Schema as the industry standard by the W3C will enable SQL NOT NULL clauses etc to be available for XML. Thanks again
- What about XML Extensibility?
2001-06-26 04:33:25 Martin Vanderjagt [Reply]
I agree fully with your remarks. We are using Excelon to full satisfaction. However, a problem arises when objects in the XML DTD have to be mapped to SQL elements (as in SAP or other legacy systems).
Should I try to map them, or to store related XML in the SQL system after all?
This, in my opinion, is the most fundamental problem in the industry today...
Kind regards,
Martijn van der Jagt
- What about XML Extensibility?
2001-06-26 11:54:55 Vittorio Viarengo [Reply]
I'm not sure I understand your question.
If the end target of your data flow is an SQL database, you have to map it at some point.
We see "native" XML storage systems as
1 - staging area for business documents and XML data as it flows through various systems and databases (or a network of partners)
2 - storage of semi-structured data that does not have a place elsewhere
3 - middle-tier cache for application data
4 - repository of meta-data
5 - primary XML storage system for highly dynamic applications that use XML as their extensible information model
Basically whenever you are using XML to its full potential. For applications where a thin XML layer on top of a relational database is sufficient, you should just go with it. As we was from this article, it is still relatively painful but people are doing it.
When you look at the requirements of n-tier/highly de-coupled systems, you'll see that there is going to be a lot of XML data floating around that by nature is going to change all the time. To effectively capture that data (transactionally) so that an application can go back in time and do something about it, you need a flexible storage system for XML that overcome all the limitations of current solution. That the basic value proposition of native XML storage systems.
I personally think this is only the first step in building an XML architecture. What are the (new) things that you can do once you have aggregated data and content into a common information model using XML? What kind of new applications can you enable with an extensible data model based on XML? How can you use move XML documents from system to system to model and automate business processes that span different systems/partners?
What kind of tools do you need to model, transform, manage, present and debug XML data?
Providing answers to these questions is what is going to take "native" XML system to the next level.
Regards
Vittorio
http://www.exceloncorp.com
http://www.stylusstudio.com
- What about XML Extensibility?
- What about XML Extensibility?
- Generic approaches to storing XML in RDBs
2001-06-22 05:25:53 Richard Edwards [Reply]
See also our paper "Persistent DOM: An Architecture for XML Repositories in Relational Databases", Lecture Notes in Computer Science, vol. 1983, pp416-421, 2000, and my XMLDevCon presentation "XML Repositories in Relational Databases" (downloadable from the "Research" section of http://www.rgedwards.com/ ).
We propose a similar generic approach to x-Persistence and XMLDB, except that our nodes are stored with two coordinates (a left and a right node ID) that facilitate rapid serialisation of documents (and fragments). Systems that store simple parent-child relationships will require sequential node tree traversal in order to serialise documents - a slow process for large documents.
- Oracle Internet File System
2001-06-22 03:41:25 Peter Bull [Reply]
Are there any clear advantages or disadvantages of using this file system as a repository for XML documents. Arbortext has worked with Oracle to use this file system and this is being sold as a content management system. Does this solution offer advantages over other products, which use a relational database.
- XTM Topic Maps in relational database
2001-06-20 21:28:32 Jack Park [Reply]
At http://nexist.sourceforge.net, we are developing an engineering testbed to see how far we can push the XTM standard as the backbone of an API for web collaboratories. The system uses HSQLDB (also at sourceforge) though I suspect that just about any rdbms can be used. XTM elements are stored in separate tables in a schema I call "abnormal form"; it was cheap, quick, and works.
- XMLDB
2001-06-20 20:42:00 Pete Ryland [Reply]
Hi, FYI there are several excellent open source products out there which do similar things to those commercial products described. One of these is an exciting new product I am leading development on. It's called xmldb (http://xmldb.sourceforge.net/) and although still in early development it is already becoming quite popular. It uses a fixed table layout similar to x-Persistence. A new release is due at the end of next week, and it will feature the importing and exporting of xml schemas, and improved document searching capabilities.
