2009-01-26

How to export data to one big XML file


My company had to add extra export format for data in a system I'm working on. I get specification and XML schema and started analyzing it for best fast solution. Data was ready to export so there left decision about choosing right way to make XML. Deadline was imminent, and everybody was really busy at work. Making export file - was good task for our newest programmer. He never parsed/created XML using SAX/DOM or other libraries, but was envy to use and learn one of the standard.

The main technological problem was the size of output XML. For biggest data case it would generate over 500MB XML with about 50000 nodes at highest level. It would be efficiently generated using only writing partially to stream technique like SAX. I have chose two options- use SAX or own similar simple API - realization time was crucial. Because XML structure was relatively easy - all tags without attributes, so we decided to use plain string buffer technique with addition of couple helper functions.

At the beginning I had objections about not using standard libraries that should help. But the implementation of XML format moved forwards, output file was easy to validate with XML schema. The most time was committed to handle non existing or incomplete data cases. The export function was ready before time. That is one of examples where using simplest tools, meets desired goal.

What are your choices in similar situation?

1 comment:

Anonymous said...

You may also want to look at vtd-xml,the latest and most advanced xml technology

http://vtd-xml.sf.net