Umm, no, or rather most emphatically; NO!  XML is a poor mans graph at best. For flat data (which it sounds like this mostly is) XML makes even less sense.  But let's consider the more complex case: real graph traversal algorithms come pre-built for things like Neo4J and Titan and things like Gremlin beat the heck out of xPath, XSLT, xQuery, et al (and I'm an Apache Cocoon committer so I do believe in using those for the right problem!).  Titan wasn't considered possible when XSLT was first conceived, the state of the art has progressed considerably since then. Graph databases, Hadoop and it's related infrastructure aren't the flavor of the month and are not going anywhere.  They package up entire generations of Computer Sciences best practices into well thought out, incredibly powerful, easily deployable systems.  If Roger is truly asking about a big data problem then the fact that his data arrived in the form of XML should not influence his choice of tool chain.  Rather, he should be using the tools that are designed to deal with data volumes the size he mentions and solve the real problem, not just an intermediate step.

BTW, this arrived off list, feel free to put it back on list if you wish...

Peter Hunsberger

On Mon, Mar 9, 2015 at 6:36 PM, Ihe Onwuka <ihe.onwuka@g...> wrote:

On Mon, Mar 9, 2015 at 6:06 PM, Peter Hunsberger <peter.hunsberger@g...> wrote:

Yes, unless there is a need to forward on the XML to some other endpoint I can't really see why it would need to stay as XML?  

Because it's easy to get it out of XML into whatever shape or form your analytics idea of the day/week/month/epoch needs it?