Subject: RE: Performance tips to speed up multiple transforms
From: <vojtech.toman@xxxxxxx>
Date: Sun, 7 Nov 2010 10:34:15 -0500
|
> > I'm transforming around 500 individual XML log files - ~250MB worth
- into 4 output
> files. Each log file contributes to all 4 output files. At present,
I'm running a Saxon
> transform within VBScript with one XSLT file for each log file. It
processes around 2
> log files per second. I guess it's slow 'cos I'm shelling out to a
hidden command box
> every time I transform a log file(open a shell, start java, open the
Saxon .jar file, run
> the transform, append the 4 results to the 4 'master' output files,
close everything,
> repeat ). It works, but I guess it could be significantly more
efficient.
> >
> > What would the smart person be looking at to speed up the process?
Write the
> whole script in Java? .NET? C++... Or just make a Transform 'do' the
whole thing?
What you are describing sounds like a perfect use case for an XProc
pipeline (that would loop over the input XML documents, transform them,
and generate the output files.)
Regards,
Vojtech
--
Vojtech Toman
Consultant Software Engineer
EMC | Information Intelligence Group
vojtech.toman@xxxxxxx
http://developer.emc.com/xmltech
|