Subject: Floating point precision error
From: "Thang Le" <tle@xxxxxxxxxxx>
Date: Wed, 25 Jan 2006 20:16:26 -0500
|
This is something I don't understand. Given this XSL snippet:
<xsl:value-of select="$nodes[1]/unitamount" />_
<xsl:value-of select="$nodes[1]/quantity" />_
<xsl:value-of select="($nodes[1]/quantity * $nodes[1]/unitamount)" />_
I'm getting the printout as:
6.4_
3000_
19199.908447265625_
The literal values in XML is 6.4 and 3000. If I round the result to 2
decimal places, I still get 19199.91, a far cry from what it should be
(19200).
I'm using dom4j 1.6, so I guess that's XSLT 1.0? Also this only happens
on SunOS and not on Windows. Still, the error is huge and obvious.
Isn't there a way to force the calculation to be more exact?
Thanks,
Thang Le
--
NeoMail - http://neomail.sourceforge.net
|
Cart
