Maybe I'm being really dumb here, but what is the difference between a
decimal number and a floating point number?


-----Original Message-----
From: Jeff Kenton [mailto:jkenton@xxxxxxxxxxxxx] 
Sent: 29 March 2004 14:53
To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
Subject: Re:  Re: XSLT 2.0 Decimal number silliness


Pete,

You missed the point.  We're talking about decimal numbers here, not floats.

--jeff


Pete Forman wrote:
> Jeff Kenton <jkenton@xxxxxxxxxxxxx> writes:
> 
>  > I've been looking at XSLT 2.0 decimal numbers (again), and I still
>  > think the WC needs to do better.
>  > [...]
>  > <xsl:value-of select="2.0 div 3.0" />~
>  > <xsl:value-of select="0.444 div 0.666" />~
>  > [...]
>  > Both lines of output should be equal.
> 
> Why?  The majority of implementations will use a binary format such as
> IEC 60559 to hold floating point numbers.  2.0 and 3.0 can be
> converted exactly but the representations of 0.444 and 0.666 will be
> inexact (0.44400000000000001 and 0.66600000000000004 in my test).  You
> might get a better match with 0.5 div 0.75.  If you want fixed point
> number semantics try moving the decimal points to the right before
> dividing.
> 
> Floating point arithmetic is inexact by definition.  Your results
> _are_ the same to within the precision specified by the language
> and/or implementation.