Subject: Re: RE: numerical inconsistencies doing dec to hex conv.
From: Peter Doggett <ee99ppd@xxxxxxxxxxxx>
Date: Mon, 10 Mar 2003 15:05:16 +0000 (GMT)
|
Hi,
It is a linux version of Xalan that uses C++, not sure what
version it is though (im at uni at the mo it is installed t
home ).
Thanks everyone for the sanity check!
Peter.
On Mon, 10 Mar 2003 12:59:30 -0000 Andrew Welch
<AWelch@xxxxxxxxxxxxxxx> wrote:
>
> Works fine for me with msxml3, 4, saxon 6.5.2, 7.2, and xalan 2.4.1
>
> I know there are issues with rounding .5's, but I think floor() is constant.
>
> Which processor are you using??
>
> cheers
> andrew
>
> > -----Original Message-----
> > From: Peter Doggett [mailto:ee99ppd@xxxxxxxxxxxx]
> > Sent: 10 March 2003 12:05
> > To: XSL-List@xxxxxxxxxxxxxxxxxxxxxx
> > Subject: numerical inconsistencies doing dec to hex conv.
> >
> >
> > Hi,
> >
> > I am using the below template to convert a decimal number
> > into a hex number. It seems to work correctly apart from
> > when I want to calculate the hex equivelent of the decimal
> > number - 4294967295. This number is of particular
> > significance as in Hex this should be FFFFFFFF (a 32 bit,
> > bitmask; i am using xml to describe registers in h/w).
> > The template outputs FFFFFFF0 abnd I dont know why.
> >
> > The template seems to output the incorrect value for
> > 1099511627775, which is the decimal representation of
> > FFFFFFFFFF in hex.
> >
> > The recursive template is as follows:
> >
> >
> > <xsl:variable name="hexDigits" select="'0123456789ABCDEF'" />
> > <xsl:template name="toHex">
> > <xsl:param name="decimalNumber" />
> > <xsl:if test="$decimalNumber >= 16">
> > <xsl:call-template name="toHex">
> > <xsl:with-param name="decimalNumber"
> > select="floor($decimalNumber div 16)" />
> > </xsl:call-template>
> > </xsl:if>
> > <xsl:value-of select="substring($hexDigits,
> > ($decimalNumber mod 16) + 1, 1)" />
> > </xsl:template>
> >
> > Can anybody figure out why this is happening or have an
> > equivelent template that can calculate the conversion
> > correctly?
> >
> >
> > Thankyou for your time.
> >
> >
> > ----------------------
> > Peter Doggett
> > ee99ppd@xxxxxxxxxxxx
> >
> >
> > XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
> >
> >
> >
> >
> >
> > ---
> > Incoming mail is certified Virus Free.
> > Checked by AVG anti-virus system (http://www.grisoft.com).
> > Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
> >
> >
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
>
>
> XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
>
----------------------
Peter Doggett
ee99ppd@xxxxxxxxxxxx
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
|