Subject: RE: numerical inconsistencies doing dec to hex conv.
From: "Andrew Welch" <AWelch@xxxxxxxxxxxxxxx>
Date: Mon, 10 Mar 2003 12:59:30 -0000
|
Works fine for me with msxml3, 4, saxon 6.5.2, 7.2, and xalan 2.4.1
I know there are issues with rounding .5's, but I think floor() is constant.
Which processor are you using??
cheers
andrew
> -----Original Message-----
> From: Peter Doggett [mailto:ee99ppd@xxxxxxxxxxxx]
> Sent: 10 March 2003 12:05
> To: XSL-List@xxxxxxxxxxxxxxxxxxxxxx
> Subject: numerical inconsistencies doing dec to hex conv.
>
>
> Hi,
>
> I am using the below template to convert a decimal number
> into a hex number. It seems to work correctly apart from
> when I want to calculate the hex equivelent of the decimal
> number - 4294967295. This number is of particular
> significance as in Hex this should be FFFFFFFF (a 32 bit,
> bitmask; i am using xml to describe registers in h/w).
> The template outputs FFFFFFF0 abnd I dont know why.
>
> The template seems to output the incorrect value for
> 1099511627775, which is the decimal representation of
> FFFFFFFFFF in hex.
>
> The recursive template is as follows:
>
>
> <xsl:variable name="hexDigits" select="'0123456789ABCDEF'" />
> <xsl:template name="toHex">
> <xsl:param name="decimalNumber" />
> <xsl:if test="$decimalNumber >= 16">
> <xsl:call-template name="toHex">
> <xsl:with-param name="decimalNumber"
> select="floor($decimalNumber div 16)" />
> </xsl:call-template>
> </xsl:if>
> <xsl:value-of select="substring($hexDigits,
> ($decimalNumber mod 16) + 1, 1)" />
> </xsl:template>
>
> Can anybody figure out why this is happening or have an
> equivelent template that can calculate the conversion
> correctly?
>
>
> Thankyou for your time.
>
>
> ----------------------
> Peter Doggett
> ee99ppd@xxxxxxxxxxxx
>
>
> XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
>
>
>
>
>
> ---
> Incoming mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
>
>
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.458 / Virus Database: 257 - Release Date: 24/02/2003
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
|