Sorry for the noise. I got it working right after posting for help.
The solution was to swap the quotes around in the statement, which I thought
I'd already tried, but it worked this time.

Eg:

<xsl:variable name="newString" select='replace($oldString, "&apos;" ,
"XXX")'/>

Thanks,
  Craig