• From: Michael Kay <mike@s...>
• To: xml-dev@l...
• Date: Fri, 04 Jan 2013 23:51:55 +0000

 >According to the book, Fonts & Encodings (p. 61, first paragraph): ... 
we select a substring that begins with a combining character, this new 
string will not be a valid string in Unicode.

Does the book say where it gets its definition of "valid string" from? 
I'm not aware that Unicode itself defines this term. In most definitions 
of string, a string is simply a sequence of characters, and there are no 
constraints on the characters that can be included.

W3C defined a concept of "fully normalized strings" [1], which must not 
begin with combining characters. But most libraries of operations on 
strings (and in particular, the XPath function library) make no attempt 
to ensure that all operations produce fully normalized strings; if they 
tried to do this, the resulting operations would be a lot more complex 
and a lot more expensive.

Of course, the concepts of characters and strings are very fuzzy around 
the edges. How many characters are there in the "ffi" ligature? Because 
the concepts are fuzzy, there have been debates for decades about 
whether an accented character is one character or two. Unicode, because 
it was the successful result of some major political compromises, says 
it can be either (there is a composed representation as one character, 
and a decomposed representation as two). Both forms have practical 
advantages and disadvantages; you have highlighted one of the well-known 
disadvantages of the "decomposed" representation.

Michael Kay
Saxonica

[1] http://www.w3.org/TR/charmod-norm/#sec-FullyNormalized

• References:
  • An element's value is an invalid Unicode string ... how can it bewell-formed?
    • From: "Costello, Roger L." <costello@m...>