From: Richard Wordingham (email@example.com)
Date: Sun Jun 04 2006 - 04:18:00 CDT
Theodore H. Smith wrote on Saturday, June 03, 2006 at 12:56 PM
>>> And the other point is that a character (aka unicode glyph)
>> This is a misusage of the term "glyph" here, I believe.
Yes. In Unicode terms, 'glyph' refers to the character shape, and
generally non-semantically significant glyph differences within a language
are not encoded for the sake of that language. Furthermore, as I understand
it, general stylistic features, which convey information above the level of
letters, such as italics, are also not encoded. One generally italicises
words, rather than letters. The overlap of symbols and letters complicates
There is a feeling that the Unicode character encoding standard is being
converted to a glyph encoding standard.
>> The semantics, which
>> you need to access tables for, inhere to the code points, so
>> you can't just treat a UTF-8 string as a bag o' bytes for
>> processing. <Counterargument snipped> (Except for trival operations
>> like string copying,
>> string length for buffer size, and so on.)
> But I already said I have Unicode correct upper casing and lowercasing
> code on UTF-8.
> What if I compile my source code and put it on my server host, to do
> uppercasing and lowercasing of UTF-8? And then post the address here. I'm
> no web monkey, more of a desktop developer, but I can probably handle an
> uppercase and lowercase button and a text field :)
Unnecessary. Just sketch the solutions.
> Would that prove to you that you can do uppercasing and lowercasing on
> UTF-8 without worrying about the codepoints?
Here's a test case -
U+1FA6 GREEK SMALL LETTER OMEGA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI
U+1FA6 decomposes to <U+03C9, U+0313, U+0342, U+0345> (combining classes 0,
230, 230 and 240 respectively).
How do you, Theodore Smith, go about converting <U+0369, U+0345, U+0313,
U+0342> to upper case (and not title case)?
The correct upper case form (see
http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt ) has three
canonically equivalent encodings:
<U+1F6E GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI, U+0399 GREEK
CAPITAL LETTER IOTA>
<U+1F68, U+0342, U+0399>
<U+03A9, U+0313, U+0342, U+0399>
Aside: What is the correct upper case form of <U+03B1, U+033D, U+0345> and
U+03B1, U+0345, U+033D>? Is it truly <U+0391, U+033D, U+0399>? I suspect
it depends on the semantics being applied to U+033D COMBINING X ABOVE.
Conversion to normal form D sounds rather brute force. By my calculation,
for Unicode 4.1 you have 55,903 pairs of characters to swap round, composed
from the 384 characters not of combining class 0.
Normal Form C is even worse for brute force. Just to compose U+1FB3 GREEK
SMALL LETTER ALPHA WITH YPOGEGRAMMENI you have to have 384-8 = 376 3-element
substitutions, such as <U+03B1, U+033D, U+0345> to <U+1FB3, U+033D>, 376 *
376 = 141,376 4-element substitutions,... (It has been suggested that it is
unreasonable to ask for sequences of more than 30 combining characters to be
This archive was generated by hypermail 2.1.5 : Sun Jun 04 2006 - 04:29:43 CDT