Re: 'code unit' and 'code point' meaning check

From: Philippe Verdy (verdy_p@wanadoo.fr)
Date: Thu May 15 2003 - 13:56:44 EDT

  • Next message: Michael \(michka\) Kaplan: "Re: 'code unit' and 'code point' meaning check"

    From: "Otto Stolz" <Otto.Stolz@uni-konstanz.de>
    > Ben Dougall wrote:
    >
    > > <guess> that area is full of surrogates. so they need another code point
    > > to make up a single character. on their own 0xd800-0xdfff are 1/2
    > > characters :) </guess>
    >
    > Oh, no! Again, you are confusing code-points and code-units
    > (in other words: Unicode and its UTFs).
    >[snip]
    > In a nutshell: Unicode is not UTF-16.

    Or in other words, Unicode defines *code points* only, not code units (this is left to specific encodings used to serialize it, including UTF-*, and "compressed" BOCU and CESU encodings, which can all be computed algorithmically from Unicode code points).

    Note that some UTF-* encodings are now described by Unicode.org as standards, but is technically an annex to the standard, and not necessary to its definition.



    This archive was generated by hypermail 2.1.5 : Thu May 15 2003 - 14:59:41 EDT