Re: Unicode Collation Algorithm

From: Daniel Ehrenberg (microdan@gmail.com)
Date: Fri May 16 2008 - 01:57:38 CDT

  • Next message: Jukka K. Korpela: "Re: Exemplifying apostrophes"

    Oops, thank you! I missed that.

    On Fri, May 16, 2008 at 1:52 AM, Åke Persson <ake.persson@mimer.se> wrote:
    >
    >
    >
    > Daniel Ehrenberg wrote:
    >
    >
    > > I'm trying to implement the Unicode Collation Algorithm, and I'm a
    > > little confused by line 36099 of CollationTest_SHIFTED.txt. It is:
    > >
    > > 006C 00B7 0021; # (l·) LATIN SMALL LETTER L, MIDDLE DOT [1262 | 0020
    > > 01AF | 0002 0002 | FFFF FFFF 0258]
    > >
    > > Here are the collation keys for the characters that it uses:
    > >
    > > 006C ; [.1262.0020.0002.006C] # LATIN SMALL LETTER L
    > > 00B7 ; [*0279.0020.0002.00B7] # MIDDLE DOT
    > > 0021 ; [*0258.0020.0002.0021] # EXCLAMATION MARK
    > >
    > > All elements have combining class 0 and the string is already in NFD.
    >
    > > The asterisks indicate that an element is variable-weighted. Why,
    > > then, in the key given, is U+00B7 treated as if it is not
    > > variable-weighted? I'm treating variable weighted elements as shifted,
    > > not non-ignorable, and as far as I can tell there's no way for a
    > > variable-weighted element to not get shifted based on the context. So,
    > > by my calculations, the actual collation key should be [1262 | 0020 |
    > > 0002 | FFFF 0279 0258]. This would make it precede the previous line
    > > in sort order. Could somebody help me figure this out?
    >
    >
    >
    >
    > The following lines from allkeys-5.1.0.txt is the answer to your confusion:
    >
    > 0140 ; [.24C4.0020.0002.0140][.0000.01AF.0002.0140] # LATIN SMALL LETTER L
    > WITH MIDDLE DOT; QQKL
    >
    > 006C 00B7 ; [.24C4.0020.0002.0140][.0000.01AF.0002.0140] # LATIN SMALL
    > LETTER L WITH MIDDLE DOT
    >
    >
    >
    > Surprisingly, U+00B7 is always treated as an accent when preceded by LETTER
    > L, a behaviour that belongs to a tailoring for Catalan.
    >
    >
    >
    > The expectation was something like the following line from
    > allkeys-4.0.0.txt:
    >
    > 0140 ; [.1E5C.0020.0004.0140][*0167.0020.0004.0140] # LATIN SMALL LETTER L
    > WITH MIDDLE DOT; QQKN
    >
    >
    >
    > Kind regards,
    >
    > Åke Persson
    >
    >

    This archive was generated by hypermail 2.1.5 : Fri May 16 2008 - 02:02:53 CDT