US7ASCII, UTF8 and ZHT16BIG5 in database configurations

From: Magda Danish (Unicode) (v-magdad@microsoft.com)
Date: Thu Oct 24 2002 - 13:29:08 EDT

  • Next message: Yung-Fong Tang: "need open source tools to convert indic font encoding into ISCII or Unicode"

    Ronald,

    I am forwarding your question to the Unicode list. If you're not
    subscribed to the list please go to
    http://www.unicode.org/unicode/consortium/distlist.html#4 and
    self-subscribe so you can follow the thread.

    Regards,

    Magda Danish
    Administrative Director
    The Unicode Consortium
    650-693-3921

    > -----Original Message-----
    > Date/Time: Thu Oct 24 05:11:30 EDT 2002
    > Contact: ronald.to@hthk.com
    > Report Type: Other Question, Problem, or Feedback
    >
    > Hi,
    >
    > This is Ronald TO working on a data warehouse project.
    > Following is a problem that we have.
    >
    > Source systems are using different character sets for their
    > database configurations. They are namely, US7ASCII, UTF8 and
    > ZHT16BIG5, so far! The target repository is tentatively UTF8.
    > We are using a ETL tool called DataStage.
    >
    > A simple test has been performed between UTF8 and ZHT16BIG5.
    > I guess you knew the result already, Chinese characters that
    > move across are wrongly recognized when transfer from UTF8 to
    > ZHT16BIG5 and length error if transfer from ZHT16BIG5 to
    > UTF8. English characters transfer produce no error in either
    > directions.
    >
    > The immediate solution is of course, no Chinese character
    > data. But then source system need not configure their
    > database with ZHT16BIG5. Alternatively, use ZHT16BIG5 as the
    > configuration of the data warehouse. Again system that rely
    > on output from the data warehouse has to be ZHT16BIG5 as
    > well. A simple rule of thumb: given the data warehouse is
    > ZHT16BIG5, source system can be either ZHT16BIG5, UTF8 or
    > perhaps US7ASCII and systems that rely on output from data
    > warehouse has to be ZHT16BIG5.
    >
    > Please enlighten.
    >
    > Regards,
    > Ronald TO
    >
    >
    > -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
    > (End of Report)
    >
    >



    This archive was generated by hypermail 2.1.5 : Thu Oct 24 2002 - 15:36:51 EDT