RE: Roundtripping in Unicode

From: Mike Ayers (
Date: Wed Dec 15 2004 - 11:47:27 CST

  • Next message: Mark Davis: "Re: Roundtripping in Unicode"

    > From: Peter Kirk []
    > Sent: Wednesday, December 15, 2004 3:52 AM

    > But surely octets 0x80 to 0x9f are (at least mostly) invalid
    > in ISO 8859?

            They are in fact valid. However, because they are control
    characters, they are not considered displayable.

    > While some applications may choose to process
    > these invalid characters as if they were valid, but display
    > them as boxes or not at all (and this is a security risk),
    > others and especially those concerned with security do in
    > fact treat them as errors, in one way or another.
    > For example, Marcin noted for Mozilla:
    > >If a filename ... can be
    > >converted but contains characters like 0x80-0x9F in ISO-8859-2, they
    > >are displayed as question marks and the file is inaccessible.

            This is a good policy and is what Lars should consider. It places
    the responsibility for the filename where it belongs: on the file's creator.

    > It should be treated as a general issue with ALL locales and
    > character sets (with perhaps just a few exceptions) that not
    > all sequences of octets represent valid character strings.
    > UTF-8 is by no means a special case here.

            Exactly. Which underscores just how silly these threads are.


    "Tumbleweed E-mail Firewall <>" made the following
     annotations on 12/15/04 09:50:11
    This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

    This archive was generated by hypermail 2.1.5 : Wed Dec 15 2004 - 11:54:45 CST