Re: Unicode lexer

From: Hans Aberg (
Date: Sat Apr 16 2005 - 16:57:55 CST

  • Next message: Edward H. Trager: "Re: PHP (Re: wiki?)"

    At 15:00 -0700 2005/04/16, Tex Texin wrote:
    >I am looking for a lexer that is unicode based. Any suggestions?
    >Preferably open source, but I'll compile all suggestions and post it
    >back to the list.

    I posted in the Flex list
         List-Archive: <>
    some Haskell code that admits one to generate Flex-like regular
    expressions from Unicode character number classes, in a way that the
    generated lexer parses your choice of UTF-8 or UTF-32 (big or little
    endian). So you might be able to use Flex or some similar lexer
    generator by entering those regular expressions by hand into the
    lexer source file. Feedback welcome.

       Hans Aberg

    This archive was generated by hypermail 2.1.5 : Sat Apr 16 2005 - 17:02:47 CST