Re: Non-ascii string processing?

From: Peter Kirk (peterkirk@qaya.org)
Date: Tue Oct 07 2003 - 05:20:27 CST


On 07/10/2003 02:35, Jill Ramonsky wrote:

>
> Knowing the number of characters won't help you one iota. What you
> need to know here is the number of default grapheme clusters.
> I still have yet to hear a useful purpose for counting the number of
> /characters/.
>
> Jill
>
Suppose I have a UTF-8 string and want to know how many default grapheme
clusters it contains. How do I do so? Well, I step through the string
character by character, combining successive characters into grapheme
clusters. To do this without having to decode the UTF-8 myself, I need
to be able to get at the string character by character, and very likely
use a loop based on the number of characters in the string, e.g. the
following Basic (horrid language but good for making my point here):

For i% = 1 to Len(utf8string$)
    c$ = Mid(utf8string$, i%, 1)
    Process c$
Next i%

Such a loop would be more efficient in UTF-32 of course, but this is
still a real need for working with character counts.

-- 
Peter Kirk
peter@qaya.org (personal)
peterkirk@qaya.org (work)
http://www.qaya.org/


This archive was generated by hypermail 2.1.5 : Thu Jan 18 2007 - 15:54:24 CST