Target Audience: Software Engineer
Level of Session: Intermediate, Advanced
For many operations on text, knowing the language of the text can improve results. Invoking the right spell checker, language specific layout module, or text-to-speech module are examples of this. Markup languages like HTML and rich text formats like RTF provide means to 'tag' all parts of the text with a language designator. These features are not always used, and other data formats such as plain text, don't provide information on the language. When it would be useful to know the language of a piece of text, but language tags are not available, the language of the text can be guessed with surprising accuracy from even short runs of text. There are several common techniques that can be used to guess language.
|When the world wants to talk, it speaks Unicode|
International Unicode Conferences are organized by Global Meeting Services, Inc., (GMS).
GMS is pleased to be able to offer the International Unicode Conferences under an exclusive
license granted by the Unicode Consortium. All responsibility for conference finances and
operations is borne by GMS. The independent conference board serves solely at the pleasure
of GMS and is composed of volunteers active in Unicode and in international software
development. All inquiries regarding International Unicode Conferences should be addressed
Unicode and the Unicode logo are registered trademarks of Unicode, Inc. Used with permission.
5 July 1999, Webmaster