UnicodeIUC22
Program Showcase Registration Accommodation Travel Sponsors
Unicode Standard Conference Board Conference CD Last Conference Past Conferences Next Conference
Abstract

Text Boundary Analysis in ICU

Andy Heninger - IBM Corporation

Intended Audience: Software Engineers
Session Level: Intermediate

Text boundary analysis provides the ability to locate significant boundaries within a region of text, such as boundaries between words or sentences, or candidate positions for line breaks when laying out text on a screen or page. The Unicode Consortium technical reports TR 14 and TR 29 provide basic definitions for identifying boundaries or break positions.

The ICU Unicode support library from IBM provides a very high performance and customizable implementation of boundary analysis. The rules for identifying break positions within the text are expressed in a form of regular expression, allowing for special purpose text breaking by means of rules specified by the application. ICU also includes a dictionary based word break for use with Thai. New boundary analysis features for ICU 2.2 include support for Unicode 3.2, user-customizable boundary rules in C/C++, and tagging identified break positions with additional information for use by the application.

This paper will review the basics of boundary analysis, and describe the features and implementation approach in ICU 2.2. Future directions, including possible new features for later versions of ICU, will also be discussed.


Unicode
When the world wants to talk, it speaks Unicode

UnicodeIUC22
Program Showcase Registration Accommodation Travel Sponsors
Unicode Standard Conference Board Conference CD Last Conference Past Conferences Next Conference
International Unicode Conferences are organized by Global Meeting Services, Inc., (GMS). GMS is pleased to be able to offer the International Unicode Conferences under an exclusive license granted by the Unicode Consortium. All responsibility for conference finances and operations is borne by GMS. The independent conference board serves solely at the pleasure of GMS and is composed of volunteers active in Unicode and in international software development. All inquiries regarding International Unicode Conferences should be addressed to info@global-conference.com.

Unicode and the Unicode logo are registered trademarks of Unicode, Inc. Used with permission.

23 May 2002, Webmaster