[Unicode]  Technical Reports
 

Unicode Technical Standard #35

Unicode Locale Data Markup Language (LDML)
Part 2: General

Version 24
Editors CLDR committee members
Date 2013-09-18
This Version http://www.unicode.org/reports/tr35/tr35-33/tr35.html
Previous Version http://www.unicode.org/reports/tr35/tr35-31/tr35.html
Latest Version http://www.unicode.org/reports/tr35/
Corrigenda http://unicode.org/cldr/corrigenda.html
Latest Proposed Update http://www.unicode.org/reports/tr35/proposed.html
Namespace http://cldr.unicode.org/
DTDs http://unicode.org/cldr/dtd/24/
Revision 33

Summary

This document describes parts of an XML format (vocabulary) for the exchange of structured locale data. This format is used in the Unicode Common Locale Data Repository.

This is a partial document, describing general parts of the LDML: display names & transforms, etc. For the other parts of the LDML see the main LDML document and the links above.

Status

This document has been reviewed by Unicode members and other interested parties, and has been approved for publication by the Unicode Consortium. This is a stable document and may be used as reference material or cited as a normative reference by other specifications.

A Unicode Technical Standard (UTS) is an independent specification. Conformance to the Unicode Standard does not imply conformance to any UTS.

Please submit corrigenda and other comments with the CLDR bug reporting form [Bugs]. Related information that is useful in understanding this document is found in the References. For the latest version of the Unicode Standard see [Unicode]. For a list of current Unicode Technical Reports see [Reports]. For more information about versions of the Unicode Standard, see [Versions].

Parts

The LDML specification is divided into the following parts:

Contents of Part 2, General

1 Display Name Elements

Display names for scripts, languages, countries, currencies, and variants in this locale are supplied by this element. They supply localized names for these items for use in user-interfaces for various purposes such as displaying menu lists, displaying a language name in a dialog, and so on. Capitalization should follow the conventions used in the middle of running text; the <contextTransforms> element may be used to specify the appropriate capitalization for other contexts (see Section 12 ContextTransform Elements ). Examples are given below.

Note: The "en" locale may contain translated names for deprecated codes for debugging purposes. Translation of deprecated codes into other languages is discouraged.

Where present, the display names must be unique; that is, two distinct code would not get the same display name. (There is one exception to this: in time zones, where parsing results would give the same GMT offset, the standard and daylight display names can be the same across different time zone IDs.)

Any translations should follow customary practice for the locale in question. For more information, see [Data Formats].

<localeDisplayPattern>

<!ELEMENT localeDisplayPattern ( alias | (localePattern*, localeSeparator*, localeKeyTypePattern*, special*) ) >

For compound language (locale) IDs such as "pt_BR" which contain additional subtags beyond the initial language code: When the <languages> data does not explicitly specify a display name such as "Brazilian Portuguese" for a given compound language ID, "Portuguese (Brazil)" from the display names of the subtags.

It includes three sub-elements:

For example, for the locale identifier zh_Hant_CN_co_pinyin_cu_USD, the display would be "Chinese (Traditional, China, Pinyin Sort Order, Currency: USD)". The key-type for co_pinyin doesn't use the localeKeyTypePattern because there is a translation for the key-type in English:

<type type="pinyin" key="collation">Pinyin Sort Order</type>

<languages>

This contains a list of elements that provide the user-translated names for language codes, as described in Section 3, Unicode Language and Locale Identifiers.

<language type="ab">Abkhazian</language>
<language type="aa">Afar</language>
<language type="af">Afrikaans</language>
<language type="sq">Albanian</language>

The type can actually be any locale ID as specified above. The set of which locale IDs is not fixed, and depends on the locale. For example, in one language one could translate the following locale IDs, and in another, fall back on the normal composition.

type translation composition
nl_BE Flemish Dutch (Belgium)
zh_Hans Simplified Chinese Chinese (Simplified)
en_GB British English English (United Kingdom)

Thus when a complete locale ID is formed by composition, the longest match in the language type is used, and the remaining fields (if any) added using composition.

Alternate short forms may be provided for some languages (and for territories and other display names), for example.

<language type="az">Azerbaijani</language>
<language type="az" alt="short">Azeri</language>
<language type="en_GB">British English</language>
<language type="en_GB" alt="short">U.K. English</language>
<language type="en_US">American English</language>
<language type="en_US" alt="short">U.S. English</language>

<scripts>

This element can contain an number of script elements. Each script element provides the localized name for a script code, as described in Section 3, Unicode Language and Locale Identifiers (see also UAX #24: Script Names [UAX24]). For example, in the language of this locale, the name for the Latin script might be "Romana", and for the Cyrillic script is "Kyrillica". That would be expressed with the following.

<script type="Latn">Romana</script>
<script type="Cyrl">Kyrillica</script>

The script names are most commonly used in conjunction with a language name, using the <localePattern> combining pattern, and the default form of the script name should be suitable for such use. When a script name requires a different form for stand-alone use, this can be specified using the "stand-alone" alternate:

<script type="Hans">Simplified</script>
<script type="Hans" alt="stand-alone">Simplified Han</script>
<script type="Hant">Traditional</script>
<script type="Hant" alt="stand-alone">Traditional Han</script>

This will produce results such as the following:

<territories>

This contains a list of elements that provide the user-translated names for territory codes, as described in Section 3, Unicode Language and Locale Identifiers.

<territory type="AD">Andorra</territory>
<territory type="AF">Afghanistan</territory>
<territory type="AL">Albania</territory>
<territory type="AO">Angola</territory>
<territory type="DZ">Algeria</territory>
<territory type="GB">United Kingdom</territory>
<territory type="GB" alt="short">U.K.</territory>
<territory type="US">United States</territory>
<territory type="US" alt="short">U.S.</territory>

<variants>

This contains a list of elements that provide the user-translated names for the variant_code values described in Section 3, Unicode Language and Locale Identifiers.

<variant type="nynorsk">Nynorsk</variant>

<keys>

This contains a list of elements that provide the user-translated names for the key values described in Section 3, Unicode Language and Locale Identifiers.

<key type="collation">Sortierung</key>

<types>

This contains a list of elements that provide the user-translated names  for the type values described in Section 3, Unicode Language and Locale Identifiers. Since the translation of an option name may depend on the key it is used with, the latter is optionally supplied.

<type type="phonebook" key="collation">Telefonbuch</type>

<measurementSystemNames>

This contains a list of elements that provide the user-translated names for systems of measurement. The types currently supported are "US", "metric", and "UK".

<measurementSystemName type="US">U.S.</type>

Note: In the future, we may need to add display names for the particular measurement units (millimeter versus millimetre versus whatever the Greek, Russian, etc are), and a message format for positioning those with respect to numbers. For example, "{number} {unitName}" in some languages, but "{unitName} {number}" in others.

<transformNames>

This contains a list of elements that provide the user-translated names for transforms that are not script or locale-based, such as FULLWIDTH.

<transformName type="Numeric">Numeric</type>

<codePatterns>

<codePattern type="language">Language: {0}</type>

2 Layout Elements

<!ELEMENT layout ( alias | (orientation*, inList*, inText*, special*) ) >

This top-level element specifies general layout features. It currently only has one possible element (other than <special>, which is always permitted).

<!ELEMENT orientation ( characterOrder*, lineOrder*, special* ) >
<!ELEMENT characterOrder ( #PCDATA ) >
<!ELEMENT lineOrder ( #PCDATA ) >

The lineOrder and characterOrder elements specify the default general ordering of lines within a page, and characters within a line. The possible values are:

Direction Value
Vertical top-to-bottom
bottom-to-top
Horizontal left-to-right
right-to-left

If the value of lineOrder is one of the vertical values, then the value of characterOrder must be one of the horizontal values, and vice versa. For example, for English the lines are top-to-bottom, and the characters are left-to-right. For Mongolian (in the Mongolian Script) the lines are right-to-left, and the characters are top to bottom. This does not override the ordering behavior of bidirectional text; it does, however, supply the paragraph direction for that text (for more information, see UAX #9: The Bidirectional Algorithm [UAX9]).

For dates, times, and other data to appear in the right order, the display for them should be set to the orientation of the locale.

<inList> (deprecated)

The <inList> element is deprecated and has been superseded by the <contextTransforms> element; see Section 12 ContextTransform Elements.

This element controls whether display names (language, territory, etc) are title cased in GUI menu lists and the like. It is only used in languages where the normal display is lower case, but title case is used in lists. There are two options:

<inList casing="titlecase-words">
<inList casing="titlecase-firstword">

In both cases, the title case operation is the default title case function defined by Chapter 3 of [Unicode] . In the second case, only the first word (using the word boundaries for that locale) will be title cased. The results can be fine-tuned by using alt="list" on any element where titlecasing as defined by the Unicode Standard will produce the wrong value. For example, suppose that "turc de Crimée" is a value, and the title case should be "Turc de Crimée". Then that can be expressed using the alt="list" value.

<inText> (deprecated)

The <inList> element is deprecated and has been superseded by the <contextTransforms> element; see Section 12 ContextTransform Elements.

This element indicates the casing of the data in the category identified by the inText type attribute, when that data is written in text or how it would appear in a dictionary. For example :

<inText type="languages">lowercase-words</inText>

indicates that language names embedded in text are normally written in lower case. The possible values and their meanings are :

3 Character Elements

<!ELEMENT characters (alias | (exemplarCharacters*, ellipsis*, moreInformation*, stopwords*, indexLabels*, mapping*, special*)) >

The <characters> element provides optional information about characters that are in common use in the locale, and information that can be helpful in picking resources or data appropriate for the locale, such as when choosing among character encodings that are typically used to transmit data in the language of the locale. It may also be used to help reduce confusability issues: see [UTR39]. It typically only occurs in a language locale, not in a language/territory locale. The stopwords are an experimental feature, and should not be used.

<exemplarCharacters>[a-zåæø]</exemplarCharacters>

The basic exemplar character sets (main and auxiliary) contain the commonly used letters for a given modern form of a language, which can be for testing and for determining the appropriate repertoire of letters for charset conversion or collation. ("Letter" is interpreted broadly, as anything having the property Alphabetic in the [UAX44], which also includes syllabaries and ideographs.) It is not a complete set of letters used for a language, nor should it be considered to apply to multiple languages in a particular country. Punctuation and other symbols should not be included in the main and auxiliary sets. In particular, format characters like CGJ are not included.

There are four sets altogether: main, auxiliary, punctuation, and index. The main set should contain the minimal set required for users of the language, while the auxiliary exemplar set is designed to encompass additional characters: those non-native or historical characters that would customarily occur in common publications, dictionaries, and so on. Major style guidelines are good references for the auxiliary set. So, for example, if Irish newspapers and magazines would commonly have Danish names using å, for example, then it would be appropriate to include å in the auxiliary exemplar characters; just not in the main exemplar set. Thus English has the following:

<exemplarCharacters>[a b c d e f g h i j k l m n o p q r s t u v w x y z]</exemplarCharacters>
<exemplarCharacters type="auxiliary">[á à ă â å ä ã ā æ ç é è ĕ ê ë ē í ì ĭ î ï ī ñ ó ò ŏ ô ö ø ō œ ú ù ŭ û ü ū ÿ]</exemplarCharacters>

For a given language, there are a few factors that help for determining whether a character belongs in the auxiliary set, instead of the main set:

For example, the exemplar character set for en (English) is the set [a-z]. This set does not contain the accented letters that are sometimes seen in words like "résumé" or "naïve", because it is acceptable in common practice to spell those words without the accents. The exemplar character set for fr (French), on the other hand, must contain those characters: [a-z é è ù ç à â ê î ô û æ œ ë ï ÿ]. The main set typically includes those letters commonly "alphabet".

The punctuation set consists of common punctuation characters that are used with the language (corresponding to main and auxiliary). Symbols may also be included where they are common in plain text, such as ©. It does not include characters with narrow technical usage, such as dictionary punctuation/symbols or copy-edit symbols. For example, English would have something like the following:

- ‐ – —
, ; : ! ? . …
' ‘ ’ " “ ” ′ ″
( ) [ ] { } ⟨ ⟩
© ® ™ @ & ° ‧ ·/ # % ¶ § * † ‡
+ − ± × ÷ < ≤ = ≅ ≥ > √

When determining the character repertoire needed to support a language, a reasonable initial set would include at least the characters in the main and punctuation exemplar sets, along with the digits and common symbols associated with the numberSystems supported for the locale (see Numbering Systems).

The index characters are a set of characters for use as a UI "index", that is, a list of clickable characters (or character sequences) that allow the user to see a segment of a larger "target" list. For details see the Unicode LDML: Collation document. The index set may only contain characters whose lowercase versions are in the main and auxiliary exemplar sets, though for cased languages the index exemplars are typically in uppercase. Characters from the auxiliary exemplar set may be necessary in the index set if it needs to properly handle items such as names which may require characters not included in the main exemplar set.

Here is a sample of the XML structure:

<exemplarCharacters type="index">[A B C D E F G H I J K L M N O P Q R S T U V W X Y Z]</exemplarCharacters>

The display of the index characters can be modified with the Index labels elements, discussed in Section 5.6.4.

3.1 Exemplar Syntax

In all of the exemplar characters, the list of characters is in the Unicode Set format, which allows boolean combinations of sets of letters, including those specified

Sequences of characters that act like a single letter in the language — especially in collation — are included within braces, such as [a-z á é í ó ú ö ü ő ű {cs} {dz} {dzs} {gy} ...]. The characters should be in normalized form (NFC). Where combining marks are used generatively, and apply to a large number of base characters (such as in Indic scripts), the individual combining marks should be included. Where they are used with only a few base characters, the specific combinations should be included. Wherever there is not a precomposed character (for example, single codepoint) for a given combination, that must be included within braces. For example, to include sequences from the Where is my Character? page on the Unicode site, one would write: [{ch} {tʰ} {x̣} {ƛ̓} {ą́} {i̇́} {ト゚}], but for French one would just write [a-z é è ù ...]. When in doubt use braces, since it does no harm to include them around single code points: for example, [a-z {é} {è} {ù} ...].

If the letter 'z' were only ever used in the combination 'tz', then we might have [a-y {tz}] in the main set. (The language would probably have plain 'z' in the auxiliary set, for use in foreign words.) If combining characters can be used productively in combination with a large number of others (such as say Indic matras), then they are not listed in all the possible combinations, but separately, such as:

[‌ ‍ ॐ ०-९ ऄ-ऋ ॠ ऌ ॡ ऍ-क क़ ख ख़ ग ग़ घ-ज ज़ झ-ड ड़ ढ ढ़ ण-फ फ़ ब-य य़ र-ह ़ ँ-ः ॑-॔ ऽ ् ॽ ा-ॄ ॢ ॣ ॅ-ौ]

The exemplar character set for Han characters is composed somewhat differently. It is even harder to draw a clear line for Han characters, since usage is more like a frequency curve that slowly trails off to the right in terms of decreasing frequency. So for this case, the exemplar characters simply contain a set of reasonably frequent characters for the language.

The ordering of the characters in the set is irrelevant, but for readability in the XML file the characters should be in sorted order according to the locale's conventions. The main and auxiliary sets should only contain lower case characters (except for the special case of Turkish and similar languages, where the dotted capital I should be included); the upper case letters are to be mechanically added when the set is used. For more information on casing, see the discussion of Special Casing in the Unicode Character Database.

Restrictions

  1. The main, auxiliary and index sets are normally restricted to those letters with a specific Script character property (that is, not the values Common or Inherited) or required Default_Ignorable_Code_Point characters (such as a non-joiner), or combining marks, or the Word_Break properties Katakana, ALetter, or MidLetter.
  2. The auxiliary set should not overlap with the main set. There is one exception to this: Hangul Syllables and CJK Ideographs can overlap between the sets.
  3. Any Default_Ignorable_Code_Points should be in the auxiliary set , or, if they are only needed for currency formatting, in the currency set. These can include characters such as U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK which may be needed in bidirectional text in order for date, currency or other formats to display correctly.

3.2 Mapping

<mapping registry="iana" type="iso-2022-jp utf-8" alt="email"/>

The mapping element describes character conversion mapping tables that are commonly used to encode data in the language of this locale for a particular purpose. Each encoding is identified by a name from the specified registry. If more than one encoding is used for a particular purpose, the encodings are listed in the type attribute in order, from most preferred to least. An alt tag is used to indicate the purpose ("email" or "www" being the most frequent); if it is absent, then the encoding(s) may be used for all purposes not explicitly specified.

Each locale may have at most one mapping element tagged with a particular purpose, and at most one general-purpose mapping element. Inheritance is on an element basis; an element in a sub-locale overrides an inherited element with the same purpose.

For email usage (alt="email") the list begins with encodings that should be tried for outgoing mail; these encodings should be tried in order until one is found that can represent the message text. Typically, this section of the encoding list terminates with encoding "utf-8", which can represent any message text. Any encodings listed after "utf-8" may be encountered in incoming messages (along with the encodings in the first section) and should be handled for incoming messages, but should not be used for outgoing messages.

Currently the only registry that can be used is "iana", which specifies use of  an IANA name.

Note: While IANA names are not precise for conversion (see UTS #22: Character Mapping Tables [UTS22]), they are sufficient for this purpose.

3.3 Index Labels

The elements described in this section are not currently used in CLDR data nor in CLDR implementations. These elements should be considered as "beta" and implementers should not depend on them yet.

<!ELEMENT indexLabels (indexSeparator*, compressedIndexSeparator*, indexRangePattern*, indexLabelBefore*, indexLabelAfter*, indexLabel*) >

<!ELEMENT indexSeparator ( #PCDATA ) >

<!ELEMENT compressedIndexSeparator ( #PCDATA ) >

<!ELEMENT indexRangePattern ( #PCDATA ) >

<!ELEMENT indexLabelBefore ( #PCDATA ) >

<!ELEMENT indexLabelAfter ( #PCDATA ) >

<!ELEMENT indexLabel ( #PCDATA ) >
<!ATTLIST indexLabel indexSource CDATA #IMPLIED >
<!ATTLIST indexLabel priority ( 1 | 2 | 3 ) #IMPLIED >

The index label elements provide information for modifying the index exemplar characters in display. In particular, they are used to indicate how index exemplar characters can be compressed where screen real estate is limited. For example, A B C D E F G H I J K L M N O P Q R S T U V W X Y Z can be represented as A • E • I • N • S • Z.

The indexSeparator can be used to separate the index characters if they occur in free flowing text (instead of, say, on buttons or in cells). The default (root) is a space. Where the index is compressed (by omitting values — see the priority attribute below), the compressedIndexSeparator can be used instead.

The indexRangePattern is used for dynamic configuration. That is, if there are few items in X, Y, and Z, they can be grouped into a single bucket with <indexRangePattern>{0}-{1}</separator>, giving "X-Z". The indexLabel can either be applied to a single string from the exemplars, or to the result of an indexRangePattern; so the localizer can turn "X-Z" into "XYZ" if desired.

The indexLabelBefore and indexLabelAfter are used before and after a list. The default (root) value is an ellipsis, as in the example at the top.

When displaying index characters with multiple scripts, the main language can be used for all characters from the main script. For other scripts there are two possibilities:

  1. Use the primary characters from the UCA. This has the disadvantage that many very uncommon characters show up.
  2. Use the likely-subtags language for each scripts. For example, if the main language is French, and Cyrillic characters are present, then the likely subtags language for Cyrillic is "ru" (derived by looking up "und-Cyrl").

The indexLabel is used to display characters (if it is available). That is, when displaying index characters, if there is an indexLabel with non-empty text contents, display that text instead. For example, for Hungarian, we could have A => "A, Á".

The priority is used where not all of the index characters can be displayed. In that case, only the higher priorities (lower numbers) would be displayed.

3.4 Ellipsis

<!ELEMENT ellipsis ( #PCDATA ) >
<!ATTLIST ellipsis type ( initial | medial | final | word-initial | word-medial | word-final ) #IMPLIED >

The ellipsis element provides patterns for use when truncating strings. There are three versions: initial for removing an initial part of the string (leaving final characters); medial for removing from the center of the string (leaving initial and final characters), and final for removing a final part of the string (leaving initial characters). For example, the following uses the ellipsis character in all three cases (although some languages may have different characters for different positions).

<ellipsis type="initial">…{0}</ellipsis>
<ellipsis type="medial">{0}…{1}</ellipsis>
<ellipsis type="final">{0}…</ellipsis>

There are alternatives for cases where the breaks are on a word boundary, where some languages include a space. For example, such as case would be:

<ellipsis type="word-initial">… {0}</ellipsis>

3.5 More Information

The moreInformation string is one that can be displayed in an interface to indicate that more information is available. For example:

<moreInformation>?</moreInformation>

4 Delimiter Elements

<!ELEMENT delimiters (alias | (quotationStart*, quotationEnd*, alternateQuotationStart*, alternateQuotationEnd*, special*)) >

The delimiters supply common delimiters for bracketing quotations. The quotation marks are used with simple quoted text, such as:

He said, “Don’t be absurd!”

When quotations are nested, the quotation marks and alternate marks are used in an alternating fashion:

He said, “Remember what the Mad Hatter said: ‘Not the same thing a bit! Why you might just as well say that “I see what I eat” is the same thing as “I eat what I see”!’”

<quotationStart> </quotationStart>
<quotationEnd> </quotationEnd>
<alternateQuotationStart> </alternateQuotationStart>
<alternateQuotationEnd> </alternateQuotationEnd>

5 Measurement System Data

<!ELEMENT measurementData ( measurementSystem*, paperSize* ) >

<!ELEMENT measurementSystem EMPTY >
<!ATTLIST measurementSystem type ( metric | US | UK ) #REQUIRED >
<!ATTLIST measurementSystem territories NMTOKENS #REQUIRED >

<!ELEMENT paperSize EMPTY >
<!ATTLIST paperSize type ( A4 | US-Letter ) #REQUIRED >
<!ATTLIST paperSize territories NMTOKENS #REQUIRED >

The measurement system is the normal measurement system in common everyday use (except for date/time). For example:

<measurementData>
  <measurementSystem type="metric" territories="001"/>
  <measurementSystem type="US" territories="US"/>
  <paperSize type="A4" territories="001"/>
  <paperSize type="US-Letter" territories="US"/>
</measurementData>

The values are "metric", "US", or "UK"; others may be added over time. The "metric" value indicates the use of SI [ISO1000] base or derived units, or non-SI units accepted for use with SI: For example, meters, kilograms, liters, and degrees Celsius. The "US" value indicates the customary system of measurement as used in the United States: feet, inches, pints, quarts, degrees Fahrenheit, and so on. The "UK" value indicates the customary system of measurement as used in the United Kingdom: feet, inches, pints, quarts, and so on. It is also called the Imperial system: the pint, quart, and so on are different sizes than in "US".

The paperSize attribute gives the height and width of paper used for normal business letters. The values are "A4" and "US-Letter".

For both measurementSystem entries and paperSize entries, later entries for specific territories such as "US" will override the value assigned to that territory by earlier entries for more inclusive territories such as "001".

The measurement information was formerly in the main LDML file, and had a somewhat different format.

5.1 Measurement Elements (deprecated)

<!ELEMENT measurement (alias | (measurementSystem?, paperSize?, special*)) >

The measurement element is deprecated in the main LDML files, because the data is more appropriately organized as connected to territories, not to linguistic data. Instead, the measurementData element in the supplemental data file should be used.

6 Unit Elements

<!ELEMENT units (alias | (unit*, unitLength*, durationUnit*, special*)) >

<!ELEMENT unitLength (alias | (compoundUnit*, unit*, special*)) >
<!ATTLIST unitLength type ( long | short | narrow ) #REQUIRED >

<!ELEMENT compoundUnit (alias | (compoundUnitPattern*, special*)) >
<!ATTLIST compoundUnit type NMTOKEN #REQUIRED >

<!ELEMENT unit (alias | (unitPattern*, special*)) >
<!ATTLIST unit type NMTOKEN #REQUIRED >

<!ELEMENT durationUnit (alias | (durationUnitPattern*, special*)) >
<!ATTLIST durationUnit type NMTOKEN #REQUIRED >

<!ELEMENT unitPattern ( #PCDATA ) >
<!ATTLIST unitPattern count (0 | 1 | zero | one | two | few | many | other) #REQUIRED >

<!ELEMENT compoundUnitPattern ( #PCDATA ) >

<!ELEMENT durationUnitPattern ( #PCDATA ) >

These elements specify the localized way of formatting quantities of units such as years, months, days, hours, minutes and seconds— for example, in English, "1 day" or "3 days". The English rules that produce this example are as follows ({0} indicates the position of the formatted numeric value):

<unit type="day">
  <unitPattern count="one">{0} day</unitName>
  <unitPattern count="other">{0} days</unitName>
</unit>

Units, like other values with a count attribute, use a special inheritance. See Part 1: Core: Section 4.1 Multiple Inheritance .

The units in CLDR are not comprehensive; it is anticipated that more will be added over time. They currently include the following:

Type Unit Sample Format
acceleration g-force {0} G
angle arc-minute {0}′
angle arc-second {0}″
angle degree {0}°
area acre {0} ac
area hectare {0} ha
area square-foot {0} ft²
area square-kilometer {0} km²
area square-meter {0} m²
area square-mile {0} mi²
duration day {0} d
duration hour {0} h
duration millisecond {0} ms
duration minute {0} min
duration month {0} m
duration second {0} s
duration week {0} w
duration year {0} y
length centimeter {0} cm
length foot {0} ft
length inch {0} in
length kilometer {0} km
length light-year {0} ly
length meter {0} m
length mile {0} mi
length millimeter {0} mm
length picometer {0} pm
length yard {0} yd
mass gram {0} g
mass kilogram {0} kg
mass ounce {0} oz
mass pound {0} lb
power horsepower {0} hp
power kilowatt {0} kW
power watt {0} W
pressure hectopascal {0} hPa
pressure inch-hg {0} inHg
pressure millibar {0} mbar
speed kilometer-per-hour {0} km/h
speed meter-per-second {0} m/s
speed mile-per-hour {0} mi/h
temperature celsius {0}°C
temperature fahrenheit {0}°F
volume cubic-kilometer {0} km³
volume cubic-mile {0} mi³
volume liter {0} l

There are three widths: long, short, and narrow. As usual, the narrow forms may not be unique: in English, 1′ could mean 1 minute of arc, or 1 foot. Thus narrow forms should only be used where the context makes the meaning clear.

Where the unit of measurement is one of the International System of Units (SI), the short and narrow forms will typically use the international symbols, such as “mm” for millimeter. They may, however, be different if that is customary for the language or locale. For example, in Russian it may be more typical to see the Cyrillic characters “мм”.

Units are included for translation even where they are not typically used in a particular locale, such as kilometers in the US, or inches in Germany. This is to account for use by travelers and specialized domains, such as the German “̌Fernseher von 32 bis 55 Zoll (80 bis 140 cm)” for TV screen size in inches and centimeters.

compoundUnitPattern

The compoundUnitPattern for "per" is used to construct a pattern such as "feet per second" or "ft/s". Some units already have 'precomputed' forms, such as kilometer-per-hour; where such units exist, they should be used in preference to composing with compoundUnitPattern.

For the divisor, the "one" plural category should be used, while for the dividend the appropriate plural form according the placeholder number should be used.

[There is a known problem with some languages in the long form in that the divisor should be inflected. This will probably require the future addition of a special 'divisor' form of units commonly used in the divisor.]

Unit Sequences

Units may be used in composed sequences, such as 5° 30′ for 5 degrees 30 minutes, or 3 ft 2 in.For that purpose, the appropriate width of the unit listPattern can be used to compose the units in a sequence.

<listPattern type="unit"> (for the long form)
<listPattern type="unit-narrow">
<listPattern type="unit-short">

durationUnit

The durationUnit is a special type of unit used for composed time unit durations.

<durationUnit type="hms">
  <durationUnitPattern>h:mm:ss</durationUnitPattern> <!-- 33:04:59 -->
</durationUnit>   

The type contains a skeleton, where 'h' stands for hours, 'm' for minutes, and 's' for sections. These are the same symbols used in availableFormats, except that there is no need to distinguish different forms of the hour.

7 POSIX Elements

<!ELEMENT posix (alias | (messages*, special*)) >
<!ELEMENT messages (alias | ( yesstr*, nostr*)) >

The following are included for compatibility with POSIX.

<posix>
    <posix:messages>
        <posix:yesstr>ja</posix:yesstr>
        <posix:nostr>nein</posix:nostr>
    </posix:messages>
<posix>

  1. The values for yesstr and nostr contain a colon-separated list of strings that would normally be recognized as "yes" and "no" responses. For cased languages, this shall include only the lower case version. POSIX locale generation tools must generate the upper case equivalents, and the abbreviated versions, and add the English words wherever they do not conflict. Examples:
    • ja → ja:Ja:j:J:yes:Yes:y:Y
    • ja → ja:Ja:j:J:yes:Yes // exclude y:Y if it conflicts with the native "no".
  2. The older elements yesexpr and noexpr are deprecated. They should instead be generated from yesstr and nostr so that they match all the responses.

So for English, the appropriate strings and expressions would be as follows:

yesstr "yes:y"
nostr "no:n"

The generated yesexpr and noexpr would be:

yesexpr "^([yY]([eE][sS])?)"
This would match y,Y,yes,yeS,yEs,yES,Yes,YeS,YEs,YES.

noexpr "^([nN][oO]?)"
This would match n,N,no,nO,No,NO.

8 Reference Element

(Use only in supplemental data; deprecated for ldml.dtd and locale data)

<!ELEMENT references ( reference* ) >
<!ELEMENT reference ( #PCDATA ) >
<!ATTLIST reference type NMTOKEN #REQUIRED>
<!ATTLIST reference standard ( true | false ) #IMPLIED >
<!ATTLIST reference uri CDATA #IMPLIED >

The references section supplies a central location for specifying references and standards. The uri should be supplied if at all possible. If not online, then a ISBN number should be supplied, such as in the following example:

<reference type="R2" uri="http://www.ur.se/nyhetsjournalistik/3lan.html">Landskoder på Internet</reference>
<reference type="R3" uri="URN:ISBN:91-47-04974-X">Svenska skrivregler</reference>

9 Segmentations

<!ELEMENT segmentations ( alias | segmentation*) >

<!ELEMENT segmentation ( alias | (variables?, segmentRules? , exceptions?) | special*) >
<!ATTLIST segmentation type NMTOKEN #REQUIRED >

<!ELEMENT variables ( alias | variable*) >

<!ELEMENT variable ( #PCDATA ) >
<!ATTLIST variable id CDATA #REQUIRED >

<!ELEMENT segmentRules ( alias | rule*) >

<!ELEMENT rule ( #PCDATA ) >
<!ATTLIST rule id NMTOKEN #REQUIRED >

<!ELEMENT exceptions ( exception* ) >

<!ELEMENT exception ( #PCDATA ) >

The segmentations element provides for segmentation of text into words, lines, or other segments. The structure is based on [UAX29] notation, but adapted to be machine-readable. It uses a list of variables (representing character classes) and a list of rules. Each must have an id attribute.

The rules in root implement the segmentations found in [UAX29] and [UAX14], for grapheme clusters, words, sentences, and lines. They can be overridden by rules in child locales.

Here is an example:

<segmentations>
  <segmentation type="GraphemeClusterBreak">
    <variables>
      <variable id="$CR">\p{Grapheme_Cluster_Break=CR}</variable>
      <variable id="$LF">\p{Grapheme_Cluster_Break=LF}</variable>
      <variable id="$Control">\p{Grapheme_Cluster_Break=Control}</variable>
      <variable id="$Extend">\p{Grapheme_Cluster_Break=Extend}</variable>
      <variable id="$L">\p{Grapheme_Cluster_Break=L}</variable>
      <variable id="$V">\p{Grapheme_Cluster_Break=V}</variable>
      <variable id="$T">\p{Grapheme_Cluster_Break=T}</variable>
      <variable id="$LV">\p{Grapheme_Cluster_Break=LV}</variable>
      <variable id="$LVT">\p{Grapheme_Cluster_Break=LVT}</variable>
    </variables>
    <segmentRules>
      <rule id="3"> $CR × $LF </rule>
      <rule id="4"> ( $Control | $CR | $LF ) ÷ </rule>
      <rule id="5"> ÷ ( $Control | $CR | $LF ) </rule>
      <rule id="6"> $L × ( $L | $V | $LV | $LVT ) </rule>
      <rule id="7"> ( $LV | $V ) × ( $V | $T ) </rule>
      <rule id="8"> ( $LVT | $T) × $T </rule>
      <rule id="9"> × $Extend </rule>
    </segmentRules>
  </segmentation>
...

Variables: All variable ids must start with a $, and otherwise be valid identifiers according to the Unicode definitions in [UAX31]. The contents of a variable is a regular expression using variables and UnicodeSets. The ordering of variables is important; they are evaluated in order from first to last (see Section 9.1 Segmentation Inheritance). It is an error to use a variable before it is defined.

Rules: The contents of a rule uses the syntax of [UAX29]. The rules are evaluated in numeric id order (which may not be the order in which the appear in the file). The first rule that matches determines the status of a boundary position, that is, whether it breaks or not. Thus ÷ means a break is allowed; × means a break is forbidden. It is an error if the rule does not contain exactly one of these characters (except where a rule has no contents at all, or if the rule uses a variable that has not been defined.

There are some implicit rules:

Note: A rule like X Format* -> X in [UAX29] and [UAX14] is not supported. Instead, this needs to be expressed as normal regular expressions. The normal way to support this is to modify the variables, such as in the following example:

<variable id="$Format">\p{Word_Break=Format}</variable>
<variable id="$Katakana">\p{Word_Break=Katakana}</variable>
...
<!-- In place of rule 3, add format and extend to everything -->
<variable id="$X">[$Format $Extend]*</variable>
<variable id="$Katakana">($Katakana $X)</variable>
<variable id="$ALetter">($ALetter $X)</variable>
...

9.1 Segmentation Inheritance

Variables and rules both inherit from the parent.

Variables: The child's variable list is logically appended to the parent's, and evaluated in that order. For example:

// in parent
<variable id="$AL">[:linebreak=AL:]</variable>
<variable id="$YY">[[:linebreak=XX:]$AL]</variable>
// adds $AL

// in child
<variable id="$AL">[$AL && [^a-z]]</variable> // changes $AL, does not affect $YY
<variable id="$ABC">[abc]</variable>
// adds new rule

Rules: The rules are also logically appended to the parent's. Because rules are evaluated in numeric id order, to insert a rule in between others just requires using an intermediate number. For example, to insert a rule before id="10.1" and after id="10.2", just use id="10.15". To delete a rule, use empty contents, such as:

<rule id="3"/> // deletes rule 3

9.2 Segmentation Exceptions

Note: As of CLDR 24, the <exceptions> data is to be considered a technology preview. Data currently in CLDR was extracted from the Unicode Localization Interoperability project, or ULI. See http://uli.unicode.org for more information on the ULI project.

The segmentation exceptions list provides a set of cases which, though otherwise identified as a segment by rules, should be skipped during segmentation.

For example, in the English phrase "Mr. Smith", CLDR segmentation rules would normally find a Sentence Break between "Mr" and "Smith". However, typically, "Mr." is just an abbreviation for "Mister", and not actually the end of a sentence.

Each exception has a separate <exception> element, whose contents are the exceptional break to be skipped.

Example:

    <segmentation type="SentenceBreak">
      <exceptions>
        <exception draft="provisional">Maj.</exception>
        <exception draft="provisional">Mr.</exception>
        <exception draft="provisional">Lt.Cdr.</exception>
	. . .
      </exceptions>
    </segmentation>

10 Transforms

Transforms provide a set of rules for transforming text via a specialized set of context-sensitive matching rules. They are commonly used for transliterations or transcriptions, but also other transformations such as full-width to half-width (for katakana characters). The rules can be simple one-to-one relationships between characters, or involve more complicated mappings. Here is an example:

<transform source="Greek" target="Latin" variant="UNGEGN" direction="both">
...
  <comment>Useful variables</comment>
  <tRule>$gammaLike = [ΓΚΞΧγκξχϰ] ;</tRule>
  <tRule>$egammaLike = [GKXCgkxc] ;</tRule>
...
  <comment>Rules are predicated on running NFD first, and NFC afterwards</comment>
  <tRule>::NFD (NFC) ;</tRule>
...
  <tRule>λ ↔ l ;</tRule>
  <tRule>Λ ↔ L ;</tRule>
...
  <tRule>γ } $gammaLike ↔ n } $egammaLike ;</tRule>
  <tRule>γ ↔ g ;</tRule>
...
  <tRule>::NFC (NFD) ;</tRule>
...
</transform>

The source and target values are valid locale identifiers, where 'und' means an unspecified language, plus some additional extensions.

Inheritance

The CLDR transforms are built using the following locale inheritance. While this inheritance is not required of LDML implementations, the transforms supplied with CLDR may not otherwise behave as expected without some changes.

For either the source or the target, the fallback starts from the maximized locale ID (using the likely-subtags data). It also uses the country for lookup before the base language is reached, and root is never accessed: instead the script(s) associated with the language are used. Where there are multiple scripts, the maximized script is tried first, and then the other scripts associated with the language (from supplemental data).

For example, see the bolded items below in the fallback chain for az_IR.

  Locale ID Comments
1 az_Arab_IR The maximized locale for az_IR
2 az_Arab Normal fallback
3 az_IR Inserted country locale
4 az Normal fallback
5 Arab Maximized script
6 Cyrl Other associated script

The source, target, and variant use "laddered" fallback, where the source changes the most quickly (using the above rules), then the target (using the above rules), then the variant if any, is discarded. That is, in pseudo code:

For example, here is the fallback chain for ru_RU-el_GR/BGN.

source   target variant
ru_RU - el_GR /BGN
ru - el_GR /BGN
Cyrl - el_GR /BGN
ru_RU - el /BGN
ru - el /BGN
Cyrl - el /BGN
ru_RU - Grek /BGN
ru - Grek /BGN
Cyrl - Grek /BGN
ru_RU - el_GR
ru - el_GR
Cyrl - el_GR
ru_RU - el
ru - el
Cyrl - el
ru_RU - Grek
ru - Grek
Cyrl - Grek

Variants

Variants used in CLDR include UNGEGN and BGN, both indicating sources for transliterations. There is an additional attribute private="true" which is used to indicate that the transform is meant for internal use, and should not be displayed as a separate choice in a UI.

There are many different systems of transliteration. The goal for the "unqualified" script transliterations are

  1. to be lossless when going to Latin and back
  2. to be as lossless as possible when going to other scripts
  3. to abide by a common standard as much as possible (possibly supplemented to meet goals 1 and 2).

Language-to-language transliterations, and variant script-to-script transliterations are generally transcriptions, and not expected to be lossless.

Additional transliterations may also be defined, such as customized language-specific transliterations (such as between Russian and French), or those that match a particular transliteration standard, such as the following:

The rules for transforms are described in Section 10.1 Transform Rules. For more information on Transliteration, see Transliteration Guidelines.

10.1 Transform Rules

<!ELEMENT transforms ( transform*) >
<!ELEMENT transform ((comment | tRule)*) >
<!ATTLIST transform source CDATA #IMPLIED >
<!ATTLIST transform target CDATA #IMPLIED >
<!ATTLIST transform variant CDATA #IMPLIED >
<!ATTLIST transform direction ( forward | backward | both ) "both" >
<!ATTLIST transform visibility ( internal | external ) "external" >

<!ELEMENT comment (#PCDATA) >
<!ELEMENT tRule (#PCDATA) >

The transform rules are similar to regular-expression substitutions, but adapted to the specific domain of text transformations. The rules and comments in this discussion will be intermixed, with # marking the comments. In the xml format these in separate elements: comment and tRule. The simplest rule is a conversion rule, which replaces one string of characters with another. The conversion rule takes the following form:

xy → z ;

This converts any substring "xy" into "z". Rules are executed in order; consider the following rules:

sch → sh ;
ss → z ;

This conversion rule transforms "bass school" into "baz shool". The transform walks through the string from start to finish. Thus given the rules above "bassch" will convert to "bazch", because the "ss" rule is found before the "sch" rule in the string (later, we'll see a way to override this behavior). If two rules can both apply at a given point in the string, then the transform applies the first rule in the list.

All of the ASCII characters except numbers and letters are reserved for use in the rule syntax, as are the characters →, ←, ↔. Normally, these characters do not need to be converted. However, to convert them use either a pair of single quotes or a slash. The pair of single quotes can be used to surround a whole string of text. The slash affects only the character immediately after it. For example, to convert from a U+2190 ( ← ) LEFTWARDS ARROW to the string "arrow sign" (with a space), use one of the following rules:

\←   →  arrow\ sign ;
'←'   →   'arrow sign' ;
'←'   →   arrow' 'sign ;

Spaces may be inserted anywhere without any effect on the rules. Use extra space to separate items out for clarity without worrying about the effects. This feature is particularly useful with combining marks; it is handy to put some spaces around it to separate it from the surrounding text. The following is an example:

 → i ; # an iota-subscript diacritic turns into an i.

For a real space in the rules, place quotes around it. For a real backslash, either double it \\, or quote it '\'. For a real single quote, double it '', or place a backslash before it \'.

Any text that starts with a hash mark and concludes a line is a comment. Comments help document how the rules work. The following shows a comment in a rule:

x → ks ; # change every x into ks

The "\u" notation can be used instead of any letter. For instance, instead of using the Greek π, one could write:

\u03C0 → p ;

One can also define and use variables, such as:

$pi = \u03C0 ;
$pi → p ;

10.1.1 Dual Rules

Rules can also specify what happens when an inverse transform is formed. To do this, we reverse the direction of the "←" sign. Thus the above example becomes:

$pi ← p ;

With the inverse transform, "p" will convert to the Greek p. These two directions can be combined together into a dual conversion rule by using the "↔" operator, yielding:

$pi ↔ p ;

10.1.2 Context

Context can be used to have the results of a transformation be different depending on the characters before or after. The following means "Remove hyphens, but only when they follow lower case letters":

[:lowercase letter:] } '-' → '' ;

The context itself ([:lowercase letter:]) is unaffected by the replacement; only the text between the curly braces is changed.

10.1.3 Revisiting

If the resulting text contains a vertical bar "|", then that means that processing will proceed from that point and that the transform will revisit part of the resulting text. Thus the | marks a "cursor" position. For example, if we have the following, then the string "xa" will convert to "w".

x → y | z ;
z a → w;

First, "xa" is converted to "yza". Then the processing will continue from after the character "y", pick up the "za", and convert it. Had we not had the "|", the result would have been simply "yza". The '@' character can be used as filler character to place the revisiting point off the start or end of the string. Thus the following causes x to be replaced, and the cursor to be backed up by two characters.

x → |@@y;

10.1.4 Example

The following shows how these features are combined together in the Transliterator "Any-Publishing". This transform converts the ASCII typewriter conventions into text more suitable for desktop publishing (in English). It turns straight quotation marks or UNIX style quotation marks into curly quotation marks, fixes multiple spaces, and converts double-hyphens into a dash.

# Variables

$single = \' ;
$space = ' ' ;
$double = \" ;
$back = \` ;
$tab = '\u0008' ;

# the following is for spaces, line ends, (, [, {, ...
$makeRight = [[:separator:][:start punctuation:][:initial punctuation:]] ;

# fix UNIX quotes

$back $back → “ ; # generate right d.q.m. (double quotation mark)
$back → ‘ ;

# fix typewriter quotes, by context

$makeRight { $double ↔ “ ; # convert a double to right d.q.m. after certain chars
^ { $double → “ ; # convert a double at the start of the line.
$double ↔ ” ; # otherwise convert to a left q.m.

$makeRight {$single} ↔ ‘ ; # do the same for s.q.m.s
^ {$single} → ‘ ;
$single ↔ ’;

# fix multiple spaces and hyphens

$space {$space} → ; # collapse multiple spaces
'--' ↔ — ; # convert fake dash into real one

10.1.5 Rule Syntax

The following describes the full format of the list of rules used to create a transform. Each rule in the list is terminated by a semicolon. The list consists of the following:

The filter rule, if present, must appear at the beginning of the list, before any of the other rules.  The inverse filter rule, if present, must appear at the end of the list, after all of the other rules.  The other rules may occur in any order and be freely intermixed.

The rule list can also generate the inverse of the transform. In that case, the inverse of each of the rules is used, as described below.

10.1.6 Transform Rules

Each transform rule consists of two colons followed by a transform name, which is of the form source-target. For example:

:: NFD ;
:: und_Latn-und_Greek ;
:: Latin-Greek; # alternate form

If either the source or target is 'und', it can be omitted, thus 'und_NFC' is equivalent to 'NFC'. For compatibility, the English names for scripts can be used instead of the und_Latn locale name, and "Any" can be used instead of "und". Case is not significant.

The following transforms are defined not by rules, but by the operations in the Unicode Standard, and may be used in building any other transform:

Any-NFC, Any-NFD, Any-NFKD, Any-NFKC - the normalization forms defined by [UAX15].

Any-Lower, Any-Upper, Any-Title - full case transformations, defined by [Unicode] Chapter 3.

In addition, the following special cases are defined:

Any-Null - has no effect; that is, each character is left alone.
Any-Remove - maps each character to the empty string; this, removes each character.

The inverse of a transform rule uses parentheses to indicate what should be done when the inverse transform is used. For example:

:: lower () ; # only executed for the normal
:: (lower) ; # only executed for the inverse
:: lower ; # executed for both the normal and the inverse

10.1.7 Variable Definition Rules

Each variable definition is of the following form:

$variableName = contents ;

The variable name can contain letters and digits, but must start with a letter. More precisely, the variable names use Unicode identifiers as defined by [UAX31]. The identifier properties allow for the use of foreign letters and numbers.

The contents of a variable definition is any sequence of Unicode sets and characters or characters. For example:

$mac = M [aA] [cC] ;

Variables are only replaced within other variable definition rules and within conversion rules. They have no effect on transliteration rules.

10.1.8 Filter Rules

A filter rule consists of two colons followed by a UnicodeSet. This filter is global in that only the characters matching the filter will be affected by any transform rules or conversion rules. The inverse filter rule consists of two colons followed by a UnicodeSet in parentheses. This filter is also global for the inverse transform.

For example, the Hiragana-Latin transform can be implemented by "pivoting" through the Katakana converter, as follows:

:: [:^Katakana:] ; # do not touch any katakana that was in the text!
:: Hiragana-Katakana;
:: Katakana-Latin;
:: ([:^Katakana:]) ; # do not touch any katakana that was in the text
                     # for the inverse either!

The filters keep the transform from mistakenly converting any of the "pivot" characters. Note that this is a case where a rule list contains no conversion rules at all, just transform rules and filters.

10.1.9 Conversion Rules

Conversion rules can be forward, backward, or double. The complete conversion rule syntax is described below:

Forward

A forward conversion rule is of the following form:

before_context { text_to_replace } after_context → completed_result | result_to_revisit ;

If there is no before_context, then the "{" can be omitted. If there is no after_context, then the "}" can be omitted. If there is no result_to_revisit, then the "|" can be omitted. A forward conversion rule is only executed for the normal transform and is ignored when generating the inverse transform.

Backward

A backward conversion rule is of the following form:

completed_result | result_to_revisit ← before_context { text_to_replace } after_context ;

The same omission rules apply as in the case of forward conversion rules. A backward conversion rule is only executed for the inverse transform and is ignored when generating the normal transform.

Dual

A dual conversion rule combines a forward conversion rule and a backward conversion rule into one, as discussed above. It is of the form:

a { b | c } d ↔ e { f | g } h ;

When generating the normal transform and the inverse, the revisit mark "|" and the before and after contexts are ignored on the sides where they do not belong. Thus, the above is exactly equivalent to the sequence of the following two rules:

a { b c } d  →  f | g  ;
b | c  ←  e { f g } h ; 

10.1.10 Intermixing Transform Rules and Conversion Rules

Transform rules and conversion rules may be freely intermixed. Inserting a transform rule into the middle of a set of conversion rules has an important side effect.

Normally, conversion rules are considered together as a group.  The only time their order in the rule set is important is when more than one rule matches at the same point in the string.  In that case, the one that occurs earlier in the rule set wins.  In all other situations, when multiple rules match overlapping parts of the string, the one that matches earlier wins.

Transform rules apply to the whole string.  If you have several transform rules in a row, the first one is applied to the whole string, then the second one is applied to the whole string, and so on.  To reconcile this behavior with the behavior of conversion rules, transform rules have the side effect of breaking a surrounding set of conversion rules into two groups: First all of the conversion rules before the transform rule are applied as a group to the whole string in the usual way, then the transform rule is applied to the whole string, and then the conversion rules after the transform rule are applied as a group to the whole string.  For example, consider the following rules:

abc → xyz;
xyz → def;
::Upper;

If you apply these rules to “abcxyz”, you get “XYZDEF”.  If you move the “::Upper;” to the middle of the rule set and change the cases accordingly, then applying this to “abcxyz” produces “DEFDEF”.

abc → xyz;
::Upper;
XYZ → DEF;

This is because “::Upper;” causes the transliterator to reset to the beginning of the string. The first rule turns the string into “xyzxyz”, the second rule upper cases the whole thing to “XYZXYZ”, and the third rule turns this into “DEFDEF”.

This can be useful when a transform naturally occurs in multiple “passes.”  Consider this rule set:

[:Separator:]* → ' ';
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';

If you apply this rule to “high school”, you get “H.S.”, but if you apply it to “high  school” (with two spaces), you just get “high school” (with one space). To have “high  school” (with two spaces) turn into “H.S.”, you'd either have to have the first rule back up some arbitrary distance (far enough to see “elementary”, if you want all the rules to work), or you have to include the whole left-hand side of the first rule in the other rules, which can make them hard to read and maintain:

$space = [:Separator:]*;
high $space school → 'H.S.';
middle $space school → 'M.S.';
elementary $space school → 'E.S.';

Instead, you can simply insert “ ::Null; ” in order to get things to work right:

[:Separator:]* → ' ';
::Null;
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';

The “::Null;” has no effect of its own (the null transform, by definition, does not do anything), but it splits the other rules into two “passes”: The first rule is applied to the whole string, normalizing all runs of white space into single spaces, and then we start over at the beginning of the string to look for the phrases. “high    school” (with four spaces) gets correctly converted to “H.S.”.

This can also sometimes be useful with rules that have overlapping domains.  Consider this rule set from before:

sch → sh ;
ss → z ;

Apply this rule to “bassch” results in “bazch” because “ss” matches earlier in the string than “sch”. If you really wanted “bassh”—that is, if you wanted the first rule to win even when the second rule matches earlier in the string, you'd either have to add another rule for this special case...

sch → sh ;
ssch → ssh;
ss → z ;

...or you could use a transform rule to apply the conversions in two passes:

sch → sh ;
::Null;
ss → z ;

10.1.11 Inverse Summary

The following table shows how the same rule list generates two different transforms, where the inverse is restated in terms of forward rules (this is a contrived example, simply to show the reordering):

Original Rules Forward Inverse
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x ↔ y ;
z → w ;
r ← m ;
:: upper;
a → b ;
c ↔ d ;
:: any-publishing ;
:: ([:Number:]) ;
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x → y ;
z → w ;
:: upper ;
a → b ;
c → d ;
:: any-publishing ;
:: [:Number:] ;
:: publishing-any ;
d → c ;
:: lower ;
y → x ;
m → r ;
:: japanese-greek ;
:: greek-latin ;

Note how the irrelevant rules (the inverse filter rule and the rules containing ←) are omitted (ignored, actually) in the forward direction, and notice how things are reversed: the transform rules are inverted and happen in the opposite order, and the groups of conversion rules are also executed in the opposite relative order (although the rules within each group are executed in the same order).

11 List Patterns

<!ELEMENT listPatterns (alias | (listPattern*, special*)) >

<!ELEMENT listPattern (alias | (listPatternPart*, special*)) >
<!ATTLIST listPattern type (NMTOKEN) #IMPLIED >

<!ELEMENT listPatternPart ( #PCDATA ) >
<!ATTLIST listPatternPart type (start | middle | end | 2 | 3) #REQUIRED >

List patterns can be used to format variable-length lists of things in a locale-sensitive manner, such as "Monday, Tuesday, Friday, and Saturday" (in English) versus "lundi, mardi, vendredi et samedi" (in French). For example, consider the following example:

<listPatterns>
 <listPattern>
  <listPatternPart type="2">{0} and {1}</listPatternPart>
<listPatternPart type="start">{0}, {1}</listPatternPart> <listPatternPart type="middle">{0}, {1}</listPatternPart> <listPatternPart type="end">{0}, and {1}</listPatternPart>
</listPattern> </listPatterns>

The data is used as follows: If there is a type type matches exactly the number of elements in the desired list (such as "2" in the above list), then use that pattern. Otherwise,

  1. Format the last two elements with the "end" format.
  2. Then use middle format to add on subsequent elements working towards the front, all but the very first element. That is, {1} is what you've already done, and {0} is the previous element.
  3. Then use "start" to add the front element, again with {1} as what you've done so far, and {0} is the first element.

Thus a list (a,b,c,...m, n) is formatted as: start(a,middle(b,middle(c,middle(...end(m, n))...)))

11.1 Gender of Lists

<!-- Gender List support -->
<!ELEMENT gender ( personList+ ) >
<!ELEMENT personList EMPTY >
<!ATTLIST personList type ( neutral | mixedNeutral | maleTaints ) #REQUIRED >
<!ATTLIST personList locales NMTOKENS #REQUIRED >

This can be used to determine the gender of a list of 2 or more persons, such as "Tom and Mary", for use with gender-selection messages. For example,

  <supplementalData>
    <gender>
      <!-- neutral: gender(list) = other -->
      <personList type="neutral" locales="af da en..."/>

      <!-- mixedNeutral: gender(all male) = male, gender(all female) = female, otherwise gender(list) = other -->
      <personList type="mixedNeutral" locales="el"/> 

      <!-- maleTaints: gender(all female) = female, otherwise gender(list) = male -->
      <personList type="maleTaints" locales="ar ca..."/> 
    </gender>
  </supplementalData>

There are three ways the gender of a list can be formatted:

  1. neutral: A gender-independent "other" form will be used for the list.
  2. mixedNeutral: If the elements of the list are all male, "male" form is used for the list. If all the elements of the lists are female, "female" form is used. If the list has a mix of male, female and neutral names, the "other" form is used.
  3. maleTaints: If all the elements of the lists are female, "female" form is used, otherwise the "male" form is used.

12 ContextTransform Elements

<!ELEMENT contextTransforms ( alias | (contextTransformUsage*, special*)) >
<!ELEMENT contextTransformUsage ( alias | (contextTransform*, special*)) >
<!ATTLIST contextTransformUsage type CDATA #REQUIRED >
<!ELEMENT contextTransform ( #PCDATA ) >
<!ATTLIST contextTransform type ( uiListOrMenu | stand-alone ) #REQUIRED >

CLDR locale elements provide data for display names or symbols in many categories. The default capitalization for these elements is intended to be the form used in the middle of running text. In many languages, other capitalization may be required in other contexts, depending on the type of name or symbol.

Each <contextTransformUsage> element’s type attribute specifies a category of data from the table below; the element includes one or more <contextTransform> elements that specify how to perform capitalization of this category of data in different contexts. The <contextTransform> elements are only needed for cases in which the capitalization is other than the default form used in the middle of running text. The only value currently defined for the <contextTransform> element is the transformation "titlecase-firstword", covering the case in which text that is otherwise lowercase needs to have its first word titlecased. No other necessary case transforms have been identified.

Four contexts for capitalization behavior are currently identified. Two need no data, and hence have no corresponding <contextTransform> elements:

Two other contexts require <contextTransform> elements if their capitalization behavior is other than the default for running text. The context is identified by the type attribute, as follows:

Example:

    <contextTransforms>
        <contextTransformUsage type="languages">
             <contextTransform type="uiListOrMenu">titlecase-firstword</contextTransform>
             <contextTransform type="stand-alone">titlecase-firstword</contextTransform>
        </contextTransformUsage>
        <contextTransformUsage type="month-format-except-narrow">
             <contextTransform type="uiListOrMenu">titlecase-firstword</contextTransform>
        </contextTransformUsage>
        <contextTransformUsage type="month-standalone-except-narrow">
             <contextTransform type="uiListOrMenu">titlecase-firstword</contextTransform>
        </contextTransformUsage>
    </contextTransforms>
<contextTransformUsage> type attribute values
type attribute value Description
all Special value, indicates that the specified transformation applies to all of the categories below
language localeDisplayNames language names
script localeDisplayNames script names
territory localeDisplayNames territory names
variant localeDisplayNames variant names
key localeDisplayNames key names
type localeDisplayNames type names
month-format-except-narrow dates/calendars/calendar[type=*]/months format wide and abbreviated month names
month-standalone-except-narrow dates/calendars/calendar[type=*]/months stand-alone wide and abbreviated month names
month-narrow dates/calendars/calendar[type=*]/months format and stand-alone narrow month names
day-format-except-narrow dates/calendars/calendar[type=*]/days format wide and abbreviated day names
day-standalone-except-narrow dates/calendars/calendar[type=*]/days stand-alone wide and abbreviated day names
day-narrow dates/calendars/calendar[type=*]/days format and stand-alone narrow day names
era-name dates/calendars/calendar[type=*]/eras (wide) era names
era-abbr dates/calendars/calendar[type=*]/eras abbreviated era names
era-narrow dates/calendars/calendar[type=*]/eras narrow era names
quarter-format-wide dates/calendars/calendar[type=*]/quarters format wide quarter names
quarter-standalone-wide dates/calendars/calendar[type=*]/quarters stand-alone wide quarter names
quarter-abbreviated dates/calendars/calendar[type=*]/quarters format and stand-alone abbreviated quarter names
quarter-narrow dates/calendars/calendar[type=*]/quarters format and stand-alone narrow quarter names
calendar-field dates/fields/field[type=*]/displayName field names
(for relative forms see type "tense" below)
zone-exemplarCity dates/timeZoneNames/zone[type=*]/exemplarCity city names
zone-long dates/timeZoneNames/zone[type=*]/long zone names
zone-short dates/timeZoneNames/zone[type=*]/short zone names
metazone-long dates/timeZoneNames/metazone[type=*]/long metazone names
metazone-short dates/timeZoneNames/metazone[type=*]/short metazone names
symbol numbers/currencies/currency[type=*]/symbol symbol names
displayName-count numbers/currencies/currency[type=*]/displayName[count=*] currency names for use with count
displayName numbers/currencies/currency[type=*]/displayName currency names
tense dates/fields/field[type=*]/relative and dates/fields/field[type=*]/relativeTime relative field names
unit-pattern units/unitLength[type=*]/unit[type=*]/unitPattern[count=*] unit names

13 Choice Patterns

A choice pattern is a string that chooses among a number of strings, based on numeric value. It has the following form:

<choice_pattern> = <choice> ( '|' <choice> )*
<choice> = <number><relation><string>
<number> = ('+' | '-')? ('∞' | [0-9]+ ('.' [0-9]+)?)
<relation> = '<' | '
≤'

The interpretation of a choice pattern is that given a number N, the pattern is scanned from right to left, for each choice evaluating <number> <relation> N. The first choice that matches results in the corresponding string. If no match is found, then the first string is used. For example:

Pattern N Result
0≤Rf|1≤Ru|1<Re -∞, -3, -1, -0.000001 Rf (defaulted to first string)
0, 0.01, 0.9999 Rf
1 Ru
1.00001, 5, 99, Re

Quoting is done using ' characters, as in date or number formats.