13 Localized Applications

When you write an application (or any other type of software) that is going to be localized, you need to take special care when working with characters, strings, numbers, and dates, as different countries represent these items in different ways. This chapter describes how to write code for localized applications, focusing on the text manager and international manager, which were introduced in Palm OS^® version 3.1, and the overlay manager, which is introduced in Palm OS version 3.5. The chapter covers:

• Localization Guidelines
  
  
• Using Overlays to Localize Resources
  
  
• Text Manager and International Manager
  
  
• Characters
  
  
• Strings
  
  
• Dates
  
  
• Numbers
  
  
• Compatibility Information
  
  
• Notes on the Japanese Implementation
  
  
• Summary of Localization
  

This chapter does not cover how to actually perform localization of resources. For more information on this subject, see your tools documentation.

Localization Guidelines

When you start planning for the localized version of your application, follow these guidelines:

• If you use the English language version of the software as a guide when designing the layout of the screen, try to allow:
  
  • extra space for strings
  • larger dialogs than the English version requires
  
  
• Don't put language-dependent strings in code. If you have to display text directly on the screen, remember that a one-line warning or message in one language may need more than one line in another language. See the section "Strings" in this chapter for further discussion.
  
  
• Don't depend on the physical characteristics of a string, such as the number of characters, the fact that it contains a particular substring, or any other attribute that might disappear in translation.
  
  
• Use the functions described in this chapter when working with characters, strings, numbers, and dates.
  
  
• Consider using string templates as described in the section "Dynamically Determining a String's Contents" in this chapter. Use as many parameters as possible to give localizers greater flexibility. Avoid building sentences by concatenating substrings together, as this often causes translation problems.
  
  
• Abbreviations may be the best way to accommodate the particularly scarce screen real estate on the Palm OS device.
  
  
• Remember that user interface elements such as lists, fields, and tips scroll if you need more space.
  

The chapter "Good Design Practices" provides further user interface guidelines.

Using Overlays to Localize Resources

Palm OS version 3.5 adds support for localizing resource databases through overlays. Localization overlays provide a method for localizing a software module without requiring a recompile or modification of the software. Each overlay database is a separate resource database that provides an appropriately localized set of resources for a single software module (the PRC file, or base database) and a single target locale (language and country). Note that each Palm OS device supports a single locale.

No requirements are placed on the base database, so for example, third parties can construct localization overlays for existing applications without forcing any modifications by the original application developer. In rare cases, you might want to disable the use of overlays to prevent third parties from creating overlays for your application. To do so, you should include an 'xprf'=0 resource (symbolically named sysResTExtPrefs) in the database and set its disableOverlays flag. This resource is defined in UIResources.r.

An overlay database has the same creator as the base database, but its type is 'ovly', and a suffix identifying the target locale is appended to its name. For example, Datebook.prc might be overlaid with a database named Datebook_jpJP, which indicates that this overlay is for Japan. Each overlay database has an 'ovly'=1000 resource specifying the base database's type, the target locale, and information necessary to identify the correct version of the base database for which it was designed.

The Palm OS SDK provides tools that you can use to create overlays. See Using the PRC to Overlay Tool for more information on creating overlays.

When a PRC file is opened on a system that supports overlays, the overlay manager determines what the locale is for this device and it looks for an overlay matching the base database and the locale. The overlay database's name must match the base database's name, its suffix must match the locale's suffix, and it must have an 'ovly'=1000 resource that matches the base database. If the name, suffix, and overlay resource are all correct, the overlay is opened in addition to the PRC file. When the PRC file is closed, its overlay is closed as well.

The overlay is opened in read-only mode and is hidden from the programmer. When you request a database pointer, you'll receive a pointer to the base database, not the overlay. You can simply make resource manager calls like you normally would, and the resource manager accesses the overlay where appropriate.

When accessing a localizable resource, do not use functions that search for a resource only in the database you specify. For example:

// WRONG! searches only one database.

DmOpenRef dbP = DmNextOpenResDatabase(NULL);
UInt16 resIndex = DmFindResource(dpP, strRsc,
    strRscID);
MemHandle resH = DmGetResourceIndex(dbP,
    resIndex);

In the example above, dbP is a pointer to the most recently opened database, which is typically the overlay version of the database. Passing this pointer to DmFindDatabase means that you are searching only the overlay database for the resource. If you're searching for a non-localized resource, DmFindResource won't be able to locate it. Instead, you should use DmGet1Resource, which searches the most recently opened database and its overlay for a resource, or DmGetResource, which searches all open databases and their overlays.

// Right. DmGet1Resource searches both

// databases.
MemHandle resH = DmGet1Resource(strRsc,
    strRscID);

// Or use DmGetResource to search all open

// databases.
MemHandle resH = DmGetResource(strRsc,
    strRscID);

The data manager only opens an overlay if the resource database is opened in read-only mode. If you open a resource database in read-write mode, the associated overlay is not opened. What's more, if you modify the an overlaid resource in the base database, the checksum in the overlay's 'ovly' resource becomes invalid, which prevents the overlay from being used at all. Thus if you change the resource database, you must also change the overlay database.

You typically don't work with the overlay manager directly although it does provide a few public functions. One potentially useful function is OmGetCurrentLocale, which returns a structure identifying the locale on this device.

Text Manager and International Manager

The Palm OS provides two managers that help you work with localized strings and characters. These managers are called the text manager and the international manager.

Computers represent the characters in an alphabet with a numeric code. The set of numeric codes for a given alphabet is called a character encoding. Of course, a character encoding contains more than codes for the letters of an alphabet. It also encodes punctuation, numbers, control characters, and any other characters deemed necessary. The set of characters that a character encoding represents is called, appropriately enough, a character set.

As you know, different languages use different alphabets. Most European languages use the Latin alphabet. The Latin alphabet is relatively small, so its characters can be represented using a single-byte encoding ranging from 32 to 255. On the other hand, Asian languages such as Chinese, Korean, and Japanese require their own alphabets, which are much larger. These larger character sets are represented by a combination of single-byte and double-byte numeric codes ranging from 32 to 65,535.

A given Palm OS device supports one language and one character encoding to represent the characters required by that language. Although the Palm OS supports multiple character encodings, a given device uses only one of those encodings. For example, a French device would probably use a character encoding similar to the Microsoft^® Windows^® code page 1252 character encoding (an extension of ISO Latin 1), while a Japanese device would use a character encoding similar to Microsoft Windows code page 932 (an extension of Shift JIS). Code page 932 is not supported on the French device, and code page 1252 is not supported on the Japanese device even though they both use the same version of Palm OS. No matter what the encoding is on a device, Palm guarantees that the low ASCII characters (0 to 0x7F) are the same. The exception to this rule is 0x5C, which is a yen symbol on Japanese devices and a backslash on all others.

The text manager allows you to work with text, strings, and characters independent of the character encoding. If you use text manager routines and don't work directly with string data, your code should work on any system, regardless of which language and character encoding the device supports (as long as it supports the text manager).

The international manager's job is to detect which character encoding a device uses and initialize the corresponding version of the text manager. The international manager also sets system features that identify which encoding and fonts are used. For the most part, you don't work with the international manager directly.

The text manager and international manager are supported starting in Palm OS version 3.1. If your application should work on older systems, you should test for the presence of these managers before using text manager calls. Listing 13.1 shows how.

Listing 13.1 Testing for text and international managers

---

NOTE:  

You can still use the text manager and be compatible with earlier releases if you link your application with the PalmOSGlue library. See the section "Compatibility Information" for more information.

---

Characters

Depending on the device's supported language, the Palm OS may encode characters using either a single-byte encoding or a multi-byte encoding. Because you do not know which character encoding is used until runtime, you should never make an assumption about the size of a character.

For the most part, your application does not need to know which character encoding is used, and in fact, it should make no assumptions about the encoding or about the size of characters. Instead, your code should use text manager functions to manipulate characters. This section describes how to work with characters correctly in a localized application. It covers:

• Declaring Character Variables
  
  
• Using Character Constants
  
  
• Missing and Invalid Characters
  
  
• Retrieving a Character's Attributes
  
  
• Virtual Characters
  
  
• Retrieving the Character Encoding
  

Declaring Character Variables

Declare all character variables to be of type WChar. WChar is a 16-bit unsigned type that can accommodate characters of any encoding. Don't use Char. Char is an 8-bit variable that cannot accommodate larger character encodings. The only time you should ever use Char is to pass a parameter to an older Palm OS function.

WChar ch; // Right. 16-bit character.

Char ch; // Wrong. 8-bit character.

When you receive input characters through the keyDownEvent, you'll receive a WChar value. (That is, the data.keyDown.chr field is a WChar.)

Even though character variables are now declared as WChar, string variables are still declared as Char *, even though they may contain multi-byte characters. See the section "Strings" for more information on strings.

Using Character Constants

Character constants are defined in several header files. The header file Chars.h contains characters that are guaranteed to be supported on all systems regardless of the encoding. Other header files exist for each supported character encoding and contain characters specific to that encoding. The character encoding-specific header files are not included in the PalmOS.h header by default because they define characters that are not available on every system.

To make it easier for the compiler to find character encoding problems with your project, make a practice of using the character constants defined in these header files rather than directly assigning a character variable to a value. For example, suppose your code contained this statement:

WChar ch = 'å'; // WRONG! Don't use.

This statement may work on a Latin system, but it would cause problems on an Asian-language system because the å character does not exist. If you instead assign the value this way:

WChar ch = chrSmall_A_RingAbove;

you'll find the problem at compile time because the chrSmall_A_RingAbove constant is defined in CharLatin.h, which is not included by default.

Missing and Invalid Characters

If during application testing, you see an open rectangle, a shaded rectangle, or a gray square displayed on the screen, you have a missing character.

A missing character is one that is valid within the character encoding but the current font is not able to display it. In this case, nothing is wrong with your code other than you have chosen the wrong font. The system displays a gray square in place of a missing double-byte character and an open rectangle in place of a missing single-byte rectangle (see Figure 13.1).

Figure 13.1 Missing/invalid characters

In multi-byte character encodings, a character may be missing as described above, or it may be invalid. In single-byte character encodings, there's a one-to-one correspondence between numeric values and characters to represent. This is not the case with multi-byte character encodings. In multi-byte character encodings, there are more possible values than there are characters to represent. Thus, a character variable could end up containing an invalid character--a value that doesn't actually represent a character.

If the system is asked to display an invalid character, it prints an open rectangle for the first invalid byte. Then it starts over at the next byte. Thus, the next character displayed and possibly even the remaining text displayed is probably not what you want. Check your code for the following:

• Truncating strings. You might have truncated a string in the middle of a multi-byte character.
  
  
• Appending characters from one encoding set to a string in a different encoding.
  
  
• Arithmetic on character variables that could result in an invalid character value.
  
  
• Arithmetic on a string pointer that could result in pointing to an intra-character boundary. See "Performing String Pointer Manipulation" for more information.
  
  
• Assumptions that a character is always a single byte long.
  

Use the text manager function TxtCharIsValid to determine whether a character is valid or not.

Retrieving a Character's Attributes

The text manager defines certain functions that retrieve a character's attributes, such whether the character is alphanumeric, etc. You can use these functions on any character, regardless of its size and encoding.

A character also has attributes unique to its encoding. Functions to retrieve those attributes are defined in the header files specific to the encoding.

---

WARNING!

In previous versions of the Palm OS, the header file CharAttr.h defined character attribute macros such as IsAscii. Using these macros on double-byte characters produces incorrect results. Use the text manager macros instead of the CharAttr.h macros.

---

Virtual Characters

Virtual characters are nondisplayable characters that trigger special events in the operating system, such as displaying low battery warnings or displaying the keyboard dialog. Virtual characters should never occur in any data and should never appear on the screen.

The Palm OS uses character codes 256 decimal and greater for virtual characters. The range for these characters may actually overlap the range for "real" characters (characters that should appear on the screen). The keyDownEvent distinguishes a virtual character from a displayable character by setting the command bit in the event record.

The best way to check for virtual characters, including virtual characters that represent the hard keys, is to use the TxtGlueCharIsVirtual function defined in the PalmOSGlue library. (See "Compatibility Information" for more information on the PalmOSGlue library.)

Therefore, when you check for a virtual character, first check the command bit in the event record. If the command bit is set, then the character is virtual. See Listing 13.2.

Listing 13.2 Checking for virtual characters

Retrieving the Character Encoding

Occasionally, you may need to determine which character encoding is being used. For example, your application may need to do some unique text manipulation if it is being run on a European device. You can retrieve the character encoding from the system feature set using the FtrGet function as shown in Listing 13.3.

Listing 13.3 Retrieving the character encoding

Strings

On systems that support the international manager and the text manager, strings are made up of characters that are either a single-byte long or multiple bytes long, up to four bytes. As stated previously, character variables are always two bytes long. However, when you add a character to a string, the operating system may shrink it down to a single byte if it's a low ASCII character. Thus, any string that you work with may contain a mix of single-byte and multi-byte characters.

Using characters of different sizes in a string has implications for manipulating strings, searching strings, and implementing the global find facility in your application. This section describes how to perform all of these tasks using text manager functions. It also describes how to create and display dynamically computed strings and how to display error messages.

• Manipulating Strings
  
  
• Performing String Pointer Manipulation
  
  
• Truncating Displayed Text
  
  
• Comparing Strings
  
  
• Global Find
  
  
• Dynamically Determining a String's Contents
  

  
  

  ---

  
  TIP: Many of the existing Palm OS functions have been modified to work with strings containing multi-byte characters. All Palm OS functions that return the length of a string, such as FldGetTextLength and StrLen, always return the size of the string in bytes, not the number of characters in the string.
  

  ---

  
  

Manipulating Strings

Any time that you want to work with character pointers, you need to be careful not to point to an intra- character boundary (a middle or end byte of a multi-byte character). For example, any time that you want to set the insertion point position in a text field or set the text field's selection, you must make sure that you use byte offsets that point to inter-character boundaries. (The inter-character boundary is both the start of one character and the end of the previous character, except when the offset points to the very beginning or very end of a string.)

Suppose you want to iterate through a string character by character. Traditionally, C code uses a character pointer or byte counter to iterate through a string a character at a time. Such code will not work properly on systems with multi-byte characters. Instead, if you want to iterate through a string a character at a time, use text manager functions:

• TxtGetNextChar retrieves the next character in a string.
  
  
• TxtGetPreviousChar retrieves the previous character in a string.
  
  
• TxtSetNextChar changes the next character in a string and can be used to fill a string buffer.
  

Each of these three functions returns the size of the character in question, so you can use it to determine the offset to use for the next character. For example, Listing 13.4 shows how to iterate through a string character by character until a particular character is found.

Listing 13.4 Iterating through a string or text

The text manager also contains functions that let you determine the size of a character without iterating through the string:

• TxtCharSize returns how much space a given character will take up inside of a string.
  
  
• TxtCharBounds determines the boundaries of a given character within a given string.
  

Listing 13.5 Working with arbitrary limits

Performing String Pointer Manipulation

Never perform any pointer manipulation on strings you pass to the text manager unless you use text manager calls to do the manipulation. For text manager functions to work properly, the string pointer must point to the first byte of a character. If you use text manager functions when manipulating a string pointer, you can be certain that your pointer always points to the beginning of a character. Otherwise, you run the risk of pointing to an inter-character boundary.

// WRONG! buffer + kMaxStrLength is not

// guaranteed to point to start of character.
buffer[kMaxStrLength] = '\0';

// Right. Truncate at a character boundary.

UInt32 charStart, charEnd;
TxtCharBounds(buffer, kMaxStrLength,
    &charStart, &charEnd);
TxtSetNextChar(buffer, charStart, chrNull);

Truncating Displayed Text

If you're performing drawing operations, you often have to determine where to truncate a string if it's too long to fit in the available space. Two functions help you perform this task on strings with multi-byte characters:

• WinDrawTruncChars - This function draws a string within a specified width, determining automatically where to truncate the string. If it can, it draws the entire string. If the string doesn't fit in the space, it draws one less than the number of characters that fit and then ends the string with an ellipsis (...).
  
  
• FntWidthToOffset - This function returns the byte offset of the character displayed at a given pixel position. It can also return the width of the text up to that offset.
  

Comparing Strings

Use the text manager functions TxtCompare and TxtCaselessCompare to perform comparisons of strings.

In character encodings that use multi-byte characters, some characters are accurately represented as either single-byte characters or multi-byte characters. That is, a character might have both a single-byte representation and a double-byte representation. One string might use the single-byte representation and another might use the multi-byte representation. Users expect the characters to match regardless of how many bytes a string uses to store that character. TxtCompare and TxtCaselessCompare can accurately match single-byte characters with their multi-byte equivalents.

Because a single-byte character might be matched with a multi-byte character, two strings might be considered equal even though they have different lengths. For this reason, TxtCompare and TxtCaselessCompare take two parameters in which they pass back the length of matching text in each of the two strings. See the function descriptions in the Palm OS SDK Reference for more information.

Note that StrCompare and StrCaselessCompare are equivalent, but they do not pass back the length of the matching text.

Global Find

A special case of performing string comparison is implementing the global system find facility. To implement this facility, you should call TxtFindString. As with TxtCompare and TxtCaselessCompare, TxtFindString accurately matches single-byte characters with their corresponding multi-byte characters. Plus, it passes back the length of the matched text. You'll need this value to highlight the matching text when the system requests that you display the matching record.

Older versions of Palm OS use the function FindStrInStr. FindStrInStr is not able to return the length of the matching text. Instead, it assumes that characters within the string are always one byte long.

Listing 13.6 and Listing 13.7 show how to implement a global find facility on all systems (whether the text manager exists or not), and how to implement a response to sysAppLaunchCmdGoto, which is the system's request that the matching record be displayed. These two listings are only code excerpts. For the complete implementation of these two functions, see the example code in your development environment.

Listing 13.6 Implementing global find

Listing 13.7 Displaying the matching record

Dynamically Determining a String's Contents

When working with strings in a localized application, you never hard code them. Instead, you store strings in a resource and use the resource to display the text. If you need to create the contents of the string at runtime, store a template for the string as a resource and then substitute values as needed.

For example, consider the Edit view of the Memo application. Its title bar contains a string such as "Memo 3 of 10." The number of the memo being displayed and the total number of memos cannot be determined until runtime.

To create such a string, use a template resource and the text manager function TxtParamString. TxtParamString allows you to search for the sequence ^0, ^1, up to ^3 and replace each of these with a different string. (If you need more parameters, you can use TxtReplaceStr, which allows you to replace up to ^9; however, TxtReplaceStr only allows you to replace one of these sequences at a time.) The PalmOSGlue library defines a function TxtGlueParamString, which calls TxtParamString if it exists or else implements the Latin equivalent of it.

In the Memo title bar example, you'd create a string resource that looks like this:

Memo ^0 of ^1

And your code might look like this:

Listing 13.8 Using string templates

Dates

If your application deals with dates and times, it should abide by the values the user has set in the system preference for date and time display. The default preferences at startup are different for the different languages, though they can be overridden.

To check the system preferences call PrefGetPreference with one of the values listed in the second column of Table 13.1. The third column lists an enumerated type that helps you interpret the value.

Table 13.1 Date and time preferences 

Preference	Name	Returns a value of type
Date formats (i.e., month first or day first)	prefDateFormat	DateFormatType
Time formats (i.e., use a 12-hour clock or use a 24-hour clock)	prefTimeFormat	TimeFormatType
Start day of week (i.e., Sunday or Monday)	prefWeekStartDay	0 (Sunday) or 1 (Monday)

To work with dates in your code, use the Date and Time Manager API. It contains functions such as DateToAscii, DayOfMonth, DayOfWeek, and DaysInMonth, which allow you to work with dates independent of the user's preference settings.

Numbers

If your application displays large numbers or floating-point numbers, you must check and make sure you are using the appropriate thousands separator and decimal separator for the device's country by doing the following (see Listing 13.9):

Listing 13.9 Working with numbers

Compatibility Information

If you want to maintain backward compatibility with earlier releases but you still want to use the localization features described in this chapter, you can link your application with the library PalmOSGlue (PalmOSGlue.lib or libPalmOSGlue.a). This library provides these features for versions 2.0 and 3.0.

When you use PalmOSGlue, you use the text manager in the same way as described in this chapter, but the names of the functions are different. For example, TxtFindString is named TxtGlueFindString in the PalmOSGlue. (See the chapter "PalmOSGlue Library" of the Palm OS SDK Reference for a complete mapping table.) When you make a call to a glue function (TxtGlueFunc, FntGlueFunc, or WinGlueFunc), the code in PalmOSGlue either uses the text manager or international manager on the ROM or, if the managers don't exist, executes a simple Latin equivalent of the function.

PalmOSGlue is a linkable library that is bound to your project at link time. It is not a shared library. PalmOSGlue will increase your application's code size. The exact amount by which your code size increases depends on the number of library functions you call; the linker strips any unused routines and data.

Palm OS version 3.1 contains the following changes from previous releases that affect strings, text, and localization. These changes may affect you if you're updating an application written to run on a prior release or if you want to maintain backward compatibility with prior releases:

• The keyDownEvent structure's chr field (which contains the input character) has been changed from a Word to a WChar. The chr field may contain a multi-byte character, so you should never copy the chr field into a Char variable or pass it to a function using a Char parameter. Always use WChar.
  
  
• Some of the special Palm OS glyphs in the high ASCII range (such as the shortcut stroke and the command stroke) have been moved down into the control code range, and other characters (such as the numeric space and horizontal ellipsis) have been copied into the control range so that they're guaranteed to exist in every encoding. For the numeric space and horizontal ellipsis, you can use the macros ChrNumericSpace and ChrHorizEllipsis to return the appropriate character regardless of the character map. In PalmOSGlue, these two macros are named TxtGlueGetNumericSpaceChar and TxtGlueGetHorizEllipsisChar, respectively.
  
  
• The four playing-card characters have been moved from the high ASCII range in the standard four fonts to the 9-point Symbol font.
  
  
• Character attribute functions and macros are now obsolete and have been replaced by functions and macros in the text manager.
  
  
• The String Manager functions StrChr and StrStr now treat buffers as characters, not arbitrary byte arrays. If you previously used these functions to search data buffers, your code may no longer work.
  

Notes on the Japanese Implementation

This section describes programming practices for applications that are to be localized for Japanese use. It covers:

• Japanese Character Encoding
  
  
• Japanese Character Input
  
  
• Displaying Japanese Strings on UI Objects
  
  
• Displaying Error Messages
  

Japanese Character Encoding

The character encoding used on Japanese systems is based on Microsoft code page 932. The complete 932 character set (JIS level 1 and 2) is supported in both the standard and large font sizes. The bold versions of these two fonts contain bolded versions of the glyphs found in the 7-bit ASCII range, but the single-byte Katakana characters and the multi-byte characters are not bolded.

Japanese Character Input

On current Japanese devices, users enter Japanese text using Latin (ASCII) characters, and special software called a front-end processor (FEP) transliterates this text into Hiragana or Katakana characters. The user can then ask the FEP to phonetically convert Hiragana characters into a mixture of Hiragana and Kanji (Kana-Kanji conversion).

Four silkscreen buttons added to the Japanese device control the FEP transliteration and conversion process. These four FEP buttons are arranged vertically between the current left-most silkscreen buttons and the Graffiti^® area. The top-most FEP button tells the FEP to attempt Kana-Kanji conversion on the inline text. The next button confirms the inline text and terminates the inline conversion session. The third button toggles the transliteration mode between Hiragana and Katakana. The last button toggles the FEP on and off.

Japanese text entry is always inline, which means that transliteration and conversion happen directly inside of a field. The field code passes events to the FEP, which then returns information about the appropriate text to display.

During inline conversion, the Graffiti space stroke acts as a shortcut for the conversion FEP button and the Graffiti return stroke acts as a shortcut for the confirm FEP button.

Displaying Japanese Strings on UI Objects

To conserve screen space, you should use half-width Katakana characters on user interface elements (such as buttons, menu items, labels, and pop-up lists) whenever the string contains only Katakana characters. If the string contains a mix of Katakana and either Hiragana, Kanji, or Romaji, then use the full-width Katakana characters instead.

Displaying Error Messages

You may have code that uses the macros ErrFatalDisplayIf and ErrNonFatalDisplayIf to determine error conditions. If the error condition occurs, the system displays the file name and line number at which the error occurred along with the message that you passed to the macro. Often these messages are hard-coded strings. On Japanese systems, the Palm OS traps the messages passed to these two macros and displays a generic message explaining that an error has occurred.

You should only use ErrFatalDisplayIf and ErrNonFatalDisplayIf for totally unexpected errors. Do not use them for errors that you believe your end users will see. If you wish to inform your users of an error, use a localizable resource to display the error message instead of ErrFatalDisplayIf or ErrNonFatalDisplayIf.

Summary of Localization