Issue 125483 - Corrupted font display on Japanese OS/2 systems
Summary: Corrupted font display on Japanese OS/2 systems
Alias: None
Product: General
Classification: Code
Component: ui (show other issues)
Version: 4.1.0
Hardware: PC OS/2
: P3 Major (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
Depends on:
Blocks: 126518
  Show dependency tree
Reported: 2014-08-21 13:08 UTC by Alex Taylor
Modified: 2015-09-14 11:47 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Note You need to log in before you can comment on or make changes to this issue.
Description Alex Taylor 2014-08-21 13:08:43 UTC
Entering Japanese text in a text document on a Japanese language version of OS/2 results in unreadable characters on the screen. Specifically, kanji and fullwidth kana characters either do not appear at all, or else are drawn superimposed on top of one another in the first character column.  This occurs with any Japanese-capable TrueType font, including "GOTHIC", "MINCHO", "MS 明朝", "M+ 2p", "Times New Roman WT J", "Arial Unicode MS", etc.

Steps to Reproduce:
1. Open or create a text (Writer) document.
2. Copy some Japanese text to the clipboard in Firefox (or some other application supporting the Unicode clipboard). 
3. Paste the text into Writer.

(Use of the clipboard is necessary to enter kanji, due to IME conversion support not working as per ticket #125469. For fullwidth kana, the system input buttons for hiragana or katakana at the bottom of the desktop can be used for direct entry.)

Actual Results:
CJK characters other than halfwidth katakana either do not display, or appear drawn on top of one another in a single column.

Expected Results:
The CJK characters should be displayed correctly.

OpenOffice 4.1.1 OS/2 RC3 (running on IBM ThinkPad X60, eComStation 2.x Japanese).

Additional Builds & Platforms:
Another user has confirmed the problem with OpenOffice 2.4 on IBM ThinkPad R51.

Additional Information:
The issue does NOT occur when pasting Japanese text into OpenOffice Writer on a non-Japanese version of OS/2. The problem appears specific to the Japanese DBCS OS version.

The issue occurs regardless of whether Innotek Font Engine version 2.4 or 2.6beta is installed.
Comment 1 Alex Taylor 2014-09-13 15:09:38 UTC
A correction to the above: I have re-tested using several different Japanese/CJK fonts. In fact, Japanese text does display correctly if a Unicode-capable TrueType font (such as Arial Unicode MS or Droid Sans Fallback) is used.

The confusion was caused by attempting to change the font AFTER pasting by highlighting the text (this was apparently failing because text selection seems unreliable when attempting to highlight the corrupted text). However, setting the font and THEN pasting the text demonstrates that it does work with those fonts.

The problem appears to occur when using a font with a native CJK cmap, such as MS Mincho.  (Again, however, this ONLY occurs when running on the Japanese OS version.)
Comment 2 Alex Taylor 2014-09-13 15:24:39 UTC
To be more specific, it appears that the font handler (FT2LIB) is treating these fonts differently.  I suspect that it detects when the font has a Shift-JIS cmap, notes that the primary OS codepage is also Shift-JIS, and accordingly fails to use the Unicode cmap as it should.
Comment 3 Alex Taylor 2014-10-12 13:56:22 UTC
After some more testing...

The 1993 versions of MSMINCHO.TTF and MSGOTHIC.TTF which are included in OS/2 Japanese (and other DBCS versions) exhibit this problem.  The problem MIGHT be related to the fact that they include non-Latin Japanese names (MS明朝 and MSゴシック) which are (only) used on DBCS OS/2 systems.  Since they in fact contain only Unicode cmaps, this is the only factor I can identify which might explain the different behaviour on DBCS vs non-DBCS systems.

If newer versions (MSMINCHO.TTC and MSGOTHIC.TTC) from e.g. Windows 7 are installed instead, those ones seem to work perfectly well. These ones have English-only names which are used on DBCS and non-DBCS systems alike.

Of the other default Japanese system fonts, Heisei Mincho and Heisei Kaku-Gothic (both versions) do not work, even though they have English names.  (However, I have checked the TTFs and they do not have Unicode cmaps.)  

So, the workaround seems to be: avoid the use of all the DBCS outline fonts included with OS/2.  This is pretty cumbersome, of course, since users are required to hunt down other DBCS fonts to use in their place.

This is almost certainly (yet another) bug in the Innotek Font Engine. This really has to be replaced, it is causing too many problems at this point!
Comment 4 Alex Taylor 2014-10-12 14:00:29 UTC
A more immediate problem, which may be easier to address, is that the OS/2 OpenOffice port for some reason uses "GOTHIC" and "MINCHO" as the default for Asian fonts when run on DBCS systems.

"GOTHIC" and "MINCHO" are aliases which are implemented through a special font driver which isn't even supported by OpenOffice (or FreeType), and which point to bitmap fonts in any case.  This certainly isn't going to work.

So, the defaults for Asian fonts should be changed to something else; I suggest "Times New Roman WT J", which is at least fairly likely to be installed on most OS/2 systems, and is a TrueType font that works.
Comment 5 Alex Taylor 2014-10-15 10:39:54 UTC
CORRECTION regarding 'MS Mincho' and 'MS Gothic'.  After more tests, it turns out that the problem isn't in their cmaps at all.  It is entirely due to the handling of the font names.

Normally I use a FreeType-based system font driver (FT2IFI), which reports the English font names of MS Mincho and MS Gothic to the OS (which in turn is picked up by OpenOffice).

However, if I revert to the default IBM TrueType driver, its name reporting behaviour is different - on DBCS systems it defaults to using the Japanese font names.  Although OpenOffice doesn't use the system font driver for rendering, it does rely on it to report the names of installed fonts, and it evidently cannot cope with the Japanese font names.

So, the workaround in the case of MS Mincho and MS Gothic is to BOTH replace the default versions with the newer (Windows XP/Vista/7) versions, AND install the FreeType IFI driver (either FREETYPE.DLL or FT2IFI.DLL) in OS/2.

This does not help with the other default OS/2 Japanese fonts, as they either lack English names entirely, do not have Unicode cmaps, or both.