Issue 87970

Summary: Hebrew Characters in Spreadsheet Names Became Misdisplayed
Product: gsl Reporter: zmanz <junkie>
Component: codeAssignee: stefan.baltzer
Status: CLOSED FIXED QA Contact: issues@gsl <issues>
Severity: Trivial    
Priority: P2 CC: hdu, issues, kaplanlior, nadavkav, oc666, rene
Version: OOo 2.4.0   
Target Milestone: OOo 3.3   
Hardware: All   
OS: Unix, all   
Issue Type: DEFECT Latest Confirmation in: ---
Developer Difficulty: ---
Attachments:
Description Flags
lots of info on OS info
none
Test Spreadsheet
none
Screenshot of the Issue
none
here's a patch for illustration purposes
none
the offending font for reference
none
update patch to apply against DEV300_m61
none
Hebrew characters are not shown in format->character dialog
none
extend patch to also take account of the font weight in the char cache none

Description zmanz 2008-04-07 02:08:52 UTC
Hello all,

I use Gentoo GNU/Linux x86_64 with kernel 2.6.25-rc8, and I do update my box,
more or less, on a weekly basis.

a few days ago I upgraded from OpenOffice.org v2.3.1 to v2.4.0, and Since I did,
All the Hebrew Characters in Spreadsheet Names suddenly became Misdisplayed.

at occasion which I upgraded OpenOffice.org, I didn't make any other change to
the system - or at least not a one which I can recall.

Any Ideas ?

Assistance would be very much appreciated :)
Comment 1 zmanz 2008-04-07 02:14:51 UTC
Created attachment 52598 [details]
lots of info on OS info
Comment 2 zmanz 2008-04-07 02:15:34 UTC
Created attachment 52599 [details]
Test Spreadsheet
Comment 3 zmanz 2008-04-07 02:16:12 UTC
Created attachment 52600 [details]
Screenshot of the Issue
Comment 4 zmanz 2008-04-07 02:19:18 UTC
as you can see, the issue only appears on the names of the inactive spreadsheets.
Comment 5 Martin Hollmichel 2008-04-07 14:06:04 UTC
issue needs confirmation before getting a target milestone.
Comment 6 hdu@apache.org 2008-04-08 08:35:00 UTC
Probably another result of the highly requested use of fontconfig for substituting font  (issue 54603).
Comment 7 kaplanlior 2008-04-09 11:07:43 UTC
This bug isn't limited to spreadsheet but seems to be a general UI problem.

Already reported in Debian at:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=433231

The Debian bug report includes more screen shots of the problem.

Please change status from UNCONFIRMED to NEW.
Comment 8 rene 2008-04-09 11:42:13 UTC
confirm
Comment 9 hdu@apache.org 2008-04-09 12:38:06 UTC
I wonder if you can reproduce it if the environment variable "SAL_DISABLE_FC_SUBST" is set before starting 
soffice?
Comment 10 zmanz 2008-04-09 14:48:01 UTC
Setting the SAL_DISABLE_FC_SUBST environment varible indeed masks the issue.
Comment 11 hdu@apache.org 2008-04-10 11:40:42 UTC
That means that fontconfig suggests a wrong font. This is probably caused by a non-Hebrew font which 
misleadinly advertises Hebrew support.

It would be interesting to know which font that would be. The easiest way to find this out is to recreate the 
problem in a Writer document. Use Format->Character to set the CTL-font to the font selected as UI font 
and check that the problem in the resulting document is similar to the spreadsheet tabs. Then export this 
document to PDF. Then use your favorite PDF viewer, open font properties and note the font names. One 
of the fonts mentioned will probably have empty-box glyphs for Hebrew. You can use the very nice 
fontforge application to analyze this font.
Comment 12 zmanz 2008-04-10 16:46:28 UTC
How can I find which fonts does the UI use ? (for looking-up the problematic one
- as suggested by 'hdu')
Comment 13 hdu@apache.org 2008-04-11 09:13:48 UTC
The UI-font depends on the option "use system font for user interface". If it is enabled (it usually is, but 
please check in the dialog Tools->Options->OpenOffice->View), then use your system configuration tool 
(kconfig for KDE, gconfedit for GNOME) to find out what your system suggests as user interface font.
Comment 14 oc666 2008-05-10 21:34:33 UTC
I use XFCE and I have the same problem.
I enabled "use system font for user interface" and after restarting openoffice I
still get squares. I'm using culmus fonts (supporting Hebrew).
Comment 15 frank.loehmann 2008-05-22 09:35:13 UTC
This issue is important and listed on the quarterly review for Calc:
http://wiki.services.openoffice.org/wiki/2008_Q2_Review_of_Spreadsheet_Project
Comment 16 nadavkav 2008-05-22 13:21:32 UTC
setting the GNOME "Application Font" to Yehuda CLM (Culmus hebrew font project)
and setting the OO to use system fonts, solve this issue.
(on Fedora 8 + OOo 2.3 , Debian Sid + OOo 2.4)

when i say solve i mean - workaround :-)

it should be working just fine with Sans or DejaVu Sans (and many other fonts)
that used to display those same text labels just fine in previous OO versions.
Comment 17 shai2platonix 2008-05-22 23:08:59 UTC
I just checked, and I think I'm on to something:

1) In OpenOffice, I can see the problem only with the DejaVu Sans font (to test
quickly: in writer, write a few hebrew words, select them, and open the
"Format->Character" dialog. In this dialog, you can see how the text will look
as you select the font).

2) So I suspected there was something wrong with the font, and tried to
reproduce in KWord. No go -- in KWord, everything stays fine

3) Then I noticed a difference in the font selection dialogs -- the list of
typefaces (KWord calls them "font styles"). OpenOffice has "Extralight", "Book",
and combinations from the set { Condensed, Bold, Oblique }, whereas KWord has  
{ Light, Regular, Bold } x { Italic, (not italic) }. These lists change from
font to font, in both programs; I don't know how consistent the lists are in all
the fonts.

4) Actually, the problem is only in the ExtraLight typeface; it is the first in
the list and selected by default when DejaVu Sans is selected. Again, I can't
say if other fonts have problematic typefaces which aren't the font's default
typeface.

For non-Hebrew users who want to test, and need a couple of Hebrew words to
paste into a doc, here: כמה מלים בעברית.

Comment 18 oc666 2008-05-23 00:11:05 UTC
I just get it work on xubuntu 8.04.
I changed in the configuration (tools->options) the replace table in fonts section.
I just configure it to add font dejavu and replace it with arial. Don't forget
to set it to always.
Comment 19 caolanm 2008-05-26 15:59:37 UTC
I believe I know what this is, the font being used is probably "DejaVu Sans" and
we look for a LIGHT font for tabs name in the background and do get it as DejaVu
Sans the extralight version. That font is then missing the Hebrew glyphs so we
ask for a fallback font to use, fontconfig says "DejaVu Sans is a good fallback
font for missing glyphs in a LIGHT DejaVu Sans". We go "but that's the same
font, no way", but it is actually a different font as fontconfig also says
"NORMAL" version which is a vital bit of info that we lose along the way.

I'll try and cook up a proto-patch for consideration

Comment 20 hdu@apache.org 2008-05-26 16:09:43 UTC
@cmc: Great analysis, thanks! Issue 79878 is biting us again under new disguise it seems...
Comment 21 nadavkav 2008-05-26 16:23:54 UTC
see a quick workaround:
https://bugzilla.redhat.com/show_bug.cgi?id=445588

removing dejavu-fonts-experimental package (for fedora machines, at least)
workedaround this issue :-)
Comment 22 caolanm 2008-05-26 16:50:50 UTC
Created attachment 53948 [details]
here's a patch for illustration purposes
Comment 23 caolanm 2008-05-26 16:51:35 UTC
Created attachment 53949 [details]
the offending font for reference
Comment 24 caolanm 2008-05-26 16:56:33 UTC
So that patch would "work", and shows that the problem is in considering all
fonts with the same name to be the same font when it comes to discarding a
fontconfig provided candidate. 

So the proto-patch bubbles back up the discovered replacements details and we
can use that to describe the font in OOo terms. I have no idea though whether
it's a good idea or not to do what this straightforward patch does and clobber
the details of the requested font with the details of the found font.
Comment 25 Martin Hollmichel 2008-06-09 08:46:13 UTC
what will be the correct target milestone for this issue.
Comment 26 hdu@apache.org 2008-06-11 11:56:31 UTC
Oops the target needs to be adjusted. There are two easy workarounds that are both confirmed to work:
- remove the dejavu-sans-experimental package
- setting the environment variable SAL_DISABLE_FC_SUBST before starting office
Comment 27 dudyk 2009-06-11 20:37:06 UTC
The workarounds may work on fedora, but not on Ubuntu (9.04 AMD64).
The variable doesn't help and the package doesn't exist and there is no package
that seems similar that is installed.
In addition, it seems this bug exists in the Xandros that comes with the eeePC 701.
Comment 28 caolanm 2009-10-09 15:15:29 UTC
Created attachment 65273 [details]
update patch to apply against DEV300_m61
Comment 29 caolanm 2009-10-09 15:16:54 UTC
just in case it wasn't clear the first time round (protofix.patch was a stupid
name), this patch does fix the immediate problem
Comment 30 rene 2009-12-03 13:47:13 UTC
*** Issue 107296 has been marked as a duplicate of this issue. ***
Comment 31 rene 2009-12-03 13:55:00 UTC
dudyk: use your tools, luke ;-)

$ apt-file search DejaVuSans-ExtraLight.ttf
ttf-dejavu-extra: /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-ExtraLight.ttf

(disclaimer: I didn't try the workaround)
Comment 32 dawnlight 2009-12-07 11:22:19 UTC
Dear friends,

There is now a bounty on this bug:
http://www.fossfactory.org/project/p192
Comment 33 hdu@apache.org 2010-01-18 16:00:03 UTC
Applied the latest patch in CWS fchints for target 3.3. Thanks Caolan!
Comment 34 rene 2010-01-30 14:18:01 UTC
Hmm, was this fix also supposed to fix the tabs in format->characters etc?

Then it doesn't fully fix it - I just applied the patch on my 3.2 and it still
shows squares.
Comment 35 nadavkav 2010-01-30 16:31:26 UTC
Unselected (background) Tab Names
if i am using an English UI it works fine
if i am using an Hebrew UI it work fine.

format->characters (dialog)
if i am using an English UI it works fine
if i am using an Hebrew UI it does not work fine.

see attached screen shot.

Debian sid OOo 3.2
Comment 36 nadavkav 2010-01-30 16:34:09 UTC
Created attachment 67487 [details]
Hebrew characters are not shown in format->character dialog
Comment 37 caolanm 2010-02-09 12:15:36 UTC
*** Issue 89839 has been marked as a duplicate of this issue. ***
Comment 38 caolanm 2010-02-09 12:18:19 UTC
Created attachment 67718 [details]
extend patch to also take account of the font weight in the char cache
Comment 39 caolanm 2010-02-09 12:27:47 UTC
In addition to the previously discussed stuff there's also a cache that looks up
successfully found chars and returns a fontname where they were found. But that
font might have only worked for a given weight, so e.g. if a *normal* weight
DejaVu Sans rendered some text ok, then if we try *light* weight foo for the
same string later we'll be told from the cache that DejaVu Sans is good for that
string. But we'll try *light* weight DejaVu Sans which is an epic fail as that
font doesn't have Hebrew chars in the light variant.

Suggest we cache chars+weight as the cache unit.
Comment 40 hdu@apache.org 2010-02-17 08:56:54 UTC
@sba: please verify in CWS fchints01
Only the X11-ports are affected.
Comment 41 stefan.baltzer 2010-03-02 09:33:10 UTC
Verified in CWS fchints01.
Comment 42 caolanm 2010-04-29 16:42:50 UTC
closing, integrated DEV300_m75