Issue 14069

I have been testing again, and I have new conclusions: it is partially
a matter of fonts and partially a matter of keyboards. If you use both
the lucidasans font and the "Insert->Special Character" menu the
vowels are placed perfectly beneath and over the consonants. If you
try to use any other unicode font (including the MSWin ones as Tahoma
or Times New Roman) the vowels are not correctly handle. Those MS
fonts work well in OO.org for Windows.

The standard Hebrew keyboard which comes with X is not working for the
vowels in OO.org but it works perfectly with any other Gnome2
application, so you have to use the "Insert->Special Character" menu
to get them (besides lucidasans, of course).

OO.org 1.1 Beta - 2 has solved the keyboard problem, so you can input
the Hebrew vowels as in any other Gnome2 application. Thanks!

However, the fonts problem remains. I have been testing again and I
have  new conclusions: the problem is just a matter of screen
rendering (or visualization). As I have metioned the only font is
redered OK is the one called "Lucida sans", but I have realized that
the version which comes with j2re is the only lucky one.

The vowels are not really put in the wrong place as I thought, they
are just viewed in the wrong place. If you chage the document view to
Web, then everything un every font is shown perfect.

DL->HDU: Could you please takeover?

probably more of issue 13254 and issue 14072, i'll have a look

We will not support a full hebrew version in OOo1.1. Out of this
reason I set this task to OOo 2.0.

pl->hdu: welcome back

Just to note- this is not just a font problem. The same fonts in
windows wich displays Hebrew diacritics ok, fails to do so on linux.

Yes, I see the problem when the menu View->OnlineLayout mode is disabled. 
 

The simple patch at
http://www.jtcsv.com/cgibin/icu-bugs/incoming?id=3306, which works for
ICU 2.2 and 2.6.1, causes LayoutEngine to use OpenType to position the
vowels which makes a huge difference in legibility using the ICU
sample application gnomelayout (source/samples/layout).

So this gets us one step closer to having decent vowel rendering, but
this same patch for OOo's libicule results in messed up output. Why is
OOo bothered by ICU using the OpenType path for Hebrew rather than the
non-OpenType? Hopefully only a minor tweak is needed to get OOo and
ICU to cooperate better in this case. I would appreciate any pointers
to figure out this problem.

For comparison, I attach a png of OOo without a patch; of OOo with the
patch; and of the Windows version of OOo showing the same file as a
benchmark of working Hebrew vowels. Note that daghesh and hiriq are
especially troublesome, as well as seghol with consonants that have
descenders. Also attached is the sxw file used to generate these
screenshots.

The screenshots were made with the font "Ezra SIL" (SILEOT.ttf)
available at http://www.sil.org/computing/fonts/silhebruni/download.htm.

See ICU bug 3306
(http://www.jtcsv.com/cgibin/icu-bugs/incoming?id=3306) for more
information and screenshots. Here's the patch, which is the same for
ICU 2.2 or 2.6.1:

--- HebrewShaping.cpp.orig      Tue Jun  3 15:58:22 2003
+++ HebrewShaping.cpp   Fri Oct  3 00:07:21 2003
@@ -11,12 +11,14 @@
 
 U_NAMESPACE_BEGIN
 
-const LETag ligaFeatureTag  = 0x6C696761; // 'liga'
+const LETag ccmpFeatureTag  = 0x63636d70; // 'ccmp'
+const LETag kernFeatureTag  = 0x6b65726e; // 'kern'
+const LETag markFeatureTag  = 0x6d61726b; // 'mark'
 const LETag emptyTag        = 0x00000000; // ''
 
 const LETag hebrewTags[] =
 {
-    ligaFeatureTag, emptyTag
+    ccmpFeatureTag, kernFeatureTag, markFeatureTag, emptyTag
 };
 
 void HebrewShaping::shape(const LEUnicode * /*chars*/, le_int32
/*offset*/, le_
int32 charCount, le_int32 /*charMax*/,
--- LayoutEngine.cpp.orig       Thu Oct  2 00:05:44 2003
+++ LayoutEngine.cpp    Thu Oct  2 23:44:37 2003
@@ -428,6 +428,7 @@
             break;
 
         case arabScriptCode:
+        case hebrScriptCode:
             result = new ArabicOpenTypeLayoutEngine(fontInstance,
scriptCode, l
anguageCode, gsubTable);
             break;
 

Created attachment 9920 [details]
stock OOo incorrectly rendering Hebrew vowels

Created attachment 9921 [details]
OOo with patched ICU rendering Hebrew vowels with garbage

Created attachment 9923 [details]
Windows version of OOo correctly rendering Hebrew vowels

Created attachment 9925 [details]
OOo sxw document used to generate previous screenshots

HDU->Matt: Did you use the original ICU 2.2 or OOo's patched ICU for
your test? With your patch on OOo's ICU it looks good to me, so if you
can confirm that it works I'd be happy to put it into OOo1.1.1 or
1.1.2. I'm attaching the (OOo+your) patched linux libicule binary.

Are you sure that replacing the use of OpenType's "liga" feature by
the "ccmp"+"kern"+"mark" features would not cause problems with some
fonts?

Also Eric's reply in http://www.jtcsv.com/cgibin/icu-bugs/incoming?id=3306
worries me. Of course he has right. To get important fonts like
"David" to work faking GSUB/GPOS etc. is a good idea.

What do you suggest?

Created attachment 10491 [details]
patched libicule.so.22.0

Herbert, thanks for the libicule.so. Your version no longer draws
boxes. After wasting much time I realized that OOo's icu-2.2-patch is
a little different than IBM's version, and includes the code to remove
"deleted" characters. So the GSUB stuff (dagesh, sin/shin dots, etc.)
seems to work fine.

But even with the boxes gone, there is still a problem with the
rendering of vowel points and with text selection. Many of the vowels
are misplaced, but more distracting is that some consonants end up
with extra space between them which makes them look like separate
words. I'm attaching another screenshot to show this.

Text selection: OOo fails to highlight the entire line. This is
visible in the screenshot; I highlighted the line by pressing Home
then Shift-End, and also tried the mouse to the same effect. Somehow
the effects of ICU rearranging the vowel glyphs using the font's GPOS
tables messes up text measurement. A line of text without any vowels
can be selected perfectly.

These two problems are probably related to measuring glyph width and
advance. I'm not sure how to proceed from here. Can you suggest where
I might look next?

I do not recommend committing my patch until these two problems are
resolved. The pre-patch output is the lesser of these two evils.

Matt

Created attachment 11283 [details]
Same file rendered using HDU's patched libicule, showing selection and rendering defects

BTW, with regard to my last comment, compare the screenshot of faulty
rendering to the attachment "Windows version of OOo correctly
rendering Hebrew vowels."

With regard to your other questions: "Are you sure that replacing the
use of OpenType's "liga" feature by the "ccmp"+"kern"+"mark" features
would not cause problems with some fonts?..."

Since "kern" and "mark" are GPOS features they don't conflict with
"liga" (which is a GSUB feature). These two shouldn't break rendering
with other fonts, as long as the layout engine only applies those
features for fonts that have those tables. So "kern" and "mark" should
be safe for any font.

Regarding "liga," I would suggest using the font's GSUB tables unless
the font doesn't have any, in which case we should fall back to using
the fake 'liga' table.

Eric said he thought he could change ICU to query the font for the
existence of "ccmp" tables without breaking any public API, and that
he's going to think about what should be done about "ccmp" and "dlig"
vs "liga".

I hope this helps.

Matt

dina: fwi

>The standard Hebrew keyboard which comes with X is not working for the
>vowels in OO.org but it works perfectly with any other Gnome2

the lyx keyboard variant under Xfree 4.3 does allow to enter Hebrew diacritics
under linux, however they do indeed go after the charachter instead of
under/over it.

Basicly, entering Hebrew diacritics (or viewing Hebrew diacritics) under linux
is completely broken.

----
To setup the lyx Hebrew keyboard variant under Xfree 4.3, use something similar
to my setting in XF86Config:

Section "InputDevice"
	Identifier  "Keyboard0"
	Driver      "keyboard"
	Option      "XkbLayout" "us,il"
	Option  "XkbOptions" "grp:switch,grp:alt_shift_toggle,grp_led:scroll"
	Option "XkbVariant" ",lyx"
EndSection

Hebrew diacritics are in Shifted Hebrew positions

*** Issue 26065 has been marked as a duplicate of this issue. ***

cp: retargeted to Office-Later due to limited ressources

sforbes said: "Basicly, entering Hebrew diacritics (or viewing Hebrew
diacritics) under linux is completely broken."

I set the keyboard distribution and variant with KDE. I´m using Israeli keyboard
with lyx variant and I can write  vowels in konqueror, evolution and many other
softwares. The only problem is in OOo.

Is anyone looking into this? A school here wants to deploy OOo on Linux as a
"model project" (which will be deployed in more schools if this goes well), and
they need diacritic support under linux.

HDU: Do you have plans to upgrade to the latest ICU (3.2)? It renders Hebrew
diacritics without any patching.

sforbes: I'm tinkering with it again. I don't have much skill inside OOo, but
maybe I can diagnose the problem for others to fix.

As one of the main authors of the Hebrew rendering module in pango I wish that
OO would be using pango. As I am aware that that is not going to happen in the
near future, may I suggest that the heuristics for wovel placement that I
created in the Hebrew rendering module be copied. It is obviously not as good as
real OpenType GPOS table support, but it has the advantage that it works also
with fonts without such tables.

Look at hebrew_shaper_get_cluster_kerning() in
pango/modules/hebrew/brew-shaper.c in the pango cvs.

when will you fix the tham thing??!??!!

Well, it's been 20 months since I first reported this problem. I remember that
solving this issue was targeted OOo 2.0 but I have just tried the 1.9.65
developer version and the issue is still there. What it surprised me is that it
keeps working perfectly with my Lucida Sans Unicode font. I can provide the font
so you can figure out what it is going *right* with it. Let me remind you that
this is an OO.org (Linux Version) issue and not a general Linux problem since
Gnome (thanks Pango) and KDE handle Hebrew diacritics very nicely.

This issue also affects OS X.

OOo'S text layouter is ready for ICU 3.2, but AFAIK some other parts of OOo are
not ready for an update... Since ICU's layout engine is somewhat independent of
other parts of ICU it may be possible to have a mix of new layout engine and old
ICU 2.6. But this would mean a support nightmare. I'd much rather wait for the
other parts of OOo to become ready...

Well, Pango and Qt handle the screen well, except for the dagesh (shift-d on 
the lyx variant of the Israeli keyboard). Pango can even handle the printing 
well, but only gedit seems to use that faculty. Abiword and all kde apps choke 
a bit on printing by shifting all the diacritics in the OOo way. I filed a bug 
report on the kde site (http://bugs.kde.org/show_bug.cgi?id=96858 ) and 
eventually with Trolltech, they confirmed it, and are going to produce a patch 
for the next version of QT (or so I was told. If OOo has such trouble with 
this issue, I hope this doesn't mean that it is programatically too difficult 
for rapid fixing). 
 
I downloaded the Lucida font and will test them later. However, I fear I know 
too little about fonts to figure out what that font does right. 

I just tested Lucida Sans and a few other fonts, and believe I understand how 
LS works. First of all, it doesn't work well; the diacritics aren't well 
centered. Second, I believe I understand somewhat why. The diacritics seem to 
have their own length, and that length is better set than in other font 
collections. However, since the diacritics have their own length (so I 
speculate) instead of their length and position being proportional to the 
letter they relate to, a diacritic may shift near a yod or similarly 
non-average width letters. That is a solution that works somewhat and it is 
the best we can do with OOo, however, I can't wait for the new ICU (3.2) to be 
integrated in OOo. 
 
Is my analysis kind of right? 

Another data point: with the patch I mentioned previously, and with a good
OpenType font (SBL Hebrew, Ezra SIL, or Cardo), in "Online Layout" mode the
vowels/nikud render perfectly. Actually this isn't really a new data point,
because HDU mentioned earlier that he sees this rendering problem with Online
Layout disabled; just wanted to make clearer that OOo can in some instances
render Hebrew nikud/vowels properly.

Hi, I'm suffering from this problem as well - I do a lot of writing with vowels
and cantilation marks and currently can only do so in MS word in windows. I'd
prefer to be able to use OO in linux, since I'd like to be MS-free.  Is this
going to be fixed anytime soon?

oh, one other comment - the bible with cantilation marks from mechon mamre works
fine in konqueror. but not in mozilla, and not in scribus.

Reading the bugreport for ICU 
(http://www.jtcsv.com/cgibin/icu-bugs/incoming?id=3306) revealed that there 
are diff's associated to it.   
 
It looks like these changes will fix a error in the GPOS and GSUB processing 
of ICU. As far as see is the changed code also in ICU 3.2  
 
It might be of interest to add this patch to the ICU 2.6 patch of OOo? 
 
I have combined the diff's and will attach them to this bugreport.  
 
Jens 

Created attachment 23011 [details]
the patch from ICU bugdatabase

Does anyone know whether this problem has finally been fixed in 2.0 Beta? If
not, will it be eventually fixed in 2.0? 

How can I rebuild ICU 2.6 using the above patch? (I mean, where should I get the
"original" source?)

For now I am stuck with Windows.... very sad :(

Thanks!

The vowels problem is solved long ago, as far as I am concerned. I use
OpenOffice.org 1.1.4 - binary installation on Gentoo Linux, and it is perfect. I
have edited the whole Pessach-Haggada with Nikud (I took the Haggada text using
cut&paste from
http://he.wikisource.org/wiki/%D7%94%D7%92%D7%93%D7%94_%D7%A9%D7%9C_%D7%A4%D7%A1%D7%97

and I had no problem editing and printing (to pdf) the all document, full of
voweled hebrew.

FYI

Shahar Livne
Open Source Development and Support
http://livnex.com

as of m100 (tested on Linux) the problem remains as it was. I also tested,
perhaps m95 or m93 on windows and the problem still exists there as well.
I can't help you with any compilation/patching problem, I CAN, however, give you
this link:
ftp://mirrors.dotsrc.org/openoffice/developer/680_m100/
for instance. any other mirror would do. the source files are there.. (change
the /680_mXXX/ to current milestone)
good luck! :)

The patch for IZ 50571 seems to fix Hebrew vowel rendering, at least for my m77
build. I'll try to build a clean m109 with and without this patch.

I have verified that Hebrew vowels/diacritics are rendered properly now, with four patches against m110:

1) my ICU patch, posted here Oct 2003
2) the ICU patch from IZ 50571
3 & 4) two small hdu patches against sallayout.cxx and gcach_layout.cxx.

Retargeting, since all patches needed to solve this are now available.

Fixed in CWS gslpatches4.

HDU->US: Please verify in CWS gslpatches4.

Testing hints: Install the SILEOT.ttf and SILEOTSR.ttf fonts. Load the
Genesis.sxw bugdoc.

re-open issue and reassign to us@openoffice.org

reassign to us@openoffice.org

reset resolution to FIXED

verified in 'gslpatches4'.

I am trying ubuntu breezy's newest packages for OpenOffice.org2 (based on build
129 I think) and the problem with vowels remains. Any clue about when paches
will be applied to the main tree? will your fix make it for the final release?

I think the problem is still there. Something has changed though. In my last
post I had only tried using Times New Roman, Tahoma, FreeSans, Babel Unicode,
etc. and all of them still do have the same  placement problem. Now I have just
tried using Ezra SIL and this one looks OK (as well as Lucida Sans ... which
always did) ... I think Ezra was used for testing the fix and not the more
popular all-purpose MS fonts.

Those fonts probably don't have GPOS tables to use to position the vowels, 
whereas fonts like Ezra SIL, SBL Hebrew, and Cardo do. Because of this, those 
Microsoft fonts aren't good for Hebrew. 
 
Did previous versions of OOo render properly with those fonts? 
 
I think ICU may have used a heuristic for older fonts without GPOS tables, but 
this heuristic method of vowel positioning is no longer used. 

I tried Cardo, Ezra SIL, BSL Hebrew and Lucida Sans. All of them look good now,
so the problem remains with fonts not really designed for Hebrew (Times, Tahoma,
etc.). I am still a little confused because I have tried Cardo in OOo 1.1.4 and
also looks OK (as well as Lucida Sans). So AFAK the situation from OOo1 to OOo2
has changed only for the Ezra and BSLHebrew fonts.

verified in mws m130.

I am using OOo2.0 for Fedora Core 4, and just tested all fonts on my machine 
(including the ms-webfonts) and the problem remains. Yet this bug report has a 
status of closed and fixed. What's going on? 

I would guess that what's going on is that you have no OpenType Hebrew fonts that have the built-in 
smarts (GPOS 'mark' tables, IIRC) to position vowel marks etc. Have you tried the test document with a real 
Hebrew font like SIL Hebrew, Cardo, or Ezra SIL?

I think was happening before is that when rendering with fonts that did not have vowel positioning 
information, ICU would use a heuristic method of guessing where vowel marks should go. A compromise 
might be to detect whether the font has 'mark' tables, and to use the heuristic method as a fallback if it 
doesn't.

Nevertheless, for accurate Hebrew rendering an OpenType font with vowel positioning tables should be 
used.

So this means there was a regression in ICU? I understand we keep a copy of ICU
in our tree so during the regular syncing of ICU version, we picked up this
regression as well.

One would think that feature was stripped from layout/LayoutEngine.cpp at some
point:
http://dev.icu-project.org/cgi-bin/viewcvs.cgi/icu/source/layout/LayoutEngine.cpp

Don't have the time nor the experience to verify it, but this could give a
headstart.

[user@localhost ~]$ rpm -qa|grep fonts 
<SNIP> 
fonts-hebrew-0.100-4 
msfonts-1.2.1-1 
^^^^^^^^^^^^^ 
 
Wouldn't you expect at least the fonts-hebrew-0.100-4 to be OpenType? It is, 
after all, from the Culmus project. 

I don't think the MS fonts have the 'mark' GPOS tables. And as far as I can 
tell, all the fonts are Postscript fonts (not OpenType), except for the alpha 
font Frank Ruehl OT. 

I just installed SPezra, SPdamascus and SPtiberian, tried to write "ani" with  
nikud, and saw the beautiful result: perfectly placed vowels, but ... under  
the wrong letters. The patach was under the nun and the chirik under the yod.  
Something isn't right here. I then downloaded SIL Ezra from  
http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=SILHebrUnic2#3fee5fec  
but couldn't get that one to work. Any other link available for a good  
opentype font?  

I know this is the wrong place to ask, but I couldn't find an answer for this
question anywhere. How do I place nikud in linux?

Probably the best is SBL Hebrew, available at 
http://www.sbl-site.org/Resources/default.aspx. 
 
Also, make sure you start by rendering a known-good document, to make sure the 
characters are in the correct order; i.e., generally the vowels are typed 
*after* the consonant. The SBL Hebrew documentation has a lot of information 
about Hebrew Unicode issues. 

Well, had the vowels appeared to the right of where they were expected, you 
could think I typed them before the letters. However,, given that they 
appeared to the left of the letters, one can conclude that I didn't type 'em 
too early. In fact, I checked with Lucida Sans Unicode and there it works, 
only there. I also downloaded the SBL fonts, but there too, nikud is off. HELP 
 
PS: using the OOo 2.0 for FC4. 

fqs7432: Have you tried loading the test document "Genesis 1.sxw"? 

mreimer, thanks for your suggestion. I now have five supposedly OpenType 
fonts, Ezra SIL, three from the society of Biblical literature (SPDamascus, 
SPTimerian and SPEzra) and Lucida Sans Unicode. Genesis.sxw is displayed 
properly, because it uses the best of these fonts: Ezra SIL. the three SBL 
fonts show the problem I described, of having beautiful nikud, but under the 
wrong letter. The Culmus fonts are not [yet, hopefully soon will be] OpenType, 
and neither are the ms-webfonts. Lucida Sans Unicode works, as is well known, 
but doesn't work well, as the dagesh is too far to the left. 
 
Interestingly, yesterday evening, I couldn't get Ezra to work, even after 
numerous `xset fp rehash` and numerous restarts of OOo. Today, not having 
rebooted, I started OOo again, to display Genesis.sxw, and the whole thing 
works, even in other documents. 
 
In conclusion, it is indeed the fonts that are broken, not OOo, although it 
would be nice if (a) Culmus becomes OpenType, and (b) OOo gets a better 
heuristic algorithm to correctly deal with non-OpenType fonts. 
 
While bemoaning the lack of OpenType fonts, I will say that better two working 
fonts than none. 
 
Thanks to you all! 

fqs7432: The SBL fonts you mention are not the ones you want; you want the font 
named "SBL Hebrew" available at 
http://www.sbl-site.org/Fonts/SBLHebrew-Distributionv1.11.zip. 
 
FWIW, Ezra SIL works fine for me here. Note that there is another SIL font 
named (I think) SIL Ezra that is *not* OpenType. 

Was the support for proper placement of nikud with stock Microsoft Office fonts
(such as Narkisim) declared WONTFIX?

Closed??? Why is this issue closed?

As of OpenOffice 2.01, the problem still persists. See for yourself in this
snapshot:

http://img2.tapuz.co.il/forums/1_80288281.png

This is with "David CLM" - a font designed especially for Hebrew.

While it is possible that some fonts are more problematic than others, we laymen
users don't know how to distinguish between "well behaving fonts" and "ill
behaving fonts". If the problem lies indeed in "bad fonts", could you please
release some requirement or criteria list for which fonts to use?

Also, how could a keyboard effect the location of the vowels???

I would love to see this issue fully resolved, so that I can dump Windows for
good. At the moment I have no choice buy continue using OpenOffice on Windows
(the problem does not exist there).

Thanks!

This bug is alive and kicking, and should not be closed. The CLM family fonts
(known as CULMUS) are by far the best looking and most popular Hebrew fonts for
Linux. Moreover, word vowels work properly with these fonts both with gedit text
editor and firefox browser with Pango support. This problem occure only in
OpenOffice, and unfortunately it is the place where word vowels are most needed.
Word vowels are used not only for biblical text; they are essential for poetry
and for any kind of text produced for young people. I, for example, translate
poetry, and feel really crippled by that OpenOffice bug. Please, please re-open
it and solve it.

Indeed, it seems that the fix here is too narrow in scope.
Although it works for biblical uses with fonts designed for that, it fails to apply to "regular" uses with 
"regular" Hebrew fonts (which work OK under OpenOffice for Windows).

re-opening 

Shoshana, all,
has anyone tried to reproduce this bug against official OO.o 2.0.2 build from
the OO.o webside?
Thx, Ulf.

transferring to ES.

Besides the four fonts mentioned by mreimer, it is true that there aren't many 
freely available OpenType fonts. Indeed, some applications properly display 
vowels on non-opentype fonts, and this should become possible OpenOffice. 
However, perhaps we should also ask the Culmus project whether they could 
OpenType their fonts?

My system exhibit this issue with a RedHat packaged OpenOffice.org which claims
to be 2.0.3-0.169. Is it comparable (in the context of this bug) to the official
OpenOffice.org build 2.0.2 (*)?

(*) which I downloaded but my system complains this would be a downgrade so I
wanted to ask here first.

If someone can instruct me how to test the 'nikud', I'll be glad to test it on
my system (FC5).
Which font to use? How to install it? how to eneter Nikud?

aviel,

You need to set the right keyboard map and install some OpenType fonts. If you 
use kde, you can select the Israeli keyboard of the lyx variant. Then, shift 
quf will be qamatz, shift-samakh segol, shift-tsadi tseyre, etc. As for 
OpenType fonts, mreimer has provided four names and some links in this bug 
report, just search the page for Ezra, Cardo, SBL. You can also use Lucida 
Sans Unicode, which, I believe, is distributed by MS.

>You need to set the right keyboard map and install some OpenType fonts

This is what's bothering me: The Windows version of OpenOffice supports Nikud
with non-open type fonts, this bug was originally opened for all fonts, not just
open type, and yet a fix which it's scope was smaller (ie, for open type only
fonts) marked the full bug as closed.

Can someone please refer me to where it was discussed that support for Nikud
with non open type fonts would be a Windows-only feature?

If no such decision was made, IMO we should have this bug open until we have a
fix for non open type fonts as well, especially considering the fact that most
free (both as in beer and as in speech) fonts available for Hebrew are not open
type.

The *only* way to have truly correct placement of nikud/vowel points/diacritics
is to use an OpenType font, which is what OOo currently does.

The old OOo used to use a heuristic method of positioning nikud/vowel
points/diacritics by guesswork, which is prone to breakage. I think it does not
work with multiple marks on a single consonant, for instance. The problem is
that non-OpenType fonts only have glyphs for nikud, but no positioning
information--this is why non-OpenType fonts are inadequate for proper Hebrew
nikud support. The best that can be done with these fonts is to guess, which is
at best completely inadequate for serious Hebrew word processing and publishing,
and at worst completely broken for more complex rendering such as multiple marks
on a consonant.

To satisfy both needs (proper rendering on the one hand, and better rendering
with non-OpenType fonts on the other hand), ICU could be patched to use the old
heuristic method if the font used for rendering does not have OpenType tables
(particularly the 'mark' and 'kern' GPOS tables). For someone who knows ICU,
this would probably involve just a couple of lines in the right place to query
the font for the existence of these GPOS tables.

> IMO we should have this bug open until we have a
> fix for non open type fonts as well.

Or... open a *new* bug? More focused and more specific? With as much information
as possible, as to guide the experts as much as possible?

I wish I could help with fixing the bug, but unfortunately I don't know much
about font technology, not to mention the specific piece in the OpenOffice
source code that handles Nikud (Hebrew diacritics).

My gut feeling tells me that if OpenOffice used the Pango library, the problem
would be easily fixed.

ES->HDSU: please evaluate.
Note: this issues is a "patch" set as "NEW", part CWS "gslpatches4" which is
already integrated... there's something wrong here!

ayaniger->hdu:
Have you had a chance to consider how to deal with this issue? As Shoshannah has
pointed out, at present support for Nikud with non open type fonts is a
Windows-only feature. This is a severe limitation for the use of Hebrew OOo
under Linux.

adjust target to 2.x

I agree with mreimer on what the status is and what should be done.
1.) split this issue up into the fix for OpenType fonts and non-Opentype fonts
=> this issue 14069 can get closed since it works since CWS gslpatches for OTF
2.) wait for the CWS "icuupgrade" to be integrated, then test with the new version
3.) if the problem is still there then open an issue for the non-OTF scenario
and attach the patch for ICU that fixes it

Here is the status from the three action items mentioned above:
1) is still solved
2) CWS icuupgrade is integrated in builds SRC680_m197 and newer
3) X11 fonts are disabled by default now, so there should not be much of a
problem left.

I tested with OpenType fonts on SRC680_m197 on FreeBSD and 2.2 on OS X, and it
looks fine.

hdu, thanks for all your work on this over the years!

Great! So I can finally close this long and complex issue... ;-)

i'm using OO localized (Hebrew) version 2.2 based on OOo
and this issue is still unsolved for TrueType fonts.

please reopen 
or have a new (more specific) bug issue for TT fonts ?

nadavkav: Have you tried using a font SBL Hebrew? Nikud/diacritics only get
placed properly for TrueType/OpenType fonts that have GPOS tables specifying
nikud placement.

Hi Nadav, others,
We are not going to re-open this issue because there is no problem when you use
OpenType fonts with correct GPOS positioned diacritics. This is a font issue,
period.

The Hebrew OOo project is not going to spend any more resources on this topic.
This is not a bug.

There is no heuristic that places Hebrew diacritics correctly or even
satisfactorily for all fonts in all sizes. It is not possible to write such a
heuristic because of the irregularities in the glyphs between fonts and between
sizes within fonts. Only an artist can place the diacritics correctly.
OpenType GPOS tables provide the solution to this problem.

Try MasterFont's new Hebrew fonts with GPOS diacritics.

I understand that you do not work on this issue due to limited resources. This
is perfectly fine, I would never vote to expend valuable resources on it.
However I still think that this is a bug (enhancement bug at least?) that other
word processors (even the humble gedit or abiword in linux, not to speak of MS
Word in Windows) do not have. It is in fact possible to have decent nikud
without proper font GPOS tables (pango does just that). There are not
free/opensource Opentype Hebrew fonts with proper GPOS tables for nikud, so we
can not use OOo to write Hebrew with diacritics in a truly free software
environment (and that could be considered a bug).

pueyo: There are indeed free/opensource fonts with GPOS tables--in fact, the
best fonts are free: SBL Hebrew and Ezra SIL. At one point SBL Hebrew was the
only OpenType font that could properly render every combination of nikud in the
Hebrew Bible. Cardo also has GPOS tables for Hebrew. All are free for
non-commercial use. Ezra SIL is available under the Open Font License. Check
these out:

http://www.sbl-site.org/educational/BiblicalFonts_SBLHebrew.aspx
http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=EzraSIL_Home
http://scholarsfonts.net/cardofnt.html

It's likely there are others.

mreimer: I was not aware that Ezra SIL was available under the Open Font
License, which is a truly free licence. SBL Hebrew and Cardo have "freeware"
licences so they can be used for no commercial purposes but can not be modified.

So I gladly correct my previous comment: OOo+Ezra SIL can be used to type Hebrew
with nikud in a truly free software environment. Thank you.

yba->pueyo:
There is indeed a dearth of OpenType Hebrew fonts with diacritics at this time.
No one disagrees that at the present time this is a serious problem with Hebrew
in OOo. However, this is an economic problem, not a software problem. It does
not make sense to try to solve an economic problem by writing code, especially
since there is no possibility of writing satisfactory code to solve what is in
fact an esthetic problem. The appropriate technology to solve this problem is
available in OpenType GPOS and GSUB together. Since I think that writing code to
solve this problem is inappropriate, and that the resources to write and test
that code should be spent on font enhancement, I do not agree that this is a
feature enhancement, it's just adding cruft to OOo and delaying implementation
of the correct solution.

There have been many atttempts to write heuristics for Hebrew diacritic
placement. I don't know of any of them that purport to be anything more than
hacks or workarounds. As far as I know, only one of them was written since the
publication of the OpenType standard. The best of these heuristics work only
passibly well and only for diacritics below the line of the glyphs, not in or
above the glyphs (CHOLAM, DAGESH), and they require fonts with an inordinate
amount of space below the glyphs in order to trick the eye into thinking that
the diacritics are placed correctly when in fact they are just centered below
the glyph.

Hi everyone, I agree with most of what was said here, I'd just like to add my
two cents:

The automatic niqqud placement heuristics may never be good enough for a
beautiful rendition of the bible or poetry (although I never quite understood
why this has to be the case - nice niqqud placement isn't more an art than nice
line breaking is). But is quite sufficient for what 90% of the users need niqqud
for - for disambiguating one or two words in a long text. You know, adding one
chirik here, one kamatz there, and so on, to make ambiguous-looking words a bit
clearer. For this use (which again, I believe is the most common use for in
niqqud for OpenOffice users), it is useful to allow the user to use whatever
font he chose (which has niqqud, of course), rather than tell him: "you want to
add one chiriq? then switch to the Ezra Sil (or whatever) font". And you want
the resulting output to look "acceptable", not more and not less.
Wouldn't it make sense to do what Dov suggested in a comment here in 2005, which
is to copy his existing vowel placement heuristic? It (possibly) won't be
perfect, but should be better than the existing situation. The fact that you
consider this problem low priority doesn't mean it has been "FIXED" (which is
how this bug is marked now).

By the way, pueyo's first comment (from May 6 2003) makes me wonder. How could
it be that the same fonts (the MS Windows fonts like Times New Roman) place
vowels correctly in Windows, but not in Linux? If these fonts have the correct
"OpenType" niqqud placement, doesn't it help in Linux?