Created attachment 25307 [details]
web wizard works with soffice.bin

Created attachment 25308 [details]
beanshell macro don't work with soffice

Created attachment 25309 [details]
beanshell macro works with soffice.bin

at cc to Florian Heckl

-- 
ericb

add cc to patrick luby

issue started 

First fix works fine, but Tino told me it was not accepted by udk team :

"""""""""""""""""""" quote """""""""""""""""""""""""""""""""""""""
Hi Eric,

just talked to the Java guys here about your proposed patch for i47888.
The bad news: Your proposed patch is not the appropriate solution
according to our judgment. Sorry for the bad news.

But everything is not lost we just need to fix the problem in another
area of the code which is probably the project 'jvmfwk' there is
a method 'jfw_startVM'which actually starts the JVM.

Further it is possible to create plugins which can be integrated into the
JavaFramework (see: jvmfwk/plugins/...
The sunjavaplugin may serve as an example. For details see also:
http://udk.openoffice.org/common/man/spec/javavendorextension.sxw

We still need to figure out where is the optimal place to start the JVM
in a separate thread. Maybe 'jvmaccess' is the place to consider but I'm
not yet sure.

If I got a little time over the weekend I will further investigate the problem,
maybe you are faster than me.

Kind regards,
Tino 

"""""""""""""""""""""""""""  end quote """""""""""""""""""""""""""""""""""""""""

eric:
could you please describe this issue more in detail?
I couldn't reproduce your problem with m117.

do you mean
File->Wizards->Web page
immedately crashes?

if it is possible please attach beanshell macro as well
so that I can reproduce your situation...

ericb -> maho 

Yes, for example File->Wizards->Web page

About macros, you just have to use : 
Tools -> Macros -> manage** macros -> Beanshell   and test some of them
You'll observe the screenshot I send you.

**not sure, I only have french version here

As summary :

It is URGENT to solve this issue because this is an enormous stopper (even in
1.1.5 )

The solution, proposed by Florian, helped by Apple engineers, works. Florian
reported me some crashes I was not able to reproduce.  So we must take care.

The problem is, *how* implement a Core Foundation Run loop (the solution) in
another thread, in main ... but  not in main  :-)  just using jvmfwk (or ?) with
a "plugin".  This is exactly the problem we have to solve.

Last but not least : the problem is twice  : 1.1.5 / Tiger does have exactly the
same.

I ran into this same problem in NeoOffice/J with Java 1.4.x and I have code that
fixes this problem.

The NeoOffice/J code is intermixed with a lot of code that you won't want. So,
below is a stripped down version of the code changes that you will need. Note: I
haven't tested this code as I don't have an OOo 2.0 build so someone who has
such a build will have to try this code out:

1. In vcl/unx/inc/salinst.h, add the following new function declaration:

#ifndef _SV_SVAPP_HXX      
#include <svapp.hxx>
#endif

void ExecuteApplicationMain( Application *pApp );

2. In vcl/source/app/svmain.cxx, in the SVMain() function replace the following
line:

    pSVData->mpApp->Main();

with the following lines:

#ifdef MACOSX
    ExecuteApplicationMain( pSVData->mpApp );
#else // MACOSX
    pSVData->mpApp->Main();
#endif // MACOSX

3. In vcl/unx/source/app/salinst.cxx, add the following code:

#include <CoreFoundation/CoreFoundation.h>

class SVMainThread : public ::vos::OThread
{ 
    Application*            mpApp;
    CFRunLoopRef            maRunLoop;

public:
                            SVMainThread( Application* pApp, CFRunLoopRef
aRunLoop ) : ::vos::OThread(), mpApp( pApp ), maRunLoop( aRunLoop ) {}

    virtual void            run();
};

void SVMainThread::run()
{
    Application::GetSolarMutex().acquire();
    mpApp->Main();
    CFRunLoopStop( maRunLoop );
    Application::GetSolarMutex().release();
}

void sourceCallBack( void *info ) {}

void ExecuteApplicationMain( Application *pApp )
{
    // TODO: Determine if we are running Java 1.3.1 as this code is currently
hardcoded for Java 1.4 or higher
    jint jniVersion = JNI_VERSION_1_4;

    if ( jniVersion >= JNI_VERSION_1_4 )
    {
        // Create the thread to run the Main() method in
        SVMainThread aSVMainThread( pApp, CFRunLoopGetCurrent() );
        aSVMainThread.create();

        ULONG nCount = Application::ReleaseSolarMutex();

        // Start the CFRunLoop
        sourceContext.version = 0;
        sourceContext.info = NULL;
        sourceContext.retain = NULL;
        sourceContext.release = NULL;
        sourceContext.copyDescription = NULL;
        sourceContext.equal = NULL;
        sourceContext.hash = NULL;
        sourceContext.schedule = NULL;
        sourceContext.cancel = NULL;
        sourceContext.perform = &sourceCallBack;
        CFRunLoopSourceRef sourceRef = CFRunLoopSourceCreate( NULL, 0,
&sourceContext );
        CFRunLoopAddSource( CFRunLoopGetCurrent(),sourceRef,kCFRunLoopCommonModes );
        CFRunLoopRun();
 
        aSVMainThread.join();

        Application::AcquireSolarMutex( nCount );
    }
    else
    {
        pApp->Main();
    }

    return;
}

ericb -> pluby

Thank you very much for your proposition :-)  Looks very interessant and I'll
try to implement it ith m117.

FYI, what we use will be attached. I think you 'll immediatly see the problem.

Regards,
eric bachard

Created attachment 27997 [details]
part1 : svmain

Created attachment 27998 [details]
part2 : necessary modifications in solenv for objc++

Created attachment 27999 [details]
part3 : makefile.mk in desktop/source/app

Created attachment 28000 [details]
part4 :  mainmac.mm

Created attachment 28001 [details]
part5 makefile.mk for desktop/util

I would recommend *not* putting patches into the desktop module for the
following two reasons:

1. Putting patches in desktop only fixes a single executable whereas putting
patches in vcl fixes all executables that use any GUI APIs. In other words, if
you put your patches in desktop, you will need to duplicate this solution for
every executable that might inadvertently use the Java AWT APIs.

2. My vcl code does not need Objective-C++ support to compile and run.

JL->ericb: I think modifiying code outside the jvmfwk project to get Java to
function is bad. This is what jvmfwk is for. It is intenteded to encapsulate all
Java - vendor related stuff. We try to avoid platform dependent code as much as
we can because this makes maintainance very hard. This is why we have projects,
such as sal and jvmfwk.

As I understood the problem here is that the VM cannot be started in the main
thread. My suggestion is:

-create a jvm plugin library
-modify jfw_plugin_startJavaVirtualMachine so that it starts the VM in a
different thread.

The problem is, that jvmfwk unloads the plugins. However, to test if your plugin
works you could modify line 389 of jvmfwk/source/framework.cxx:

//The Module class unloads the library in its destructor
//		osl::Module modulePlugin(sLibPath);
osl_loadModule(sLibPath.pData, SAL_LOADMODULE_DEFAULT);


If this work I would introduce a optional function for the plugins:

int jfw_plugin_canUnloadAfterVMCreation()

Your plugin would return 0. The framework would take this into account (needs to
be implemented).

Please create a new plugin and do not use sunjavaplugin for this. A good place
for the code would be jvmfwk/plugins/mac or jvmfwk/plugins/mac/pluginlib. 
BTW. you could still use javaldx.

Let me know if this works for you.

pluby->jl

I understand your desire to put this code in one of the jvm frameworks. However,
I think that you misunderstand what the problem is. The problem is that with
Apple's Java 1.4 or higher JVM, any AWT code will block forever if the
primordial thread is not running a native event loop (i.e. CFRunLoopRun()) which
means that such an event loop must be started in the primordial thread and
before any AWT code is executed. If this native event loop is not started or is
started in a non-primordial thread, the first AWT call will block.

In other words, a native event loop absolutely must be run on the primordial
thread for Java 1.4 and higher to work on Mac OS X. I consider this a major
design flaw in Apple's JVM, but whether you load the JVM or use the JVM plugin,
you will need this native event loop to be running on the primordial thread
because this requirement exists with the AWT code itself.

Why not run the native event loop on a different thread like other JVMs? Sorry,
but I already tried it and it does not work. The reason is that Apple's JVM 1.4
and higher implementation executes all AWT calls within timers that the JVM
posts explicitly to the primordial thread's native event queue. So, any native
event loop started on a non-primordial thread will just sit idle and the timers
on the primordial thread will never fire which will result in the same blocking
behavior.

The hanging that ericb is seeing occurs when one of the timers that the JVM
posts has a wait condition to force the timer to run synchronously. In this case
(which happens for a surprisingly large number of AWT calls), the SVMain()
thread will block waiting for the timer posted to the primordial thread to fire.

Given that this is a JVM restriction, the number of options are fairly limited.
I think that implementing the native event loop creation and the move of
Application::Main() to a non-primordial thread entirely within vcl makes the
most sense since vcl is a location of platform dependent code and the location
of my changes are in a position where Java is safe for OOo to load.

jl->pluby: Thanks very much for explaining. Is there a chance that the Mac JRE
will be changed /fixed in the foreseeable future? 

Unfortunately, I would guess that this behavior is permanent based on the fact
that this behavior (i.e. a CFRunLoop must be running in the primordial thread)
exists in all Cocoa applications. Starting with Java 1.4, Apple reimplemented
their JVM's AWT code using the Cocoa APIs and future JVM versions will likely
remain Cocoa. For the last few years now, this behavior has seemed to be pretty
deeply embedded with Cocoa. It may have something to do with the fact that Cocoa
is written in Objective-C, but I don't know for sure.

Interestingly, Java 1.3.1 which was written using the Carbon APIs behaves like
Sun's JVM in that it starts a native event loop in a secondary thread that it
spawns. So, apparently Carbon (which is older and is not written in Objective-C)
does not suffer from this behavior.

It seems there is no other way then if we want Java to work on Mac. 

Hi,

Yes, I think too there's no other possibility.  Waiting, I have add vcl module
to macosx10 cws, and with Tino, we're trying to make Patrick's proposition work.
We'll have to think to a necessary backport for 1.1.5 too.

Again, thank you very much Patrick.

I have a couple of details to add:

1. In my proposed code, there is a line missing. Replace the following lines:

        // Start the CFRunLoop
        sourceContext.version = 0;

   with:

        // Start the CFRunLoop
        CFRunLoopSourceContext aourceContext;
        sourceContext.version = 0;

2. If you don't plan on supporting Mac OS X 10.2 (Jaguar), you don't need to
include the following lines:

    // TODO: Determine if we are running Java 1.3.1 as this code is currently
hardcoded for Java 1.4 or higher
    jint jniVersion = JNI_VERSION_1_4;

    if ( jniVersion >= JNI_VERSION_1_4 )

Instead, you just need to make the following changes to force Mac OS X to load
Java 1.4 or higher. In the jvmfwk/plugins/sunmajor/pluginlib/sunjavaplugin.cxx
file (OOo 2.0) or the stoc/source/javavm/javavm.cxx file (OOo 1.1.5), replace
the following line:

    vm_args.version= JNI_VERSION_1_2;

with:

    #ifdef MACOSX
    vm_args.version= JNI_VERSION_1_4;
    #else
    vm_args.version= JNI_VERSION_1_2;
    #endif

Hi,

I have still problems for building. I must be blind, but i don't find why the
build breaks. ExecuteApplicationMain() is not known for final linking, symbols
are missing...

Attached patch comes

Created attachment 28179 [details]
actual patch for use java + CFRunLoop

I forgot to mention : the update of vos::OThread (the use of vos is deprecated)
for osl Threads is still missing in the precedent patch.

fheckl->ericb:
Looking at your last patch I see the following line:
+//void ExecuteApplicationMain( Application *pApp )
where you comment out ExecuteApplicationMain
Is that intended?

ericb->fheckl

Ooops, yes,my mistake I don't know why, I commented the bad
ExecuteApplicationMain(). 
Thank you Florian :-)

Please comment out this lline !

I think I simply have not defined ExecuteApplicationMain() in svmain.hxx, but I
still have not found how do it correctly.

I think that you might have not added the changes in vcl/unx/inc/salinst.h:

> 1. In vcl/unx/inc/salinst.h, add the following new function declaration:

> #ifndef _SV_SVAPP_HXX      
> #include <svapp.hxx>
> #endif

> void ExecuteApplicationMain( Application *pApp );

ericb->pluby

No, I have really added these changes, but since I have added ' #include
<salinst.h> '
in vcl/source/app/svmain.cxx, svmain.o builds correctly. 

The pchanges are more simple, but build still breaks :

ld: Undefined symbols:
__Z22ExecuteApplicationMainP11Application
/usr/bin/libtool: internal link edit command failed
dmake:  Error code 1, while making '../unxmacxp.pro/lib/libvcl680mxp.dylib'
'---* tg_merge.mk *---'

ERROR: Error 65280 occurred while making
/Volumes/sauvegarde/Openoffice.org680/vcl/util

Attached patch comes

Created attachment 28190 [details]
new patch, more simple build still breaking)

TRA: The proposed VCL hack should only be the last resort. Nobody inside Sun is really able to predict 
what other things get broken when introducing the VCL patch. Provided that the problem arises only 
when using Java AWT there are some alternatives to be considered first:

1. Make the Wizard code and any other Java code in OOo AWT free. 
In the Wizard only a very small portion of the code uses AWT this code 
can easily be replaced according to the responsible person inside Sun.
All wizards except the Web wizard do work even without the VCL patch.

2. Force OOo to use Java 1.3.x 
At least the Wizards are Java 1.3.x compatible and don't need a hight Java version.
I don't know if there is any other code (except the Accessibility stuff which doens't 
work anyway on Mac according to responsible person) which still depends on a 
higher Java version.
In order to realize this solution a small portion of the jvmfwk project need to be 
adapted (a function like compareJavaVersion, which is currently not implemented 
for Mac OS X) and a XML file which dtermines the min and max Java version to be 
used by OOo.

3. Remove the menu entry for the Web wizard from the file menu.
I guess nobody will really miss this wizard on Mac OS X or am I wrong.

Kind regards,
Tino

Issue retargeted for 2.0.1

Summary modified to avoid confusion with #i48186#
IMHO : this is sure a main issue for Mac OS X port

ericb ->pluby

1)  in vcl/unx/source/app/salinst.cxx, you propose to add 

#include <CoreFoundation/CoreFoundation.h>

Because of build problems, and reading what was needed, I have changed in :

#include <premac.h>
#include <CoreFoundation/CoreFoundation.h>
#include <postmac.h>

Is this correct, or not ? Or is something else still  missing ?


2)  Excepted the bad commented definition of ExecuteApplicationMain() , have you
seen some trivial error in my patch ? 

Thank's in advance

Created attachment 29143 [details]
Patch for svmain.cxx. This patch replaces all of my previous patches for this bug.

1)  in vcl/unx/source/app/salinst.cxx, you propose to add 

#include <CoreFoundation/CoreFoundation.h>

Because of build problems, and reading what was needed, I have changed in :

pluby -> ericb

1. Adding the premac.h/postmac.h includes is correct.

2. I have been working with this code on a daily basis with Neo/J (I am
upgrading Neo/J to Java 1.4.2 AWT) and, from that, I found that my original
patch prevents OOo from exiting. Apparently, CFRunLoopStop() doesn't work as
expected.

So, to fix this problem, I have moved my original patch to be entirely in the
svmain.cxx file and discarded the ExcecuteApplicationMain() function. The new
patch is attached (see the svmain.cxx.diff patch). This patch is much simpler
than my original patch and it will not block OOo from exiting.

ericb -> pluby

I remember Tino was testing a solution only modifying svmain.cxx (the most logic
in fact), but we had not found how to go out of CFRunLoop(). Happy to see a
solution exists ! 

Of course, I'll try very soon (time to read what is modified), because I'm
impatient to try :-).  Last but not least, if it works and if I'm not wrong,
this fix will be helpfull too in 1.1.x. too.
I'd just suggest to include unistd.h header in svmain.cxx, ( for the " _exit(0);
" ), but it's secondary here.

Thank you very much !

Created attachment 29151 [details]
Updated patch that adds <unistd.h> include so that it will build on Tiger

pluby -> ericb

I have posted a new patch that has the <unistd.h> include. For some reason,
<unistd.h> must be included on Tiger, but it isn't needed on Panther.

ericb->pluby

Ok, I have supposed it was to include for Panther too, so I'll modify the test :-)

About CFRunLoop(), I have tested your changes, and everything is ok for both
1.9.125 and 1.1.5. Just SHL1STDLIBS has to be modified in util/makefile.mk for
linking with CoreFoundation Framework
Last problem remaining, sound does still not work :  crash with 1.9.125. I'll
rebuild vcl with debeug enabled to see. I think JMF is not a good thing for OOo...

Waiting, thank you very much, great work :-)

Created attachment 29161 [details]
the exact patch I use for my tests

Changes backported to 1.1.5 are working fine (excepted sound, one more time).
Just waiting for 53334 fix, to rebuild the complete m57.

Created attachment 29187 [details]
Fix potential race condition in SVMain()

ericb->pluby

Thank's for the fix. Tino writes me there's maybe another possibility of race
condition between threads.

set cc

Created attachment 29222 [details]
Another race condition fix

TRA: There is another race lurking around in my opinion. The second thread need to wait until the
native run loop in the primordial thread is actually running before calling SVMain. In theory it might
be possible that Java AWT will be used already with the know unpleasant consequences. I use busy 
waiting to synchronize both thread which should be no problem in this case. Any comments?

ericb->tra

Just changing bool in BOOL (ie for some false and true), I have tested all
adapted for 1.1.5 all give correct results :
- opening all Applets fond in demo.zip (from
http://java.sun.com/openstudio/demo.zip) using  Safari
- testing all available wizards.

I have even tested first fix from Patrick with 1.9.125 and last from Tino with
1.1.5rc3 in same time ! Opening both a clock and a banner (looks like two clocks
in the same time is not possible ;-)  )

Found nothing else, excepted, playing withsound,  a (non reproductible) : 
soffice PaHost_StartEngine: AudioDeviceAddIOProc primary failed : error =
0x6E6F7065 = 'kAudioHardwareIllegalOperationError' 

I found no documentation about kAudioHardwareIllegalOperationError in XCode
documentation, and "PaHost_StartEngine: AudioDeviceAddIOProc primary failed"
comes from portaudio library, and is located in pa_mac_core.c  Maybe a little
bug with sound  ? (or a bad move from my part)
 
I'm not sure this is the final state of this fix (too complicated for me), but
sure, you Tino, and Patrick, have made a great work. 

Thank you very much!

tra->ericb: I wonder why the bug target is 2.0.1. Is this intended? I'm still
not satisfied that this platform-dependet fix is in the platform-independent
part of VCL. I think we should think about a better solution yet.

add myself to cc 

I think that tra's additions are really needed because Apple's  JVM itself already blocks until the CFRunloop 
is available on the primordial thread. At least that's what Apple's engineers told me at WWDC and it's the 
reason why OOo so far always hung at the VM's initialization.
So there should be no race conditions there.

In order to avoid the main thread patch I tried to use Java 1.3.1, instead of
1.4. If I understood things correctly the AWT threading issue should not apply
there, right ?
I added the path of the 1.3.1 installation to the directory search list in
module jvmfwk (the g_arSearchPaths[] in util.cxx). Building the whole module
with debug shows a lot of useful debugging messages, so I could verify that the
1.3.1 VM was actually started when I clicked on Tools->Options->Java (note, you
have to copy the dynlibs, a softlink results in config files not being found at
runtime).
However, and this clearly contradicts what I initially thought,  the very same
message about AWT being started from the wrong thread appeared and the 1.3.1 VM
was not considered useful. Moreover, this exception appeared regardless if I
applied the svmain patch or not!

Patrick, do you have an idea why the 1.3.1 VM cannot be successfully loaded ? 
Do you think this approach would be useful at all ? All OOo Java code should be
1.3.1 compliant, so we could, at least for now, avoid the patch and limit
ourself to the 1.3.1 VM. As NeoOffice/J runs successfully with 1.3 there must be
a way...

ericb->ssa

Thank you for your work.  I just want to add that the use of java1.4 is really
needed for future developpements of OOo. No matter if all calls are 1.3
compatibles, but it's really important to run 1.4.x at least.

I think we should concentrate on a feasible solution for current problems and
always choose the one that is most likely to break nothing else. Just because a
WebWizard, for instance, makes a single 1.4 call it seems to be overkill to
potentially change the whole application's behaviour. (This is only an example
and not the real truth) Java is used in only a few places in OOo and the fact
that applets and the MediaPlayer will probably never work in the Mac/X11
environment (see issue 54020) indicates that we should look for a simple
solution. If we could solve our problems by restricting to 1.3 I would like this
solution.

reset target to 2.0

I suggest to try : I didn't see any problem with latest change, just this last
minute change today :

not force Java 1.4 with 10.3.9 
force java1.4 with 10.4.x  

and everything works fine (already verified with 1.1.5)

So, I cleaned up the VCL part of the patch by doing a few changes:
svmain() is now separated in two parts, a hook and the real method. The hook is
used to create a new thread which in turn does not call svmain() again but only
the second part. So the race condition is gone and I removed the corresponding
static variable. The hook comes in its own file (svmainhook.cxx) which is
included in the attachement together with the patch. Additionally I removed a
warning and changed sleep() into usleep() because sleep() only accepts integers,
i.e. full seconds.
This patch is now always active on MacOSX and does no version check. I think
this is good, because then we have the same (critical) code running on all
platform which will help us to detect potential problems easier.

In jvmfwk I would always use the 1_4 constant when running on MACOSX and remove
the MAJOR/MINOR BUILD stuff.

Created attachment 29426 [details]
Cleanup - you must place svmainhook.cxx into vcl/source/app

TRA->SSA: Very good! I will test your changes. Who is gonna commit this stuff?

If you guys tell me that it works for you, I'll happily commit..

ericb -> ssa 

Yes, great work ! Very good idea to separate things.
Up to you for the commit  :-)

I'll give it a test this evening, and with 1.1.5 too.

Thank you !

I commited my changes to CWS macosx11.

tra: verified on cws macosx11 (all wizards work, bean shell macros work) -> ok.

closing