Issue 107558

Summary: A hidden step while writing OOo files?
Product: General Reporter: akawohl <andreas.kawohl>
Component: codeAssignee: AOO issues mailing list <issues>
Status: CONFIRMED --- QA Contact:
Severity: Trivial    
Priority: P2 CC: issues, jr, Mathias_Bauer, nesshof, ooo, pats
Version: OOo 3.1.1Keywords: oooqa
Target Milestone: ---   
Hardware: All   
OS: All   
Issue Type: DEFECT Latest Confirmation in: ---
Developer Difficulty: ---

Description akawohl 2009-12-09 13:13:51 UTC
Since our Impress-Poweruser works with OpenOffice 2.3.1 we received many calls
about broken Impress-files, while working on the network. These files are
usually very large > 100MB. A lot of photos or graphics are used in the files. 
The broken files are not only unreadable for OpenOffice, the zipcontainer is
broken and only zip-repair-tools can bring back a small range of the original
content. We searched a long time without success for errors on our
Novell-Network. We have finally found an indicative. 

A hidden step while writing files?
We observed, that OpenOffice is saving files in two steps:
The first step is shown by the blue “file saving” bar - Let me call it
“preparing the file for storage”
The second step is usually invisible - Let me call it „writing the file“.
On a pc there is only a small time between closing the blue bar and writing the
file to the hdd, you hardly notice it. If you have a slow network and a very
large file, it will be measurable.

Let me describe in an example: 
1.The User is closing OpenOffice by the Windows X upper right
2. He says yes to “save changes”
3. The blue “file saving”-bar appears and ends
4 Now OpenOffice should close (remember step one - he used the Windows X).
But OpenOffice seems frozen. After two or three click on the OpenOffice-Window.,
the windows taskmanager appears with the message “application is not responding“
and the user gets the dialog to cancel the program immediately. If he does, he
will destroy the file, because now the file is written.

Unbelievable? No, you can watch the writing with the file explorer!
If the progress is watched in the fileexplorer (F5), then you see, that the
filesize is not changing in step 3 . The file is changing in step 4 and you can
see it grow step by step from 0 Byte to the original size.

I stopped the time for a 100MB Impress file over one of our slow network
connections:
Step 3 22 seconds
Step 4 50 seconds

I stopped the time for a Writer 80MB file (some photos inside) over the same
connections: 
I pushed die “Save-Button” after the "safe bar" ends, the writer was frozen
(every button was gray, no coursor) for nearly 120 seconds.

I stopped the time for a Impress 15MB file at a very critical location: 
Step 3 15 seconds
Step 4 25 seconds (remember the file size 15MB!)

Most of our user react like this: “I have saved my OpenOffice file (Step3) and
now OpenOffice hangs. I can cancel the program, without any risk.” And thus,
they destroy their files.
I can reproduce this on Windows XP and Ubuntu. Most Impress-files were converted
from powerpoint to OpenOffice. I have tested it on different fileservers. The
worst result for step 4 was 150 seconds for 100MB.

My Question 
1. It is true, that the storage process (as described) consists of two steps?
2. If my description is correct, can we make a change to this process (e.g.
Options-Menue)and reduce the time OpenOffice is blocked?
3. Any other good idea to keep the user from cancelling the process?

I've created a screenmovie, so you can see our problem live. I can send in need. 

Technical description 
Fileserver-operating system: Novell-Netware: 6.5SP8
Main locations: VM unter ESX3.5/4, 1 CPU, 4GB RAM. 
4 smaller location with Hardware-Server like FSUWA: HP ProLiant ML370G3, 2 x
XEON2,8, 3GB RAM, 2 x 36GB RAID1-SYS-Volume, 4 x 72GB RAID5-Datenvolume. Netware
6.5SP8, Filesystem NSS, eDirectory 8.85.
- OpenOffice 2.3.1 under Windows 2000 und Windows XP Java 1.5.0_10 and ubuntu.
Comment 1 hwdooo 2010-01-30 11:46:30 UTC
we also encounter this problem while saving xxl-sized presentations (50MB or
more). With OOo 3.1.1 we can watch a 3-phase saving process.
1. Nothing seems to happen after confirming the "Save to"-dialog
2. After a long period of hdd activities (producing a lot of files in the user´s
temp directory), you can watch the "file saving" bar
3. Then again: nothing happens until the program finally terminates

Depending on the OS (we tested on XP an Windows 7), if a user hits the "closing
X" in the upper right corner again, the OS asks to terminate the program
immediately. Confirming this after step 2 (user thinks the saving is done) can
result in corrupt files.

Even worse:
With very large presentations (> 100MB) and embedded objects (bitmaps) OOo 3.1.1
crashes while loading the original powerpoint presentation.
"Microsoft Visual C++ runtime library" Runtime error! (soffice.bin)"
Comment 2 riessooo 2010-02-03 13:42:46 UTC
We can confirm this behavior, even with an .odp in an low performance network
there is this behavior. At least we would expect, that the sandbox is
deactivating the mouse click in OOo during the finishing phase for saving the file.
Comment 3 jr 2010-02-04 09:27:56 UTC
Confirmed, reproducible with OOo 2.x and 3.1.1
Setting priority to P2 since this issue leads to heavy data loss
Comment 4 jr 2010-02-04 09:29:18 UTC
changed subcomponent
Comment 5 wolframgarten 2010-02-04 09:56:56 UTC
Has anyone tried the 3.2 RC version on this problem? Has anyone been asked to
send a crash report after Office has crashed and if so, which error id did you
receive? 
@jrahemipour: does this happen only with impress files or also with large writer
files? If so, framework was the right component.
Comment 6 akawohl 2010-02-04 13:05:38 UTC
I tested it today with OOo 3.2 RC 4 - same result as described in my original
message.

- Please remember my description OOo ist  n o t  crashing. 
It is a unclear situation for the end user. The saving bar is closed and OOo
seems to be froozen. And if a user hits the "closing X" in the upper right
corner again, the OS asks to terminate the program immediately. 

- I can confirm this also for writer (50MB). The real saving of the file happens
after closing the saving bar. But the saving/hanging-times are much shorter.

- Last: It is not a problem of Microsoft-files, I can reproduce whith an filetype.



Comment 7 clippka 2010-02-05 09:16:50 UTC
cl->mba: I'm assuming a framework issue? the 'second save step' should be the
copy of the temporary storage to the target location?
I remembered we had some corruption issues with samba but I have forgotten the
context.
Comment 8 Mathias_Bauer 2010-02-05 09:26:33 UTC
Indeed, the second step is the move of the temporary file to the target
location, perhaps combined with a creation of a backup file in case that is
activated in the settings. But this step is not performed asynchronously, so at
the time where the application is able to receive user actions (e.g. mouse
clicks) again the file should be saved already. A possible problem (especially
in networks) can be that the target file system fails to flush the file buffer
to disk, but OOo plans for that and explicitly asks the OS to flush this buffer,
so that potential errors will be reported (and should give an error message in
OOo). So whatever happens here, it is "a little bit" more complex.
We have to study that.

I agree that this problem most probably is a framework problem.
Comment 9 akawohl 2010-02-05 15:04:36 UTC
Perhaps the following is still useful:
I focused my search for a long time on network errors.
But I can reproduce this problem, however, already with a simple usb-stick.
Comment 10 mikhail.voytenko 2011-01-13 13:23:10 UTC
Sorry, but if I understand the description correctly the user just kills the
office process while the saving is still not ended ( great idea, it is like
shutting down the computer by unplugging the electricity cable ). Of course the
data is not transported to the file, it happens at the end of the saving
process. Sorry, but the framework is not able to prevent killing of the office
process.

The only thing that could be done is to show the progressbar longer, so that the
user do not try to kill the process. I am in doubt that this is P2 issue.
Comment 11 akawohl 2011-01-14 15:30:03 UTC
Ok,
let me explain it one more time. 

Graphical user-interfaces use easy elements to communicate with the user. One
very easy element is a progress-bar. It begins at the left and when it's closed,
the message to the user is: 'task successfully completed'. The opposite of this
is an application showing an hourglass and shows no reaction on any user action,
like a mouse-click. The message is: 'Oops, there is something wrong, this
application is stalling'.

Saving a file is one of the most important tasks a program has to do. It must be
controlled and monitored carefully, completely and free of any irritation.

At the moment OpenOffice controls only a part of the process 'saving a file', a
second step is uncontrolled (OOo stalls and shows an hourglass). 

The progress-bar must be open as long as OpenOffice is saving the file.

Because closing the progress-bar is a clear signal to the user, that his task is
fulfilled successfully. Giving this signal before the process is completed, is
nothing else but a bug.

There is the Danger of Data loss and this is without any doubt a P2 issue.

If you are unable (it seems to me more unwilling) to fix this bug, maybe you can
create a new kind of pop-up-art. You can show the following message before
saving: “Please do not unplug the powercabel. This application is not hanging,
it's saving that way”.
Comment 12 Mathias_Bauer 2011-03-23 13:13:52 UTC
The severity of this issue in combination with the effort to fix it does not justify to keep the 3.4 target. According to our new handling of critical issues we either must remove the "data_loss" keyword or downgrade the prio to P4. I prefer the former; in fact it is not a real data loss, only if the user reacts insanely. To differentiate this issue from other *real* data loss issues that deserve much higher visibility, I remove the keyword.
Comment 13 Marcus 2017-05-20 10:48:10 UTC
Reset assigne to the default "issues@openoffice.apache.org".
Comment 14 Patricia Shanahan 2017-07-08 13:52:08 UTC
(In reply to mikhail.voytenko from comment #10)
> Sorry, but if I understand the description correctly the user just kills the
> office process while the saving is still not ended ( great idea, it is like
> shutting down the computer by unplugging the electricity cable ). Of course
> the
> data is not transported to the file, it happens at the end of the saving
> process. Sorry, but the framework is not able to prevent killing of the
> office
> process.
> 
> The only thing that could be done is to show the progressbar longer, so that
> the
> user do not try to kill the process. I am in doubt that this is P2 issue.

I don't think showing the progressbar longer is a complete fix, though it would improve the situation. The computer could crash for unrelated reasons, such as a power failure, at any time no matter how much the user wants it to go on running. We need to ensure the data always exists, and the user knows what to do to recover it.

The best way I know to ensure this is to write to a temporary file. After the temporary file is completely written and flushed, rename to make it the primary file. There is a risk of losing changes since the last completed save, but that always exists and is generally understood.

For clarity, the progress bar should remain visible and incomplete until the rename has been done.