Issue 122827

38 seconds to save a change in attachment using Rev. 1505643 Win 7.

Intel i3 2.93 GHz 4 GB RAM

I can confirm that. I looks like a memory leak is also involved. Saving a 77 kB Excel document with 30 simple sheets that sums one colum of ~50 values allocates  1 GB virtual memory. I have killed the soffice process after a minute. Saving the same document takes ~3 sec in OO 3.4.1.

I converted the existing XLS file to ODS format. Saving the ODS document is fast but loading the document is slow, slower than the XLS version. It seems as reverting to OO 3.4.1 is the only solution now.

Created attachment 81182 [details]
XLS format slow saving and memory leak sample document

I have attached another simple XLS document. Steps to reproduce:

1. Start Calc 4.0.0, start Task Manager, select soffice.bin process, watch VM Size column
2. Open the MemoryLeak.xls file in Calc
3. Make a modification to a sheet
4. Save

Watch allocated amount of VM Size, it goes above 300 MB.

5. Close MemoryLeak.xls document
6. Close Calc

Closing soffice.bin process also takes ages and allocated amount of VM Size goes above 600 MB.

Going to revert to ver. 3.4.1

(In reply to Petr Vones from comment #2)
> I can confirm that. I looks like a memory leak is also involved. Saving a 77
> kB Excel document with 30 simple sheets that sums one colum of ~50 values
> allocates  1 GB virtual memory. I have killed the soffice process after a
> minute. Saving the same document takes ~3 sec in OO 3.4.1.
> 
> I converted the existing XLS file to ODS format. Saving the ODS document is
> fast but loading the document is slow, slower than the XLS version. It seems
> as reverting to OO 3.4.1 is the only solution now.

Thanks Petr, I will take your advice and revert to 3.4.1, awaiting the problem in that part of Ver. 4.0 to be solved. It should be possible to pinpoint the routine which does that conversion and best find out what compiler the old routine was compiled with ... I am no programmer, cannot help :-(

(In reply to pr_lich from comment #5)
> Going to revert to ver. 3.4.1

Agreed ...

Confirmed on Windows 7 64-bit with "simple cash accounting" attachment.

Note:  saving in ODF format is not slow at all, much faster than XLS.   So problem appears to be limited to the XLS writing code.

Raising severity to "critical"

*** Issue 122874 has been marked as a duplicate of this issue. ***

I am having the same problem, very slow saving........

I am having the same problem.  Saving an .xls format worksheet in ver 4 takes more than a minute as it took just seconds in ver 3.  Also I am experiencing display issues.  The screen gets all messed up.  I switch to another tab and then back to the original tab and it looks OK.  As I move around the sheet it gets messed up again.

(In reply to manager from comment #11)
> Also I am
> experiencing display issues.  The screen gets all messed up.  I switch to
> another tab and then back to the original tab and it looks OK.  As I move
> around the sheet it gets messed up again.

this looks like Bug 122801

*** Issue 122896 has been marked as a duplicate of this issue. ***

There seems to be exactly opposite issue when loading documents. The same document is loaded slowly in ODS format but fast in XLS format, see bug 122911

FWIW: The oox::xls::CellBlock cache introduced in AOO 3.4 caused some performance problems (bug 119232) that could be related. The fix I suggested (attachment 79789 [details]) was not yet reviewed by the Calc expert though.

*** Issue 122941 has been marked as a duplicate of this issue. ***

If additional confirmation needed:  My win 7 64 system saves an XLS file in version 4 in 13+ seconds.  The same file in ODS saves in less than 3 seconds.  Going the other way, that file opens the XLS version in less than a quarter of a second but takes nearly 2 seconds opening the ODT.

I have not yet reinstalled 3.4, but my recollection is that saving and opening that same file as an XLS were even faster than the best times 4.0 produces, regardless of file type

For my purposes, this bug overshadows the improvements in 4.0.  I must reluctantly go back to last 3.4 version pending a fix.

Ditto with others. xls spreadsheet used to save in 2 secs in OO 3.4.1. Exactly the same spreadsheet now takes 40 secs in AOO 4.0.

Will be reverting to 3.4.1

Reedmace

Reapplying the change of revision 1406980 (for bug 121200) that was undone in revision 1407366 (for bug 119707) gets the performance back to AOO341 levels.

The latter change seems to extend the number of rows visited per sheet
- from 134..217 135 to 65535 for the "Simple cash accounting spreadsheet"
- from 135 to 65535 for the "XLS format slow saving" sample
and that largely expanded range slows things down.

Great! Sounds good, when can we download the updated Ver.4 ?

*** Issue 123000 has been marked as a duplicate of this issue. ***

*** Issue 123025 has been marked as a duplicate of this issue. ***

approve as showstopper

Created attachment 81366 [details]
Fix patch

In fix code of 119707, it needs to calculate the last row with visible attributes in order to avoid missing such rows during XLS exporting. But the mechanism is wrong, it always return 1M row. So redundant information will be stored. That is the root cause.
Symphony doesn't have the problem. So contribute latest symphony code as fix

Comment on attachment 81366 [details]
Fix patch

Thanks for the patch! With it the performance is back to normal and first tests show no side-effects. The analysis of the root cause and the fix make very much sense to me. I'm not too proficient in Calc though so I'd like to invite further reviews by developers more familiar with Calc.

I downloaded the patch i122827. But now I have a problem: I'm only a "simple user" and I have no clue how to install this patch. Can you give me same help with it?

(In reply to hpeter.stuedemann from comment #26)
> I downloaded the patch i122827. But now I have a problem: I'm only a "simple
> user" and I have no clue how to install this patch. Can you give me same
> help with it?

The patch is a source code patch which is intended for a developer audience. As a "simple user" you'll probably want a binary that has the patch applied. When Clarence committed his patch the next pre-release build will contain it. I'll add a link here then. The alternative is to wait for the official Apache OpenOffice 4.0.1 release that will contain the patch too.

"hdu" committed SVN revision 1518253 into trunk:
#i122827# fix the performance regression in xls saving introduced by #i119707#

"hdu" committed SVN revision 1518259 into branches/AOO401:
#i122827# fix the performance regression in xls saving introduced by #i119707#

Fixed in trunk and in the AOO401 branch.

36 seconds to save a change in attachment using Rev. 1516414 Win 7.

Intel i5 2.60 GHz 4 GB RAM

Still not fixed.  Installed 4.0.1 from snapshot. I see no change in performance.  Takes about 12 seconds to save a spreadsheet that saves in 3.4.1 in about 1 second

(In reply to SVN Robot from comment #29)
> "hdu" committed SVN revision 1518259 into branches/AOO401:
> #i122827# fix the performance regression

Only AOO401-snapshots with a revision of 1518259 or higher have the problem fixed.  Older revisions such as 1516414 don't have the fix and without the fix they of course still have the problem. Newer snapshot builds (which contain the fix) will be available soon.

Thanks for the information, waiting for the build containing the fixe

This AM installed latest version.  Works as it should.  Thanks to all who contributed to this fix!!

(In reply to Gary Bennett from comment #35)
> This AM installed latest version.  Works as it should.  Thanks to all who
> contributed to this fix!!

Great! 
Please let us "mere mortals know when the update (with all fixes) is safe to download and what the Rev. Nr must be.

Thanks for great work!

(In reply to k.wehrlin from comment #36)
> Please let us "mere mortals know when the update (with all fixes) is safe to
> download and what the Rev. Nr must be.

Proposed Release Schedule
https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+4.0.1
09/19 GA

(In reply to Petr Vones from comment #37)
> (In reply to k.wehrlin from comment #36)
> > Please let us "mere mortals know when the update (with all fixes) is safe to
> > download and what the Rev. Nr must be.
> 
> Proposed Release Schedule
> https://cwiki.apache.org/confluence/display/OOOUSERS/AOO+4.0.1
> 09/19 GA

Thanks Petr,

I will be waiting patiently :-)

(In reply to k.wehrlin from comment #38)
> I will be waiting patiently :-)

For TESTING only and with the usual caveats ("These builds are intended to be used for early testing by other community volunteers. They have no release quality and should not be installed in a production environment") there is of course the snapshots page at https://cwiki.apache.org/confluence/display/OOOUSERS/Development+Snapshot+Builds which might be interesting...

It's verified fixed in build AOO401m2(Build:9711)  -  Rev. 1518667

*** Issue 123165 has been marked as a duplicate of this issue. ***

*** Issue 123381 has been marked as a duplicate of this issue. ***

(In reply to fanyuzhen from comment #40)
> It's verified fixed in build AOO401m2(Build:9711)  -  Rev. 1518667

Haven't had any news for a while, does this mean that I (mere mortal user) may now update to the last version? Would that be 4.1 ?

The problem is fixed in version 4.0.1, available since 1 October 2013 from http://www.openoffice.org/download/

(In reply to Andrea Pescetti from comment #44)
> The problem is fixed in version 4.0.1, available since 1 October 2013 from
> http://www.openoffice.org/download/

Thanks Andrea,
I downloaded, installed and tried it on the file I found the bug with and so far I am impressed :-)
Thank you all for a job well done!

Closing the resolved bug.