Issue 124999 - Huge memory consumption for very large and complex .odt
Summary: Huge memory consumption for very large and complex .odt
Status: VERIFIED FIXED
Alias: None
Product: Writer
Classification: Application
Component: code (show other issues)
Version: 4.0.0
Hardware: All All
: P2 Major (vote)
Target Milestone: 4.1.1
Assignee: Oliver-Rainer Wittmann
QA Contact:
URL:
Keywords: data_loss, regression
Depends on:
Blocks: 124985 125000
  Show dependency treegraph
 
Reported: 2014-05-28 05:34 UTC by Rainer Bielefeld
Modified: 2014-08-03 22:39 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---
jsc: 4.1.1_release_blocker+


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Rainer Bielefeld 2014-05-28 05:34:09 UTC
Steps how to reproduce with "AOO 4.2.0-Dev – German UI / German locale [AOO420m1(Build:9800)  -   Rev. 1597166  2014-05-20 1]" on German WIN7 Home Premium (64bit)", “historic” 4. User Profile used for all predecessor versions.

0. Launch WIN Task Manager
1. Download / Unzip test kit "Issue 118725 Comment 13 - images dropped by random"
2. Open "TestBook2.odt" (140MB document), watch memory consumption
   > 1,7 GB or so for opening document
3. Click into first page, then pres downpage key and keep pressed
   slowly increasing memory consumption (nearby invisible), CRASH after few pages
   (what might be a new and different problem, I will submit a separate report)

11 Compare with OOo 3.3.0 by doing steps 2,3 with that version:
   > Opening document only will rise Memory consumption by 200 KB or so,
     no crash

Additional Info:

(a) My PC:
    64 bit AMD Phenom II X4 955 Processor 3.2 GHz, 4GB RAM, 
    Graphic Card: NVIDIA GeForce GT 430, Monitor HANNS-G HL225DBB 1920 x 1080 
(b) Memory problem (without crash) already reproducible with
* server installation of "AOO 4.0.0 Final – German UI / German locale 
  [AOO400m3(Build:9702)- Rev. 1503704 2013-07-16 14:54:56 (Di, 16 Jul 2013)]"
  on German German WIN7 Home Premium (64bit)", Common 4.0-dev User Profile
* server installation of " AOO 4.0.0-Dev – English UI / German locale 
  [AOO400m1(Build:9700) - Rev. 1457992 – Rev.1457606 ((2013-03-19))]" 
  on German WIN7 Home Premium (64bit)", own separate user profile
(c) No conspicuous memory consumption until
*  server installation of "AOO 3.4.0 – German UI / German locale 
   [AOO340m1(Build:9590) - Rev.1327774]" on German WIN7 Home Premium (64bit)"
   own separate user profile
(d) CONFIRMED due to Issue 118725 Comment 13
(e) It sould be interesting to see whether other OS are affected
Comment 1 Rainer Bielefeld 2014-05-28 06:21:16 UTC
c): Still ok with 3.4.1 on WIN
Comment 2 Rainer Bielefeld 2014-05-28 06:31:49 UTC
The memory problem is related to a picture loss problem. Proceeding the test will replace lots of pictures by "read error" placeholder boxes, if you safe the document in that situation the saved document will be much smaller than the original document.

@Oliver:
May be you want to check dependencies to your other picture loss fix activities?
Comment 3 Rainer Bielefeld 2014-05-28 07:40:34 UTC
I think this one is not a particular WRITER problem, more research required.
Comment 4 hdu@apache.org 2014-06-03 15:15:04 UTC
Also seen on Linux and Mac. After closing the document the memory consumption of the Mac-app went back to normal, which means that the memory is not leaked but it is cached longer. On Linux the RSS was reduced but the virtual memory stayed the same probably because of sal not giving the memory back.
Comment 5 hdu@apache.org 2014-06-04 09:53:54 UTC
This bug has the same root cause as the crash in bug 125000 => duplicate

*** This issue has been marked as a duplicate of issue 125000 ***
Comment 6 Oliver-Rainer Wittmann 2014-06-04 13:09:19 UTC
I think we should keep this issue open on its own
This one is about high memory consumption already observed in AOO 4.0.0, while issue 125000 is a crash which occurs since AOO 4.1.0. Thus, if we fix the crash introduced in AOO 4.1.0 it will not automatically fix the high memory consumption introduced in AOO 4.0.0.
Comment 7 Oliver-Rainer Wittmann 2014-06-04 13:09:34 UTC
back to confirmed status
Comment 8 Oliver-Rainer Wittmann 2014-06-06 11:47:08 UTC
taking over to provide certain improvements on the memory consumption.

As this issue more or less caused issue 125000 and issue 125000 has been marked as a candidate for a 4.1.x I am marking this issue also as a candidate.
Comment 9 Oliver-Rainer Wittmann 2014-06-06 11:47:51 UTC
damn - wrong issue number for dependency
Comment 10 SVN Robot 2014-06-06 12:28:19 UTC
"orw" committed SVN revision 1600863 into trunk:
124999: - call <SwGrfNode::onGraphicChanged()> only when <Graphic> instance h...
Comment 11 SVN Robot 2014-06-06 13:43:40 UTC
"orw" committed SVN revision 1600883 into trunk:
124999: As Writer graphics are no longer painted via the graphic manager - se...
Comment 12 Oliver-Rainer Wittmann 2014-06-06 13:51:21 UTC
Improved the memory footprint regarding Writer graphics:
- Avoid unnecessary load of the graphic's data during the load of documents
- Assure that the graphic's data is swapped out after it had been loaded for the rendering.

The changes in the memory footprint regarding Writer graphics are related to the improvements Armin had in the last releases for the rendering - he can tell you more about these improvements.
Comment 13 Oliver-Rainer Wittmann 2014-06-16 08:31:22 UTC
requesting 4.1.1 release blocker flag as fix for issue 125000 depends on this fix.
Comment 14 jsc 2014-06-16 08:47:14 UTC
grant showstopper flag, regression and already fixed
Comment 15 SVN Robot 2014-06-19 09:50:38 UTC
"orw" committed SVN revision 1603797 into branches/AOO410:
124999: - call <SwGrfNode::onGraphicChanged()> only when <Graphic> instance h...
Comment 16 SVN Robot 2014-06-19 09:51:56 UTC
"orw" committed SVN revision 1603799 into branches/AOO410:
124999: As Writer graphics are no longer painted via the graphic manager - se...
Comment 17 Oliver-Rainer Wittmann 2014-06-19 09:53:28 UTC
fixed on branch AOO410 for planned 4.1.1 release
Comment 18 fanyuzhen 2014-07-08 09:29:02 UTC
Checked it with build AOO411m1(Build:9770)  -  Rev. 1603804
2014-06-16 14:10:45 (Mo, 16 Jun 2014) on Windows, opening this document will rise Memory consumption by 800 KB or so, no crash. My PC: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHZ 1.97 GHz, 2.98 GB of RAM
Comment 19 fanyuzhen 2014-07-08 09:58:08 UTC
Correction: opening this document will rise Memory consumption by 2.3 MB or so
Comment 20 Pedro 2014-08-03 22:39:57 UTC
Testbook2.odt is no longer available but loading the heavy 108Mb ODT from Comment 31 https://issues.apache.org/ooo/show_bug.cgi?id=118725#c31 shows an increase of some 37Mb, which seems acceptable. There is no Crash.
Therefore the bug seems to be fixed in AOO411m4(Build:9774)  -  Rev. 1614049 (aka RC1)
Tested under Win 7 x64