Issue 120839 - [From Symphony]It costs too much memory to open a large spreadsheet file containing pivot tables
Summary: [From Symphony]It costs too much memory to open a large spreadsheet file cont...
Status: CLOSED FIXED
Alias: None
Product: Calc
Classification: Application
Component: open-import (show other issues)
Version: 3.4.0
Hardware: All All
: P3 Normal (vote)
Target Milestone: 4.0.0
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-09-07 03:18 UTC by Clarence GUO
Modified: 2017-05-20 10:31 UTC (History)
1 user (show)

See Also:
Issue Type: ENHANCEMENT
Latest Confirmation in: ---
Developer Difficulty: ---

Attachments
Fix patch (6.80 KB, patch)
2012-09-07 05:55 UTC, Clarence GUO 		no flags Details | Diff
New fix patch (6.48 KB, patch)
2012-09-07 09:32 UTC, Clarence GUO 		clarence.guo.bj: review?
Details | Diff
Show Obsolete (1) Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this issue.
Description Clarence GUO 2012-09-07 03:18:53 UTC 
Open some large spreadsheet files containing big pivot tables, it costs too much memory in the peak time during loading. For some files, it even consume 1.8G memory. This issue would cost Symphony freeze or crash when loading a too large file.
Comment 1 Clarence GUO 2012-09-07 05:31:43 UTC 
The sample is very big, so post to https://docs.google.com/open?id=0B27TsJa_-7-JaU8tTFk0UUxRaTg.
It will cost 320M memory during loading. For some bigger samples, the memory consumption can even reach to 1.8G
Comment 2 Clarence GUO 2012-09-07 05:55:00 UTC 
Created attachment 79340 [details]
Fix patch
Comment 3 Clarence GUO 2012-09-07 06:09:14 UTC 
[Root Cause]
Read all caches first and then convert them later together. It will cost too much memory to store mid result
[Resolution]
Read one cache and then convert the related tables and then release memory, then read the next one then convert, release. Do it looply.
Comment 4 Clarence GUO 2012-09-07 09:32:07 UTC 
Created attachment 79354 [details]
New fix patch
Comment 5 Wang Lei 2012-09-07 09:34:07 UTC 
confirm it
Comment 6 SVN Robot 2012-09-07 09:47:26 UTC 
"leiw" committed SVN revision 1381965 into trunk:
#i120839 [From Symphony]It costs too much memory to open a large spreadsheet ...
Comment 7 Wang Lei 2012-09-07 09:48:14 UTC 
Submit in revision 1381965