Issue 97886

Summary: Datapilot uses a lot of memory in OOo Calc
Product: Calc Reporter: pvrohr <peter.vonrohr>
Component: open-importAssignee: AOO issues mailing list <issues>
Status: CONFIRMED --- QA Contact:
Severity: Trivial    
Priority: P3 CC: issues, kyoshida
Version: DEV300m37   
Target Milestone: ---   
Hardware: PC   
OS: Windows Vista   
URL: http://n.ethz.ch/student/pvrohr/OOoML/
Issue Type: DEFECT Latest Confirmation in: ---
Developer Difficulty: ---
Attachments:
Description Flags
zip-file with spreadsheet and database
none
Memory diagram none

Description pvrohr 2009-01-09 09:05:00 UTC
When starting or refreshing a series of datapilots which are accessing an OOo
Base database, OOo Calc consumes all available memory and just freezes. 

The problem occurs on:
- lenovo thinkpad t400 (4gb ram) under vista business sp1 using OOo Dev300m37
- acer aspire 5000 (1gp ram) under xp sp3 using OOo 3.0

An example spreadsheet with the respective database is available in a zip-file
under:
http://n.ethz.ch/student/pvrohr/OOoML/
Comment 1 pvrohr 2009-01-09 09:08:18 UTC
Created attachment 59258 [details]
zip-file with spreadsheet and database
Comment 2 aladdin2k7 2009-01-09 09:59:59 UTC
Confirmed on OOo 2.4.
Comment 3 ooo 2009-01-09 11:38:04 UTC
@kohei: is this related to (and maybe fixed with) issue 93998?
Comment 4 kyoshida 2009-01-09 14:36:44 UTC
@er: I'm not sure.  The fix in issue 93998 is mostly effective on sheet data
source and/or data with lots of duplicate or empty rows.  The example data is a
database with no duplicate rows, so that fix wouldn't have been effective.

Having said that, when refreshing datapilot output with Base data, it takes some
time to refresh, but eventually it finishes.  But I admit, it still uses a lot
of memory and we should probably find more ways to reduce memory footprint there.

Just to provide a data point, on my system (openSUSE 11.0), the memory use
before refreshing was 248 MB, and it rose to 1.2 GB after refreshing.

What's interesting is that this problem already existed in 2.4 (according to
aladdin2k7's comment).  That version doesn't use cache tables, so the problem
may not be just in the cache table implementation.  Maybe the result calculation
code is using too much memory?  I don't know...

Sounds a little like Issue 55266 at first glance.
Comment 5 kyoshida 2009-01-09 14:53:44 UTC
Changing the title.  The term "memory leak" has a different meaning & it has not
been proven that Calc leaks memory.  (It's possible, just not proven yet.)
Comment 6 pvrohr 2009-01-09 14:57:30 UTC
Created attachment 59269 [details]
Memory diagram
Comment 7 pvrohr 2009-01-09 15:00:20 UTC
i just added a memory diagram from the vista machine as an attachment to this
issue. the high demand in memory is one thing, but to me the fact that memory is
never returned seams not ok to me. 

but in any case thanks a lot for all the good work in such a short time.
Comment 8 ooo 2009-01-09 18:10:21 UTC
@pvrohr: The Windows task manager's performance overview is not suited to
determine actual memory consumption of a process. What you see may just be some
contiguous space of reserved memory, only partly committed memory. Freeing
allocated memory does not necessarily decrement that value. A slightly more
accurate value may be obtained by monitoring the VM Size of a process.