Issue 64721 - improve memory management of pdf converter; crashes
Summary: improve memory management of pdf converter; crashes
Status: CONFIRMED
Alias: None
Product: Writer
Classification: Application
Component: code (show other issues)
Version: OOo 3.0 Beta 2
Hardware: PC Windows 2000
: P3 Trivial with 2 votes (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
: 84949 (view as issue list)
Depends on:
Blocks:
 
Reported: 2006-04-25 06:16 UTC by matthias007
Modified: 2017-05-20 11:15 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
shows error message with newly installed win2k (33.10 KB, image/jpeg)
2006-05-02 20:01 UTC, matthias007
no flags Details
shows crash report and usage of memory at crash (78.37 KB, image/jpeg)
2006-05-09 05:33 UTC, matthias007
no flags Details
shows memory usage going up before crash when exporting pdf (93.04 KB, image/jpeg)
2008-07-30 11:55 UTC, matthias007
no flags Details
testfile that crashed when pdf was exported (109.93 KB, application/vnd.oasis.opendocument.text)
2008-07-30 11:58 UTC, matthias007
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description matthias007 2006-04-25 06:16:55 UTC
I have a small text of about 100k with 163 jpg pictures
of a digital camera outside in an extra folder.

Each time when opening the file OO opens up each picture
and places all the data in memory.

The draw backs are 
- time consuming opening of file
- very large amount of memory necessary and 
  lange virtual memory file
- when running out of memory
  - crashing
  - try to repair undamaged 100kB file which 
    may result in another crash  
  - no pdf export possible
   
A possible workaround could be that
+ OO initially only opens these pictures that are 
  shown immediately on the screen. When one scrolls
  other needed pictures are loaded.  
+ OO frees memory from not needed pictures when 
  running out of memory.
+ OO does not load all pictures in memory for doing
  a printout. But instead uses just the needed ones for 
  the page in printing process.
+ OO reduces picture size when opening.
Comment 1 lars 2006-04-25 11:51:02 UTC
possible duplicate to issue 57955
Comment 2 michael.ruess 2006-04-25 12:37:41 UTC
Some kind of "load on demand" for Graphics (and even OLE'S) is desired here.
Comment 3 matthias007 2006-04-25 19:03:09 UTC
this issue is not meant as a duplicate of 57955
because in 57955 only load time is mentioned.

Here also other problems are addressed like possible
system crash and pdf output crash. 
Comment 4 merschmann 2006-04-26 22:04:25 UTC
As this is from the german user list,
(started with http://de.openoffice.org/servlets/ReadMsg?list=users&msgNo=48504)
I should add some information:
- matthias007 has a 768MB-machine and tries to resize the photos in OOo.
- We calculated at least 1,5-2 GB for the whole document with pictures without
compression.
- The graphics-cache has been increased, the crash comes later then.
- Similiar file could be loaded on equivalent Linux system by another user
Comment 5 matthias007 2006-04-26 22:22:18 UTC
to add some more info:
The system is Win2k. 768MB Ram. AthlonXP2000.
System installed new. Same problem. Run time error.
It does not seem that the system runs out of hard 
disk cache. The system crashed when showing 68% usage. 

My brother has a similar system using Win2k
and 1GB memory. He got a memory warning and crashed
one time. The next time it worked and it was possible
to open the 108kB dokument and do a pdf-export. 
On his system this worked. 

One guy in the list "Rainer" tried loading a file with 
many pictures but was not using the cut feature of OO.
It seemed to work with more than 400 smaller pictures 
of each 750kB.

But it crashed on a similar windows system of his wife
when he integrated the pictures.  
Comment 6 matthias007 2006-05-02 20:01:36 UTC
Created attachment 36221 [details]
shows error message with newly installed win2k
Comment 7 matthias007 2006-05-02 20:03:05 UTC
sometimes I am able to open the file and work with it, sometimes I get this
runtime error. I will retest with more memory.  
Comment 8 matthias007 2006-05-08 17:34:36 UTC
now I have 1.8GB memory built inside my PC.
The same problem occurs: Run time error!

I tried a simple workaround:

I took half of the external pictures in one 
folder and the other half in another.
I renamed the folders, so that OO could 
only access the pictures of one of them
at a time. This works !

Benefit:
+ highly improved start up time
+ it works and shows the graphics
  in the first half of the document
+ I can choose the second half afterwards

Possible improvement to implement:
+ OO could do this quite simple procedure for me:
+ OO could create 2 folders 1 and 2 and place the pictures
  inside in the order that they are called.
+ OO could choose to start with folder 1 or
  folder 2 depending on the current reading pointer. 
+ If one scrolls then OO could 
  load the next folder and may throw out the first one.
+ there could be also 3 folders instead of using 2. 
+ doing this way even documents with many pictures
  could be worked with without any handling discomfort.
+ this procedure could be patented if wanted. 

Best Regards
  Matthias Weisser      
Comment 9 matthias007 2006-05-09 05:33:28 UTC
Created attachment 36326 [details]
shows crash report and usage of memory at crash
Comment 10 matthias007 2006-05-09 05:38:55 UTC
to be sure that the crash problem is not a 
problem of my new installation of OO2.02
I removed my english OO2.02 and the german lang pack.

I rebooted and installed a new german OO2.02.  

I am still getting a crash, but this time 
not with a window showing RunTimeError.

As you can see in the second attachment
my pc uses nearly the whole memory and
only a part of the virtual memory. 

So why it is then crashing?
Comment 11 michael.ruess 2008-01-03 09:23:39 UTC
*** Issue 84949 has been marked as a duplicate of this issue. ***
Comment 12 matthias007 2008-04-19 20:41:31 UTC
I switched to OO2.4 and tried to export my old text to pdf. 

I am getting a run time error. OO crashes. 
The task manager shows memory usage going up and up 
before the crash. My system has 2GB of memory.
My online crash report went to Sun microsystems. 

At the moment I do not seem to be able to do an pdf export
which worked with the older versions of OO. 

This is a show stopper for me, because I do need a 
really working pdf export.   
Comment 13 matthias007 2008-07-29 19:25:03 UTC
Today I installed OOo_3.0.0beta2_20080707_Win32Intel_install_wJRE_de.exe
and tried opening the above old text.

It works opening the odt-file and the pictures are shown correctly.

If I push the pdf-Button the pdf-Export is started correctly,
but more and more memory is used as has been before just
opening the odt-file. At some time OO crashes with a run time error
similar to the above crash report from may 9th.  

The problem seems to be related to a memory management problem using 
the pdf-converter. The same problem had been in OO2.4. But it had
been ok some time before 2.4. Maybe the 2.1, 2.2 or 2.3 had been ok. 
Comment 14 michael.ruess 2008-07-30 07:20:04 UTC
Duplicate of issue 34715.

*** This issue has been marked as a duplicate of 34715 ***
Comment 15 michael.ruess 2008-07-30 07:21:02 UTC
Closed.
Comment 16 matthias007 2008-07-30 09:19:36 UTC
this is not the same behaviour as Issue 34715.

Issue 34715 talks about a memory loss of 10kB
per file. This is not the case here for crashing
my file. 

As can be seen here the memory usage is due 
to the pictures coming up until there is a crash. 

There is no crash reported with Issue 34715.
Therefore its not the same.

I see several possibilities:
+ close this Issue when the behaviour is 
  correctly added to Issue 34715. At the 
  moment I do not see one doing so. 
+ leave this as it is, because it describes 
  the problem correctly.
+ close this issue and open up a new issue
  describing the crash behaviour that is 
  not given in Issue 34715.

Anyway the problem seems to be a memory 
management problem, but not due to a memory
leak of 10k per usage as in Issue 34715.    
Comment 17 michael.ruess 2008-07-30 10:03:03 UTC
Then please attach you file here, so that we are able to reproduce and fix the
issue.
I tried with a manual containing about 150 graphics and it worked fine. So
please, provide you r doc, other wise we are not able to do anything on this.
Comment 18 matthias007 2008-07-30 11:55:38 UTC
Created attachment 55447 [details]
shows memory usage going up before crash when exporting pdf
Comment 19 matthias007 2008-07-30 11:58:02 UTC
Created attachment 55448 [details]
testfile that crashed when pdf was exported
Comment 20 matthias007 2008-07-30 12:16:34 UTC
There are 2 attachments:
+ in the odt-file I changed the characters
  to be able to give it here. The pictures
  are embedded as links. They are as big as
  1.2 MB and 1.5MB each. Therefore I am not 
  able to do an attachment of these.
 
  I could do a DVD and send it somethere, if 
  there is no other possibility to check this. 

+ the jgp-file shows the exporting of the pdf
  file with the memory running up and then the 
  crash message being shown.

I also tried building a similar file using only
one picture - being imported the same way about
120 times. This file does not crash and the memory
will not go up continuously the same way. 
Therefore I do not add this file here. 
Comment 21 matthias007 2008-07-30 16:56:44 UTC
looking at the pdf-file sizes I get:
+ about 61MB pdf file size for a file
  containing all pictures which now 
  crashes when I try producing it.
+ about 1MB pdf file size for a similar
  file, when only one external jpg file
  instead of 150 different is used. 

Producing the 1MB pdf there seems to be 
no crash and no error.  
Comment 22 michael.ruess 2008-08-05 11:52:30 UTC
MRU->HI: now there is an attachment available; please have a look at it.
Comment 23 Marcus 2017-05-20 11:15:32 UTC
Reset assigne to the default "issues@openoffice.apache.org".