Apache OpenOffice (AOO) Bugzilla – Issue 101491
performance: metafile creation got extreme slow - Calc hangs trying to open particular EXCEL document with charts
Last modified: 2009-07-30 12:27:17 UTC
I was not able to test sample document from Issue 101434 with "Ooo 3.1.0 RC2 multilingual version German UI WIN XP: [OOO310m11 (Build 9399)]", OOo hangs with 100% CPU load. Document can be opened with 2.0.2
Same problem with "Ooo Dev 3.2.0 multilingual version English UI WIN XP: [DEV300m44 (Build 9395)]": OOo hangs in step "calculating" during open process. "Ooo 3.0.0 (DE) Multilingual version German UI WIN XP: [OOO300m9 (Build9358)]" on an other PC) opens the document without any problems.
After a while (1/4 hour) CPU load decreases to app. 0, but document will not be opened.
Confirmed with Kubuntu 9.04 -> OS all The file is large and has some wideranged charts -> performace problem
This smells much like Issue 99928, though I haven't confirmed whether this is a duplicate of that.
I added the keyword regression as it was possible to open the document with a previous version. With OOo3.0 and dev300m29 it was possible to open the document in less a minute on my computer and even switch to sheet 'BoE-iFWDr' which contains a big line chart. In dev300m30 it was possible to load the document in less then one minute but switching to sheet 'BoE-iFWDr' hangs. Within OOo 3.1 the document has not finished loading after 40 minutes and the memory consumption has raised by more than 1GB. In the end there is not much processor usage. Smells like a memory reallocation problem. Maybe a memory leak also? In sum there are two problems: 1) The metafile creation has decreased dramatically in performance. 2) All metafiles are created allready while the document is loaded even if the according chart is not visible at all. @aw, please have a look at the performance collapse of the metafile creation. This issue should concentrate on problem 1. I created a reduced example document which does contain only one chart, so you can identify the problematic meta file creation call easily. @kohei, I think this has noting to do with issue 99928.
Created attachment 62374 [details] example with one big line chart
For the second problem I created a separate issue 102063.
AW: Adding to CWS aw074...
AW: Looking what happens internally. The MetaFile creation has to do some deep decompositions; there seem to be 81 main objects and each of them has a PolygonStrokePrimitive which decomposes to 36679 filled polygons to create the stroke geometry. This seems to be the expensive part. For MetaFile creation this may be implemented using MetaLineAction/MetaPolyLineAction where LineInfo is used. The problem here are the edge roundings, which would not be supported by the MetaFile. Since this was not supported in the past, this may not be a problem...
AW: Experimented with directly creating MetaPolyLineAction(s) which host the fat lines (35.0 units wide). This works in principle and is much faster, but the MetaFile created does not use AA when it gets painted since the methods in VCL which work with LineInfo (DrawPolyLine, DrawLine) are not adapted to AA usage. If they were, they would again create fat line geometry to implement this. If we can live with the fact that fat lines in charts will not be displayed AAed, we have a solution which is much faster and needs much less memory, too... Experimenting with how we could get this AAed eventually...
Fat lines are default in the chart so it would be a great pity to loose AA here completely. Maybe it is possible to introduce a threshold value for the number of points in a polygon, the like that below the threshold a nice AA graphic is produced and if the number of points is too high the not AA'ed but fast version is produced?
AW->iha: I agree, losing AA would be ugly; but in the end it's the reason for the performance task and the difference between 3.0 and 3.1 here. AA does not come for free. Without AA, i can creach the same speed as before (to say: not really primitive problem). The basic problem are fat lines on various sytems. For WIN and MAC we can directly use the system (also not blazing fast, but...), but for UNIX (linux, solaris) the line geometry HAS to be broken down to be painted as polygons -> this is exactly the 'slow' operation, no difference if it happens at MetaFile creation or later at MetaFile paint. Continuing experimenting...
AW: I have a partial solution, need to check on Linux and Mac...
AW: The speed problem is pretty well solved, there are two problems remaining: (a) The edge roundings under unix/linux sometimes look like something is missing (asked hdu already) (b) On the mac there is no basegfx::B2DLINEJOIN_NONE, instead kCGLineJoinMiter is used. This is bigger than the BoundRect, so somehow a NONE needs to be realized. Looking further...
AW: Did some finetuning; in principle the paths in VCL painting line polygons with linewidth (referred to as LPWL) have to be completely supported; unfortunately this leads to a lot of changes (which are mostly used in AAed cases, so not too dangerous when thinking about the whole VCL UI stuff). The basic change is to not export decomposed geometry in MetaFile processor, starting from a specific treshhold (used 1000 points in polygon for now). Another change is to support LPWL in Pixel processor by using the same treshhold to fallback to temporary (only for paint) decompositions. This also needed to support AA when needed. A third change is to support LPWL in MetaFile replay, especially MetaFile entries with LineInfo data. These were not supported for AA before at all, so that whole support (also using fallbacks) is added now. Further changes look for making all this and all fallbacks work on all systems with AA and without AA. Looks good so far; i will need to re-check unix and mac versions. Checking in fine tunings...
AW: For (b) i will use the fallback, this will work. For (a) i will have to take another look at basegfx::tools::solveCrossovers() and the edge rounding creations...
AW: Checked line geometry creation for a critical example, no problems found. The involved points are exactly equally calculated and are equal. Checking again and looking at the addPointsAtCutsAndTouches method, too...
AW: Found a very small difference in the example; while the first edge is a curve (solving trivial bezier does nt trigger) the 2nd is not. Thus, for the 2nd edge creation, the outer points are calculated using the direct perpendicular of the edge (end - start) instead of using rEdge.getTangent(0.0/1.0) which pretty much does the same with the small difference that the result is multiplied by 0.3 to create default tangent vectors for edges. Also added a const element counter to mergeTemporaryPointsAndPolygon, just a small addon...
AW: I think i got the glitch now: In createAreaGeometryForJoin, when B2DLINEJOIN_ROUND is used, the created partial arc is appended to the simple angle-defining polygon. That addded part also has the edge start/end points, but created numerically from a angle define combined with a numerically not completely correct circle description using beziers and their cuts. This leads to slightly different positions. To solve this, it will be necessary to use the 'real' start and end points of the rounding, but add the bezier edge rounding info to them. Trying this...
AW: Works as expected, this should fix (a). Also re-checked the polygon goven by HDU in his eMail and if addPointsAtCutsAndTouches works as expected. Directly added the points to the XML content of a file with a polygon for that purpose. The method works as expected; the needed points are added. It seem this was not the problem. All direct problems of this task solved, code checked in -> done.
AW: Building mac and linux versions for testing...
AW: Ckecked with wntmsci12.pro build, works as expected.
AW->WG: Please review, e.g. with HJMemulator_reduced.zip
Verified in CWS.
Tested in m17. Closed.