Issue 101491 - performance: metafile creation got extreme slow - Calc hangs trying to open particular EXCEL document with charts
Summary: performance: metafile creation got extreme slow - Calc hangs trying to open p...
Status: CLOSED FIXED
Alias: None
Product: Draw
Classification: Application
Component: ui (show other issues)
Version: OOo 3.1 RC2
Hardware: PC All
: P3 Trivial (vote)
Target Milestone: OOo 3.1.1
Assignee: wolframgarten
QA Contact: issues@graphics
URL:
Keywords: performance, regression
Depends on:
Blocks: 101434
  Show dependency tree
 
Reported: 2009-05-02 09:16 UTC by Rainer Bielefeld
Modified: 2009-07-30 12:27 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
example with one big line chart (3.60 MB, application/x-compressed)
2009-05-19 15:16 UTC, IngridvdM
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description Rainer Bielefeld 2009-05-02 09:16:14 UTC
I was not able to test sample document from Issue 101434 with "Ooo 3.1.0 RC2
multilingual version German UI WIN XP: [OOO310m11 (Build 9399)]", OOo hangs with
100% CPU load. Document can be opened with 2.0.2
Comment 1 Rainer Bielefeld 2009-05-02 09:42:53 UTC
Same problem with "Ooo Dev 3.2.0 multilingual version English UI WIN XP:
[DEV300m44 (Build 9395)]": OOo hangs in step "calculating" during open process.

"Ooo 3.0.0 (DE) Multilingual version German UI WIN XP: [OOO300m9 (Build9358)]"
on an other PC) opens the document without any problems.
Comment 2 Rainer Bielefeld 2009-05-02 10:21:39 UTC
After a while (1/4 hour) CPU load decreases to app. 0, but document will not be
opened.
Comment 3 merschmann 2009-05-03 22:02:23 UTC
Confirmed with Kubuntu 9.04 -> OS all
The file is large and has some wideranged charts -> performace problem
Comment 4 kyoshida 2009-05-05 14:19:03 UTC
This smells much like Issue 99928, though I haven't confirmed whether this is a
duplicate of that.
Comment 5 IngridvdM 2009-05-18 16:01:04 UTC
I added the keyword regression as it was possible to open the document with a
previous version. With OOo3.0 and dev300m29 it was possible to open the document
in less a minute on my computer and even switch to sheet 'BoE-iFWDr' which
contains a big line chart. In dev300m30 it was possible to load the document in
less then one minute but switching to sheet 'BoE-iFWDr' hangs. Within OOo 3.1
the document has not finished loading after 40 minutes and the memory
consumption has raised by more than 1GB. In the end there is not much processor
usage. Smells like a memory reallocation problem. Maybe a memory leak also?
In sum there are two problems:
1) The metafile creation has decreased dramatically in performance.
2) All metafiles are created allready while the document is loaded even if the
according chart is not visible at all.
@aw, please have a look at the performance collapse of the metafile creation.
This issue should concentrate on problem 1.

I created a reduced example document which does contain only one chart, so you
can identify the problematic meta file creation call easily. 

@kohei, I think this has noting to do with issue 99928.
Comment 6 IngridvdM 2009-05-19 15:16:42 UTC
Created attachment 62374 [details]
example with one big line chart
Comment 7 IngridvdM 2009-05-19 15:18:29 UTC
For the second problem I created a separate issue 102063.
Comment 8 Armin Le Grand 2009-06-22 15:02:28 UTC
AW: Adding to CWS aw074...
Comment 9 Armin Le Grand 2009-06-24 15:30:07 UTC
AW: Looking what happens internally. The MetaFile creation has to do some deep
decompositions; there seem to be 81 main objects and each of them has a
PolygonStrokePrimitive which decomposes to 36679 filled polygons to create the
stroke geometry. This seems to be the expensive part.
For MetaFile creation this may be implemented using
MetaLineAction/MetaPolyLineAction where LineInfo is used. The problem here are
the edge roundings, which would not be supported by the MetaFile. Since this was
not supported in the past, this may not be a problem...
Comment 10 Armin Le Grand 2009-06-24 16:26:03 UTC
AW: Experimented with directly creating MetaPolyLineAction(s) which host the fat
lines (35.0 units wide). This works in principle and is much faster, but the
MetaFile created does not use AA when it gets painted since the methods in VCL
which work with LineInfo (DrawPolyLine, DrawLine) are not adapted to AA usage.
If they were, they would again create fat line geometry to implement this.

If we can live with the fact that fat lines in charts will not be displayed
AAed, we have a solution which is much faster and needs much less memory, too...

Experimenting with how we could get this AAed eventually...
Comment 11 IngridvdM 2009-06-24 17:02:47 UTC
Fat lines are default in the chart so it would be a great pity to loose AA here
completely. Maybe it is possible to introduce a threshold value for the number
of points in a polygon, the like that below the threshold a nice AA graphic is
produced and if the number of points is too high the not AA'ed but fast version
is produced?
Comment 12 Armin Le Grand 2009-06-25 10:49:32 UTC
AW->iha: I agree, losing AA would be ugly; but in the end it's the reason for
the performance task and the difference between 3.0 and 3.1 here. AA does not
come for free. Without AA, i can creach the same speed as before (to say: not
really primitive problem).

The basic problem are fat lines on various sytems. For WIN and MAC we can
directly use the system (also not blazing fast, but...), but for UNIX (linux,
solaris) the line geometry HAS to be broken down to be painted as polygons ->
this is exactly the 'slow' operation, no difference if it happens at MetaFile
creation or later at MetaFile paint.

Continuing experimenting...
Comment 13 Armin Le Grand 2009-06-25 16:03:15 UTC
AW: I have a partial solution, need to check on Linux and Mac...
Comment 14 Armin Le Grand 2009-06-25 17:52:34 UTC
AW: The speed problem is pretty well solved, there are two problems remaining:
(a) The edge roundings under unix/linux sometimes look like something is missing
(asked hdu already)
(b) On the mac there is no basegfx::B2DLINEJOIN_NONE, instead kCGLineJoinMiter
is used. This is bigger than the BoundRect, so somehow a NONE needs to be realized.

Looking further...
Comment 15 Armin Le Grand 2009-06-26 12:19:05 UTC
AW: Did some finetuning; in principle the paths in VCL painting line polygons
with linewidth (referred to as LPWL) have to be completely supported;
unfortunately this leads to a lot of changes (which are mostly used in AAed
cases, so not too dangerous when thinking about the whole VCL UI stuff).

The basic change is to not export decomposed geometry in MetaFile processor,
starting from a specific treshhold (used 1000 points in polygon for now).

Another change is to support LPWL in Pixel processor by using the same treshhold
to fallback to temporary (only for paint) decompositions. This also needed to
support AA when needed.

A third change is to support LPWL in MetaFile replay, especially MetaFile
entries with LineInfo data. These were not supported for AA before at all, so
that whole support (also using fallbacks) is added now.

Further changes look for making all this and all fallbacks work on all systems
with AA and without AA.

Looks good so far; i will need to re-check unix and mac versions. Checking in
fine tunings...
Comment 16 Armin Le Grand 2009-06-30 10:32:18 UTC
AW: For (b) i will use the fallback, this will work. For (a) i will have to take
another look at basegfx::tools::solveCrossovers() and the edge rounding creations...
Comment 17 Armin Le Grand 2009-07-01 12:17:19 UTC
AW: Checked line geometry creation for a critical example, no problems found.
The involved points are exactly equally calculated and are equal. Checking again
and looking at the addPointsAtCutsAndTouches method, too...
Comment 18 Armin Le Grand 2009-07-01 14:31:11 UTC
AW: Found a very small difference in the example; while the first edge is a
curve (solving trivial bezier does nt trigger) the 2nd is not. Thus, for the 2nd
edge creation, the outer points are calculated using the direct perpendicular of
the edge (end - start) instead of using rEdge.getTangent(0.0/1.0) which pretty
much does the same with the small difference that the result is multiplied by
0.3 to create default tangent vectors for edges.
Also added a const element counter to mergeTemporaryPointsAndPolygon, just a
small addon...
Comment 19 Armin Le Grand 2009-07-01 14:34:48 UTC
AW: I think i got the glitch now: In createAreaGeometryForJoin, when
B2DLINEJOIN_ROUND is used, the created partial arc is appended to the simple
angle-defining polygon. That addded part also has the edge start/end points, but
created numerically from a angle define combined with a numerically not
completely correct circle description using beziers and their cuts. This leads
to slightly different positions.
To solve this, it will be necessary to use the 'real' start and end points of
the rounding, but add the bezier edge rounding info to them. Trying this...
Comment 20 Armin Le Grand 2009-07-01 15:47:09 UTC
AW: Works as expected, this should fix (a).
Also re-checked the polygon goven by HDU in his eMail and if
addPointsAtCutsAndTouches works as expected. Directly added the points to the
XML content of a file with a polygon for that purpose. The method works as
expected; the needed points are added. It seem this was not the problem.
All direct problems of this task solved, code checked in -> done.
Comment 21 Armin Le Grand 2009-07-02 11:06:31 UTC
AW: Building mac and linux versions for testing...
Comment 22 Armin Le Grand 2009-07-06 16:33:34 UTC
AW: Ckecked with wntmsci12.pro build, works as expected.
Comment 23 Armin Le Grand 2009-07-07 09:55:59 UTC
AW->WG: Please review, e.g. with  HJMemulator_reduced.zip
Comment 24 wolframgarten 2009-07-09 10:02:49 UTC
Verified in CWS.
Comment 25 wolframgarten 2009-07-30 12:27:17 UTC
Tested in m17. Closed.