Bug 152213

Ideally we should have this fixed by patch_1.

please, attach zipped the header file.
Or steps how produce the header file.

Created attachment 73316 [details]
Header file for SolidWorks COM-Client

May be also responsible for the problem is my crude folder structure:

home
 |-NetbeansProject1
 |-NetbeansProject2
 `-src
    |-main.cc
    |-sldworks.h
    |- ...

I. e. I have two Netbeans projects using some same code located in folder src "besides" the project folders; this is
where my big header file is in.
I just figured out that the debugger is not stopping in my main.cc, it seems that Netbeans don't like such a structure?
I will file another issue for that soon.

It seems
Please,
- provide NB additional memory in file ${NB}/etc/netbeans.conf: "-J-Xmx800M".
- do not open the header in editor.
Is problem gone away?
Also, could you try NB 6.5?

I just tried the setting -J-Xmx800M, it seems to be much better: I have only short peaks with 100% CPU load now. My old
setting has been -J-Xmx300M; since the memory toolbar displays about 250/280 MB I thought it to be enough. (But: My
development Notebook has only 1GB RAM, and I have to run SolidWorks (a resource-hungry CAD application) together with
Netbeans ...)
The problem with the crash when opening the file still remains.

I don't want to install Netbeans 6.5 now because I fear it could interfere with my current installation, sorry. I will
install the new version on my computer at home during the next days and try again there.

Request of another user:

Unfortunately, the project I work on is proprietary and I cannot share any of the source code. I have no performance
issues working on a normal project, but source files that I deal with at work can exceed 50K lines in size.

To use netbeans at my work, I manually increase the maximum memory of the process to 1 GB so that I have no problems
viewing any files or navigating through the code. However, whenever I need to edit a large file I usually revert to
using gvim because the reparse cycles take too long while I'm editing. Also, one notices a similar degradation editing
header files, regardless of their size, that are included in every single source file in the project, because netbeans
appears to be reparsing every file that includes the header file while I edit the header file.  Finally, I periodically,
usually once every other day, need to shut down and restart netbeans because its minimum memory usage after a garbage
collection grows too high. I think this is related to large files as well because memory usage usually only spikes while
editing large files.

Looking at the ticket you referenced in your email, I would say that my issue is either similar or exactly the same as
the issue described in the ticket, but at a smaller scale because he is working with a 300k line header file that is
included in other files, as opposed to the 50K+ line source files that I see in my project.

This scalability issue with large files is the only one I deal with when using cnd. A similar open source project I
haved worked on that is over half the size of the proprietary project I deal with at work but has more reasonably sized
source files, usually never exceeding 1k lines per file, causes no trouble for netbeans at all for either memory usage
or performance.

Let me know if there is  any other useful information I can provide.

Sean Reque

- use file reference content in mark occurrences reference visitor.
Change set: http://hg.netbeans.org/main?cmd=changeset;node=c1060ab7617c

Integrated into 'main-golden', will be available in build *200901230201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/a72b99cdd834
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#152213:Large header files cause massive performance problem
- provide direct access to file declarations by offset and kind

Created attachment 76175 [details]
biggest objects when 300K line file is opened in editor

Integrated into 'main-golden', will be available in build *200902180201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/a1c192fbfe59
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing: IZ#152213:Large header files cause massive performance problem
- fix low performance of c/c++ code folding

I checked out the revision a1c192fbfe59 and netbeans cnd had the best performance I have seen to date. I think the
biggest improvement came from the fact that the IDE was no longer re-parsing files automatically, but only when I saved
a file. I was able to edit both large source files (<= 70K lines) and large ubiquitously-included projected header files
with only moderate slowdown. However, after encountering an unrelated bug I upgraded to a later revision, e27585b9d94d,
and performance has again degraded to unusable levels. It seems that cnd is also again re-parsing files automatically,
not waiting until I save them.

Integrated into 'main-golden', will be available in build *200903191401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/8a872e037ac9
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#152213:Large header files cause massive performance problem
- increase repase delay on file edit for large files

Integrated into 'main-golden', will be available in build *200903210201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/930382d09944
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#152213:Large header files cause massive performance problem
- change algorithm of counting of repase delay

Integrated into 'main-golden', will be available in build *200903281400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/496658d9f689
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#152213:Large header files cause massive performance problem
- remove expensive macro iterator by file

Looks like the most critical part of the issue has been fixed. Downgrading to P3. Alexander, please consider if we can
close this IZ and open some more specific IZ (may be a few of them).

Integrated into 'main-golden', will be available in build *200904220201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/5b4bfb0ef497
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#152213:Large header files cause massive performance problem
- fix rendering of typedefs (wrong algorithm consumes 15% of parsing time)

first implementation of "precompiled" header in the change set:
http://hg.netbeans.org/cnd-main/rev/d96cb7fcc78a
- fixed responsiveness problem in following case:
Source file includes "large" header as first included file.
In this case reparsing time of source file do not depend on size of first header file.

sreque,
now you can forbids reparsing on document changed (see IZ#168651).

IMHO performance is good enough, dynamic reparsing can be forbidding, so I close issue as fixed.

fixed

Integrated into 'main-golden', will be available in build *200910221401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/58f474b9552a
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing IZ#168651 Add an option to stop the dynamic code indexing
fixing IZ#152213 Large header files cause massive performance problem