This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 250470 - Slow and Weird auto-suggest for Exception class names in New Breakpoint dialog
Summary: Slow and Weird auto-suggest for Exception class names in New Breakpoint dialog
Status: CLOSED INVALID
Alias: None
Product: debugger
Classification: Unclassified
Component: Java (show other bugs)
Version: Dev
Hardware: PC Windows 7
: P2 normal with 3 votes (vote)
Assignee: Martin Entlicher
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-17 17:49 UTC by NukemBy
Modified: 2017-11-27 17:08 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Autosuggest in New Breakpoint (22.63 KB, image/png)
2015-02-17 17:49 UTC, NukemBy
Details
temp-NB-code-completion (178.50 KB, application/octet-stream)
2015-05-01 11:25 UTC, NukemBy
Details

Note You need to log in before you can comment on or make changes to this bug.
Description NukemBy 2015-02-17 17:49:32 UTC
Created attachment 152034 [details]
Autosuggest in New Breakpoint

When I need to set breakpoint on exception I always have to write full name "by hand" - because auto-suggest just does NOT work :(

- it is slow - "some" result appear in around 5 seconds after typing (intellisense in text editor work almost instantly)

- provided results are irrelevant - see the screenshot, i was trying to set breakpoint NullPointerException
Comment 1 Martin Entlicher 2015-03-04 10:56:30 UTC
I've reproduced the problematic behavior.
- One problem seems to be the case sensitivity, "nullp"... versus "NullP"...
- The "Please wait..." message is sometimes not replaced with the results
- I'll check the performance.
Comment 2 Martin Entlicher 2015-03-04 15:59:10 UTC
The content of the completion is fixed by http://hg.netbeans.org/core-main/rev/b032c222faca
Working on the performance issue...
Comment 3 Quality Engineering 2015-03-06 04:11:27 UTC
Integrated into 'main-silver', will be available in build *201503060001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/b032c222faca
User: mentlicher@netbeans.org
Log: #250470: Provide correct completion content.
Comment 4 Martin Entlicher 2015-03-06 11:23:22 UTC
Code-completion was further improved by changeset:   285051:cc43715c7ab0
http://hg.netbeans.org/core-main/rev/cc43715c7ab0

It's hopefully accurate and sufficiently performant now.
Comment 5 Quality Engineering 2015-03-07 08:40:56 UTC
Integrated into 'main-silver', will be available in build *201503070001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/cc43715c7ab0
User: mentlicher@netbeans.org
Log: #250470: Better and faster code-completion.
Comment 6 NukemBy 2015-03-12 16:36:28 UTC
Seems not to be working for me. I've got this slowness report while trying to add breakpoint on Exception
http://statistics.netbeans.org/analytics/exception.do?id=770184 and
http://statistics.netbeans.org/analytics/exception.do?id=770189

Was typing "nullp" - message Please Wait was showing for around 5 seconds, i retyped the same in case sensitive manner "NullP" - same result - endless "Please wait ..." message.

I'm using latest dev build - 20150312
Comment 7 NukemBy 2015-05-01 11:25:32 UTC
Created attachment 153490 [details]
temp-NB-code-completion
Comment 8 NukemBy 2015-05-01 11:33:11 UTC
It looks like that problem may also negatively affect other functionality.

The steps i've done recently:

1. tried to add exception breakpoint - was typing "nullp" in "add exception breakpoint" dialog. Could not wait until autosuggest to complete as usual

2. copy/pasted exception name from log (to avoid typos)

3. attached to process - stopped on exception breakpoint, but ... NB could not show neither current stack trace nor current variables. 

After waiting for half minute i've started self profiler for half minute. It shows that Code Completion thread was running all that time. Not sure if it was holding some global locks that may prevent other threads to run normally.

Selfsampler is attached.

PS: raising priority to P2 since that problem may affect not just "add breakpoint" dialog.

PS: currently running on NetBeans IDE Dev (Build 201504280001)
Comment 9 Martin Entlicher 2015-05-13 15:21:03 UTC
I do not think I can do something about this. From the snapshots it's visible that the class index query and source for binary query is causing the slowness. I've made issue #252383 from your report.
Comment 10 NukemBy 2016-09-29 20:39:04 UTC
I'd like to reopen this issues - actually it was never fixed for me (in term of performance). Now I tried to find root cause be debugging NetBeans and here are my finding:

Performance issues starts in this line

    http://hg.netbeans.org/main/file/ef35591e2d0f/debugger.jpda.ui/src/org/netbeans/modules/debugger/jpda/ui/completion/ExceptionCompletionProvider.java#l146
    public class ExceptionCompletionProvider implements CompletionProvider {

        @Override
        public void run(CompilationController cc) throws Exception {
            LOG.log(Level.FINE, "  Running CompletionUserTask, compilation controller = {0}", cc);
            cc.toPhase(Phase.RESOLVED);
 -->        TypeElement thr = cc.getElements().getTypeElement("java.lang.Throwable"); //NOI18N
            LOG.log(Level.FINE, "  Filling results, throwable element = {0}", thr);
            if (thr != null) {

this causes full rescan of all source roots in a rather non-optimal way. Root cause of the problem is somewhere here

    http://hg.netbeans.org/main/file/ef35591e2d0f/java.source.base/src/org/netbeans/modules/java/source/parsing/FolderArchive.java#l144
    
    public Iterable<JavaFileObject> getFiles(String folderName, ClassPath.Entry entry, Set<JavaFileObject.Kind> kinds, JavaFileFilterImplementation filter) throws IOException {
        ...
        if (entry == null || entry.includes(folderName)) {
            File folder = new File (this.root, folderName.replace('/', File.separatorChar));      //NOI18N
            ...
 -->        final File[] content = folder.listFiles();

            if (content != null) {

    with my current set of opened projects this line is invoked ... ohhh ... 396867 times. This is really overwhelming and just can't go quicky (finished in my case in around 35-40 seconds).

So what's happening, as far as I see - there are 2 steps:

1. Netbeans collects all currently available sources roots (debugger shows something around 150+)
2. After that it feeds them into some 'javac' tooling API, which ...
3. ... tries to resolve available packages by calling back into NetBeans-provided class - to check if and where the package are available.

And with dummy implementation in ProxyFileManager this just enumerates through combination of (N-of-roots * N-of-packages) triggering ~400K of folder.listFiles();

Notes:

- my production project has around 350 modules - they are mostly small (less than 30 files), 
  but each of them adds it source root into above mentioned calculation - so in real life timing is much worse
  
- I've logged all requested path to file - and it appears that 90% of the listing points to folders like this
    my-workspace\cache\index\s507\java\15\classes\

    as far as I understand - this is cache of the embedded Lucene and, I'm not sure, if these paths should be fed into lookup.
    Anyway - since this is 'cache', whose state is managed in memory - there should be quicker way to query for list of files than going 
    directly into file system.
    
And finally 

1. I think that this whole thing will work much quicker just by scanning all available source roots into temp HashMap beforehand.

    My current my-workspace\cache\index\... contains roughly 20000K of files + folder, file manager scans it in less than 5 seconds.
    Opened 10 projected with their sources roots would add at most additional 2 seconds.

2. since NetBeans already holds somewhere indices of everything - there is no actual need in file access for that operation at all.

...

Providing call stack up until 'SourceFileManager.list' - just for reference.
                    
"Code Completion"
	at org.netbeans.modules.java.source.parsing.SourceFileManager.list(SourceFileManager.java:99)
	at org.netbeans.modules.java.source.parsing.ProxyFileManager.list(ProxyFileManager.java:153)
	at com.sun.tools.javac.code.ClassFinder.scanUserPaths(ClassFinder.java:604)
	at com.sun.tools.javac.code.ClassFinder.fillIn(ClassFinder.java:538)
	at com.sun.tools.javac.code.ClassFinder.complete(ClassFinder.java:305)
	at com.sun.tools.javac.code.ClassFinder.access$000(ClassFinder.java:75)
	at com.sun.tools.javac.code.ClassFinder$1.complete(ClassFinder.java:169)
	at com.sun.tools.javac.code.Symbol.complete(Symbol.java:591)
	at com.sun.tools.javac.code.ClassFinder.completeOwners(ClassFinder.java:317)
	at com.sun.tools.javac.code.ClassFinder.complete(ClassFinder.java:289)
	at com.sun.tools.javac.code.ClassFinder.access$000(ClassFinder.java:75)
	at com.sun.tools.javac.code.ClassFinder$1.complete(ClassFinder.java:169)
	at com.sun.tools.javac.code.Symbol.complete(Symbol.java:591)
	at com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:1106)
	at com.sun.tools.javac.code.ClassFinder.loadClass(ClassFinder.java:418)
	at com.sun.tools.javac.comp.Resolve.loadClass(Resolve.java:1917)
	at com.sun.tools.javac.comp.Resolve.findIdentInPackage(Resolve.java:2154)
	at com.sun.tools.javac.comp.Attr$IdentAttributer.visitMemberSelect(Attr.java:356)
	at com.sun.tools.javac.comp.Attr$IdentAttributer.visitMemberSelect(Attr.java:347)
	at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2083)
	at com.sun.source.util.SimpleTreeVisitor.visit(SimpleTreeVisitor.java:81)
	at com.sun.tools.javac.comp.Attr$IdentAttributer.visitMemberSelect(Attr.java:350)
	at com.sun.tools.javac.comp.Attr$IdentAttributer.visitMemberSelect(Attr.java:347)
	at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2083)
	at com.sun.tools.javac.comp.Attr.attribIdent(Attr.java:343)
	at com.sun.tools.javac.main.JavaCompiler.resolveIdent(JavaCompiler.java:697)
	at com.sun.tools.javac.model.JavacElements.binaryNameToClassSymbol(JavacElements.java:131)
	at com.sun.tools.javac.model.JavacElements.getTypeElementByBinaryName(JavacElements.java:117)
	at org.netbeans.api.java.source.ElementHandle.getTypeElementByBinaryName(ElementHandle.java:623)
	at org.netbeans.api.java.source.ElementHandle.resolveImpl(ElementHandle.java:161)
	at org.netbeans.api.java.source.ElementHandle.resolve(ElementHandle.java:137)
	at org.netbeans.modules.debugger.jpda.ui.completion.ExceptionCompletionProvider$CompletionUserTask.fillSubTypes(ExceptionCompletionProvider.java:249)
	at org.netbeans.modules.debugger.jpda.ui.completion.ExceptionCompletionProvider$CompletionUserTask.run(ExceptionCompletionProvider.java:172)
	at org.netbeans.modules.debugger.jpda.ui.completion.ExceptionCompletionProvider$CompletionUserTask.run(ExceptionCompletionProvider.java:126)
	at org.netbeans.modules.java.source.parsing.MimeTask.run(MimeTask.java:83)
	at org.netbeans.modules.parsing.impl.TaskProcessor.callUserTask(TaskProcessor.java:609)
	at org.netbeans.modules.parsing.api.ParserManager$MimeTaskAction.run(ParserManager.java:380)
	at org.netbeans.modules.parsing.api.ParserManager$MimeTaskAction.run(ParserManager.java:363)
	at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:204)
	at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:201)
	at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176)
	at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360)
	at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141)
	at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88)
	at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:201)
	at org.netbeans.modules.parsing.api.ParserManager.parse(ParserManager.java:334)
	at org.netbeans.api.java.source.JavaSource.runUserActionTaskImpl(JavaSource.java:415)
	at org.netbeans.api.java.source.JavaSource.runUserActionTask(JavaSource.java:407)
	at org.netbeans.modules.debugger.jpda.ui.completion.ExceptionCompletionProvider$1.query(ExceptionCompletionProvider.java:114)
	at org.netbeans.spi.editor.completion.support.AsyncCompletionTask.run(AsyncCompletionTask.java:223)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
Comment 11 NukemBy 2016-09-29 20:41:34 UTC
If someone would like to log these folder.listFiles() - here is the code for copy/paste


    public static boolean loggingOn = true;
    public static long lastWrite = System.currentTimeMillis();
    
    private static PrintStream log; 
    static {
        try {
            OutputStream os = new FileOutputStream("c:\\temp\\FolderArchive.getFiles-master.txt");
            log = new PrintStream(os);
        }
        catch( FileNotFoundException ex ) {
            Exceptions.printStackTrace(ex);
        }
    }
    
    @Override
    public Iterable<JavaFileObject> getFiles(String folderName, ClassPath.Entry entry, Set<JavaFileObject.Kind> kinds, JavaFileFilterImplementation filter) throws IOException {
        assert folderName != null;
        if (folderName.length()>0) {
            folderName+='/';                                                                            //NOI18N
        }
        if (entry == null || entry.includes(folderName)) {
            File folder = new File (this.root, folderName.replace('/', File.separatorChar));      //NOI18N
            //Issue: #126392 on Mac
            //The problem when File ("A/").listFiles()[0].equals(new File("a/").listFiles[0]) returns false
            //Normalization is slow - turn on this workaround only for users which require it.
            //The problem only happens in case when there is file with wrong case in import.
            if (normalize) {
                folder = FileUtil.normalizeFile(folder);
            }
long started = System.currentTimeMillis();    
            final File[] content = folder.listFiles();

if( loggingOn )  {
    long now = System.currentTimeMillis();    
    if( now - lastWrite > 1000 ) {
        log.println("");
        log.println("   -------------------------->");
        log.println("");
    }
        
    log.println(
            LocalDateTime.now().toLocalTime() + ": \""
            + Thread.currentThread().getName() + "\" "
            + folder + ": " + (content == null ? 0 : content.length) + " files, " + (now - started) + "ms");
    
    lastWrite = now;
}
Comment 12 Christian Lenz 2017-11-27 17:08:06 UTC
I will close it, because it will handled now at the JIRA board: https://issues.apache.org/jira/browse/NETBEANS-183. Please feel free to change the state of the ticket.
Comment 13 Christian Lenz 2017-11-27 17:08:33 UTC
Please handle everything there now.