This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 139776 [details] 3 log files in zip file A java web project with JPA breaks after minimal JPQL editing in a JPA entity. The IDE becomes unresponsive. I have multi-line JPQL. The only non-standard JPQL I use contains CAST like: + "and mytable.someIdId = CAST(:someParameterId INTEGER)\n" which is an Eclipse extension I must use. The log contains exceptions and many different types of warnings. The log directory contains many fresh .sig.dump files with stack traces. Product Version: NetBeans IDE Dev (Build 201309060001) Java: 1.7.0_25; Java HotSpot(TM) Client VM 23.25-b01 Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b17 System: Windows XP version 5.1 running on x86; Cp1252; en_NZ (nb) Menu|Tools|Options|Miscellaneous|Files|"Enable auto scanning of files" disabled. I cannot explain why in the Projects window java many files show the red error badges because I only changed a JPQL string in a @NamedQuery in the entity class. When I open a file in the editor that shows an error badge in the Projects window, then the IDE starts background scanning for 20 minutes. When it stops scanning, then a few files are fixed. If again I open an erroneous file then the IDE start again scanning for 20 minutes. Finally, after an hour, all errors are fixed. The error hints in the source code are like: method mountPackage in class WebApplication cannot be applied to given types; required: String,Class<P> found: String,Class<SignInPage> reason: inferred type does not conform to upper bound(s) inferred: SignInPage upper bound(s): Page where P is a type-variable: P extends Page declared in method <P>mountPackage(String,Class<P>) ---- (Alt-Enter shows hints) That is an error about the use of an ant library API which the IDE seems to get confused about. Project clean and build work fine. I have to say again that it would be a good idea if JPQL parsing in Java files could be turned off because even though I use JPQL heavily, the parsing causes more problems than what it is worth.
Please report one problem per issue http://wiki.netbeans.org/IssueReportingGuidelines
I saw can't add watch sometimes, but it seems unrelated. Also I see a lot of other warnings in the log from the beginning, regarding different jars/modules, it's unclear if jpql validation cause problems or contrary a problem somewhere else cause issue with jpql. regarding hints, for now it's first item in plans, I hope i will be finally done. going to evaluate remaining items. can you provide nps snapshot after jpql edition, do you mean if there is no "cast", there will be no problem, and all badges appears only after jpql edition? after file open? is it 20 minutes for any file even without jpql?
JPQL editing was the trigger of this. It took 20 minutes for the IDE to re-scan with the result that only part of the error badges disappeared. Then I had to open one of the erroneous files (not related to the JPQL) before the IDE started scanning once again, only to scan for another 20 minutes with another bach of files fixed. Then I opened another of the remaining files. Subsequently the IDE started scanning again to fix the reining files. Then 60 minutes were over. This is a different problem. The question is how can JPQL editing corrupt the IDE so much. It would have been much faster if I had just closed the IDE, deleted the cache folder and re-started. It probably takes 5 to 10 minutes for the IDE to re-scan everything. I am not saying that there is no problem without cast. I am just saying that cast is the only non-standard code in the JPQL. There is an exception in the log. We should be able to tell what it means, and how such an exception can get the IDE corrupted. I may be able to provide more info later but I cannot promise. I would suggest that you try to produce a scenario that produces JPL exception with multi-line JPQL as I wrote in my 1st comment.
one of log files contain too much errors, from javac, from refactoring etc, as jpql validation use java tree a lot, it may be a problem also. also, even before first npe from jpql, I see number of logs like WARNING [org.openide.filesystems.FileUtil]: FileUtil.normalizeFile(C:\_dt\app\gt1\jprj\G1\G1_W\src\java\ejb\session\Gee1SessionLocal.java) took 1,390 ms. Result is C:\_dt\app in my opinion, it may mean overloaded hdd, and may mean 10 similar calls anywhere may cause significant slowdown and if it happens before jpql change/validation triggering, main problem may be somewhere else. nps may help here, it's hard to decide so far what is required to fix here and why this may happens as there was no changes in jpql validation in this release, also tried to add "cast" and it doesn't matter, even validator is eclipselink validator, it seems to validate generic jpql without extensions and it's one more item to made hints configurable. Tried also multiline/splitted named query and no prblem, will play with this option a bit more, as it's risky parsing area with possible infinite loops etc cases. is it reproducible if you will open the same file again and try to edit jpql? Anyway will try to reduce number of npe in jpql as first step on persistence side, it should help but may be a bit as I have no a project to reproduce any slowness, http://hg.netbeans.org/web-main/rev/c7863e07382b
Perhaps the IDE needs better logging so you can exploit the log files more. This is a fresh dev build so all the necessary logging should be turned on. So far, I have been able to blow up the IDE after every minor release. A little re-factoring usually does the trick to get OutOfMemoryError or other major issues. The team has criticized me for my slow computer so I bought a new one. Only to discover that the culprit was NOT the computer but TWO memory leaks that are now fixed. You are saying that the harddrive is overloaded. Well, that is what NetBeans is doing, and then perhaps something needs to be done about it. No surprise with 2 million I/Os during such a session. I suspect that part of the problem is normalizeFile() which is known to be slow, and I further suspect that the use of JNA for Windows I/O is not helping, either. The problem is that one probably needs to do a little bit of coding to confuse the IDE. Unfortunately, there is no easy way to reproduce this and make test cases. For example, this session I did only with the intention to reproduce an OutOfMemoryError I got with the same sequence of events a week earlier. So the results are quite different each time, perhaps depending on the build. What I am trying to say is that these log files that you have now, and potentially a heap dump in case there is a leak, is all that I would normally be able to produce in any case. So the logging needs to improve or the analysis of it so that the work that I am doing is not wasted. There are so many files like *.sig_27.dump in the log directory that were created during this session. I still have a full copy of the log directory, so if you need more then please provide me with an email address and I send all of it to you. I just can't believe that one cannot at least pinpoint and fix one of the many problems from the abundance of information in all of these files.
Regarding npes, it may be easy to fix, but it really shouldn't(may not) be main issue here, memory dump may help more to find out if persistence of any other code do smth wrong. You may try to submit separate issue with same log for coupling errors, and may be jars problems etc, other teams may look if there is anything they can evaluate fix.
so far, played a bit with line breaks, with refactorings, with something "enum like", fix one more npe, initially thought to be ok to have these npes, all are catched logged with info level, but may affect something, somewhere internally in eclipsleink validation. but still no problem similar to memory leak, may be I need a bigger sample project, or evaluate memory dump if will be provided. the main issue for now is to evaluate if fix do somthing good and if it's a stopper, otherwise all minor improvements will go in next release.
Thanks. I watched memory consumption carefully. Memory consumption would always go back to initial value + - 10MBytes - could not see anything like a memory leak. A week ago I discovered that the IDE was using 80MBytes more than 7.3.1 but not with this build anymore.
I suppose my fixes are not in the build yet, so it may mean something special may happen with cache/other nb option, I'm not sure each build have the same cache for example, or just something goes corrupted by unknown initial reason.
Integrated into 'main-silver', will be available in build *201309100337* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/c7863e07382b User: Sergey B. Petrov <sj-nb@netbeans.org> Log: #235622 should reduce npe rate, even when scan is cancelled
unfortunately I don't know what else I can do on persistence side without memory dump/snapshot. set to P3 for now, feel fre to rise back if will have the issue again and will be able to provide snapshot, thanks.
regarding memory usage, it may be occational hit https://netbeans.org/bugzilla/show_bug.cgi?id=235649
I have run the same procedure with the latest build 201309110001 without this problem. There are some exceptions in the log, but with no effect on IDE stability: java.lang.NullPointerException at org.netbeans.modules.j2ee.persistence.spi.jpql.ManagedTypeProvider.getEmbeddable(ManagedTypeProvider.java:162) at org.netbeans.modules.j2ee.persistence.spi.jpql.ManagedTypeProvider.getEmbeddable(ManagedTypeProvider.java:156) at org.eclipse.persistence.jpa.jpql.tools.GenericSemanticValidatorHelper.getEmbeddable(GenericSemanticValidatorHelper.java:206) at org.eclipse.persistence.jpa.jpql.tools.GenericSemanticValidatorHelper.getEmbeddable(GenericSemanticValidatorHelper.java:1) at org.eclipse.persistence.jpa.jpql.tools.DefaultSemanticValidator.validateNullComparisonExpression(DefaultSemanticValidator.java:901) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2935) at org.eclipse.persistence.jpa.jpql.parser.NullComparisonExpression.accept(NullComparisonExpression.java:75) at org.eclipse.persistence.jpa.jpql.parser.CompoundExpression.acceptChildren(CompoundExpression.java:66) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:63) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateAndExpression(AbstractSemanticValidator.java:670) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2571) at org.eclipse.persistence.jpa.jpql.parser.AndExpression.accept(AndExpression.java:61) at org.eclipse.persistence.jpa.jpql.parser.CompoundExpression.acceptChildren(CompoundExpression.java:65) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:63) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateAndExpression(AbstractSemanticValidator.java:670) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2571) at org.eclipse.persistence.jpa.jpql.parser.AndExpression.accept(AndExpression.java:61) at org.eclipse.persistence.jpa.jpql.parser.AbstractSingleEncapsulatedExpression.acceptChildren(AbstractSingleEncapsulatedExpression.java:53) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:526) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:3103) at org.eclipse.persistence.jpa.jpql.parser.SubExpression.accept(SubExpression.java:49) at org.eclipse.persistence.jpa.jpql.parser.CompoundExpression.acceptChildren(CompoundExpression.java:66) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:442) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateOrExpression(AbstractSemanticValidator.java:1938) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2999) at org.eclipse.persistence.jpa.jpql.parser.OrExpression.accept(OrExpression.java:59) at org.eclipse.persistence.jpa.jpql.parser.CompoundExpression.acceptChildren(CompoundExpression.java:65) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:442) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateOrExpression(AbstractSemanticValidator.java:1938) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2999) at org.eclipse.persistence.jpa.jpql.parser.OrExpression.accept(OrExpression.java:59) at org.eclipse.persistence.jpa.jpql.parser.AbstractSingleEncapsulatedExpression.acceptChildren(AbstractSingleEncapsulatedExpression.java:53) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:526) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:3103) at org.eclipse.persistence.jpa.jpql.parser.SubExpression.accept(SubExpression.java:49) at org.eclipse.persistence.jpa.jpql.parser.CompoundExpression.acceptChildren(CompoundExpression.java:65) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:63) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateAndExpression(AbstractSemanticValidator.java:670) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2571) at org.eclipse.persistence.jpa.jpql.parser.AndExpression.accept(AndExpression.java:61) at org.eclipse.persistence.jpa.jpql.parser.CollectionExpression.acceptChildren(CollectionExpression.java:119) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:112) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2628) at org.eclipse.persistence.jpa.jpql.parser.CollectionExpression.accept(CollectionExpression.java:99) at org.eclipse.persistence.jpa.jpql.parser.AbstractConditionalClause.acceptChildren(AbstractConditionalClause.java:72) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:624) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateWhereClause(AbstractSemanticValidator.java:2511) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:3215) at org.eclipse.persistence.jpa.jpql.parser.WhereClause.accept(WhereClause.java:59) at org.eclipse.persistence.jpa.jpql.parser.AbstractSelectStatement.acceptChildren(AbstractSelectStatement.java:94) at org.eclipse.persistence.jpa.jpql.parser.SelectStatement.acceptChildren(SelectStatement.java:110) at org.eclipse.persistence.jpa.jpql.AbstractValidator.visit(AbstractValidator.java:743) at org.eclipse.persistence.jpa.jpql.parser.AnonymousExpressionVisitor.visit(AnonymousExpressionVisitor.java:470) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.validateSelectStatement(AbstractSemanticValidator.java:2009) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:3038) at org.eclipse.persistence.jpa.jpql.parser.SelectStatement.accept(SelectStatement.java:102) at org.eclipse.persistence.jpa.jpql.AbstractSemanticValidator.visit(AbstractSemanticValidator.java:2837) at org.eclipse.persistence.jpa.jpql.parser.JPQLExpression.accept(JPQLExpression.java:136) at org.eclipse.persistence.jpa.jpql.tools.AbstractJPQLQueryHelper.validateSemantic(AbstractJPQLQueryHelper.java:518) at org.eclipse.persistence.jpa.jpql.tools.AbstractJPQLQueryHelper.validate(AbstractJPQLQueryHelper.java:462) at org.eclipse.persistence.jpa.jpql.tools.AbstractJPQLQueryHelper.validate(AbstractJPQLQueryHelper.java:449) [catch] at org.netbeans.modules.j2ee.jpa.verification.rules.entity.JPQLValidation.apply(JPQLValidation.java:112) at org.netbeans.modules.j2ee.jpa.verification.rules.entity.JPQLValidation.apply(JPQLValidation.java:81) at org.netbeans.modules.j2ee.jpa.verification.common.Rule.execute(Rule.java:68) at org.netbeans.modules.j2ee.jpa.verification.common.RulesEngine.visitTypeAsClass(RulesEngine.java:70) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder.processClass(JPAProblemFinder.java:174) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder.access$000(JPAProblemFinder.java:95) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder$1.run(JPAProblemFinder.java:146) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder$1.run(JPAProblemFinder.java:134) at org.netbeans.modules.j2ee.persistenceapi.metadata.orm.annotation.EntityMappingsMetadataModelImpl$2.call(EntityMappingsMetadataModelImpl.java:89) at org.netbeans.modules.j2ee.metadata.model.api.support.annotation.AnnotationModelHelper.runCallable(AnnotationModelHelper.java:256) at org.netbeans.modules.j2ee.metadata.model.api.support.annotation.AnnotationModelHelper.access$100(AnnotationModelHelper.java:79) at org.netbeans.modules.j2ee.metadata.model.api.support.annotation.AnnotationModelHelper$3.run(AnnotationModelHelper.java:218) at org.netbeans.modules.j2ee.metadata.model.api.support.annotation.AnnotationModelHelper$3.run(AnnotationModelHelper.java:215) at org.netbeans.modules.java.source.parsing.MimeTask.run(MimeTask.java:83) at org.netbeans.modules.parsing.impl.TaskProcessor.callUserTask(TaskProcessor.java:593) at org.netbeans.modules.parsing.api.ParserManager$MimeTaskAction.run(ParserManager.java:382) at org.netbeans.modules.parsing.api.ParserManager$MimeTaskAction.run(ParserManager.java:365) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:206) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:203) at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176) at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360) at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:74) at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:203) at org.netbeans.modules.parsing.impl.RunWhenScanFinishedSupport.runWhenScanFinished(RunWhenScanFinishedSupport.java:136) at org.netbeans.modules.parsing.api.ParserManager.parseWhenScanFinished(ParserManager.java:359) at org.netbeans.api.java.source.JavaSource.runWhenScanFinished(JavaSource.java:583) at org.netbeans.modules.j2ee.metadata.model.api.support.annotation.AnnotationModelHelper.runJavaSourceTaskWhenScanFinished(AnnotationModelHelper.java:215) at org.netbeans.modules.j2ee.persistenceapi.metadata.orm.annotation.EntityMappingsMetadataModelImpl.runReadActionWhenReady(EntityMappingsMetadataModelImpl.java:87) at org.netbeans.modules.j2ee.metadata.model.api.MetadataModel.runReadActionWhenReady(MetadataModel.java:185) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder.run(JPAProblemFinder.java:134) at org.netbeans.modules.j2ee.jpa.verification.JPAProblemFinder$ProblemFinderCompInfo.run(JPAProblemFinder.java:389) at org.netbeans.modules.java.source.JavaSourceAccessor$CancelableTaskWrapper.run(JavaSourceAccessor.java:298) at org.netbeans.modules.parsing.impl.TaskProcessor.callParserResultTask(TaskProcessor.java:568) at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:744) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1432) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2042)
ok, I will look into this npe to see if it have sense to remove/handle it or keep as npe and will lose the issue then
as it stated to work, I'm closing as wfm instead of fixed, but ar the same time provided npe behaviour should change http://hg.netbeans.org/web-main/rev/6e8d68ad5784 nb code will not throw npe here.