This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 140940 [details] Thread stack + Class histogram + System properties reports NetBeans using project directory which is also a git working copy(with standard .git repository subfolder). Within 15 minutes NetBeans fills-up all of it's available heap,slowdowns and after a while crashes. WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread null WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread null WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread org.n etbeans.modules.sampler.InternalSampler@31798748 WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread null WARNING [org.netbeans.core.TimableEventQueue]: too much time in AWT thread null java.lang.OutOfMemoryError: Java heap space Dumping heap to C:\Users\xxxxxUser\AppData\Roaming\NetBeans\dev\var\log\heapdump.h prof ... Heap dump file created [1639747359 bytes in 13.846 secs] I can't attach a heap dump unfortunately because it is too large(About 1.4GB). I've opened it with MAT and those are it's conclusions: Problem Suspect 1: The thread org.openide.util.RequestProcessor$Processor @ 0x77eb4ebb8 GitRefresh keeps local variables with total size 1,457,449,584 (93.97%) bytes. The memory is accumulated in one instance of "java.util.LinkedHashMap" loaded by "<system class loader>". The stacktrace of this Thread is available. Thread Stack / Class histogram / System Properties: Attached in ZIP archive. If you need some other info which can be exported from MAT just ask
Please attach also the full messages.log http://wiki.netbeans.org/FaqLogMessagesFile From the stacktrace it looks as some sort of infinite recursion. Does it happen only for this particular project? Do you have any cyclic symlinks in your project? Can you run with less memory (so the heap dump is not too big), zip the heap dump and upload it to https://netbeans.org/projects/versioncontrol/downloads
I'll need some time to get back to this as i've didn't stored the messages.log back then and currently git module is just disabled so i can somehow work. I've already tried to run it with less memory but it still reaches 1.4GB and does it quite fast. I don't have symlinks(especially circular) in the project folder. I'm not sure if it's possible to create small heap dump in my case... Need to say i also have my custom patch to masterfs module applied(See bug 236930) - it can be related but i'm not sure.
Additional answers: "Does it happen only for this particular project?": I don't have other project here(Only one). "Do you have any cyclic symlinks in your project?" Project tree doesn't have any symlinks. I've managed to get relatively small heap dump(250Mb pure - 25Mb 7z-ipped) and preserve messages.log. I can't share the messages.log however - unfortunately it has too much sensitive information about the project. It's anyway of no much use for this issue: During background scanning GitInterceptor is waiting and log file/console is filled with scanning details. Once scanning finishes - console and log stop receiving updates and Git starts to leak in background(After 50-100Mb of leak i've took heap snapshot). Only then IDE is closed console and log receive few lines of stuff IDE outputs while shutting down. Looks like the link you gave(versioncontrol) doesn't allow me to do uploads.
Forgot to mention - While GitInterceptor is waiting for background/indexes scanning no leaks occur - They start after background scanning finishes.
added logging to the status command: core-main #02b3ff8da357 please, victor, download and install a daily build [1] in a few days (you'll be notified in this issue) and run the IDE with -J-Dorg.netbeans.libs.git.jgit.commands.StatusCommand.lel=FINE and attach the log when you see the memory usage growing. The log will contain some info about scanned files. That could reveal at least if the command scans files it shouldn't be.
... [1] - http://bits.netbeans.org/download/trunk/nightly/latest/
Integrated into 'main-silver', will be available in build *201310310001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/02b3ff8da357 User: Ondrej Vrabec <ovrabec@netbeans.org> Log: #236936 - Memory leak in GitRefresh logging added
I've updated the sources several times(Last free included your patch since it landed main-silver). Today i've enabled git plugin and got the behaviour of high leaking. I didn't got the logging working right away with your StatusCommand.lel=FINE so i changed that into StatusCommand.level=FINE instead(Like it is in other modules with logging). At this stage i've got a log for the leak in a whole messages.log: FINE [org.netbeans.libs.git.jgit.commands.StatusCommand]: Inspecting file tests/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/1o/31-2949-attachment(
I've updated the sources several times(Last free included your patch since it landed main-silver). Today i've enabled git plugin and got the behaviour of high leaking. I didn't got the logging working right away with your StatusCommand.lel=FINE so i changed that into StatusCommand.level=FINE instead(Like it is in other modules with logging). At this stage i've got a log for the leak in a whole messages.log: FINE [org.netbeans.libs.git.jgit.commands.StatusCommand]: Inspecting file tests/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/data/1o/31-2949-attachment(UTF8CHARSxls ---- W:\tests\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\data\1o\31-2949-attachment(UTF8CHARSxls The file names are in UTF8 encoding(Non latin). "tests" folder has a data subfolder which has 0,0o,1,1o subfolders and files without any links or recursions or subfolders with same name. However in the log the "data" is present like million times in a sequence like code appends same directory name to the concatenated path in some sort of loop which i assume is happening in the just above your new logger line: lastPath = path; File file = new File(workTreePath + File.separator + path); Possibly workTreePath at some point already includes part of path causing strange concatenations. Note that i have this folder for quite a lot of time and didn't touch it for more than a year while issue started not quite long time before. There are no other entries appearing in the log.
Looking at the git ecosystem tree history i see commits which are at least 2 month old and not relevant and commits which are at most two weeks old which start from "Upgrade Jgit to 3.x" which i remember i've pulled after i detected the leak. Between those 2 groups there is only one commit which landed 5 weeks ago - 25.09(And quite matches the period i've made the report - 09.10). Possibly the problem is there - That was attempted fix for bug 235338
(In reply to victork from comment #9) > I didn't got the logging working right away with your StatusCommand.lel=FINE > so i changed that into StatusCommand.level=FINE instead(Like it is in other > modules with logging). Sure, "level" is correct, my mistake. > "tests" folder has a data subfolder which has 0,0o,1,1o subfolders and files > without any links or recursions or subfolders with same name. > > However in the log the "data" is present like million times in a sequence > like code appends same directory name to the concatenated path in some sort > of loop which i assume is happening in the just above your new logger line: > lastPath = path; > File file = new File(workTreePath + File.separator + path); There must be some kind of a symlink in either your worktree or directly in the repository. Otherwise i have no idea how git could recursively traverse tests/data/data... indefinitely. If there was an error in the code i am pretty sure someone would have noticed for the past several years. Please pack test/data folder and send it to my e-mail. You can delete all files from the folder, i am just interested in the structure. (In reply to victork from comment #10) > Possibly the problem is there - That was attempted fix for bug 235338 nonsense, the fix comes for a completely different module and is absolutely irrelevant. Maybe someone added a symlink on linux and you simply cannot see it on windows (btw W: looks like a network drive, is it a mapped non-ntfs partition?), have you tried to clone the project on linux via "git clone" in commandline? Maybe the project contains a symlink just in HEAD and index but not in worktree.
Created attachment 141817 [details] Folder structure
"There must be some kind of a symlink in either your worktree or directly in the repository. Otherwise i have no idea how git could recursively traverse tests/data/data... indefinitely. If there was an error in the code i am pretty sure someone would have noticed for the past several years. Please pack test/data folder and send it to my e-mail. You can delete all files from the folder, i am just interested in the structure." Added a folder structure package as attachment(Made in Linux). "Maybe someone added a symlink on linux and you simply cannot see it on windows (btw W: looks like a network drive, is it a mapped non-ntfs partition?), have you tried to clone the project on linux via "git clone" in commandline? Maybe the project contains a symlink just in HEAD and index but not in worktree." Correct. W: is a network drive mapped to a folder on ext3fs via SMB. However all listings and checks that i've performed to check the contents are done via SSH on a linux box as i know quite well FS/OS differences features and internals(ls -la /var/www/data). "someone added a symlink" Impossible. This VM is used only by me. "Maybe the project contains a symlink just in HEAD and index but not in worktree." Tests folder is unversioned. "to clone the project on linux via "git clone" Tested this option specifically by your request - Everything is fine and no tests folder is present(Like i said above it's not versioned). "nonsense, the fix comes for a completely different module and is absolutely irrelevant." Who knows - software architecture is a complex thingy, not? Modifying A can break seemingly unrelated B or even everything :). I don't know - this is the only patch landed in the respective period.
btw, why are you using this setup (git controlled tree mounted remotely in another machine)? I believe the correct way to use git is to clone the project in Windows and commit/push changes back. Just like if you were two different persons working on two different machines. Honestly you're kind of abusing git this way. Could you please clone the project in Windows (do not mount the remote project but clone it normally into C:\ somewhere), create the test folder and its subfolders and let me know if it helps or still parses git statuses indefinitely? It may be related to smb share.
"btw, why are you using this setup (git controlled tree mounted remotely in another machine)? I believe the correct way to use git is to clone the project in Windows and commit/push changes back. Just like if you were two different persons working on two different machines. Honestly you're kind of abusing git this way." Believe me i know what - however lot's of things and setup layout are enforced by management ;) If it was depend on me i would use Linux/BSD directly and not via slow SMB protocol ;) However this way has some pluses - Tortoise Git has direct access to this tree and thus to repository(Without creating additional layer). The workflow is "Done working on updates/fixes -> Commit". I've worked the way you suggest then i worked from home an year ago. Having "Done working -> Commit -> Push changes to upstream" made me a lot of headache especially then i needed to "Hard Reset" the branch back in time while also making the commit operations chain longer.
i have an idea what might help you. If all files under tests/data are new (not committed) and you do not intend to ever commit them you might ignore the folder tests/data (which could result in skipping the ignored folder and instruct our client not to recursively traverse the subtree). So try adding "tests/data/" to .gitignore file and restart the IDE, maybe it helps.
I've finally found a cause of this strange behavior. "data" folder had some file with unreadable characters in it's name(Non normal UTF-8 but a corrupted one. Samba displayed it with an completely empty name and this was probably the reason for the recursion). I've managed to remove the file on Linux via "rm ./*.msg" with questions and answering "Y" on this problematic file(Removing it directly by name was not possible because it's name was corrupted). Now Git doesn't leak anymore and data folder is no more recursed. Probably the leaking caused by an unaccesible object with an empty name(Which may also incorrectly interpreted as "link"). I think this issue can be closed as it's quite rare and can be caused by filesystem corruption(Or bad encoding used in a file name) + samba(And/or possibly other remote access protocols).