Hi Team, I am using JSR223 Sampler with Groovy 3.03 for sending messages to Kafka topic. (Attaching Script that i have used, I am using a basic setup of Kafka single node) I am using kafka-clients-2.5.0.jar placed under lib\ext for sending messages to Kafka. Issue: When running simple test with 5 users i have observed memory being accumulated for JMeter process and eventually JMeter stops responding. Observations: I have monitored JMeter process using VisualVM, memory is piling up over time, attaching snapshot and Heap dump for the same. I have done some investigation on Heap dump and i see that memory leak is happening due to UI update. When i run the same test in NON-GUI mode i don't see any issue issue.
Thanks for the report. To work on this, we need more information. Can you provide * the script that triggers the leak (or better a minimal example that triggers it) * more details on the heap debugging you already did (for example, where exactly the references pile up) * jmeter.log (there might be hints about errors in there) * JDK and OS version (problem could be specific to any of those) * any setting, that differs from the default setup, like a JVM heap setting
Created attachment 37347 [details] Jmeter.log and kafka Script Jmeter.log and kafka Script
* the script that triggers the leak --> Included in zip file Script does the following below for multiple iterations and each users: 1. Establish connection with Kafka 2. Send a record 3. Close connection * jmeter.log --> Included in zip file * JDK and OS version --> Oracle JDK: java version "1.8.0_251" Java(TM) SE Runtime Environment (build 1.8.0_251-b08) Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode) --> OS: Windows 10 Build 19041.329, Last update KB4557959 * any setting, that differs from the default setup, like a JVM heap setting --> I am using default settings, no changes made apart from kafka-clients-2.5.0.jar * more details on the heap debugging --> Top 5 Memory Heap Consumers Class Name | Shallow Heap | Retained Heap | Percentage ---------------------------------------------------------------------------------------------------- org.apache.jorphan.gui.ui.TextComponentUI$1 @ 0xc1d41150| 40 | 809,020,752 | 88.88% org.apache.jmeter.DynamicClassLoader @ 0xc01e8138 | 80 | 4,670,392 | 0.51% org.apache.jmeter.gui.LoggerPanel @ 0xc1d5cd90 | 360 | 732,784 | 0.08% org.apache.jmeter.gui.util.JSyntaxTextArea @ 0xc0c618d8 | 712 | 100,272 | 0.01% org.apache.jmeter.gui.action.ActionRouter @ 0xc1c049c0 | 32 | 83,776 | 0.01% ---------------------------------------------------------------------------------------------------- --> Below is the breakdown for TextComponentUI Class Name | Shallow Heap | Retained Heap | Retained Heap ------------------------------------------------------------------------------------------------------------------------------------------ org.apache.jorphan.gui.ui.TextComponentUI$1 @ 0xc1d41150 | 40 | 809,020,752 | 809,020,752 '- [1] java.lang.Object[14] @ 0xc1c8a220 | 72 | 72 | 72 '- listenerList javax.swing.event.EventListenerList @ 0xc1d41818 | 16 | 88 | 88 '- listenerList org.fife.ui.rsyntaxtextarea.RSyntaxDocument @ 0xc1d414d0 | 112 | 240,176 | 240,176 |- <Java Local> java.awt.EventDispatchThread @ 0xc03af950 AWT-EventQueue-0 Thread| 136 | 1,391,400 | 1,391,400 |- model org.apache.jmeter.gui.util.JSyntaxTextArea @ 0xc1d32260 | 712 | 9,648 | 9,648 ------------------------------------------------------------------------------------------------------------------------------------------
Hi Felix, On doing some investigation I found that "TextComponentUI" refers to area where we see the logs in UI, I have disabled logging in GUI from log4j2.xml <Loggers> <Root level="info"> <AppenderRef ref="jmeter-log" /> <!--<AppenderRef ref="gui-log-event" />--> </Root> </Loggers> After which I don't any memory leak even on long run. I think it might be due to logging that is being done from external JAR that I am using for Kafka, there by accumulating heap due to update in UI. Is Logback enabled in Async mode for Jmeter!! Regards, Karthik
Updating Status, apologies forgot to read instructions. Updated with requested info and further observations in earlier comments, please look into them.
Hi Team, Any update on the issue. Regards, Karthik S
Hello, The Logger panel holds up to 1000 events by default which you can adjust with: jmeter.loggerpanel.maxlength=1000 You can disable this receiving events through: -jmeter.loggerpanel.enable_when_closed=false You should not run a load test in GUI mode, so if you do and have a high throughput + low log level, you might end up with such situation. Switching to Async logging won't help here. PS: You should close Kafka producer in a finally
1) Your script has a problem as you use below code inside Groovy, so you'll end up caching multiple versions of a different compiled script: ${__Random(1,100000000,)} Can you try my new attached script 2) Are you sure that non responsive JMeter is not related to those warnings ? 2020-07-03 16:23:43,318 WARN o.a.k.c.NetworkClient: [Producer clientId=producer-6501] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. 2020-07-03 16:23:43,318 WARN o.a.k.c.NetworkClient: [Producer clientId=producer-6501] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
Created attachment 37373 [details] Fixed script
When JMeter is frozen I checked Kafka using batch utility, i am able to send messages without issue. The warning below is from when Kafka is not started and i tried to run test, resulted in below error. 2020-07-03 16:23:43,318 WARN o.a.k.c.NetworkClient: [Producer clientId=producer-6501] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. With the fixed script i don't see that Complete freeze of JMeter, but it is momentary. This would work for the initial scripting. Thanks for letting me know the issue with the script.
So what happens now with fixed script? Is there a memory leak ?
Period has prolonged but still I see a memory leak and it leads to the same class org.apache.jorphan.gui.ui.TextComponentUI. Memory is being accumulated and I see the JMeter freezes because of high GC activity.
I did try to add the two arguments as suggested, I missed one of them in the earlier test. jmeter.loggerpanel.maxlength=1000 (Added new) jmeter.loggerpanel.enable_when_closed=false With max length added I don't see a memory leak, I ran the test in GUI for an hour and it seems really fine. Thanks for your help.
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/5352