Bug 55932

Summary: Create a Async BackendListener to allow easy plug of new listener (Graphite, JDBC, Console,...)
Product: JMeter - Now in Github Reporter: Philippe Mouawad <p.mouawad>
Component: MainAssignee: JMeter issues mailing list <issues>
Status: RESOLVED FIXED    
Severity: enhancement CC: maklaut, p.mouawad
Priority: P2    
Version: 2.10   
Target Milestone: ---   
Hardware: All   
OS: All   
Bug Depends on: 57321    
Bug Blocks: 57246    
Attachments: Graphite Dashboard for cumulated data
Implementation
Elements list with my test

Description Philippe Mouawad 2013-12-25 23:14:14 UTC
Created attachment 31154 [details]
Graphite Dashboard for cumulated data

Graphite is an interesting way to have graphs rendered live and on demand:
http://graphite.readthedocs.org/en/latest/overview.html

With this Bugzilla I propose to create the following:
A new listener that will send the following metrics

1) Per second:
- min response time
- max response time
- percentile 90 for response time rolling over last 100 values
- percentile 95 for response time rolling over last 100 values
- successful req
- failed req
- total req

2) Listener will also send the following metrics which are not based on a variation per second:
- ActiveThreads
- Stopped Threads
- Started Threads

I suggest to use commons-math for percentile computations (which by the way could be useful in other parts of JMeter).

I also suggest using Pickle protocol as it seems to be the most efficient as per Graphite docs:
http://graphite.readthedocs.org/en/latest/feeding-carbon.html

Reviewing existing libraries, I didn't find a reference for Graphite client in Java but my final implementation is partly based on this:
- https://github.com/BrightcoveOS/metrics-graphite-pickle/blob/master/src/main/java/com/brightcove/metrics/reporting/GraphitePickleReporter.java 

Only using the pickle protocol part and adding:
- Socket Pooling based on commons-pool2 to avoid overcreating sockets


Additional dependencies:
- Apache commons-pool2
- Apache commons-math3


See attached example of a generated Dashboard
Comment 1 Philippe Mouawad 2013-12-25 23:18:49 UTC
Created attachment 31155 [details]
Implementation
Comment 2 Milamber 2013-12-31 05:36:41 UTC

I've tested the listener with a Graphite server on a virtual machine on my machine.
Seems works fine, but I don't find the response time graph in elements list (see screenshot).

I've following this tutorial to install the graphite server (on debian 7)
https://www.digitalocean.com/community/articles/installing-and-configuring-graphite-and-statsd-on-an-ubuntu-12-04-vps

Perhaps I missing something?


Note, the ant tests task don't work. I put the trace in another comment.
Comment 3 Milamber 2013-12-31 05:37:48 UTC
(missing doc reference, saveservice, ...)


     [java] Created: 89 tests including 8 suites
     [java] Starting test run, test count = 2530
     [java] .......................................F..
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........F................................
     [java] .........................................
     [java] ..F.......................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] ........................E.E..E.E.E.E.E.E.E.E.E......
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .........................................
     [java] .............................
     [java] Time: 77.02
     [java] There were 11 errors:
     [java] 1) org.apache.jmeter.reporters.graphite.GraphiteListener - en - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 2) org.apache.jmeter.reporters.graphite.GraphiteListener - en - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 3) org.apache.jmeter.reporters.graphite.GraphiteListener - de - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 4) org.apache.jmeter.reporters.graphite.GraphiteListener - no - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 5) org.apache.jmeter.reporters.graphite.GraphiteListener - pl - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 6) org.apache.jmeter.reporters.graphite.GraphiteListener - pt - BR(org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 7) org.apache.jmeter.reporters.graphite.GraphiteListener - es - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 8) org.apache.jmeter.reporters.graphite.GraphiteListener - tr - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 9) org.apache.jmeter.reporters.graphite.GraphiteListener - ja - (org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 10) org.apache.jmeter.reporters.graphite.GraphiteListener - zh - CN(org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 11) org.apache.jmeter.reporters.graphite.GraphiteListener - zh - TW(org.apache.jmeter.testbeans.gui.PackageTest)java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key graphiteHost.displayName
     [java] 	at java.util.ResourceBundle.getObject(ResourceBundle.java:374)
     [java] 	at java.util.ResourceBundle.getString(ResourceBundle.java:334)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.checkAllNecessaryKeysPresent(PackageTest.java:157)
     [java] 	at org.apache.jmeter.testbeans.gui.PackageTest.runTest(PackageTest.java:112)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] There were 3 failures:
     [java] 1) testMaven(org.apache.jmeter.JMeterVersionTest)junit.framework.AssertionFailedError: Should have no names left: [commons-pool2, commons-math3]. Check ApacheJMeter_parent.pom
     [java] 	at org.apache.jmeter.JMeterVersionTest.testMaven(JMeterVersionTest.java:167)
     [java] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     [java] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     [java] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 2) runTestElement(org.apache.jmeter.junit.JMeterTest)junit.framework.AssertionFailedError: SaveService nameMap (saveservice.properties) should contain org.apache.jmeter.reporters.graphite.GraphiteListener
     [java] 	at org.apache.jmeter.junit.JMeterTest.checkElementAlias(JMeterTest.java:553)
     [java] 	at org.apache.jmeter.junit.JMeterTest.runTestElement(JMeterTest.java:540)
     [java] 	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
     [java] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] 3) runGUITitle(org.apache.jmeter.junit.JMeterTest)junit.framework.AssertionFailedError: component_reference.xml needs 'Graphite_Listener' anchor for org.apache.jmeter.testbeans.gui.TestBeanGUI
     [java] 	at org.apache.jmeter.junit.JMeterTest.runGUITitle(JMeterTest.java:368)
     [java] 	at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
     [java] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)
     [java] FAILURES!!!
     [java] Tests run: 2530,  Failures: 3,  Errors: 11

BUILD FAILED
/home/milamber/W-workspaces/Workspaces-JMeter/Jmeter/build.xml:2677: Java returned: 1

Total time: 3 minutes 12 seconds
Comment 4 Milamber 2013-12-31 05:38:43 UTC
Created attachment 31166 [details]
Elements list with my test
Comment 5 Philippe Mouawad 2013-12-31 13:40:17 UTC
Hello Milamber,
Thanks for tests, there is no response time, just percentile95 and percentile90.

Regarding tests failure, I was planning to fix this if we agree to commit patch.

Is it ok for commiting it ?
Comment 6 Sebb 2014-01-03 12:57:20 UTC
I have been having a look at the implementation.

I don't really see that it needs Commons Math; we aleady have StatCalculator which handles percentiles and more.

Likewise, does it really need Commons Pool?
It seems wrong to have to have 2 separate pools of SocketOutputStream instances.
How many of these would there be?

Also, DescriptiveStatistics is not thread-safe (nor is StatCalculator).

If we do implement something like this, I think the data processing needs either to be carefully synchronised, or the raw data should be sent to a separate singleton background thread.

Follow-ups to the dev list please.
Comment 8 Philippe Mouawad 2014-11-22 15:41:54 UTC
Date: Sat Nov 22 15:36:37 2014
New Revision: 1641081

URL: http://svn.apache.org/r1641081
Log:
Bug 55932 - Create a Async BackendListener to allow easy plug of new listener (Graphite, JDBC, Console,...)
Bugzilla Id: 55932

Added:
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/AbstractBackendListenerClient.java   (with props)
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java   (with props)
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListenerClient.java   (with props)
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListenerContext.java   (with props)
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListenerGui.java   (with props)
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/SamplerMetric.java   (with props)
Modified:
    jmeter/trunk/bin/saveservice.properties
    jmeter/trunk/build.properties
    jmeter/trunk/build.xml
    jmeter/trunk/eclipse.classpath
    jmeter/trunk/res/maven/ApacheJMeter_parent.pom
    jmeter/trunk/src/core/org/apache/jmeter/resources/messages.properties
    jmeter/trunk/src/core/org/apache/jmeter/resources/messages_fr.properties
    jmeter/trunk/src/core/org/apache/jmeter/samplers/SampleResult.java
    jmeter/trunk/src/core/org/apache/jmeter/save/SaveService.java
    jmeter/trunk/xdocs/changes.xml
    jmeter/trunk/xdocs/usermanual/component_reference.xml
Comment 9 Philippe Mouawad 2014-11-22 15:59:21 UTC
Date: Sat Nov 22 15:55:27 2014
New Revision: 1641086

URL: http://svn.apache.org/r1641086
Log:
Bug 55932 - Create a Async BackendListener to allow easy plug of new listener (Graphite, JDBC, Console,...)
Fix mistake in imports
Bugzilla Id: 55932

Modified:
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java
Comment 10 Philippe Mouawad 2014-11-22 20:03:25 UTC
Date: Sat Nov 22 20:01:50 2014
New Revision: 1641118

URL: http://svn.apache.org/r1641118
Log:
Bug 55932 - Create a Async BackendListener to allow easy plug of new listener (Graphite, JDBC, Console,...)
Fix javadocs , naming
But more importantly fix multi-threading issues on instance variables as there is only 1 BackendListener shared by all threads
Bugzilla Id: 55932

Modified:
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java
Comment 11 Milamber 2014-11-25 07:38:45 UTC

When I run the ant distribution task, I have this error:

     [java] There was 1 error:
     [java] 1) GUIComponents2(org.apache.jmeter.junit.JMeterTest)java.lang.NumberFormatException: For input string: ""
     [java] 	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
     [java] 	at java.lang.Integer.parseInt(Integer.java:470)
     [java] 	at java.lang.Integer.parseInt(Integer.java:499)
     [java] 	at org.apache.jmeter.visualizers.backend.BackendListenerGui.modifyTestElement(BackendListenerGui.java:268)
     [java] 	at org.apache.jmeter.visualizers.backend.BackendListenerGui.createTestElement(BackendListenerGui.java:257)
     [java] 	at org.apache.jmeter.junit.JMeterTest.GUIComponents2(JMeterTest.java:445)
     [java] 	at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
     [java] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [java] 	at org.apache.jorphan.test.AllTests.main(AllTests.java:236)



If the queueSize is empty, the NFE occurs.
Several solutions (in my mind):
1/ add a try catch block (and ignore the NFE or set a default value)
2/ use a if/then/else block (set a default value if null or emtpy)

and change the docs (not require, indicate the default value)
Comment 12 Philippe Mouawad 2014-12-27 15:07:09 UTC
Date: Sun Nov 30 19:35:07 2014
New Revision: 1642600

URL: http://svn.apache.org/r1642600
Log:
Store queue size as a String (allows use of variables etc)

Modified:
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java
    jmeter/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListenerGui.java
Comment 13 The ASF infrastructure team 2022-09-24 20:37:56 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/3291