Bug 61725

Summary: Response time table in dashboard report don't match with aggregate report
Product: JMeter - Now in Github Reporter: Aj <ajay.singhes>
Component: MainAssignee: JMeter issues mailing list <issues>
Severity: normal CC: p.mouawad
Priority: P2    
Version: 4.0   
Target Milestone: ---   
Hardware: PC   
OS: All   
Attachments: Image file
sample csv file
jmeter log in debug mode
aggregate report screenshot
html report screenshot
calc document, that shows the differences of excl. and incl. percentile

Description Aj 2017-11-06 04:51:48 UTC
Created attachment 35492 [details]
Image file

Statistics table of dashboard generator doesn't match with response time table by aggregate report plugin by Jmeter GUI.
Comment 1 Philippe Mouawad 2017-11-08 21:12:58 UTC
In order to work with this , can you provide your CSV file and user.properties so that we can see from where it comes ?
Comment 2 Philippe Mouawad 2018-01-14 17:05:15 UTC
Probably this:

- https://dzone.com/articles/how-to-achieve-better-accuracy-in-latency-percenti
Comment 3 Vvxxsz 2018-03-22 13:48:37 UTC

Same behavior have been observed on JMeter 3.1-4.0 Java 8
Adjusting jmeter.reportgenerator.statistic_window property to exact samples amount, -1 or 20000000 didn't solve the problem.

Report has been generated using jmeter.bat -g /path/to/csv -o /path/to/html/ cmd

In terms to the reproduce the issue here is:
- reportgenerator.properties
- jmeter.log with debug logging
- sample_html_report.csv
- Aggregate Report and HTML report screenshots
Comment 4 Vvxxsz 2018-03-22 13:50:12 UTC
Created attachment 35804 [details]
sample csv file
Comment 5 Vvxxsz 2018-03-22 13:51:01 UTC
Created attachment 35805 [details]
jmeter log in debug mode
Comment 6 Vvxxsz 2018-03-22 13:51:29 UTC
Created attachment 35806 [details]
aggregate report screenshot
Comment 7 Vvxxsz 2018-03-22 13:52:14 UTC
Created attachment 35807 [details]
html report screenshot
Comment 8 Vvxxsz 2018-03-22 13:54:13 UTC
Created attachment 35808 [details]
Comment 9 Felix Schumacher 2018-03-30 09:46:55 UTC
I have had time to look at the reported numbers and I think that we have two problems here.

First is a that we have different algorithms for calculating the percentiles in those two parts. The visualizer uses a discrete algorithm that uses the lowest number, that is recorded and inside of the 90th percentile. The reporter calculates a number, that lies on some thought point, where the 90th percentile most probably would land, if all recorded numbers where following some ideal curve. Therefore it is expected, that those values differ, even if it is not what the users seem to expect.

Second, given the different algorithms, one could expect, that the calculated percentile from the reporter would be "next" to the selected value from the visualizer. This is not the case with the reported values. And this is because the reporter seems to use the exclusive percentile, while the visualizer uses the inclusive percentile.

As this is not the first report related to the usage of different algorithms for calculating the percentiles, I think it would be most appropriate to discuss this problem on the dev (or users) list and use one algorithm for both components.
Comment 10 Felix Schumacher 2018-03-30 09:49:44 UTC
Created attachment 35827 [details]
calc document, that shows the differences of excl. and incl. percentile

I have enhanced the given csv file with a block, that calculates the percentiles for the "/" requests in different ways. None of those is a "wrong" value, but we probably should choose one of those as our "right" value.
Comment 11 Felix Schumacher 2018-03-30 09:53:36 UTC
As this seems to be about the usage of different algorithms for calculating the percentiles, I think bug 61130 is the same bug as this one.

*** This bug has been marked as a duplicate of bug 61130 ***
Comment 12 The ASF infrastructure team 2022-09-24 20:38:10 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/4568