Bug 61725 - Response time table in dashboard report don't match with aggregate report
Summary: Response time table in dashboard report don't match with aggregate report
Status: RESOLVED DUPLICATE of bug 61130
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 4.0
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-06 04:51 UTC by Aj
Modified: 2018-03-30 09:53 UTC (History)
1 user (show)



Attachments
Image file (605.90 KB, image/jpeg)
2017-11-06 04:51 UTC, Aj
Details
sample csv file (43.12 KB, application/vnd.ms-excel)
2018-03-22 13:50 UTC, Vvxxsz
Details
jmeter log in debug mode (49.80 KB, text/plain)
2018-03-22 13:51 UTC, Vvxxsz
Details
aggregate report screenshot (47.82 KB, image/jpeg)
2018-03-22 13:51 UTC, Vvxxsz
Details
html report screenshot (86.33 KB, image/jpeg)
2018-03-22 13:52 UTC, Vvxxsz
Details
user.properties (6.08 KB, text/plain)
2018-03-22 13:54 UTC, Vvxxsz
Details
calc document, that shows the differences of excl. and incl. percentile (35.24 KB, application/octet-stream)
2018-03-30 09:49 UTC, Felix Schumacher
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Aj 2017-11-06 04:51:48 UTC
Created attachment 35492 [details]
Image file

Statistics table of dashboard generator doesn't match with response time table by aggregate report plugin by Jmeter GUI.
Comment 1 Philippe Mouawad 2017-11-08 21:12:58 UTC
In order to work with this , can you provide your CSV file and user.properties so that we can see from where it comes ?
Comment 2 Philippe Mouawad 2018-01-14 17:05:15 UTC
Probably this:

- https://dzone.com/articles/how-to-achieve-better-accuracy-in-latency-percenti
Comment 3 Vvxxsz 2018-03-22 13:48:37 UTC
Greetings

Same behavior have been observed on JMeter 3.1-4.0 Java 8
Adjusting jmeter.reportgenerator.statistic_window property to exact samples amount, -1 or 20000000 didn't solve the problem.

Report has been generated using jmeter.bat -g /path/to/csv -o /path/to/html/ cmd

In terms to the reproduce the issue here is:
- reportgenerator.properties
- jmeter.log with debug logging
- sample_html_report.csv
- Aggregate Report and HTML report screenshots
Comment 4 Vvxxsz 2018-03-22 13:50:12 UTC
Created attachment 35804 [details]
sample csv file
Comment 5 Vvxxsz 2018-03-22 13:51:01 UTC
Created attachment 35805 [details]
jmeter log in debug mode
Comment 6 Vvxxsz 2018-03-22 13:51:29 UTC
Created attachment 35806 [details]
aggregate report screenshot
Comment 7 Vvxxsz 2018-03-22 13:52:14 UTC
Created attachment 35807 [details]
html report screenshot
Comment 8 Vvxxsz 2018-03-22 13:54:13 UTC
Created attachment 35808 [details]
user.properties
Comment 9 Felix Schumacher 2018-03-30 09:46:55 UTC
I have had time to look at the reported numbers and I think that we have two problems here.

First is a that we have different algorithms for calculating the percentiles in those two parts. The visualizer uses a discrete algorithm that uses the lowest number, that is recorded and inside of the 90th percentile. The reporter calculates a number, that lies on some thought point, where the 90th percentile most probably would land, if all recorded numbers where following some ideal curve. Therefore it is expected, that those values differ, even if it is not what the users seem to expect.

Second, given the different algorithms, one could expect, that the calculated percentile from the reporter would be "next" to the selected value from the visualizer. This is not the case with the reported values. And this is because the reporter seems to use the exclusive percentile, while the visualizer uses the inclusive percentile.

As this is not the first report related to the usage of different algorithms for calculating the percentiles, I think it would be most appropriate to discuss this problem on the dev (or users) list and use one algorithm for both components.
Comment 10 Felix Schumacher 2018-03-30 09:49:44 UTC
Created attachment 35827 [details]
calc document, that shows the differences of excl. and incl. percentile

I have enhanced the given csv file with a block, that calculates the percentiles for the "/" requests in different ways. None of those is a "wrong" value, but we probably should choose one of those as our "right" value.
Comment 11 Felix Schumacher 2018-03-30 09:53:36 UTC
As this seems to be about the usage of different algorithms for calculating the percentiles, I think bug 61130 is the same bug as this one.

*** This bug has been marked as a duplicate of bug 61130 ***