Bug 64617 - HTML report: In graph Response Time Percentiles Over Time 90,95,99th percentile correspond in reality to 0.90, 0.95 and 0.99 percentiles
Summary: HTML report: In graph Response Time Percentiles Over Time 90,95,99th percenti...
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.3
Hardware: All All
: P2 regression (vote)
Target Milestone: JMETER_5.4
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2020-07-21 09:47 UTC by Hiroyoshi Mitsumori
Modified: 2020-07-27 02:41 UTC (History)
1 user (show)



Attachments
Differences between the graphs (Response Time Percentiles Over Time) (235.89 KB, image/png)
2020-07-21 09:47 UTC, Hiroyoshi Mitsumori
Details
jmx file and jtl file (15.07 KB, application/x-zip-compressed)
2020-07-23 18:37 UTC, Hiroyoshi Mitsumori
Details
jmeter.log (4.58 KB, application/x-zip-compressed)
2020-07-23 18:53 UTC, Hiroyoshi Mitsumori
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hiroyoshi Mitsumori 2020-07-21 09:47:59 UTC
Created attachment 37365 [details]
Differences between the graphs (Response Time Percentiles Over Time)

In the HTML report, the Response Time Percentiles Over Time (successful responses) graph displays three percentile values: 90th percentile, 95th percentile, and 99th percentile.
These 3 percentile values should indicate a longer response time than the median (= 50th percentile). However, I have found that in the JMeter 5.3 graphs, the three percentile values are displayed below the Median.

If I create a report in JMeter 5.2.1 from the same log, the Response Time Percentiles Over Time graph will be completely different.
Median, which is new in JMeter 5.3, is of course not shown. In addition to that, the display of the three percentile values will be different.

I am reporting to Bugzilla for the first time. 
If there is any missing information, I will do my best to respond to it.

Thanks.
Comment 1 Philippe Mouawad 2020-07-21 11:51:54 UTC
Thanks for report.

One clarification:
- Did you use the different version to generate the Graphs only 
- or did you run a test then generate with version 5.2.1 then run a test then generate with version 5.3 ?

Can you provide screenshots or the full report that shows this difference ?

Could you share:
- JMX file
- jmeter.log
- CSV file for the 2 tests ?

Thank you
Regards
Comment 2 Hiroyoshi Mitsumori 2020-07-23 18:37:42 UTC
Created attachment 37369 [details]
jmx file and jtl file
Comment 3 Hiroyoshi Mitsumori 2020-07-23 18:53:26 UTC
Created attachment 37370 [details]
jmeter.log
Comment 4 Hiroyoshi Mitsumori 2020-07-23 18:57:14 UTC
(In reply to Philippe Mouawad from comment #1)
> Thanks for report.
> 
> One clarification:
> - Did you use the different version to generate the Graphs only 
> - or did you run a test then generate with version 5.2.1 then run a test
> then generate with version 5.3 ?

I generated two reports from the one log files I created in version 5.3.

 
> Can you provide screenshots or the full report that shows this difference ?
> 
> Could you share:
> - JMX file
> - jmeter.log
> - CSV file for the 2 tests ?

I shared the two zip files.
It contains:
- JMX file
- CSV file(one file)
- jmeter.log(two files)

You can generate two different HTML reports from two versions of JMeter, 5.3 and 5.2.1, using the attached jtl file.

Thank you.
Comment 5 Philippe Mouawad 2020-07-25 20:13:03 UTC
Issue has been introduced by :
https://github.com/apache/jmeter/commit/9a8ca629dec1bf70444161988481399332434173

Related to Bug 64380
Comment 6 Philippe Mouawad 2020-07-25 20:18:26 UTC
pmouawad pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/jmeter.git


The following commit(s) were added to refs/heads/master by this push:
     new 5d47043  Bug 64617 - HTML report: In graph Response Time Percentiles Over Time 90,95,99th percentile correspond in reality to 0.90, 0.95 and 0.99 percentiles
5d47043 is described below

commit 5d470437c87f2807af317284a2d9372d03db987a
Author: pmouawad <p.mouawad@ubik-ingenierie.com>
AuthorDate: Sat Jul 25 22:14:55 2020 +0200

    Bug 64617 - HTML report: In graph Response Time Percentiles Over Time
    90,95,99th percentile correspond in reality to 0.90, 0.95 and 0.99
    percentiles
---
 .../graph/impl/ResponseTimePercentilesOverTimeGraphConsumer.java     | 5 +----
 xdocs/changes.xml                                                    | 2 ++
 2 files changed, 3 insertions(+), 4 deletions(-)
Comment 7 Philippe Mouawad 2020-07-25 20:19:10 UTC
Thanks a lot for reporting this critical regression !

Can you test nightly build that will be generated tonight and available tomorrow ?
Thanks
Comment 8 Hiroyoshi Mitsumori 2020-07-27 02:41:49 UTC
I have confirmed that the graphs are working correctly in Nightly Build.
Thanks for your great work.