Bug 57516 - HTML Report generator with APDEX
Summary: HTML Report generator with APDEX
Status: RESOLVED DUPLICATE of bug 58653
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 2.12
Hardware: All All
: P2 enhancement (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-30 21:58 UTC by Philippe Mouawad
Modified: 2015-12-27 20:45 UTC (History)
2 users (show)



Attachments
Apdex computing implementation (6.23 KB, patch)
2015-01-30 21:58 UTC, Philippe Mouawad
Details | Diff
Screenshot of Report Result (326.61 KB, image/png)
2015-02-01 18:22 UTC, Philippe Mouawad
Details
Report Generator patch (28.53 KB, patch)
2015-02-01 18:23 UTC, Philippe Mouawad
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Mouawad 2015-01-30 21:58:20 UTC
Created attachment 32414 [details]
Apdex computing implementation

As per dev mailing list discussion :
http://mail-archives.apache.org/mod_mbox/jmeter-dev/201501.mbox/browser

The idea is to compute an APDEX on a series of transactions.
Comment 1 Felix Schumacher 2015-01-31 11:42:00 UTC
Some nitpicks:
 * A few more javadoc comments would not hurt :) 
 * naming of the method computeAPDEX, should this be computeApdex?
 * we might want to be able to supply a regex to specify the samplers we want to be included in the counting
 * what to do with broken samples?

All in all a nice addition.
Comment 2 Philippe Mouawad 2015-02-01 18:22:49 UTC
Created attachment 32420 [details]
Screenshot of Report Result

Hi,
I made a kind of ugly POC that generates the attached report.
I implemented an AbstractBackendListenerClient that:
- at end of Test takes the generated results file and outputs an HTML report containing :
1/ APDEX results
2/ Summary results
3/ Graph of OK/KO 

I used:
- Freemarker as templating engine (Apache compatible license)
- Bootstrap to make page a bit sexy (I used Dashboard example) but I am not a UX expert at all 
- Chartjs then flot as Graphing library, found the latter sexier for PieCharts

This code is still very ugly but I didn't think it would be that fast to have this result.

If you  agree, I will cleanup code and commit it in trunk so that it can be enhanced by all of us.

Regards
Philippe
Comment 3 Philippe Mouawad 2015-02-01 18:23:37 UTC
Created attachment 32421 [details]
Report Generator patch

Here is the early code
Comment 4 Vladimir Sitnikov 2015-02-06 20:43:12 UTC
+1 for OK/KO

-1 for ConcurrentHashMap.get()... if (==null) { .put()}
It should be ConcurrentHashMap.putIfAbsent at least.

I think success/fail needs to be included into the map key, so response time percentiles are computed across successful and failure cases separately.
Comment 5 Philippe Mouawad 2015-02-06 21:40:20 UTC
Thanks.
As I said it was an ugly code for POC, it should be fully reworked.

What would you expect as other graphs and reports that you feel JMeter currently lacks ?

Thanks

Regards
Comment 6 Vladimir Sitnikov 2015-02-06 22:31:14 UTC
Frankly speaking, each and every my project results in different load report.

Sometimes it is focused on scalability (e.g. x=load, y=response time)
Sometimes the main focus is scenarios
Sometimes the focus is on comparison of old vs new.

Typically I use R + data.table + pandoc to build reports and munge the data.
I combine different kinds of logs, so JMeter results are not the only source.

In general:
1) -1 for pie charts. For instance: https://blogs.oracle.com/experience/entry/countdown_of_top_10_reasons_to_never_ever_use_a_pie_chart

2) It would be nice to integrate "non functional requirements" into JMeter.
APDEX is a sort of. Current AggregateReport computes percentiles (it would be nice if it could compute 95, 99), but it doesn't validate outcomes.

Often there are NFRs of "90% should be less than X seconds", and "throughput should be Y per hour", and "error rate should be less than Z%". It could be wonderful if JMeter could highlight violations.

3) Other. Well, I need to think.
Comment 7 Philippe Mouawad 2015-12-27 20:45:13 UTC

*** This bug has been marked as a duplicate of bug 58653 ***