Bug 59889

Summary: Dashboard cannot display Japanese(and maybe also Chinese) characters.
Product: JMeter Reporter: Kimono <kimono.outfit.am>
Component: MainAssignee: JMeter issues mailing list <issues>
Status: RESOLVED FIXED    
Severity: normal CC: p.mouawad
Priority: P2    
Version: 3.0   
Target Milestone: ---   
Hardware: PC   
OS: All   
Attachments: jmeter.log
jmeter.properties(not modified from original)
user.properties(not modified from original)
result jtl file(UTF-8)
Problem and Expected
Simple test plan to produce a csv file with japanese characters
Change encoding to UTF-8 for CSV Files in the report package

Description Kimono 2016-07-21 15:03:25 UTC
Dashboard cannot display Japanese(and maybe also Chinese) characters.

This is because CHARSET is specified as "ISO8859-1" in CsvSampleReader.java.
It should be taken from SaveService.

CsvSampleReader.java
----
private static final String CHARSET = "ISO8859-1";
-> private static final String CHARSET = SaveService.getFileEncoding("UTF-8");
Comment 1 Philippe Mouawad 2016-07-21 20:53:56 UTC
Hi,
Thanks for report.
could you provide a sample csv file that reproduces the issue ?
Also provide user.properties, jmeter.properties and jmeter.log

Thanks
Regards
Comment 2 Kimono 2016-07-22 13:50:18 UTC
Created attachment 34062 [details]
jmeter.log
Comment 3 Kimono 2016-07-22 13:51:34 UTC
Created attachment 34063 [details]
jmeter.properties(not modified from original)
Comment 4 Kimono 2016-07-22 13:52:11 UTC
Created attachment 34064 [details]
user.properties(not modified from original)
Comment 5 Kimono 2016-07-22 13:52:51 UTC
Created attachment 34065 [details]
result jtl file(UTF-8)
Comment 6 Kimono 2016-07-22 13:55:25 UTC
Created attachment 34066 [details]
Problem and Expected
Comment 7 Kimono 2016-07-22 13:59:30 UTC
Thanks for your reply.

Additional image/pnf file is screen capture of expected result.
Comment 8 Felix Schumacher 2016-07-24 12:18:28 UTC
Created attachment 34070 [details]
Simple test plan to produce a csv file with japanese characters

The problem seems to be, that the csv (jtl) files are written using the ResultCollector, which uses UTF-8 as default. Trouble is, that they are read using CSVSampleReader (which defaults to ISO-8859-1).

When we change CSVSampleReader to UTF-8, we should consider changing CSVSampleWriter to UTF-8, too.

CSVSampleWriter inherits from AbstractSampleWriter, which initialises its writers with ISO-8859-1. Should we change AbstractSampleWriter, too? Or only change CSVSampleWriter?

What do you think?
Comment 9 Felix Schumacher 2016-07-24 12:27:57 UTC
Created attachment 34071 [details]
Change encoding to UTF-8 for CSV Files in the report package
Comment 10 Philippe Mouawad 2016-07-24 13:05:04 UTC
+1 for your patch Felix
Comment 11 Kimono 2016-07-25 13:35:40 UTC
Felix's patch solves the problem.
Thanks.
Comment 12 Felix Schumacher 2016-07-28 17:36:41 UTC
Date: Thu Jul 28 17:36:07 2016
New Revision: 1754436

URL: http://svn.apache.org/viewvc?rev=1754436&view=rev
Log:
Change encoding to UTF-8 in reports for dashboard.

Bugzilla Id: 59889

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/report/core/AbstractSampleWriter.java
    jmeter/trunk/src/core/org/apache/jmeter/report/core/CsvSampleReader.java
    jmeter/trunk/xdocs/changes.xml