Bug 63643 - CSV Data Set Config won't read quoted UTF-8 with BOM
Summary: CSV Data Set Config won't read quoted UTF-8 with BOM
Status: RESOLVED FIXED
Alias: None
Product: JMeter - Now in Github
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.1.1
Hardware: All All
: P2 normal (vote)
Target Milestone: JMETER_5.2
Assignee: JMeter issues mailing list
URL:
Keywords: FixedInTrunk
Depends on:
Blocks:
 
Reported: 2019-08-06 12:30 UTC by havlicek.honza
Modified: 2019-09-21 16:57 UTC (History)
1 user (show)



Attachments
Test plan with a test CSV file (1.52 KB, application/x-zip-compressed)
2019-08-06 12:30 UTC, havlicek.honza
Details

Note You need to log in before you can comment on or make changes to this bug.
Description havlicek.honza 2019-08-06 12:30:46 UTC
Created attachment 36705 [details]
Test plan with a test CSV file

If a CSV is 

* encoded in UTF-8 and 
* has Byte Order Mark
* starts with a quoted field (e.g no header)
* Csv Data Set Config has Allow Quoted Data set to True

the parser will throw an exception and CSV file is not read.

2019-08-06 14:20:15,229 ERROR o.a.j.c.CSVDataSet: java.io.IOException: Cannot have quote-char in plain field:["]

Expected behavior: BOM should be ignored and CSV should be read.

I have attached a simple example where a CSV file with BOM fails and second one that passes.
Comment 1 Felix Schumacher 2019-08-15 11:57:59 UTC
Thanks for the report, should now be fixed.

@Havlicek Can you try to compile master yourself and report if it works for you?

Should be in the next nightly, but the index creation script for nightlies currently has a problem and will not show them all.

commit 3af59b954cd3d271fb189080036baddd4b4f9d31
Author: Felix Schumacher <felix.schumacher@internetallee.de>
AuthorDate: Thu Aug 15 13:52:32 2019 +0200

    Skip BOM on files opened through FileServer
    
    Use the BOM to detect the character encoding, if none is given explicitly.
    
    Reported by Havlicek Honza (havlicek.honza at gmail.com)
    
    Bugzilla Id: 63643
---
 .../src/main/java/org/apache/jmeter/services/FileServer.java     | 5 ++++-
 .../test/groovy/org/apache/jmeter/services/FileServerSpec.groovy | 9 +++++++++
 .../resources/org/apache/jmeter/services/testfiles/bomData.csv   | 1 +
 xdocs/changes.xml                                                | 3 +++
 4 files changed, 17 insertions(+), 1 deletion(-)
Comment 2 pierre.astruc 2019-08-26 20:57:19 UTC
Hum, I can not download any nightly build (404 everywhere)
https://jmeter.apache.org/nightly.html

May I ask you a .zip of your recent build ?
Comment 3 Philippe Mouawad 2019-09-21 16:57:08 UTC
(In reply to pierre.astruc from comment #2)
> Hum, I can not download any nightly build (404 everywhere)
> https://jmeter.apache.org/nightly.html
> 
> May I ask you a .zip of your recent build ?

Hello,
You can try now, issue with nightly build generation was fixed few weeks ago.

Closing issue.
Please reopen if fix is not confirmed.

Thank you
Comment 4 The ASF infrastructure team 2022-09-24 20:38:17 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/5138