Bug 66140 - Report generation fails
Summary: Report generation fails
Status: NEEDINFO
Alias: None
Product: JMeter - Now in Github
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.5
Hardware: PC All
: P2 normal (vote)
Target Milestone: JMETER_5.5
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-06-23 08:12 UTC by Frederic Girard
Modified: 2022-07-11 16:09 UTC (History)
0 users



Attachments
Simple test file to reproduce error message (299 bytes, text/plain)
2022-07-08 13:41 UTC, Felix Schumacher
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frederic Girard 2022-06-23 08:12:14 UTC
Report generation fails with the following error message :

An error occurred: Error while processing samples: Consumer failed with message :No column <timeStamp> found in sample metadata <timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;sentBytes;grpThreads;allThreads;URL;Latency;SampleCount;ErrorCount;Hostname;IdleTime;Connect>, check #jmeter.save.saveservice.* properties to add the missing column

$ head -1 20220622.200632_2207SB08_20220622200627_euld2_2207SB08_cube_10H_FR.jtl
timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;sentBytes;grpThreads;allThreads;URL;Latency;SampleCount;ErrorCount;Hostname;IdleTime;Connect

First column is "timeStamp" ...
Comment 1 Felix Schumacher 2022-07-05 14:39:27 UTC
Have you looked at the end of your csv file, if it is complete?

Can you post a sample, the shows the behaviour?
Comment 2 Frederic Girard 2022-07-05 15:02:51 UTC
The report generation works with the same .jtl using jmeter 5.4.3, so I'm sure the JTL is well formated.

Note that this failure happens at the very start of the jtl parsing.

I cannot join the JTL as it is 10 GB long, I'm sorry for that.
Comment 3 Felix Schumacher 2022-07-05 16:04:16 UTC
Great to know, that the file works with 5.4.3.

Have you tried and shorten it (say to one or two lines)? Does it fail, too?
Comment 4 Felix Schumacher 2022-07-05 19:08:11 UTC
Can you look into the log file jmeter.log and post the stack trace, that should be in there?

Which settings for the report generation are you using?

How did you install JMeter and do you have any third party plugins installed?
Comment 5 Frederic Girard 2022-07-06 08:03:53 UTC
Here's the stack trace:

2022-07-06 09:53:50,277 ERROR o.a.j.JMeter: An error occurred:
org.apache.jmeter.report.dashboard.GenerationException: Error while processing samples: Consumer failed with message :No column <timeStamp> found in sample metadata <timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;sentBytes;grpThreads;allThreads;URL;Latency;SampleCount;ErrorCount;Hostname;IdleTime;Connect>, check #jmeter.save.saveservice.* properties to add the missing column
        at org.apache.jmeter.report.dashboard.ReportGenerator.generate(ReportGenerator.java:245) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.JMeter.start(JMeter.java:462) ~[ApacheJMeter_core.jar:5.5]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_322]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_322]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
        at org.apache.jmeter.NewDriver.main(NewDriver.java:259) ~[ApacheJMeter.jar:5.5]
Caused by: org.apache.jmeter.report.core.SampleException: Consumer failed with message :No column <timeStamp> found in sample metadata <timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;sentBytes;grpThreads;allThreads;URL;Latency;SampleCount;ErrorCount;Hostname;IdleTime;Connect>, check #jmeter.save.saveservice.* properties to add the missing column
        at org.apache.jmeter.report.processor.CsvFileSampleSource$PrivateProducer.startProducing(CsvFileSampleSource.java:314) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.CsvFileSampleSource.produce(CsvFileSampleSource.java:174) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.CsvFileSampleSource.run(CsvFileSampleSource.java:235) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.dashboard.ReportGenerator.generate(ReportGenerator.java:243) ~[ApacheJMeter_core.jar:5.5]
        ... 6 more
Caused by: org.apache.jmeter.report.core.SampleException: No column <timeStamp> found in sample metadata <timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;sentBytes;grpThreads;allThreads;URL;Latency;SampleCount;ErrorCount;Hostname;IdleTime;Connect>, check #jmeter.save.saveservice.* properties to add the missing column
        at org.apache.jmeter.report.core.SampleMetadata.ensureIndexOf(SampleMetadata.java:216) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.NormalizerSampleConsumer.startConsuming(NormalizerSampleConsumer.java:67) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.CsvFileSampleSource$PrivateProducer.startProducing(CsvFileSampleSource.java:311) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.CsvFileSampleSource.produce(CsvFileSampleSource.java:174) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.processor.CsvFileSampleSource.run(CsvFileSampleSource.java:235) ~[ApacheJMeter_core.jar:5.5]
        at org.apache.jmeter.report.dashboard.ReportGenerator.generate(ReportGenerator.java:243) ~[ApacheJMeter_core.jar:5.5]
        ... 6 more
Comment 6 Frederic Girard 2022-07-06 08:41:41 UTC
We use only few plugins :

lib/ext/jmeter-plugins-casutg-2.10.jar
lib/ext/jmeter-plugins-csvars-0.1.jar
lib/ext/jmeter-plugins-dummy-0.4.jar
lib/ext/jmeter-plugins-ffw-2.0.jar
lib/ext/jmeter-plugins-fifo-0.2.jar
lib/ext/jmeter-plugins-functions-2.1.jar
lib/ext/jmeter-plugins-graphs-basic-2.0.jar
lib/ext/jmeter-plugins-manager-1.7.jar
lib/ext/jmeter-plugins-perfmon-2.1.jar
lib/ext/jmeter-plugins-prmctl-0.4.jar
lib/ext/jmeter-plugins-tst-2.5.jar
lib/jmeter-plugins-cmn-jmeter-0.7.jar
lib/json-lib-2.4-jdk15.jar
lib/perfmon-2.2.2.jar

Report generation settings:

jmeter.reportgenerator.apdex_satisfied_threshold=300
jmeter.reportgenerator.apdex_tolerated_threshold=800
jmeter.reportgenerator.statistic_window=-1
jmeter.reportgenerator.report_title=Decathlon Performance Test Dashboard
jmeter.reportgenerator.overall_granularity=60000
jmeter.reportgenerator.exclude_tc_from_top5_errors_by_sampler=true
jmeter.reportgenerator.graph.responseTimePercentiles.classname=org.apache.jmeter.report.processor.graph.impl.ResponseTimePercentilesGraphConsumer
jmeter.reportgenerator.graph.responseTimePercentiles.title=Response Time Percentiles
jmeter.reportgenerator.graph.responseTimeDistribution.classname=org.apache.jmeter.report.processor.graph.impl.ResponseTimeDistributionGraphConsumer
jmeter.reportgenerator.graph.responseTimeDistribution.title=Response Time Distribution
jmeter.reportgenerator.graph.responseTimeDistribution.property.set_granularity=500
jmeter.reportgenerator.graph.activeThreadsOverTime.classname=org.apache.jmeter.report.processor.graph.impl.ActiveThreadsGraphConsumer
jmeter.reportgenerator.graph.activeThreadsOverTime.title=Active Threads Over Time
jmeter.reportgenerator.graph.activeThreadsOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.timeVsThreads.classname=org.apache.jmeter.report.processor.graph.impl.TimeVSThreadGraphConsumer
jmeter.reportgenerator.graph.timeVsThreads.title=Time VS Threads
jmeter.reportgenerator.graph.bytesThroughputOverTime.classname=org.apache.jmeter.report.processor.graph.impl.BytesThroughputGraphConsumer
jmeter.reportgenerator.graph.bytesThroughputOverTime.title=Bytes Throughput Over Time
jmeter.reportgenerator.graph.bytesThroughputOverTime.exclude_controllers=true
jmeter.reportgenerator.graph.bytesThroughputOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.responseTimesOverTime.classname=org.apache.jmeter.report.processor.graph.impl.ResponseTimeOverTimeGraphConsumer
jmeter.reportgenerator.graph.responseTimesOverTime.title=Response Time Over Time
jmeter.reportgenerator.graph.responseTimesOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.responseTimesOverTime.exclude_controllers=true
jmeter.reportgenerator.graph.responseTimePercentilesOverTime.classname=org.apache.jmeter.report.processor.graph.impl.ResponseTimePercentilesOverTimeGraphConsumer
jmeter.reportgenerator.graph.responseTimePercentilesOverTime.title=Response Time Percentiles Over Time (successful requests only)
jmeter.reportgenerator.graph.responseTimePercentilesOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.syntheticResponseTimeDistribution.classname=org.apache.jmeter.report.processor.graph.impl.SyntheticResponseTimeDistributionGraphConsumer
jmeter.reportgenerator.graph.syntheticResponseTimeDistribution.title=Synthetic Response Times Distribution
jmeter.reportgenerator.graph.syntheticResponseTimeDistribution.exclude_controllers=true
jmeter.reportgenerator.graph.syntheticResponseTimeDistribution.property.set_satisfied_threshold=${jmeter.reportgenerator.apdex_satisfied_threshold}
jmeter.reportgenerator.graph.syntheticResponseTimeDistribution.property.set_tolerated_threshold=${jmeter.reportgenerator.apdex_tolerated_threshold}
jmeter.reportgenerator.graph.latenciesOverTime.classname=org.apache.jmeter.report.processor.graph.impl.LatencyOverTimeGraphConsumer
jmeter.reportgenerator.graph.latenciesOverTime.title=Latencies Over Time
jmeter.reportgenerator.graph.latenciesOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.connectTimeOverTime.classname=org.apache.jmeter.report.processor.graph.impl.ConnectTimeOverTimeGraphConsumer
jmeter.reportgenerator.graph.connectTimeOverTime.title=Connect Time Over Time
jmeter.reportgenerator.graph.connectTimeOverTime.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.responseTimeVsRequest.classname=org.apache.jmeter.report.processor.graph.impl.ResponseTimeVSRequestGraphConsumer
jmeter.reportgenerator.graph.responseTimeVsRequest.title=Response Time Vs Request
jmeter.reportgenerator.graph.responseTimeVsRequest.exclude_controllers=true
jmeter.reportgenerator.graph.responseTimeVsRequest.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.latencyVsRequest.classname=org.apache.jmeter.report.processor.graph.impl.LatencyVSRequestGraphConsumer
jmeter.reportgenerator.graph.latencyVsRequest.title=Latencies Vs Request
jmeter.reportgenerator.graph.latencyVsRequest.exclude_controllers=true
jmeter.reportgenerator.graph.latencyVsRequest.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.hitsPerSecond.classname=org.apache.jmeter.report.processor.graph.impl.HitsPerSecondGraphConsumer
jmeter.reportgenerator.graph.hitsPerSecond.title=Hits Per Second
jmeter.reportgenerator.graph.hitsPerSecond.exclude_controllers=true
jmeter.reportgenerator.graph.hitsPerSecond.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.codesPerSecond.classname=org.apache.jmeter.report.processor.graph.impl.CodesPerSecondGraphConsumer
jmeter.reportgenerator.graph.codesPerSecond.title=Codes Per Second
jmeter.reportgenerator.graph.codesPerSecond.exclude_controllers=true
jmeter.reportgenerator.graph.codesPerSecond.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.graph.transactionsPerSecond.classname=org.apache.jmeter.report.processor.graph.impl.TransactionsPerSecondGraphConsumer
jmeter.reportgenerator.graph.transactionsPerSecond.title=Transactions Per Second
jmeter.reportgenerator.graph.transactionsPerSecond.property.set_granularity=${jmeter.reportgenerator.overall_granularity}
jmeter.reportgenerator.exporter.html.classname=org.apache.jmeter.report.dashboard.HtmlTemplateExporter
jmeter.reportgenerator.exporter.html.series_filter=^(R-HTTP_.*?|TC_.*?)(-success|-failure)?$
Comment 7 Frederic Girard 2022-07-06 09:24:15 UTC
Ok I think I found the issue: the following property was missing in user.properties:

jmeter.save.saveservice.default_delimiter=;
Comment 8 Felix Schumacher 2022-07-08 13:41:28 UTC
Created attachment 38337 [details]
Simple test file to reproduce error message

When the separator doesn't match the fields in the header, we get a strange error message.
Comment 9 Felix Schumacher 2022-07-08 13:42:28 UTC
Thanks for coming back with that information. I will leave the bug open for now.
Comment 10 Felix Schumacher 2022-07-09 12:17:34 UTC
@Frederic, can you test the one of the next builds from trunk?

It should work with your file and a mis-configured delimiter, but should log another warning in the jmeter.log file.

commit b22b00c69459dad4f7b02518c6a5e1f4bcfebad4
AuthorDate: Sat Jul 9 14:14:09 2022 +0200

    Guess the delimiter of the CSV source, when configured one seems wrong.
    
    This is in line with the behaviour of CSVSaveService.
    
    Bugzilla Id: 66140
---
 .../jmeter/report/core/SampleMetaDataParser.java   | 38 ++++++++++++++++++++--
 xdocs/changes.xml                                  |  2 ++
 2 files changed, 37 insertions(+), 3 deletions(-)
Comment 11 Felix Schumacher 2022-07-09 12:50:00 UTC
commit 157db3679a5ed14e97f4b7dd59f29b395217cd48
AuthorDate: Sat Jul 9 14:48:17 2022 +0200

    Make failing tests run again and add more tests
    
    Bugzilla Id: 66140
---
 .../apache/jmeter/report/core/SampleMetaDataParser.java   |  5 +++--
 .../jmeter/report/core/SampleMetadataParserSpec.groovy    | 15 +++++++++++++++
 2 files changed, 18 insertions(+), 2 deletions(-)
Comment 12 Frederic Girard 2022-07-11 15:55:24 UTC
Hello,

"one of the next builds from trunk"

Please where can I find them ?
Comment 14 The ASF infrastructure team 2022-09-24 20:38:23 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/5683