Bug 59973

Summary: Issue with XML version in JTL files
Product: JMeter - Now in Github Reporter: Andrey Pokhilko <apc4>
Component: MainAssignee: JMeter issues mailing list <issues>
Status: REOPENED ---    
Severity: minor CC: p.mouawad
Priority: P2    
Version: 3.0   
Target Milestone: ---   
Hardware: All   
OS: All   
Bug Depends on: 58679    
Bug Blocks:    

Description Andrey Pokhilko 2016-08-11 10:09:21 UTC
When JMeter writes XML JTL files, it uses header with XML version 1.0
declared
(https://github.com/apache/jmeter/blob/trunk/src/core/org/apache/jmeter/reporters/ResultCollector.java#L82)

Here's example that I generated with ResultCollector, reproducing the
issue from one of the users:

<?xml version="1.0" encoding="UTF-8"?>
<testResults version="1.2">
<sample t="0" it="0" lt="0" ts="1469356444856" s="false" lb="" rc=""
rm="&#x9;" tn="" dt="" by="0" ng="0" na="0"/>

</testResults>

If we will use strict XML parser, like http://www.validome.org/validate
or other strict XML reading library, we will get the error for the above
text, because &#x9 is a character entity, only allowed in XML 1.1.
XStream writer that we use generates XML 1.1 data and there is FAQ entry
for this case: http://x-stream.github.io/faq.html#XML_control_char

It is a bug of writing file body as XML 1.1 while
declaration says 1.0.
Comment 1 Andrey Pokhilko 2016-08-11 10:38:05 UTC
Date: Thu Aug 11 10:27:45 2016
New Revision: 1755929

URL: http://svn.apache.org/viewvc?rev=1755929&view=rev
Log:
Bug 59973 - Issue with XML version in JTL files

Modified:
    jmeter/trunk/bin/testfiles/BatchTestLocal.xml
    jmeter/trunk/bin/testfiles/BatchTestLocalRemote.xml
    jmeter/trunk/bin/testfiles/Bug47165.xml
    jmeter/trunk/bin/testfiles/Bug50898.xml
    jmeter/trunk/bin/testfiles/Bug52310.xml
    jmeter/trunk/bin/testfiles/Bug52968.xml
    jmeter/trunk/bin/testfiles/Bug54685.xml
    jmeter/trunk/bin/testfiles/Bug55375.xml
    jmeter/trunk/bin/testfiles/Bug56243.xml
    jmeter/trunk/bin/testfiles/Bug56811.xml
    jmeter/trunk/bin/testfiles/HTMLParserTestFile_2.xml
    jmeter/trunk/bin/testfiles/TEST_HTTPS.xml
    jmeter/trunk/src/core/org/apache/jmeter/reporters/ResultCollector.java
    jmeter/trunk/xdocs/changes.xml
Comment 2 Milamber 2016-08-17 09:16:59 UTC

The first commits introduce a regression I think.

With the trunk version, if I try to open a JTL file (recorded by trunk version too) into a listenr, I have this error message (the listener dont display the results) :

2016/08/17 10:11:28 WARN  - jmeter.reporters.ResultCollector: Failed to load /directory/Results-20160816-1625.jtl using XStream. Error was: com.thoughtworks.xstream.io.StreamException:  : only 1.0 is supported as <?xml version not '1.1' (position: START_DOCUMENT seen <?xml version="1.1"... @1:19)
Comment 3 Andrey Pokhilko 2016-08-17 09:54:41 UTC
Funny... Xstream writes XML 1.1, but requires 1.0 declaration... I'll investigate it more
Comment 4 Andrey Pokhilko 2016-08-17 10:36:56 UTC
Milamber, which version of XStream library do you have? I have 1.4.8 and don't experience this. Also, I tried searching for this kind of error message in XStream sources of 1.4.8 and did not find it.
Comment 5 Milamber 2016-08-17 18:46:22 UTC
The version from the trunk: xstream-1.4.8.jar
Comment 6 Milamber 2016-08-17 19:20:41 UTC
Test case :

- Compile JMeter from trunk (ant distribution)
- unzip the binary, launch JMeter (with Java8)
- Create a simple test plan:
Thread Group
   |-- HTTP request
   |-- View Results Tree with a file to record into JTL (inside Configure button select all options)
- Run
- Close JMeter and reopen
- Add a View Results Tree, a try to open the jtl file from the simple test.
Comment 7 Andrey Pokhilko 2016-08-17 20:03:44 UTC
Thanks for the reproducing case. I have investigated it and found that issue is in xpp3 library we use. It is 10 years old and has xml 1.0 requirement hardcoded into it. There are no newer versions of this library that could load XML 1.1.
So it's a deadlock: we write XML 1.1 as body, we state that it's 1.0 in header because reading flow is unable to accept 1.1 header. However, it perfectly eats 1.1 body, which means that xpp3 does not conform to XML standard.

From their FAQ (http://x-stream.github.io/faq.html#XML_control_char) I can assume that the only way is to rollback my changes. Also, we should document somewhere here (http://jmeter.apache.org/usermanual/listeners.html#xmlformat2.1) that JTL and JMX files produced by JMeter do not conform XML 1.0 standard and cannot be read by strict parsers. Users should be aware of this issue.
Comment 8 Andrey Pokhilko 2016-08-17 20:11:47 UTC
I have reverted the modifications
Comment 9 Philippe Mouawad 2016-08-17 20:22:01 UTC
(In reply to Andrey Pokhilko from comment #7)
> Thanks for the reproducing case. I have investigated it and found that issue
> is in xpp3 library we use. It is 10 years old and has xml 1.0 requirement
> hardcoded into it. There are no newer versions of this library that could
> load XML 1.1.
> So it's a deadlock: we write XML 1.1 as body, we state that it's 1.0 in
> header because reading flow is unable to accept 1.1 header. However, it
> perfectly eats 1.1 body, which means that xpp3 does not conform to XML
> standard.
> 
> From their FAQ (http://x-stream.github.io/faq.html#XML_control_char) I can
> assume that the only way is to rollback my changes. Also, we should document
> somewhere here
> (http://jmeter.apache.org/usermanual/listeners.html#xmlformat2.1) that JTL
> and JMX files produced by JMeter do not conform XML 1.0 standard and cannot
> be read by strict parsers. Users should be aware of this issue.

See this Bug 58679 related to this issue, this would be the final fix.
Comment 10 Philippe Mouawad 2016-08-17 20:22:56 UTC
And this https://github.com/x-stream/xstream/issues/51#issuecomment-210155359
Comment 11 Andrey Pokhilko 2016-08-17 20:35:39 UTC
Date: Wed Aug 17 20:11:30 2016
New Revision: 1756681

URL: http://svn.apache.org/viewvc?rev=1756681&view=rev
Log:
Bug 59973 - revert all modifications

Modified:
    jmeter/trunk/bin/testfiles/BatchTestLocal.xml
    jmeter/trunk/bin/testfiles/BatchTestLocalRemote.xml
    jmeter/trunk/bin/testfiles/Bug47165.xml
    jmeter/trunk/bin/testfiles/Bug50898.xml
    jmeter/trunk/bin/testfiles/Bug52310.xml
    jmeter/trunk/bin/testfiles/Bug52968.xml
    jmeter/trunk/bin/testfiles/Bug54685.xml
    jmeter/trunk/bin/testfiles/Bug55375.xml
    jmeter/trunk/bin/testfiles/Bug56243.xml
    jmeter/trunk/bin/testfiles/Bug56811.xml
    jmeter/trunk/bin/testfiles/HTMLParserTestFile_2.xml
    jmeter/trunk/bin/testfiles/TEST_HTTPS.xml
    jmeter/trunk/src/core/org/apache/jmeter/reporters/ResultCollector.java
    jmeter/trunk/xdocs/changes.xml
Comment 12 Andrey Pokhilko 2016-08-17 20:36:28 UTC
Date: Wed Aug 17 20:24:21 2016
New Revision: 1756682

URL: http://svn.apache.org/viewvc?rev=1756682&view=rev
Log:
Bug 59973 - document non-standard XML in docs and code

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/reporters/ResultCollector.java
    jmeter/trunk/xdocs/usermanual/component_reference.xml
Comment 13 The ASF infrastructure team 2022-09-24 20:38:05 UTC
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/4056