Bug 49039 - Proxy : Bug with multipart bytes
Summary: Proxy : Bug with multipart bytes
Status: RESOLVED DUPLICATE of bug 52674
Alias: None
Product: JMeter
Classification: Unclassified
Component: HTTP (show other bugs)
Version: 2.3.4
Hardware: All All
: P2 normal (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-04-02 14:23 UTC by Steph
Modified: 2012-06-27 21:17 UTC (History)
2 users (show)



Attachments
Without proxy and with proxy (108.09 KB, image/jpeg)
2010-04-02 14:23 UTC, Steph
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steph 2010-04-02 14:23:57 UTC
Created attachment 25224 [details]
Without proxy and with proxy

I send a http request with multipart data.
My data are bytes and content-type */*.
Without proxy JMeter, my request is ok.
With JMeter proxy, data are corrupted and my content-type is modified to text/plain.
I try to debug proxy in eclipse and i think the problem come from the class PostWriter.java witch create a new String from data so in my case a new String from byte.
So some characters are mismatched.
I attach two screenshots.

Thx for your help
Comment 1 Philippe Mouawad 2011-11-12 13:58:58 UTC
Hello,
Could you provide the detailed steps of your test and if possible a front facing site  for this issue ?
Otherwise a capture of the dialog with the server.

Thank you
Regards
Comment 2 jens_0 2012-01-25 12:21:54 UTC
I can confirm this.

when recording HTTP data with JMeter, I'd assume that the request is sent to the server just like the browser would send it.
This is not the case in two examples I've run across.

As you'll know, when recording with HTTP Proxy Server, JMeter will be creating HTTP Samplers and then use their sample method to realize the request, returning the return value to the browser.
So we need to create a HTTP Sampler which produces the same request as the browser would have done.

When looking at HttpRequestHdr.populateSampler, I see that ConversionUtils.getEncodingFromContentType
is called. In case the encoding is not supported (returns null) we look in the pageEncodings and formEncodings maps (when and how are they populated??) and if we find nothing here, we use:
postData = new String(rawPostData);
meaning we convert the bytes to a string based on the platform-specific default encoding.

In my examples this breaks the communication, so not only replay but record does not work.
In one case I have no Content-Type header in the request, and new String(rawPostData) seems to break it, in the other case, I have a Content-Type: application/soap+msbin1 header, where "getEncodingFromContentType" returns null.

We really should make sure JMeter sends exactly what it received to the server when acting as proxy, and even better would be if this would work when executing the test.

For the former, the solution I found seems to be to change the line HttpRequestHdr: postData = new String(rawPostData);
to:
postData = new String(rawPostData, PostWriter.ENCODING);
to match the encoding which is used when executing the request in case no encoding is set.

I also have a problem with the line 
if (firstLine && !CharUtils.isAscii((char) x)){
which I can solve by commenting this out...
Comment 3 Philippe Mouawad 2012-01-25 12:32:18 UTC
Hello Jens,
Thanks for your detailed description and investigation.

Is the site you are testing against a public one ?
If not would it be possible for you to capture your scenario with FIDDLER tool and make a Word document with screenshots of the Fiddler record. 
By having the 2 we will be able to reproduce and fix the issue.

Thank you
Regards
Philippe
Comment 4 jens_0 2012-01-30 07:19:10 UTC
The site is not a public one, but one from a customer and I don't have access any more, I'm afraid.

Probably it's enough to test if you create a http server returning some constant random byte garbage and a http client sending random bytes as post data, and checking whether there's a difference when you put JMeter as proxy in between.
As long as the Java VMs default encoding is not equal to PostWriter.ENCODING, you'll have a difference.
Comment 5 Philippe Mouawad 2012-02-06 06:52:49 UTC
(In reply to comment #2)
> I can confirm this.
> 
> when recording HTTP data with JMeter, I'd assume that the request is sent to
> the server just like the browser would send it.
> This is not the case in two examples I've run across.
> 
> As you'll know, when recording with HTTP Proxy Server, JMeter will be creating
> HTTP Samplers and then use their sample method to realize the request,
> returning the return value to the browser.
> So we need to create a HTTP Sampler which produces the same request as the
> browser would have done.
> 
> When looking at HttpRequestHdr.populateSampler, I see that
> ConversionUtils.getEncodingFromContentType
> is called. In case the encoding is not supported (returns null) we look in the
> pageEncodings and formEncodings maps (when and how are they populated??) and if
> we find nothing here, we use:
> postData = new String(rawPostData);
> meaning we convert the bytes to a string based on the platform-specific default
> encoding.
> 
> In my examples this breaks the communication, so not only replay but record
> does not work.
> In one case I have no Content-Type header in the request, and new
> String(rawPostData) seems to break it, in the other case, I have a
> Content-Type: application/soap+msbin1 header, where
> "getEncodingFromContentType" returns null.
> 
> We really should make sure JMeter sends exactly what it received to the server
> when acting as proxy, and even better would be if this would work when
> executing the test.
> 
> For the former, the solution I found seems to be to change the line
> HttpRequestHdr: postData = new String(rawPostData);
> to:
> postData = new String(rawPostData, PostWriter.ENCODING);
> to match the encoding which is used when executing the request in case no
> encoding is set.
> 
This seems reasonable, as indeed Postwriter uses ENCODING const when encoding is null.
> I also have a problem with the line 
> if (firstLine && !CharUtils.isAscii((char) x)){
> which I can solve by commenting this out...
Regarding this, would it be possible to attach to Bugzilla à fiddler capture of the failing request.
CAN you give more information about the tested app, is it a binary soap webservice ?
Comment 6 jens_0 2012-02-08 15:42:31 UTC
It was another app than the other problem, but same here no access any more.
I can confirm a binary SOAP webservice was involved though I do not remember whether it was this request. The char in question was the German umlaut "ü".
Comment 7 Philippe Mouawad 2012-06-27 21:17:24 UTC
This issue has been fixed as part of 52674 as of using :
new String(request.getRawPostData(), PostWriter.ENCODING);
when contentEncoding is not found.

Regarding  binary soap webservice , as it's a binary protocol recording would require implementing a dedicated SamplerCreator associated to mime type.

*** This bug has been marked as a duplicate of bug 52674 ***