Bug 61375 - http proxy server request fails when DNS manager custom resolver enabled
Summary: http proxy server request fails when DNS manager custom resolver enabled
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: Nightly (Please specify date)
Hardware: Macintosh All
: P2 normal (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-08-02 18:47 UTC by JJ Welch
Modified: 2017-09-02 11:08 UTC (History)
0 users



Attachments
jmeter test I used to demonstrate the defect (6.79 KB, application/xml)
2017-08-02 18:48 UTC, JJ Welch
Details
Use all custom resolvers, when more than one is given (3.96 KB, patch)
2017-08-02 20:40 UTC, Felix Schumacher
Details | Diff
try system dns as last resort (2.91 KB, patch)
2017-08-05 15:08 UTC, Felix Schumacher
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description JJ Welch 2017-08-02 18:47:29 UTC
When using a custom DNS resolver and running through a proxy, jmeter cannot connect to the host. Jmeter reports it cannot resolved the hostname of both the end server and the proxy server. I checked that this works in jmeter 2.13 and fails in both 3.2 and the nightly build Version r1803417.

Switching the DNS manager to "Use system DNS resolver" causes jmeter to work normally and route the request through the proxy successfully.

Test Setup:
DNS Manager: Clear cache each iteration: ON
Use custom DNS resolver: ON

Attached are both the JMX I used and the jmeter log it generated at DEBUG level. In the log are 10 iterations that pass and 10 iterations that fail. 
Pass = "Use system DNS resolver"
Fail = "Use custom DNS resolver"
Comment 1 JJ Welch 2017-08-02 18:48:19 UTC
Created attachment 35197 [details]
jmeter test I used to demonstrate the defect
Comment 2 Felix Schumacher 2017-08-02 20:40:46 UTC
Created attachment 35198 [details]
Use all custom resolvers, when more than one is given

I find it strange, that only the first custom resolver get tried. So I added a loop to try them all.

My setup used localhost as a proxy and that lead to the reported bug. My first solution was to add localhost as a static host entry, but that let another bug surface ( a NPE, which I fought with a null-guard in the patch).

I am not too happy with this patch, so I would be glad, if someone would take a look at the code. I think it is not completely refactored from side-effect-using code to side-effect-free code.
Comment 3 Felix Schumacher 2017-08-05 15:08:17 UTC
I think my patch doesn't help. We need more information on your setup to really reproduce your problem.

Where is your proxy located? Do you see any exceptions in the log files or in the responses?

My setup used localhost as a proxy and the custom resolver tried to resolve "localhost" using those. But those dns servers don't resolve "localhost". That is normally done by the hosts file from your computer.

So my solution was (and is) the addition of a static host resolution "localhost" to the static host table. That led to another failure as the "ip" of the static host entry was tried to be resolved.

I will attach another patch, which is hopefully a bit more correct. But it introduces a lookup with the system resolver as a last resort. That is new behaviour and could introduce other problems (which at the moment I can't think of).
Comment 4 Felix Schumacher 2017-08-05 15:08:52 UTC
Created attachment 35201 [details]
try system dns as last resort
Comment 5 JJ Welch 2017-08-09 21:06:59 UTC
I use the software package called "Charles Proxy", www.charlesproxy.com . I could try this with Fiddler on a windows machine as well.

I was thinking that there could be a logical test in the code that skips DNS when the proxy is an IP address instead of a name. Not sure how hard that would be to write a method for "isAnIP" - the regex pattern for that might introduce some other unexpected behavior or edge cases.

I do have an entry in /etc/hosts for localhost as 127.0.0.1 so I am surprised that there was an issue when I tried localhost as the proxy server name.

What was different in jmeter 2.x that allowed the proxy and the dns manager to work so well together?
Comment 6 Felix Schumacher 2017-08-29 12:13:25 UTC
(In reply to JJ Welch from comment #5)
> I use the software package called "Charles Proxy", www.charlesproxy.com . I
> could try this with Fiddler on a windows machine as well.
> 
> I was thinking that there could be a logical test in the code that skips DNS
> when the proxy is an IP address instead of a name. Not sure how hard that
> would be to write a method for "isAnIP" - the regex pattern for that might
> introduce some other unexpected behavior or edge cases.
> 
> I do have an entry in /etc/hosts for localhost as 127.0.0.1 so I am
> surprised that there was an issue when I tried localhost as the proxy server
> name.
> 
> What was different in jmeter 2.x that allowed the proxy and the dns manager
> to work so well together?

The change was introduced with bug 57447, where it was explicitly requested to ignore the systems normal dns configuration.

I think we could re-enable the usage for the static host entries as a last resort.

On a different subject. Why did you put so many dns servers in your resolver list?
Comment 7 Felix Schumacher 2017-09-02 11:08:15 UTC
Fixed in version 3.3.

Date: Sat Sep  2 11:04:06 2017
New Revision: 1807037

URL: http://svn.apache.org/viewvc?rev=1807037&view=rev
Log:
Use system DNS resolver as last resort, when resolving entries in the static host table.

Bugzilla Id: 61375

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/control/DNSCacheManager.java
    jmeter/trunk/test/src/org/apache/jmeter/protocol/http/control/TestDNSCacheManager.java
    jmeter/trunk/xdocs/changes.xml