Bug 60607

Summary:	DNS Cache Manager configuration is ignored
Product:	JMeter - Now in Github	Reporter:	Philippe Mouawad <p.mouawad>
Component:	HTTP	Assignee:	JMeter issues mailing list <issues>
Status:	RESOLVED FIXED
Severity:	regression	CC:	p.mouawad, s16260063
Priority:	P2
Version:	3.0
Target Milestone:	---
Hardware:	All
OS:	All
Attachments:	Test Plan showing issue

Description Philippe Mouawad 2017-01-19 17:01:49 UTC

Created attachment 34648 [details]
Test Plan showing issue

As reported in Bug 57447 by Sean Chang, a regression has been introduced in version 3.0 of JMeter related to migration to HttpCLient 4.5.2

Comment 1 Philippe Mouawad 2017-01-19 20:41:34 UTC

Author: pmouawad
Date: Thu Jan 19 17:11:07 2017
New Revision: 1779498

URL: http://svn.apache.org/viewvc?rev=1779498&view=rev
Log:
Bug 60607 - DNS Cache Manager configuration is ignored
Bugzilla Id: 60607

Added:
    jmeter/trunk/bin/testfiles/Bug60607.csv
    jmeter/trunk/bin/testfiles/Bug60607.jmx
    jmeter/trunk/bin/testfiles/Bug60607.xml   (with props)
Modified:
    jmeter/trunk/build.xml
    jmeter/trunk/src/protocol/http/org/apache/http/impl/conn/JMeterPoolingClientConnectionManager.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/MeasuringConnectionManager.java
    jmeter/trunk/xdocs/changes.xml

Comment 2 Philippe Mouawad 2017-01-19 20:42:56 UTC

@Sean,
Could you test nightly build ?:

-http://jmeter.apache.org/nightly.html

Thank you

Comment 3 Philippe Mouawad 2017-01-19 21:02:51 UTC

Author: pmouawad
Date: Thu Jan 19 21:02:25 2017
New Revision: 1779526

URL: http://svn.apache.org/viewvc?rev=1779526&view=rev
Log:
Bug 60607 - DNS Cache Manager configuration is ignored
Add test case for SystemDnsResolver
Bugzilla Id: 60607

Modified:
    jmeter/trunk/bin/testfiles/Bug60607.csv
    jmeter/trunk/bin/testfiles/Bug60607.jmx
    jmeter/trunk/bin/testfiles/Bug60607.xml

Comment 4 Philippe Mouawad 2017-01-22 18:31:12 UTC

Hello Sean,
Any feedback on fix ?
Thanks

Comment 5 Sean Chang 2017-01-24 05:06:32 UTC

(In reply to Philippe Mouawad from comment #4)
> Hello Sean,
> Any feedback on fix ?
> Thanks

Just tried r1779550. It works fine for DNS Cache Manager issue.
Thanks a lot for your fix!
Sorry for too late, having a fever for several days...

Comment 6 Philippe Mouawad 2017-01-24 19:31:26 UTC

(In reply to Sean Chang from comment #5)
> (In reply to Philippe Mouawad from comment #4)
> > Hello Sean,
> > Any feedback on fix ?
> > Thanks
> 
> Just tried r1779550. It works fine for DNS Cache Manager issue.
> Thanks a lot for your fix!
> Sorry for too late, having a fever for several days...

Thanks for your feedback .

Comment 7 Sean Chang 2017-01-26 04:11:18 UTC

Hi Philippe，

Sorry for one more question:

Can I just replace JMeter 3.0 with your jmeter/trunk/src/protocol/http/org/apache/http/impl/conn/JMeterPoolingClientConnectionManager.java and jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/MeasuringConnectionManager.java to make DNS Cache Manager work?

The nightly build seems to be not stable for request sending and print many warnings. I guess they are not related to DNS. I can use it to verify your fix but can not use it for load test.

Thanks.

Comment 8 Philippe Mouawad 2017-01-26 05:59:28 UTC

Hello, 
No you can't.
What are the warnings you see ?
Can you attach jmeter.log ?
Nighlty build of today should be stable if you downloaded it from jenkins.

Thanks

Comment 9 Sean Chang 2017-01-26 07:11:47 UTC

Some JMeter warnings like...

Jan 25, 2017 11:03:46 PM java.util.prefs.FileSystemPreferences checkLockFile0ErrorCode
WARNING: Could not lock User prefs.  Unix error code 24.
Jan 25, 2017 11:03:47 PM java.util.prefs.FileSystemPreferences syncWorld
WARNING: Couldn't flush user prefs: java.util.prefs.BackingStoreException: Couldn't get file lock.
Jan 25, 2017 11:07:46 PM java.util.prefs.FileSystemPreferences checkLockFile0ErrorCode
WARNING: Could not lock User prefs.  Unix error code 24.
Jan 25, 2017 11:07:46 PM java.util.prefs.FileSystemPreferences syncWorld
WARNING: Couldn't flush user prefs: java.util.prefs.BackingStoreException: Couldn't get file lock.

Comment 10 Sean Chang 2017-01-26 07:12:22 UTC

And some request failed due to file not found, but actually the file is there and succeed for some other requests.

java.io.FileNotFoundException: /home/cosben/cos_upload/2bill/200K.ts (Too many open files)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at org.apache.http.entity.FileEntity.getContent(FileEntity.java:88)
	at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sendEntityData(HTTPHC4Impl.java:1459)
	at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.handleMethod(HTTPHC4Impl.java:585)
	at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:399)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1176)
	at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1165)
	at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:473)
	at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:416)
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
	at java.lang.Thread.run(Thread.java:745)

Comment 11 Philippe Mouawad 2017-01-26 07:59:09 UTC

(In reply to Sean Chang from comment #9)
> Some JMeter warnings like...
> 
> Jan 25, 2017 11:03:46 PM java.util.prefs.FileSystemPreferences
> checkLockFile0ErrorCode
> WARNING: Could not lock User prefs.  Unix error code 24.
> Jan 25, 2017 11:03:47 PM java.util.prefs.FileSystemPreferences syncWorld
> WARNING: Couldn't flush user prefs: java.util.prefs.BackingStoreException:
> Couldn't get file lock.
> Jan 25, 2017 11:07:46 PM java.util.prefs.FileSystemPreferences
> checkLockFile0ErrorCode
> WARNING: Could not lock User prefs.  Unix error code 24.
> Jan 25, 2017 11:07:46 PM java.util.prefs.FileSystemPreferences syncWorld
> WARNING: Couldn't flush user prefs: java.util.prefs.BackingStoreException:
> Couldn't get file lock.

Hello,
This one is a java bug on linux systems, see:
http://jmeter.apache.org/changes.html => Known problems and workarounds

Comment 12 Philippe Mouawad 2017-01-26 08:02:55 UTC

(In reply to Sean Chang from comment #10)
> And some request failed due to file not found, but actually the file is
> there and succeed for some other requests.
> 
> java.io.FileNotFoundException: /home/cosben/cos_upload/2bill/200K.ts (Too
> many open files)
> 	at java.io.FileInputStream.open0(Native Method)
> 	at java.io.FileInputStream.open(FileInputStream.java:195)
> 	at java.io.FileInputStream.<init>(FileInputStream.java:138)
> 	at org.apache.http.entity.FileEntity.getContent(FileEntity.java:88)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> sendEntityData(HTTPHC4Impl.java:1459)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.handleMethod(HTTPHC4Impl.
> java:585)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:
> 399)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.
> sample(HTTPSamplerProxy.java:74)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.
> sample(HTTPSamplerBase.java:1176)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.
> sample(HTTPSamplerBase.java:1165)
> 	at
> org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.
> java:473)
> 	at
> org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:416)
> 	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
> 	at java.lang.Thread.run(Thread.java:745)

This does not mean the file is not here, it means you reach a linux limit on file descriptors.
If you don't hit this with the other version of JMeter then it could be a jmeter issue but it's not sure.
https://easyengine.io/tutorials/linux/increase-open-files-limit/


1/ Run ulimit -a with the user that runs jmeter .
2/ How many threads are you running ?
Did you compare versions of JMeter with exactly the same script and user.properties/jmeter.properties ?
Thank you

Comment 13 Sean Chang 2017-01-26 09:45:43 UTC

1)$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 386013
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

2)the running threads is 100

3)I did not compare any version, I just using latest nightly build from Jenkins and modify HEAP and NEW size. What kind of version do you want me to compare?

When I tried same test plan on 2.13, I did not remember we meet this issue.

Comment 14 Philippe Mouawad 2017-01-26 13:18:04 UTC

(In reply to Sean Chang from comment #13)
> 1)$ ulimit -a
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 386013
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 4096
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
> 
> 2)the running threads is 100
> 
> 3)I did not compare any version, I just using latest nightly build from
> Jenkins and modify HEAP and NEW size. What kind of version do you want me to
> compare?
> 
> When I tried same test plan on 2.13, I did not remember we meet this issue.

1/ Would you mind checking again on 2.13 ?
We just want to be sure there is no regression in 3.2
2/ Then run with nighlty build and do a 
lsof -p <pid of jmeter> 

Thank you

Comment 15 Felix Schumacher 2017-01-26 19:35:32 UTC

(In reply to Sean Chang from comment #10)
> And some request failed due to file not found, but actually the file is
> there and succeed for some other requests.
> 
> java.io.FileNotFoundException: /home/cosben/cos_upload/2bill/200K.ts (Too
> many open files)
> 	at java.io.FileInputStream.open0(Native Method)
> 	at java.io.FileInputStream.open(FileInputStream.java:195)
> 	at java.io.FileInputStream.<init>(FileInputStream.java:138)
> 	at org.apache.http.entity.FileEntity.getContent(FileEntity.java:88)
> 	at
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> sendEntityData(HTTPHC4Impl.java:1459)
> 	at

That should be fixed with https://bz.apache.org/bugzilla/show_bug.cgi?id=60652

Could you try the next nightly?

Thanks,
 Felix

Comment 16 Sean Chang 2017-01-27 02:36:28 UTC

Yes, did not see this problem on r1780487. Thanks Felix.

Comment 17 Sean Chang 2017-01-27 02:38:54 UTC

Hi Philippe,

Since this issue is fixed on latest nightly build as Felix's comments, I will not try 2.13 again.
Thanks for your help.

Comment 18 The ASF infrastructure team 2022-09-24 20:38:07 UTC

This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/4235