Bug 64367 - UDV defined in test plan failed to be expanded in distributed testing mode
Summary: UDV defined in test plan failed to be expanded in distributed testing mode
Status: NEW
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 5.2.1
Hardware: PC All
: P2 normal (vote)
Target Milestone: JMETER_5.3.0
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-20 03:48 UTC by Owen
Modified: 2020-05-01 11:46 UTC (History)
1 user (show)



Attachments
test plan (8.40 KB, application/xml)
2020-04-20 03:48 UTC, Owen
Details
jmeter.log (68.22 KB, application/x-zip-compressed)
2020-04-21 08:30 UTC, Owen
Details
jmeter log files on node1 (6.88 KB, application/x-gzip)
2020-04-22 08:36 UTC, Owen
Details
jmeter log files on node2 (967 bytes, application/x-gzip)
2020-04-22 08:36 UTC, Owen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Owen 2020-04-20 03:48:43 UTC
Created attachment 37184 [details]
test plan

I defined a UDV in root element of the test plan as follows:
name:  BASE_FILENAME    
value:  ${__TestPlanName}_${__P(NUM_THREAD,0)}threads_${__P(NUM_LOOP,0)}trans_${__P(TPS_MIN,0)}tpm_${__Random(0,10000)}

There is a "Simple Data Writer" listener in the test plan which refer to the variable as a file name which saved requests and responses, I ran the test with following commands:
//local mode
jmeter.bat -n -t http_bing.jmx -l http_bing.csv -JNUM_THREAD=2 -JNUM_LOOP=5 -JTPS_MIN=1800 -JTIME_RAMPUP=1 
//distributed mode, -R option follows two IP address of JMeter-server
jmeter.bat -n -t http_bing.jmx -l http_bing.csv -R "172.17.96.4,172.17.119.6" -GNUM_THREAD=2 -GNUM_LOOP=5 -GTPS_MIN=1800 -GTIME_RAMPUP=1 

I found that the variable failed to be expanded in distributed testing mode and it produced a result file "${__TestPlanName}_${__P(NUM_THREAD,0)}threads_${__P(NUM_LOOP,0)}trans_${__P(TPS_MIN,0)}tpm_${__Random(0,10000)}.xml", but it was expanded successfully in local mode testing and produced a file http_bing.jmx_2threads_5trans_1800tpm_807.xml

I guess it may a bug in variable expansion.

JMeter version: 2.5.1
Java version: 1.8.0_161
OS version: windows 10
JMX file: see attachment http_bing.jmx
Comment 1 Felix Schumacher 2020-04-20 04:58:32 UTC
Hi Owen, 

thanks for the clear description of your problem, however I think this question would have been best asked on the users mailing list (see https://jmeter.apache.org/mail2.html#JMeterUser for more info).

Parameters that are global (i.e. seen on on all cluster members) should be given with -G instead of -J. That is described in https://jmeter.apache.org/usermanual/remote-test.html

Why did you use -J? Would you like to contribute changes to the documentation, that make this clearer?

Feel free to close this bug, if -G helped to solve your problem.

PS. Another thing (that you didn't explicitly asked for): the usage of __Random should have worked in distributed mode, but you are probably better of using a node identifier like the function __machineName.
Comment 2 Owen 2020-04-20 05:51:40 UTC
(In reply to Felix Schumacher from comment #1)
> Hi Owen, 
> 
> thanks for the clear description of your problem, however I think this
> question would have been best asked on the users mailing list (see
> https://jmeter.apache.org/mail2.html#JMeterUser for more info).
> 
> Parameters that are global (i.e. seen on on all cluster members) should be
> given with -G instead of -J. That is described in
> https://jmeter.apache.org/usermanual/remote-test.html
> 
> Why did you use -J? Would you like to contribute changes to the
> documentation, that make this clearer?
> 
> Feel free to close this bug, if -G helped to solve your problem.
> 
> PS. Another thing (that you didn't explicitly asked for): the usage of
> __Random should have worked in distributed mode, but you are probably better
> of using a node identifier like the function __machineName.

Hi Felix,

Thanks for your quick reply.

Indeed I used the -J option in local running mode and -G option in distributed testing mode.
Comment 3 Owen 2020-04-20 05:58:02 UTC
(In reply to Owen from comment #0)
> Created attachment 37184 [details]
> test plan
> 
> I defined a UDV in root element of the test plan as follows:
> name:  BASE_FILENAME    
> value: 
> ${__TestPlanName}_${__P(NUM_THREAD,0)}threads_${__P(NUM_LOOP,
> 0)}trans_${__P(TPS_MIN,0)}tpm_${__Random(0,10000)}
> 
> There is a "Simple Data Writer" listener in the test plan which refer to the
> variable as a file name which saved requests and responses, I ran the test
> with following commands:
> //local mode
> jmeter.bat -n -t http_bing.jmx -l http_bing.csv -JNUM_THREAD=2 -JNUM_LOOP=5
> -JTPS_MIN=1800 -JTIME_RAMPUP=1 
> //distributed mode, -R option follows two IP address of JMeter-server
> jmeter.bat -n -t http_bing.jmx -l http_bing.csv -R
> "172.17.96.4,172.17.119.6" -GNUM_THREAD=2 -GNUM_LOOP=5 -GTPS_MIN=1800
> -GTIME_RAMPUP=1 
> 
> I found that the variable failed to be expanded in distributed testing mode
> and it produced a result file
> "${__TestPlanName}_${__P(NUM_THREAD,0)}threads_${__P(NUM_LOOP,
> 0)}trans_${__P(TPS_MIN,0)}tpm_${__Random(0,10000)}.xml", but it was expanded
> successfully in local mode testing and produced a file
> http_bing.jmx_2threads_5trans_1800tpm_807.xml
> 
> I guess it may a bug in variable expansion.
> 
> JMeter version: 2.5.1
> Java version: 1.8.0_161
> OS version: windows 10
> JMX file: see attachment http_bing.jmx

JMeter version should be 5.2.1
Comment 4 Felix Schumacher 2020-04-20 10:47:34 UTC
Are there any log messages on any of the JMeter nodes?
Comment 5 Owen 2020-04-21 08:28:17 UTC
(In reply to Felix Schumacher from comment #4)
> Are there any log messages on any of the JMeter nodes?

I ran the test with DEBUG log level and checked JMeter logs, no valuable info was found. you can search the file with the command "grep NUM_THREAD jmeter.log".

I ran the distributed testing on two containers in K8S with Debian 10 and openjdk version "11.0.6".

please refer to the attachment jmeter.log
Comment 6 Owen 2020-04-21 08:30:44 UTC
Created attachment 37186 [details]
jmeter.log
Comment 7 Felix Schumacher 2020-04-21 10:40:50 UTC
Thanks for the log file, are there any log files on the remote nodes?

Another thing I noticed in the log file. You seem to have a lot of jar files placed under /opt/jmeter_dependencies. Where do they come from? Do you really need them?

There are a lot of ClassNotFoundExceptions in the log which probably are OK, but I would try to get rid of them by adding the needed dependencies.

Interesting bit is 

08:14:32.066 [RMI TCP Connection(2)-172.17.119.16] DEBUG org.apache.jmeter.reporters.ResultCollector - Getting file: ${__TestPlanName}_${__P(NUM_THREAD,0)}threads_${__P(NUM_LOOP,0)}trans_${__P(TPS_MIN,0)}tpm_${__Random(0,10000)}.xml in thread RMI TCP Connection(2)-172.17.119.16
08:14:32.067 [RMI TCP Connection(2)-172.17.119.16] DEBUG org.apache.jmeter.reporters.ResultCollector - Getting file: http_bing_3threads_20trans_300tps.5812.csv in thread RMI TCP Connection(2)-172.17.119.16

The second line seem to have the values replaced. (I haven't looked further into this, yet)
Comment 8 Owen 2020-04-21 13:03:20 UTC
Jar files under /opt/jmeter_dependencies are JMeter plugins and their dependencies, they are parts of my testing project, those jar files are built into the docker image, it should be OK as I ran the same test with the same docker image in local running mode successfully(UDV can be expanded successfully).

I think variable expansion was failed, the variable definition is ended with ".xml", instead of ".csv", the ".csv" file is JMeter result file which contains result of each sample, the name of result file is passed to JMeter process from command line argument which is constructed by concatenation of shell variables.(I simplified the command line arguments in the description of this ticket)

I will attach log files on remote nodes later.

Thanks.
Comment 9 Owen 2020-04-22 08:36:06 UTC
Created attachment 37193 [details]
jmeter log files on node1
Comment 10 Owen 2020-04-22 08:36:38 UTC
Created attachment 37194 [details]
jmeter log files on node2
Comment 11 Owen 2020-04-22 08:42:29 UTC
Comment on attachment 37193 [details]
jmeter log files on node1

jmeter-server.log can only be displayed by the command "cat", other text viewer/editor may fail to open it. I don't know why.
Comment 12 Owen 2020-04-22 08:47:53 UTC
node1.tar.gz and node2.tar.gz contain the log files on two remote nodes, one of them also was used to execute the command line for distributed testing.

you can see the variable expansion was failed

08:24:05.690 [main] DEBUG org.apache.jmeter.engine.util.ValueReplacer - About to replace in property of type: class org.apache.jmeter.testelement.property.StringProperty: ${BASE_FILENAME}.xml
08:24:05.692 [main] DEBUG org.apache.jmeter.testelement.property.AbstractProperty - Not running version, return raw function string
08:24:05.692 [main] DEBUG org.apache.jmeter.engine.util.ValueReplacer - Replacement result: ${BASE_FILENAME}.xml
08:24:05.692 [main] DEBUG org.apache.jmeter.engine.util.ValueReplacer - About to replace in property of type: class org.apache.jmeter.testelement.property.BooleanProperty: false 

PS: the file jmeter-server.log in the tarball can only be displayed by the command "cat", other text viewer/editor can't display it normally.
Comment 13 Felix Schumacher 2020-05-01 11:46:34 UTC
I can reproduce the issue.

The problem seems to be, that the BASE_FILENAME property used in the is Simple Data Writer is evaluated on the master (not on the controlled client). But the master is not set to "RUNNING". Therefore the property is not evaluated and we get the raw string back. The corresponding log line is:

08:24:05.692 [main] DEBUG org.apache.jmeter.testelement.property.AbstractProperty - Not running version, return raw function string

I am unsure, what to do about it.