Bug 62861

(In reply to Abhishek Rao from comment #0)
> Hi,
> 
> I have a script with 5 http requests which upon executing for 10 iterations
> with keep-alive enabled for all requests, and following are my observations:
> 1. httpclient.reset_state_on_thread_group_iteration=true
> a. With 1000ms think time per iteration
> Expected Output: Connection per iteration
> Actual Output: A total of 10 connections are created, and each connection
> was iteration dependent.
> b. No think time throughout the iteration
> Expected Output: Connection per iteration
> Actual Output: A total of 10 connections are created, and each connection
> was iteration dependent.

This one looks ok, right ?

> 
> 2. httpclient.reset_state_on_thread_group_iteration=false
> a. With 1000ms think time per iteration
> Expected Output: Connection per thread, shared across multiple iterations
> Actual Output: Network inspection shows that 5 connections were created in
> total, with an interval of ~2seconds before the client (JMeter) sent a
> FIN-ACK and closed the connection, which isn't the actual "full" reuse that
> we're looking for with this feature. 
> b. No think time throughout the iteration
> Expected Output: Connection per thread, shared across multiple iterations
> Actual Output: Only one connection was created and used across all the
> iterations.
> 
> It seems though some underlying TCP configuration is being honoured, which
> is causing the client to create a new connection every 2 seconds after ~1+
> seconds of inactivity due to our think time in the script. Is there a way to
> work around this and ensure full reuse of connection? I'm not sure whether
> this is a JMeter bug in the first place.

Have a look at these 3 properties in http://jmeter.apache.org/usermanual/properties_reference.html:
httpclient4.idletimeout=0
httpclient4.validate_after_inactivity=1700
httpclient4.time_to_live=2000

Hi Philippe,

The first one is okay. 

The configuration you mentioned works, but how do I achieve pooling and indefinite reuse rather than reusing for an arbitrary time period? For example: I have scripts acting as clients using pooled connection architectures to talk to web-services internally where this feature will be really useful in simulating. Can this be a UI modifiable in future and parametric?

Also, how do I achieve hybrid simulations with one thread group using httpclient.reset_state_on_thread_group_iteration=true and one that needs it set to false?

(In reply to Abhishek Rao from comment #2)
> Hi Philippe,
> 
> The first one is okay. 
> 
> The configuration you mentioned works, but how do I achieve pooling and
> indefinite reuse rather than reusing for an arbitrary time period? For
> example: I have scripts acting as clients using pooled connection
> architectures to talk to web-services internally where this feature will be
> really useful in simulating. 

What if you set:
httpclient4.idletimeout=0
httpclient4.validate_after_inactivity=1700
httpclient4.time_to_live=Very high value


> Can this be a UI modifiable in future and
> parametric?
> Also, how do I achieve hybrid simulations with one thread group using
> httpclient.reset_state_on_thread_group_iteration=true and one that needs it
> set to false?

This would be an enhancement request.
Workaround is to use distributed testing with 1 JVM in first mode and second JVM in second mode.

(In reply to Philippe Mouawad from comment #3)
> (In reply to Abhishek Rao from comment #2)
> > Hi Philippe,
> > 
> > The first one is okay. 
> > 
> > The configuration you mentioned works, but how do I achieve pooling and
> > indefinite reuse rather than reusing for an arbitrary time period? For
> > example: I have scripts acting as clients using pooled connection
> > architectures to talk to web-services internally where this feature will be
> > really useful in simulating. 
> 
> What if you set:
> httpclient4.idletimeout=0
> httpclient4.validate_after_inactivity=1700
> httpclient4.time_to_live=Very high value
> 
> 
> > Can this be a UI modifiable in future and
> > parametric?
> > Also, how do I achieve hybrid simulations with one thread group using
> > httpclient.reset_state_on_thread_group_iteration=true and one that needs it
> > set to false?
> 
> This would be an enhancement request.
> Workaround is to use distributed testing with 1 JVM in first mode and second
> JVM in second mode.


I dont think the workaround would apply for my scenario, because there's two thread groups in the same script and each requires a different setting. Distributed mode with different settings across JVMs would mean both the thread groups are affected, when the scenario calls for just one thread group to have the setting reflected. I think I'll split it into two tests for now and run them in parallel. I think the aforementioned will make for a great enhancement, I also found out that HC4 supports PooledConnectionManager which solves the problem.

(In reply to Abhishek Rao from comment #4)
> (In reply to Philippe Mouawad from comment #3)
> > (In reply to Abhishek Rao from comment #2)
> > > Hi Philippe,
> > > 
> > > The first one is okay. 
> > > 
> > > The configuration you mentioned works, but how do I achieve pooling and
> > > indefinite reuse rather than reusing for an arbitrary time period? For
> > > example: I have scripts acting as clients using pooled connection
> > > architectures to talk to web-services internally where this feature will be
> > > really useful in simulating. 
> > 
> > What if you set:
> > httpclient4.idletimeout=0
> > httpclient4.validate_after_inactivity=1700
> > httpclient4.time_to_live=Very high value
> > 
> > 
> > > Can this be a UI modifiable in future and
> > > parametric?
> > > Also, how do I achieve hybrid simulations with one thread group using
> > > httpclient.reset_state_on_thread_group_iteration=true and one that needs it
> > > set to false?
> > 
> > This would be an enhancement request.
> > Workaround is to use distributed testing with 1 JVM in first mode and second
> > JVM in second mode.
> 
> 
> I dont think the workaround would apply for my scenario, because there's two
> thread groups in the same script and each requires a different setting.
> Distributed mode with different settings across JVMs would mean both the
> thread groups are affected, when the scenario calls for just one thread
> group to have the setting reflected. I think I'll split it into two tests
> for now and run them in parallel. I think the aforementioned will make for a
> great enhancement, I also found out that HC4 supports
> PooledConnectionManager which solves the problem.

What about my first note ?

Regarding workaround, just do the following:

- use for  threads field of each Thread Group  : 
   ${__P(threads_TG1,)}
   ${__P(threads_TG2,)}

Then run first jmeter-server with:

-Jthreads_TG1=X -Jthreads_TG2=0

And second with:

-Jthreads_TG1=0 -Jthreads_TG2=Y

I think it will work

Hello,
Any feedback on answer ?

Thanks

(In reply to Philippe Mouawad from comment #6)
> Hello,
> Any feedback on answer ?
> 
> Thanks

Hi Philippe

Thanks for the work around. It's doing what we expected. :)

Abhishek

Great, can you please answer my other questions so that I know if I can requalify bug to enhancement ?

Thanks

Requalifying to enhancement.

Note there is a relation with this topic:
http://mail-archives.apache.org/mod_mbox/jmeter-dev/201811.mbox/%3CCAH9fUpau-3YWao86yU2DhM6mCAwssa61Wqew_iMYtFbmrioRiQ%40mail.gmail.com%3E


@Reporter, it would be really nice from you (since I spent time answering and providing workaround ) to answer comment 3 and confirm provided information fixed issue 2 and it is now an enhancement request and not a bug.

(In reply to Philippe Mouawad from comment #9)
> Requalifying to enhancement.
> 
> Note there is a relation with this topic:
> http://mail-archives.apache.org/mod_mbox/jmeter-dev/201811.mbox/%3CCAH9fUpau-
> 3YWao86yU2DhM6mCAwssa61Wqew_iMYtFbmrioRiQ%40mail.gmail.com%3E
> 
> 
> @Reporter, it would be really nice from you (since I spent time answering
> and providing workaround ) to answer comment 3 and confirm provided
> information fixed issue 2 and it is now an enhancement request and not a bug.

Thought I confirmed in comment#7. Agree that this isn't necessarily a bug and just a behavior of setting TTL at HC level. It looks like we can go with the workaround for now:
a. Set TTL to desired value (long enough that connection persists throughout test) on specific jmeter-servers and leave to default for the rest
b. Configure run-time thread group enablement as mentioned in comment#5

Thanks for qualifying this as an enhancement. 

Quick question - will the enhancement be a PoolingHttpClientConnectionManager or httpclient.reset_state_on_thread_group_iteration just made threadgroup local and UI configurable?

PR available:

https://github.com/apache/jmeter/pull/470

This is an automated email from the ASF dual-hosted git repository.

pmouawad pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/jmeter.git


The following commit(s) were added to refs/heads/master by this push:
     new 629c643  This implements "Provide ability to configure whether a new iteration is a new user or same user" (BUG 62861) (#470)
629c643 is described below

commit 629c6430e17df7f7f75891d29e2129ab55a57d68
Author: Philippe M <pmouawad@users.noreply.github.com>
AuthorDate: Tue Jul 2 22:10:02 2019 +0200

    This implements "Provide ability to configure whether a new iteration is a new user or same user" (BUG 62861) (#470)

*** Bug 63535 has been marked as a duplicate of this bug. ***

@Philippe, was there a reason to implement "same user" vs "different user" as radio box?

It looks like a checkbox with label "Same user on each iteration" would be more consistent with other checkboxes.

WDYT?

(In reply to Vladimir Sitnikov from comment #14)
> @Philippe, was there a reason to implement "same user" vs "different user"
> as radio box?
> 
> It looks like a checkbox with label "Same user on each iteration" would be
> more consistent with other checkboxes.
> 
> WDYT?

We followed recommendations in UX:

https://www.nngroup.com/articles/checkboxes-vs-radio-buttons/

If you use a checkbox, are you sure users will be aware of what is done when "Same user on each iteration" is unchecked ?

Maybe it would be better to switch other controls to Radio buttons.

> We followed recommendations in UX:
> https://www.nngroup.com/articles/checkboxes-vs-radio-buttons/

Can you please clarify where is the recommendation to replace **single** checkbox with radio buttons?


Just in case, the article you mention starts with "Stay informed" sample with **two checkboxes**.

Then the very same article says that

> Ironically, with a single question, using a checkbox would have been correct

In other words, they recommend **single question using a checkbox**.


> are you sure users will be aware of
>what is done when "Same user on each iteration" is unchecked ?

1) If "same user" is checked, then the user is the same. If unchecked, then, "appareltly", the user is not the same.

2) Neither checkbox nor radio answer how "user" relates to a thread. For instance, what if I use TCP sampler. What "different user on each iteration" would mean? Nothing.

Frankly speaking, I'm not even sure the checkbox/radio should even be there, however it is already there, so we should at least try to reduce the damage.

I've realized "old" test plans are opened as "different user on each iteration".

Was it really expected?
By default JMeter creates "same user on each iteration".
However, if I take an old plan and open it with new version, then it treats the plan as "different user" which might impact users that don't even know of the feature.

This is an automated email from the ASF dual-hosted git repository.

pmouawad pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/jmeter.git


The following commit(s) were added to refs/heads/master by this push:
     new 538f8a2  Enhancement 62861: Fix old test plans opened as "different user on each iteration" while it should be "same user on each iteration"
538f8a2 is described below

commit 538f8a21ba1be380d35b5c47c137a54c099d7309
Author: pmouawad <p.mouawad@ubik-ingenierie.com>
AuthorDate: Fri Oct 4 22:02:03 2019 +0200

    Enhancement 62861: Fix old test plans opened as "different user on each
    iteration" while it should be "same user on each iteration"

    Also update release notes
    https://bz.apache.org/bugzilla/show_bug.cgi?id=62861
---
 .../java/org/apache/jmeter/threads/gui/ThreadGroupGui.java     | 10 ++++++----
 xdocs/changes.xml                                              |  1 +
 2 files changed, 7 insertions(+), 4 deletions(-)

(In reply to Vladimir Sitnikov from comment #17)
> I've realized "old" test plans are opened as "different user on each
> iteration".
> 
> Was it really expected?
> By default JMeter creates "same user on each iteration".
> However, if I take an old plan and open it with new version, then it treats
> the plan as "different user" which might impact users that don't even know
> of the feature.

Fixed, thanks for finding it

Did you just intentionally created a merge conflict with https://github.com/apache/jmeter/pull/513 ?

It is EXTREMELY sad that you create those merge conflicts for no reason.

On top of that I guess your fix is incomplete.

It does not fix "read jmx" part of the thing, so "UI is good, but actual test would go in a different way".

https://github.com/apache/jmeter/pull/513 has the proper fix, so why hassle?

(In reply to Vladimir Sitnikov from comment #20)
> Did you just intentionally created a merge conflict with
> https://github.com/apache/jmeter/pull/513 ?
> 
> It is EXTREMELY sad that you create those merge conflicts for no reason.

Vladimir, ABSOLUTELY NOT.
I fixed a bug you reported, for release preparation of tomorrow.

How can you think that ? 

We're working together in what I thought a gentle way in an OSS Team, it's not a war of EGO here...

(In reply to Vladimir Sitnikov from comment #21)
> On top of that I guess your fix is incomplete.
> 
> It does not fix "read jmx" part of the thing, so "UI is good, but actual
> test would go in a different way".
> 
> https://github.com/apache/jmeter/pull/513 has the proper fix, so why hassle?

I tested:
- reading old JMX
- creating new one , saving , reloading

I didn't find an issue, I may have missed something as I am stupid, feel free to fix it

@Philippe, would you please be so kind to add a unit test that would ensure old test plans are read appropriately?

This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/4905