Bug 60018 - Timer : Add a factor to apply on pauses
Summary: Timer : Add a factor to apply on pauses
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 3.0
Hardware: All All
: P2 enhancement (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-18 16:12 UTC by UbikLoadPack support
Modified: 2016-12-23 22:01 UTC (History)
1 user (show)



Attachments
Patch implementing first part of the feature (7.51 KB, patch)
2016-09-02 21:15 UTC, UbikLoadPack support
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description UbikLoadPack support 2016-08-18 16:12:38 UTC
This bug is related partly to Bug 58943
Sometimes you have to adapt the timers values to be half or double what they are currently.

When you have configured timers at every place this can be cumbersome as you have to touch tens of Test Elements.

It would be nice to be able to say: Take current computed pauses and multiply them by a factor.

Any other idea is welcome

If you agree to this idea, we can submit a patch.

Regards
@ubikloadpack
Comment 1 Sebb 2016-08-18 20:58:03 UTC
I'm not sure that I see a genuine use case for this. Why would every single pause have to be doubled? That does not seem like something a genuine user would do.

Most (all?) the timers can take variables, so it's possible to implement this anyway.
Comment 2 UbikLoadPack support 2016-08-18 21:07:11 UTC
The use case is testing different throughputs easily.

This frequently happens at the beginning of a campaign when you're trying to find the good think times to reach some load and you don't want to increase threads.
Frankly I believe it's a very frequent use case.


Yes it can be done by using __BeanShell or __groovy to use a property containing factor but whenever you have different think times between the samplers (We use TestAction with pause time at 0 and its child is a Timer) in a realistic test plan, it becomes a lot of work and makes test plan harder to maintain.

It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a matter of adding a property:
- thinktime_factor (for example)


Regards
Comment 3 benoit.wiart 2016-08-18 21:12:39 UTC
As a "genuine" user that's something I frequently do.

I also think timers should be moved out of the sampler and become "first level" elements.
Comment 4 Sebb 2016-08-18 21:17:15 UTC
(In reply to UbikLoadPack support from comment #2)
> The use case is testing different throughputs easily.
> 
> This frequently happens at the beginning of a campaign when you're trying to
> find the good think times to reach some load and you don't want to increase
> threads.
> Frankly I believe it's a very frequent use case.
> 
> 
> Yes it can be done by using __BeanShell or __groovy to use a property
> containing factor but whenever you have different think times between the
> samplers (We use TestAction with pause time at 0 and its child is a Timer)
> in a realistic test plan, it becomes a lot of work and makes test plan
> harder to maintain.
> 
> It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
> matter of adding a property:
> - thinktime_factor (for example)

Also documentation, testing.

Every additional function increases user support costs somewhat.


> 
> Regards
Comment 5 Philippe Mouawad 2016-08-18 21:25:52 UTC
(In reply to Sebb from comment #4)
> (In reply to UbikLoadPack support from comment #2)
> > The use case is testing different throughputs easily.
> > 
> > This frequently happens at the beginning of a campaign when you're trying to
> > find the good think times to reach some load and you don't want to increase
> > threads.
> > Frankly I believe it's a very frequent use case.
> > 
> > 
> > Yes it can be done by using __BeanShell or __groovy to use a property
> > containing factor but whenever you have different think times between the
> > samplers (We use TestAction with pause time at 0 and its child is a Timer)
> > in a realistic test plan, it becomes a lot of work and makes test plan
> > harder to maintain.
> > 
> > It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
> > matter of adding a property:
> > - thinktime_factor (for example)
> 
> Also documentation, testing.
> 
> Every additional function increases user support costs somewhat.

If this contributor contributes the full feature that they find useful, why would be refuse it ?

I don't share the reasoning regarding additional function.
If we follow it then we would stop any development on JMeter ?






> 
> 
> > 
> > Regards
Comment 6 Sebb 2016-08-19 08:26:18 UTC
(In reply to Philippe Mouawad from comment #5)
> (In reply to Sebb from comment #4)
> > (In reply to UbikLoadPack support from comment #2)
> > > It's not a big piece of code (1 line in JMeterThread#delay ) and it's only a
> > > matter of adding a property:
> > > - thinktime_factor (for example)
> > 
> > Also documentation, testing.
> > 
> > Every additional function increases user support costs somewhat.
> 

> I don't share the reasoning regarding additional function.

I was just pointing out that the amount of work involved is more than you wrote.

When deciding whether to implement a new feature, it's important to do a cost-benefit analysis. Ignoring some of the cost invalidates the analysis.

> If we follow it then we would stop any development on JMeter ?

That does not follow from what I wrote.
Comment 7 Vladimir Sitnikov 2016-08-19 17:33:58 UTC
ULP> This frequently happens at the beginning of a campaign when you're trying to find the good think times to reach some load and you don't want to increase threads.
ULP> Frankly I believe it's a very frequent use case.

I think the following approach should be used:
1) Use "Constant Throughput Timer" or alike to achieve the target load (number of iterations per hour)
2) Set "number of threads" == "number of callcenter operators" (or other kind of persons working with the system).
3) For open system (that is the case like "anybody from the internet can access the system" where the number of users is virtually unlimited), set "number of threads" == "expected maximal number of concurrent requests".

ULP>reach some load and you don't want to increase threads.

Suppose we are dealing with callcenter of 20 operators. We specify 20 threads.
If that is not enough to reach the target load, that just means "application is not responsive enough for _real_ users to reach the desired load".
It is just a valid finding: "not enough _operators_ to drive the load".
It is not something to be "cured" by "artificially decreasing think times".

Does it make sense?
Comment 8 UbikLoadPack support 2016-08-27 23:31:23 UTC
(In reply to Vladimir Sitnikov from comment #7)
> ULP> This frequently happens at the beginning of a campaign when you're
> trying to find the good think times to reach some load and you don't want to
> increase threads.
> ULP> Frankly I believe it's a very frequent use case.
> 
> I think the following approach should be used:
> 1) Use "Constant Throughput Timer" or alike to achieve the target load
> (number of iterations per hour)

If you read http://jmeter.apache.org/usermanual/component_reference.html#Constant_Throughput_Timer, 

Of course the throughput will be lower if the server is not capable of handling it, or if >>>>other timers<<<< or time-consuming test elements prevent it.



> 2) Set "number of threads" == "number of callcenter operators" (or other
> kind of persons working with the system).
> 3) For open system (that is the case like "anybody from the internet can
> access the system" where the number of users is virtually unlimited), set
> "number of threads" == "expected maximal number of concurrent requests".
> 
> ULP>reach some load and you don't want to increase threads.
> 
> Suppose we are dealing with callcenter of 20 operators. We specify 20
> threads.
> If that is not enough to reach the target load, that just means "application
> is not responsive enough for _real_ users to reach the desired load".
> It is just a valid finding: "not enough _operators_ to drive the load".
> It is not something to be "cured" by "artificially decreasing think times".
> 
> Does it make sense?

Yes.
But if may be not enough because for example in an E-Commerce load test, you can use an approach where you model the average visit.
You are given pause times , you configure them, you compute a target number of Threads.
You make a first run and see that you don't reach the number of visits expected in an hour, that might be due to response times, but that can also be due to wrong Pauses.

Isn't it interesting to be able to validate the hypothesis by multiplying all pause times (except CTP (read below proposition)) by a double (not int) factor and test again ? instead of having to do that in scripting.


We could implement it this way:
- Either we introduce a TimerConfig element or just a property
- We add a method in Timer interface canBeMultiplied (a better name is to be found) , if it returns true, we can apply the factor, if not we use timer delay as is.
- CTT would implement the method by returning false
- Other timers would implement the method by returning true


Also, as noted by Antonio Gomes Rodrigues in mailing list, this feature exists in alternative products in the same field, so it must be useful no ?
At least in our experience it would be.


Regards
Comment 9 UbikLoadPack support 2016-09-02 21:15:35 UTC
Created attachment 34195 [details]
Patch implementing first part of the feature

Hello,
Find attached a patch implementing the following:
- Add a method on Timer interface that allows it to tell wether a ratio can be applied on it
- Implement it in subclasses
- Add a new property timer.factor 
- Apply factor in JMeterThread

The patch is already usable like this and it was very useful during a load testing campaign this week to adjust timer and reach a behaviour similar to production.

Next step can be:
- Add a new Timer Config element that controls timer.factor
- Add a checkbox on all Timer GUIs to enable them to refuse adjustment

Regards
UbikLoadPack Team
Comment 10 Philippe Mouawad 2016-09-20 12:31:25 UTC
Author: pmouawad
Date: Tue Sep 20 12:28:37 2016
New Revision: 1761563

URL: http://svn.apache.org/viewvc?rev=1761563&view=rev
Log:
Bug 60018 - Timer : Add a factor to apply on pauses
Bugzilla Id: 60018

Added:
    jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java   (with props)
Modified:
    jmeter/trunk/bin/jmeter.properties
    jmeter/trunk/src/components/org/apache/jmeter/timers/RandomTimer.java
    jmeter/trunk/src/core/org/apache/jmeter/threads/JMeterThread.java
    jmeter/trunk/src/core/org/apache/jmeter/util/JMeterUtils.java
    jmeter/trunk/xdocs/changes.xml
    jmeter/trunk/xdocs/usermanual/component_reference.xml

Author: pmouawad
Date: Tue Sep 20 12:30:19 2016
New Revision: 1761564

URL: http://svn.apache.org/viewvc?rev=1761564&view=rev
Log:
Bug 60018 - Timer : Add a factor to apply on pauses
eol
Bugzilla Id: 60018

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java   (props changed)
Comment 11 Philippe Mouawad 2016-09-25 16:35:19 UTC
Author: pmouawad
Date: Sun Sep 25 16:28:59 2016
New Revision: 1762223

URL: http://svn.apache.org/viewvc?rev=1762223&view=rev
Log:
Bug 60018 - Timer : Add a factor to apply on pauses
Document property
Bugzilla Id: 60018

Modified:
    jmeter/trunk/xdocs/usermanual/properties_reference.xml
Comment 12 Philippe Mouawad 2016-12-23 22:01:45 UTC
Author: pmouawad
Date: Fri Dec 23 21:45:29 2016
New Revision: 1775911

URL: http://svn.apache.org/viewvc?rev=1775911&view=rev
Log:
Sonar : Fix errors, code smells and false-positive
Also use Timer.isModifiable() introduced within Bug 60018
Bugzilla Id: 60018

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/threads/JMeterThread.java
    
    
Author: pmouawad
Date: Fri Dec 23 21:32:13 2016
New Revision: 1775901

URL: http://svn.apache.org/viewvc?rev=1775901&view=rev
Log:
Bug 60018 - Timer : Add a factor to apply on pauses
Use Java8 default method
Bugzilla Id: 60018

Modified:
    jmeter/trunk/src/components/org/apache/jmeter/timers/RandomTimer.java
    jmeter/trunk/src/core/org/apache/jmeter/timers/ModifiableTimer.java
    jmeter/trunk/src/core/org/apache/jmeter/timers/Timer.java