Bug 60081 - Interleave Controller : Add an option to alternate across threads
Summary: Interleave Controller : Add an option to alternate across threads
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: Main (show other bugs)
Version: 3.0
Hardware: All All
: P2 enhancement (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-02 20:04 UTC by Wyatt Epp
Modified: 2016-10-02 17:15 UTC (History)
1 user (show)



Attachments
Patch implementing interleave accross threads (6.97 KB, patch)
2016-10-01 12:25 UTC, Philippe Mouawad
Details | Diff
Patch implementing feature in a simpler way as per Felix proposal (3.77 KB, patch)
2016-10-02 15:51 UTC, Philippe Mouawad
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Wyatt Epp 2016-09-02 20:04:34 UTC
Each thread simultaneously takes the same through each loop of an Interleave Controler.  This results in a spike-like pattern of execution for each child instead of an even cadence and a truly interleaved blend.

I'll try to explain further.  Consider a test plan with two alternating transaction controllers:

Thread Group (LOOPS=5)
└── Interleave Controller
    ├── pathA
    │   ├── A1
    │   └── A2
    └── pathB
        └── B1

With a single thread, this behaves as anyone could generally expect:

Loop  thread
1     pathA
2     pathB
3     pathA
4     pathB
5     pathA

But when you have three threads it looks like this:

Loop  thread1  thread2  thread3
1     pathA    pathA    pathA
2     pathB    pathB    pathB
3     pathA    pathA    pathA
4     pathB    pathB    pathB
5     pathA    pathA    pathA

Notice how thread2 and thread3 also take pathA; this generalises to any number of threads.

From a user standpoint, the expected result (in terms of both the execution dispatch timing and the execution count) is more like this:

Loop  thread1  thread2  thread3
1     pathA    pathB    pathA
2     pathB    pathA    pathB
3     pathA    pathB    pathA
4     pathB    pathA    pathB
5     pathA    pathB    pathA
Comment 1 harsh 2016-09-10 08:24:08 UTC
Hi,

According to java doc, Interleave controller "Alternate among each of the children controllers or samplers for each loop iteration".

Also in component reference 
"If you add Generative or Logic Controllers to an Interleave Controller, JMeter will alternate among each of the other controllers for each loop iteration."

So in this case behaviour is proper. as for each loop, it will interleave the samplers.
Comment 2 Wyatt Epp 2016-09-12 11:36:49 UTC
(In reply to harsh from comment #1)
> Hi,
> 
> According to java doc, Interleave controller "Alternate among each of the
> children controllers or samplers for each loop iteration".

That's not at issue here.  In fact, I was quite pleased that it honours the contract of tree semantics when it encounters controllers.

The issue is when there are multiple threads, they all interleave the _SAME_ subcontrollers and samplers at the _SAME_ time.  They're effectively in lockstep.

This makes it unusable to generate a mixed load of multiple scenarios.
Comment 3 Philippe Mouawad 2016-09-29 20:20:08 UTC
Hi,
Can I ask why you don't use RandomController in this case ?

Another alternative is to use a Counter shared accross threads that varies between 1 and the number of children of the controller and use a Switch Controller.

The spike-like pattern has less chances to occur if using Random timers and if the Controller has samplers before and after it , so I am not sure it's a problem.
AFAIU, this Interleave Controller has always worked like this and  I am afraid many users expect it to work like it does.

I personnaly never used it so I cannot tell if it suits me or not :-)

thoughts ?
Thx
Comment 4 Wyatt Epp 2016-09-29 22:25:03 UTC
(In reply to Philippe Mouawad from comment #3)
> Hi,
> Can I ask why you don't use RandomController in this case ?

Lack of determinism is an issue (it's something that was stressed repeatedly by the project lead).  But the reason we discovered this in the first place was by fixing the Weighted Switch Controller plugin to respect controllers by deriving from InterleaveControl.

It's a good idea, and I'll experiment with deriving from RandomController instead (making it a Weighted Random Controller instead, I guess), but I still think this is an issue in its own right.
 
> Another alternative is to use a Counter shared accross threads that varies
> between 1 and the number of children of the controller and use a Switch
> Controller.

I guess this could work, but it's a rather ugly hack and I'm not fond of the idea.  This sort of circumlocution makes the controller abstraction feel very leaky to me.

> The spike-like pattern has less chances to occur if using Random timers and
> if the Controller has samplers before and after it , so I am not sure it's a
> problem.

Using random timers is the sort of hack that's going to seriously play havoc with the consistency of transaction throughput, so it'll never fly with people above me.  But even if they were fine with it, I don't think it's enough to work at every scale we need because the smoothing this introduces is a function of thread count and delay range.

> AFAIU, this Interleave Controller has always worked like this and  I am
> afraid many users expect it to work like it does.

I think it probably HAS always worked this way. I disagree that these specific semantics are what any user might reasonably expect, though.  (You can sort of figure it out with a close read of the manual but I wouldn't call it clear or obvious.) If anything, the way it works now has probably driven people away from it (and is, I suspect, probably why the Weighted Switch Controller inherited from GenericController instead of InterleaveControl in the first place).

I've been thinking about it a lot, but I haven't yet been able to come up with any compelling reasons a user might want this pattern of interleaving or wouldn't want evenly interleaved requests.  I'm just one person, though; maybe this is something that needs to be brought to more eyes?

Thanks,
Wyatt
Comment 5 Philippe Mouawad 2016-10-01 12:25:23 UTC
(In reply to Wyatt Epp from comment #4)
> (In reply to Philippe Mouawad from comment #3)
> > Hi,
> > Can I ask why you don't use RandomController in this case ?
> 
> Lack of determinism is an issue (it's something that was stressed repeatedly
> by the project lead).  But the reason we discovered this in the first place
> was by fixing the Weighted Switch Controller plugin to respect controllers
> by deriving from InterleaveControl.
> 
> It's a good idea, and I'll experiment with deriving from RandomController
> instead (making it a Weighted Random Controller instead, I guess), but I
> still think this is an issue in its own right.

I am not sure. 
If you look at your initial request for Interleave accross threads, you take the hypothesis that threads run nearly sequentially while you have no guarantee on order , and as a consequence Interleave would end up generating the equivalent of RandomController.
Unless I am missing something in your report or use case.



>  
> > Another alternative is to use a Counter shared accross threads that varies
> > between 1 and the number of children of the controller and use a Switch
> > Controller.
> 
> I guess this could work, but it's a rather ugly hack and I'm not fond of the
> idea.  This sort of circumlocution makes the controller abstraction feel
> very leaky to me.

I don't think it's ugly, SwitchController is done for that. I think it will show you that the interleave you expect is not respected;

> 
> > The spike-like pattern has less chances to occur if using Random timers and
> > if the Controller has samplers before and after it , so I am not sure it's a
> > problem.
> 
> Using random timers is the sort of hack that's going to seriously play havoc
> with the consistency of transaction throughput, so it'll never fly with
> people above me.  

It depend on your use case.
When I simulate Real User behaviour, I use random timers, not using them seems to me a mistake.
If you're more in a server to server call then ok. 


>But even if they were fine with it, I don't think it's
> enough to work at every scale we need because the smoothing this introduces
> is a function of thread count and delay range.
> 
> > AFAIU, this Interleave Controller has always worked like this and  I am
> > afraid many users expect it to work like it does.
> 
> I think it probably HAS always worked this way. I disagree that these
> specific semantics are what any user might reasonably expect, though.  (You
> can sort of figure it out with a close read of the manual but I wouldn't
> call it clear or obvious.) If anything, the way it works now has probably
> driven people away from it (and is, I suspect, probably why the Weighted
> Switch Controller inherited from GenericController instead of
> InterleaveControl in the first place).

It is hard to know usage of Controllers among community.


> 
> I've been thinking about it a lot, but I haven't yet been able to come up
> with any compelling reasons a user might want this pattern of interleaving
> or wouldn't want evenly interleaved requests.  I'm just one person, though;
> maybe this is something that needs to be brought to more eyes?
> 
> Thanks,
> Wyatt


Anyway I attached a patch, can you test it and see if that's what you expect ?
Thanks
Comment 6 Philippe Mouawad 2016-10-01 12:25:56 UTC
Created attachment 34316 [details]
Patch implementing interleave accross threads
Comment 7 Philippe Mouawad 2016-10-02 15:51:57 UTC
Created attachment 34320 [details]
Patch implementing feature in a simpler way as per Felix proposal
Comment 8 Felix Schumacher 2016-10-02 16:19:10 UTC
Comment on attachment 34320 [details]
Patch implementing feature in a simpler way as per Felix proposal

Looks a lot cleaner.

I think the res strings are missing and I would prefer the modulo operator be separated with spaces from its operands.
Comment 9 Philippe Mouawad 2016-10-02 17:15:08 UTC
Author: pmouawad
Date: Sun Oct  2 17:14:45 2016
New Revision: 1763086

URL: http://svn.apache.org/viewvc?rev=1763086&view=rev
Log:
Bug 60081 Interleave Controller : Add an option to alternate across threads
Bugzilla Id: 60081

Modified:
    jmeter/trunk/src/components/org/apache/jmeter/control/InterleaveControl.java
    jmeter/trunk/src/components/org/apache/jmeter/control/gui/InterleaveControlGui.java
    jmeter/trunk/src/core/org/apache/jmeter/resources/messages.properties
    jmeter/trunk/src/core/org/apache/jmeter/resources/messages_fr.properties
    jmeter/trunk/xdocs/changes.xml
    jmeter/trunk/xdocs/usermanual/component_reference.xml