Bug 56162 - HTTP Cache Manager should not cache PUT/POST etc.
Summary: HTTP Cache Manager should not cache PUT/POST etc.
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: HTTP (show other bugs)
Version: 2.11
Hardware: PC Linux
: P2 normal (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-19 18:12 UTC by Matt Parker
Modified: 2014-03-07 18:05 UTC (History)
1 user (show)



Attachments
test plan to show inability to override if-none-match when using http cache manager (10.28 KB, text/plain)
2014-02-19 18:12 UTC, Matt Parker
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Parker 2014-02-19 18:12:02 UTC
Created attachment 31335 [details]
test plan to show inability to override if-none-match when using http cache manager

I am unable to override the 'If-None-Match' header when an HTTP Cache Manager is in play. See attached test plan. 

In my real-world test, I am doing a PUT, followed by a GET which validates changed values. The response from the PUT includes an ETag, which the Cache Manager picks up and uses in the subsequent GET. Since I specifically want to GET the full entity immediately after the PUT, I want to override the cache header.
Comment 1 Philippe Mouawad 2014-02-25 13:06:06 UTC
This is due to CacheManager being called after adding Headers from Header Manager so it overrides values defined in HeaderManager.

We could add an checkbox in CacheManager saying :
- Overridable by headers in Header Manager


Adding something in Header Manager does not seem to be a good idea for me as it is clearly related to Cache Manager.

Thoughts ?
Comment 2 Sebb 2014-02-25 17:17:29 UTC
Not sure I understand the use case.

Surely if the PUT changes the target of the GET, the Etag should change?

How does the JMeter behaviour differ from that of a browser?
[Apart from the fact that JMeter does not store the page contents, so cached gets are empty]

It would be helpful to have details of the request response headers sent by JMeter and also by a browser.
Comment 3 Matt Parker 2014-02-25 19:20:20 UTC
(In reply to Philippe Mouawad from comment #1)
> This is due to CacheManager being called after adding Headers from Header
> Manager so it overrides values defined in HeaderManager.
> 
> We could add an checkbox in CacheManager saying :
> - Overridable by headers in Header Manager
> 
> 
> Adding something in Header Manager does not seem to be a good idea for me as
> it is clearly related to Cache Manager.
> 
> Thoughts ?

Yes, putting a checkbox on CacheManager would work. Although, as an end user it would be more intuitive if JMeter were to process CacheManager headers first, then apply any headers defined explicitly in a sampler's child HeaderManager. I believe that behavior would be similar to the way JMeter allows override of other hierarchical elements.
Comment 4 Matt Parker 2014-02-25 19:44:36 UTC
(In reply to Sebb from comment #2)
> Not sure I understand the use case.
> 
> Surely if the PUT changes the target of the GET, the Etag should change?

You're correct, the ETag value does change, and the CacheManager works properly with respect to this. However, I then want to create a new GET request specifically to retrieve the full entity, but I no longer have the ability to do this once the CacheManager is there.

> 
> How does the JMeter behaviour differ from that of a browser?
> [Apart from the fact that JMeter does not store the page contents, so cached
> gets are empty]

JMeter allows me to arbitrarily change headers and their values with each request,  whereas a browser does not. This makes JMeter a much better tool for testing HTTP APIs than a web browser.

> 
> It would be helpful to have details of the request response headers sent by
> JMeter and also by a browser.

I'm sure they would look identical, but this is more about the decreased ability to change the header values in a test, than it is about how it mimics a web browser.
Comment 5 Sebb 2014-02-25 20:31:16 UTC
(In reply to Matt Parker from comment #4)
> (In reply to Sebb from comment #2)
> > Not sure I understand the use case.
> > 
> > Surely if the PUT changes the target of the GET, the Etag should change?
> 
> You're correct, the ETag value does change, and the CacheManager works
> properly with respect to this. However, I then want to create a new GET
> request specifically to retrieve the full entity, but I no longer have the
> ability to do this once the CacheManager is there.

That's the part I don't understand.
Surely the next GET after the PUT won't return the cached entry?
If it does, then perhaps CM - or the site - is faulty?

> > 
> > How does the JMeter behaviour differ from that of a browser?
> > [Apart from the fact that JMeter does not store the page contents, so cached
> > gets are empty]
> 
> JMeter allows me to arbitrarily change headers and their values with each
> request,  whereas a browser does not. This makes JMeter a much better tool
> for testing HTTP APIs than a web browser.

Yes, though of course the site must still work when used by a browser.

> > 
> > It would be helpful to have details of the request response headers sent by
> > JMeter and also by a browser.
> 
> I'm sure they would look identical, but this is more about the decreased
> ability to change the header values in a test, than it is about how it
> mimics a web browser.

That's a bit different.

As far as I understand your use case, it should not need you to override headers. If it does require this currently, then we need to establish why that is. Better to fix JMeter (if it is wrong) than to have to use a work-round.
Comment 6 Matt Parker 2014-02-25 22:41:14 UTC
(In reply to Sebb from comment #5)
> (In reply to Matt Parker from comment #4)
> > (In reply to Sebb from comment #2)
> > > Not sure I understand the use case.
> > > 
> > > Surely if the PUT changes the target of the GET, the Etag should change?
> > 
> > You're correct, the ETag value does change, and the CacheManager works
> > properly with respect to this. However, I then want to create a new GET
> > request specifically to retrieve the full entity, but I no longer have the
> > ability to do this once the CacheManager is there.
> 
> That's the part I don't understand.
> Surely the next GET after the PUT won't return the cached entry?
> If it does, then perhaps CM - or the site - is faulty?

Technically, both CM and the site are working properly. But CM's behavior is too restrictive to allow me to do what I want. After the PUT, the 2xx response from the site includes the updated ETag value. CM sees that ETag value in the PUT response, caches it, and inserts it on the next GET. From the site's perspective, the client has provided the correct cache validator, and so returns 304. What I'm trying to do is to override the etag value that CM inserts, so that I can get back a 200 instead of a 304.

> 
> > > 
> > > How does the JMeter behaviour differ from that of a browser?
> > > [Apart from the fact that JMeter does not store the page contents, so cached
> > > gets are empty]
> > 
> > JMeter allows me to arbitrarily change headers and their values with each
> > request,  whereas a browser does not. This makes JMeter a much better tool
> > for testing HTTP APIs than a web browser.
> 
> Yes, though of course the site must still work when used by a browser.

Agreed.

> 
> > > 
> > > It would be helpful to have details of the request response headers sent by
> > > JMeter and also by a browser.
> > 
> > I'm sure they would look identical, but this is more about the decreased
> > ability to change the header values in a test, than it is about how it
> > mimics a web browser.
> 
> That's a bit different.
> 
> As far as I understand your use case, it should not need you to override
> headers. If it does require this currently, then we need to establish why
> that is. Better to fix JMeter (if it is wrong) than to have to use a
> work-round.

The attached test case illustrates the problem, and I'll outline my real-world test.

Attached test case:
0. A CacheManager is added to the test plan
1. sampler performs a GET http://www.mnot.net/cache_docs/
2. the site responds with 200 OK, and an ETag value (amongst other header values)
3. CM recognizes the ETag value and copies it
4. sampler performs another GET request for the same item
5. CM intelligently inserts the etag value into the if-none-match header
6. the etag value matches what the site has, so the site returns 304
7. I want to force a 200 response, rather than a 304, so I add an if-none-match header (guaranteed to not match) to the next sampler
8. CM overrides the if-none-match value I provided, and uses its value. (This is the behavior I'd like to control)

In my real-world test, I am testing a RESTful API--I am doing a POST, followed by a GET to validate that the body matches what I previously created. Then I am validating that the server API processes the conditional GET properly, by using the CM and another GET. Then I perform a PUT, changing the contents (the server responds with a 204 and an ETag). Lastly, I want to validate that the server sends back the exact contents that I PUT (I want a 200), but since CM is automatically working its magic, I can't get anything back but a 304 No Content.
Comment 7 Matt Parker 2014-02-25 22:44:09 UTC
> anything back but a 304 No Content.

Oops, I meant "304 Not Modifed".
Comment 8 Sebb 2014-02-25 23:28:36 UTC
(In reply to Matt Parker from comment #6)
> (In reply to Sebb from comment #5)
> > (In reply to Matt Parker from comment #4)
> > > (In reply to Sebb from comment #2)
> > > > Not sure I understand the use case.
> > > > 
> > > > Surely if the PUT changes the target of the GET, the Etag should change?
> > > 
> > > You're correct, the ETag value does change, and the CacheManager works
> > > properly with respect to this. However, I then want to create a new GET
> > > request specifically to retrieve the full entity, but I no longer have the
> > > ability to do this once the CacheManager is there.
> > 
> > That's the part I don't understand.
> > Surely the next GET after the PUT won't return the cached entry?
> > If it does, then perhaps CM - or the site - is faulty?
> 
> Technically, both CM and the site are working properly. But CM's behavior is
> too restrictive to allow me to do what I want. After the PUT, the 2xx
> response from the site includes the updated ETag value.

But should a PUT response be cached?

That seems wrong.
Comment 9 Matt Parker 2014-02-26 00:43:39 UTC
(In reply to Sebb from comment #8)
> (In reply to Matt Parker from comment #6)
> > (In reply to Sebb from comment #5)
> > > (In reply to Matt Parker from comment #4)
> > > > (In reply to Sebb from comment #2)
> > > > > Not sure I understand the use case.
> > > > > 
> > > > > Surely if the PUT changes the target of the GET, the Etag should change?
> > > > 
> > > > You're correct, the ETag value does change, and the CacheManager works
> > > > properly with respect to this. However, I then want to create a new GET
> > > > request specifically to retrieve the full entity, but I no longer have the
> > > > ability to do this once the CacheManager is there.
> > > 
> > > That's the part I don't understand.
> > > Surely the next GET after the PUT won't return the cached entry?
> > > If it does, then perhaps CM - or the site - is faulty?
> > 
> > Technically, both CM and the site are working properly. But CM's behavior is
> > too restrictive to allow me to do what I want. After the PUT, the 2xx
> > response from the site includes the updated ETag value.
> 
> But should a PUT response be cached?
> 
> That seems wrong.

According to http://tools.ietf.org/html/rfc2616#section-13.10, you're correct:

"Some HTTP methods MUST cause a cache to invalidate an entity. This is
   either the entity referred to by the Request-URI, or by the Location
   or Content-Location headers (if present). These methods are:

      - PUT

      - DELETE

      - POST"

Although, I don't think CM is attempting to act as a true HTTP cache by this definition--I think it's instead acting as an embedded UA component, which is merely aware of cache-related headers. (Please correct me if I'm interpreting this wrong).

http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Cache_Manager

"If a sample is successful (i.e. has response code 2xx) then the Last-Modified and Etag (and Expired if relevant) values are saved for the URL. Before executing the next sample, the sampler checks to see if there is an entry in the cache, and if so, the If-Last-Modified and If-None-Match conditional headers are set for the request."

By that definition, CM is behaving appropriately, and it makes sense. I just think it should be expanded to also check if the sampler wants to override the cache header in use.

Sorry if I'm misinterpreting something, I'm not actively trying to confuse matters ;)
Comment 10 Sebb 2014-02-26 01:21:37 UTC
I think the cache referred to in the RFC is a proxy cache.
Local caching (in the UA) is not quite the same.

The decription of CM agrees with its behaviour, but that behaviour may be incorrect ...

The HC4 code now has a Caching implementation; hopefully we can use that when we upgrade to the next release. However that may not be usable with the other HTTP implementations.

In the meantime, I think we need to check the method as well as the status.

We certainly want to cache GET.
Not sure about HEAD, might be useful not to cache that as it is a way of testing the connection.

And some methods we don't want to cache:
PUT, POST, DELETE

Perhaps we need a list of which ones to cache; this could be overridable.

Also, maybe there should be an "Ignore Cache" option.
This would correspond with the browser Shift-Reload feature to bypass the cache.
Not sure if this should first remove any existing cache entry in case the request fails, or only update the cache on a successful response.

I expect either of those would address your use-case.
Comment 11 Matt Parker 2014-02-26 18:48:28 UTC
> I expect either of those would address your use-case.

Agreed. Thanks.
Comment 12 Sebb 2014-02-26 22:48:48 UTC
URL: http://svn.apache.org/r1572319
Log:
HTTP Cache Manager should not cache PUT/POST etc.
Bugzilla Id: 56162

Modified:
    jmeter/trunk/bin/jmeter.properties
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/control/CacheManager.java
    jmeter/trunk/xdocs/changes.xml
Comment 13 Sebb 2014-02-27 10:59:23 UTC
Note: if you want to try a nightly build with this fix, it will be in builds after version r1572319
Comment 14 Matt Parker 2014-03-05 19:34:39 UTC
Confirmed in build r1574139, the etag value from the PUT response is not re-sent on the next GET when the CM is used.

Thanks!
Comment 15 Matt Parker 2014-03-05 19:42:39 UTC
A note to anyone looking at this defect: the attached test case is not valid with respect to the fix. The attached test case still fails, because it makes the wrong assumption about the what the fix was going to be. I don't know of a public website against which I can provide a test case that matches the fix, so I validated it locally.