I would like to have the ability to optionally add the Reason Phrase to the HTTP response in Tomcat 9 (and above) like was available in Tomcat 8. I have legacy embedded firmware in devices in the field that are depending on the reason phrase. I know they are not "supposed" to, but they do, and it's multi-million dollars to replace them all, vs a small code fix. Currently Spring Boot 2.2.0 does not work with Tomcat 8, so now I'm stuck in legacy-software-hell, I can no longer upgrade my stack. Could we please have the ability to revive the "sendReasonPhrase" configuration parameter?
The documentation says: You can also deploy Spring Boot applications to any Servlet 3.1+ compatible container. Does that not work? What is wrong to deploy a Spring app as WAR file to Tomcat?
It's FAR more convenient to run Boot apps with the embedded container. Our whole infrastructure is set up that way.
First, some observations. By choosing a Spring Boot based solution on the server side (which implies regularly updating) and a client side that is essentially fixed, this design is making a number of assumptions: 1. That the protocol(s) supported on the client side will remain supported on on the server side for the life of the product. 2. That no protocol implementation issues requiring client side changes will be discovered on the client side for the life of the product. 3. The no other bugs requiring client side changes will be discovered for the life of the product. 4. That no security vulnerabilities requiring client side changes will be discovered for the life of the product. I think that assumption 1 is a reasonable assumption to make where those protocols are HTTP/1.1 and/or HTTP/2 and/or TLS. I do not think assumption 2, 3 nor 4 are reasonable. Even if the server side was similarly fixed, assumptions 3 and 4 would still apply and would still be unreasonable. I think I would be a good thing (long term) for a number of companies to be forced to go through a multi-million dollar hardware fix / replacement program so that the next time a system like this is designed more thought it is given to the question "What happens when (not if) the client needs to be updated?". The lack of reason phrase and the reasoning behind that decision has been discussed, at length, previously. See bug 60362 for one such discussion. In terms of possible workarounds, several ideas come to mind: a) Add a reverse proxy. Whether this helps will depend on the proxy but there might be one out there that inserts reason phrases. b) Spring Boot supports multiple embedded Servlet containers. Maybe one of the alternatives is more compatible with the clients. c) Figure out why Tomcat 8.5.x doesn't work with Spring Boot 2.2.x and fix it.
(In reply to Mark Thomas from comment #3) > I think I would be a good thing (long term) for a number of companies to be > forced to go through a multi-million dollar hardware fix / replacement > program so that the next time a system like this is designed more thought it > is given to the question "What happens when (not if) the client needs to be > updated?". +1 > b) Spring Boot supports multiple embedded Servlet containers. Maybe one of > the alternatives is more compatible with the clients. > > c) Figure out why Tomcat 8.5.x doesn't work with Spring Boot 2.2.x and fix > it. +1 You maybe should ask yourself whether Spring Boot is the right stack to chose if you need 10+ years stability. The regular Spring Framework and Tomcat do offer this, obviously Spring Boot does not.
(In reply to Michael Osipov from comment #4) > (In reply to Mark Thomas from comment #3) > > I think I would be a good thing (long term) for a number of companies to be > > forced to go through a multi-million dollar hardware fix / replacement > > program so that the next time a system like this is designed more thought it > > is given to the question "What happens when (not if) the client needs to be > > updated?". > > +1 So, I *completely* agree with this statement and it's sentiment, but, honestly, this response is pretty hostile to the Tomcat community. If anyone wants to know the answer to the question "why do people use Jetty instead of Tomcat?" then this kind of the response is going to be a part of the answer. I honestly don't understand why this issue is so contentious. HTTP/2 does not have reason phrases at all, and they are practically optional in HTTP/1.1 because there are no requirements for their content (https://tools.ietf.org/html/rfc7231#section-6.1). There is an interesting conversation about the reason phrase NOT being resurrected in HTTP/2 here: https://github.com/http2/http2-spec/issues/202. I find mnot's comment from 2018-02-04 to be the most compelling support for dropping the reason phrase. But when it comes down to it, inclusion of reason phrases are NOT prohibited by any version of HTTP/1.1 and some applications -- regardless of how foolish it is to have done so -- are evidently dependent upon their presence. The servlet spec has both HttpServletResponse.setStatus(int,String) which has been deprecated since spec 2.1 (release in 1998, so over 20 years ago) and HttpServletResponse.sendError(int,String). The former method argues that the usage of the "msg" parameter is ambiguous and suggests that #sendError should be user instead, but then the docs for #sendError imply that the "msg" parameter is used as the response *entity* and not as a reason phrase. So, IMO, there is still plenty of ambiguity there. If sendError(int,String) does set a reason phrase, then NOT respecting the application's attempts to set the reason phrase seems like a bug to me -- excepting of course the case where h2 is the protocol, but again, that is the application's choice. If Tomcat wants to omit reason phrases BY DEFAULT then I think that is reasonable. But if an application calls response.setStatus(200, "Super Duper") -- or response.sendError(200, "Super Duper") then I think it's reasonable to expect that the response on the wire will actually include that text.
(In reply to Christopher Schultz from comment #5) > (In reply to Michael Osipov from comment #4) > > (In reply to Mark Thomas from comment #3) > > > I think I would be a good thing (long term) for a number of companies to be > > > forced to go through a multi-million dollar hardware fix / replacement > > > program so that the next time a system like this is designed more thought it > > > is given to the question "What happens when (not if) the client needs to be > > > updated?". > > > > +1 > > So, I *completely* agree with this statement and it's sentiment, but, > honestly, this response is pretty hostile to the Tomcat community. If anyone > wants to know the answer to the question "why do people use Jetty instead of > Tomcat?" then this kind of the response is going to be a part of the answer. > > I honestly don't understand why this issue is so contentious. HTTP/2 does > not have reason phrases at all, and they are practically optional in > HTTP/1.1 because there are no requirements for their content > (https://tools.ietf.org/html/rfc7231#section-6.1). > > There is an interesting conversation about the reason phrase NOT being > resurrected in HTTP/2 here: https://github.com/http2/http2-spec/issues/202. > I find mnot's comment from 2018-02-04 to be the most compelling support for > dropping the reason phrase. > > But when it comes down to it, inclusion of reason phrases are NOT prohibited > by any version of HTTP/1.1 and some applications -- regardless of how > foolish it is to have done so -- are evidently dependent upon their presence. > > The servlet spec has both HttpServletResponse.setStatus(int,String) which > has been deprecated since spec 2.1 (release in 1998, so over 20 years ago) > and HttpServletResponse.sendError(int,String). The former method argues that > the usage of the "msg" parameter is ambiguous and suggests that #sendError > should be user instead, but then the docs for #sendError imply that the > "msg" parameter is used as the response *entity* and not as a reason phrase. > So, IMO, there is still plenty of ambiguity there. > > If sendError(int,String) does set a reason phrase, then NOT respecting the > application's attempts to set the reason phrase seems like a bug to me -- > excepting of course the case where h2 is the protocol, but again, that is > the application's choice. > > If Tomcat wants to omit reason phrases BY DEFAULT then I think that is > reasonable. But if an application calls response.setStatus(200, "Super > Duper") -- or response.sendError(200, "Super Duper") then I think it's > reasonable to expect that the response on the wire will actually include > that text. sendError() ia designed to deliver an HTML page, not the reason phrase.
Wow. That must be the biggest middle finger in software history. If I may paraphrase: "Yeah, I could fix your problem in about 10 minutes and save you a huge amount of hassle... but I'm not going to. Go screw yourself." If I spoke to one of our customers like this, it would be instant termination. Not to mention I'd feel like the biggest dick on the block. It's good to know that users in pain can count on the Tomcat dev team to turn the screws even harder. It's become clear that the use of Tomcat is a major risk to our business - not only can we not expect support, but we can expect anti-support. I will begin migrating to Jetty this week.
(In reply to Ken DeLong from comment #7) > Wow. That must be the biggest middle finger in software history. If I may > paraphrase: "Yeah, I could fix your problem in about 10 minutes and save you > a huge amount of hassle... but I'm not going to. Go screw yourself." > > If I spoke to one of our customers like this, it would be instant > termination. Not to mention I'd feel like the biggest dick on the block. > > It's good to know that users in pain can count on the Tomcat dev team to > turn the screws even harder. > > It's become clear that the use of Tomcat is a major risk to our business - > not only can we not expect support, but we can expect anti-support. I will > begin migrating to Jetty this week. You haven't even evaluated why Spring Boot requires Tomcat 9 and consider that Jetty has a much faster deprecation policy than Tomcat. We suffer from this in the Maven Wagon tests.
(In reply to Ken DeLong from comment #7) > It's good to know that users in pain can count on the Tomcat dev team to > turn the screws even harder. The pain is causing your firmware provider not us. We are just the idiots easier to talk to rather than TI or you name it.
Perhaps I should have made something clearer. While I think it is unlikely that Tomcat 9 will be adding support for an optional reason phrase, I will be looking at why Tomcat 8.5.x isn't compatible with Spring Boot. I spoke to the Spring Boot team at the end of last week and they say Boot should be making minimal use of Tomcat 9 specific features so getting Tomcat 8.5.x working with the latest Spring Boot release line shouldn't be too tricky. Timing wise I'm working through the open bugs and there are more than usual this month but the plan, as always, is to address them all - including this one - before the next release around the beginning of next month. The 9.0.x and 8.5.x code bases are pretty close so at this point, before I start work on this, I'm reasonably confident this will be fixable before the next release. If I hit something that requires, for example, a change in Spring Boot, that might slow things down. You are, of course, free to switch to Jetty if you wish. Whether or not you take up that option, I still intend to try and get Tomcat 8.5.x working with Spring Boot.
(In reply to Ken DeLong from comment #7) > Wow. That must be the biggest middle finger in software history. If I may > paraphrase: "Yeah, I could fix your problem in about 10 minutes and save you > a huge amount of hassle... but I'm not going to. Go screw yourself." > > If I spoke to one of our customers like this, it would be instant > termination. Not to mention I'd feel like the biggest dick on the block. > > It's good to know that users in pain can count on the Tomcat dev team to > turn the screws even harder. > > It's become clear that the use of Tomcat is a major risk to our business - > not only can we not expect support, but we can expect anti-support. I will > begin migrating to Jetty this week. Easy, now. If you want to rage-quit Tomcat, that's your prerogative. But let's not pretend that Apache Tomcat owes you anything at all. I haven't seen your email address on a single email to any of our mailing lists, ever. Going from "I'd like a feature reinstated" to "you guys are a risk to my business" in 3 days seems a little hostile for a product that seems to have kept your business going just fine for at least long enough for you to upgrade from Tomcat 8 to Tomcat 9. So if anything, you owe US something. That something I'd ask for is simply this: civility. We are all volunteers, here, and we are attempting to have a facts-based discussion. There are supporting arguments for all sides of this issue. There is no need to get emotional start yelling. I'd appreciate any spec-supported arguments both for and against continuing to support the reason phrase in Tomcat. So far, I only see support for removing it: 1. There is no "reason phrase" support in the Servlet Spec, so it is not accessible. 2. Tomcat historically used its own (non-customizable) reason phrases, therefore "200 Ok" is synonymous with "200 " 3. Previous version of HTTP/1.1 [https://tools.ietf.org/html/rfc2616#section-6.1.1] explicitly states that the "reason phrase" is "intended for the human user" 4. Latest version of HTTP/1.1 [rfc7231] basically says they are arbitrary, and are therefore (my conclusion) meaningless The only support for retaining it is: 1. Users are asking for it 2. No other web server seems to have removed it IOW, inertia is the only thing keeping the reason phrase around. That's a powerful motivator, honestly, IMHO.
This is a straight repost from https://bz.apache.org/bugzilla/show_bug.cgi?id=60362#c12 Note that the firmware demands "200 OK", not just "200 foo". I was pretty harsh in that other BZ. I still don't understand how the device can securely connect to any webserver, thus causing a major security risk for its unfortunate users. His only option is to dump the product he paid money for. So the reporter claims mistreatment but seems pretty good at doing worse. That being said, the reporter will likely be back again two years from now wasting our time again about his firmware, so can't we simply add a system property with an embarrassing name ? Like ADD_OK_REASON_PHRASE_FOR_BROKEN_CLIENTS.
(In reply to Remy Maucherat from comment #12) > This is a straight repost from > https://bz.apache.org/bugzilla/show_bug.cgi?id=60362#c12 > > Note that the firmware demands "200 OK", not just "200 foo". I was pretty > harsh in that other BZ. I still don't understand how the device can securely > connect to any webserver, thus causing a major security risk for its > unfortunate users. His only option is to dump the product he paid money for. > So the reporter claims mistreatment but seems pretty good at doing worse. > > That being said, the reporter will likely be back again two years from now > wasting our time again about his firmware, so can't we simply add a system > property with an embarrassing name ? Like > ADD_OK_REASON_PHRASE_FOR_BROKEN_CLIENTS. He will be back as soon as Tomcat 10 is out and Spring Boot dumps Tomcat 9. I surely that he won't have this clients updated by then.
So shall we mark this as DUPLICATE of bug #60362, or are we entertaining the idea of resurrecting the reason-phrase for Tomcat 9? I think most of the Tomcat devs are -0 at best for bringing back the reason-phrase. A formal vote seems heavy-handed.
Same discussion on the client side: https://issues.apache.org/jira/browse/MNG-6584
I've just done a simple test with the Spring Boot getting started guide (https://spring.io/guides/gs/spring-boot/) and I have overridden the Tomcat version used to specify 8.5.47 and that still works. On that basis the option recommended to the OP is to stick to Tomcat 8.5.x where the reason phrase is still supported (and will remain supported for the lifetime of 8.5.x). I appreciate that the Spring Boot getting started guide is a very simple web application. If issues are found with more complex applications / configurations not working with 8.5.x then please open a separate bug along with an example project we can use to reproduce the issue.
This is a plea for the Tomcat team to reconsider the practical implications of this decision. I agree that the HTTP/1.1 spec says that the reason phrase can be blank. However, the reality is that some clients are expecting that the reason phrase will be present. We operate a REST API and thousands of corporate/government clients connect to our API. We know that some of these clients rely on the reason phrase, but we have no way to determine how many. We currently use Tomcat 8.5, but we will need to upgrade to 9.0 at some point. That upgrade will cause clients to break, and the clients will come to us and demand that we "fix it". They won't accept arguments that it's good for them to have to change their system. All they will see is that we upgraded our system and their integration stopped working. We don't use HTTP/2 so it's irrelevant that there's no reason phrase in that version of the protocol. The reason phrase is a feature of HTTP/1.1 and I don't see any downside for Tomcat supporting it long term. Please add the flag to enable reason phrases back into to Tomcat 9 and all future versions that support HTTP/1.1.
(In reply to Nathan Clement from comment #17) > We operate a REST API and thousands of corporate/government clients connect > to our API. We know that some of these clients rely on the reason phrase, > but we have no way to determine how many. > > We currently use Tomcat 8.5, but we will need to upgrade to 9.0 at some > point. That upgrade will cause clients to break, and the clients will come > to us and demand that we "fix it". They won't accept arguments that it's > good for them to have to change their system. All they will see is that we > upgraded our system and their integration stopped working. So announce a new version of your REST API which does not include a reason phrase and ask everyone to move to it. Anyone who starts getting errors can fall-back to the older version and update their clients. Give them a timeline for when the old service will no longer be available. Tomcat can't remain static forever. > The reason phrase is a feature of HTTP/1.1 and I don't see any downside for > Tomcat supporting it long term. And yet we, the actual maintainers, *do* see downsides to supporting it long term.
The problem also is that the only would fix the symptom and push the remedy even more into the future where the problem will get worse.