Bug 56363 - OpenSSL security advisory - Heartbleed bug
Summary: OpenSSL security advisory - Heartbleed bug
Status: RESOLVED FIXED
Alias: None
Product: Tomcat Native
Classification: Unclassified
Component: Library (show other bugs)
Version: 1.1.29
Hardware: All All
: P2 critical with 80 votes (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-04-08 07:32 UTC by Mike Noordermeer
Modified: 2014-06-20 13:14 UTC (History)
7 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Noordermeer 2014-04-08 07:32:10 UTC
The binary builds of Tomcat Native 1.1.24 - 1.1.29 have been compiled with an OpenSSL version vulnerable to Heartbleed, and are thus probably vulnerable. See http://www.openssl.org/news/secadv_20140407.txt and http://heartbleed.com/

A new build using OpenSSL 1.0.1g would be very much appreciated.
Comment 1 Mladen Turk 2014-04-09 07:22:27 UTC
Working on it.
Comment 2 Mladen Turk 2014-04-10 07:00:20 UTC
Update done and tested.
We'll start release process today or tomorrow and new version containing OpenSSL 1.0.1g will be available after VOTE finishes and we create ANN message.

I'll close this issue when we send the ANN message.
Comment 3 Luke Hall 2014-04-10 14:17:30 UTC
Is there any possibility that you could provide us with a copy of the updated Tomcat Native library now? My company has several affected servers and we have been struggling to build the library ourselves, but with no success.
Comment 4 Mike Noordermeer 2014-04-10 14:48:23 UTC
While I understand that the disclosure process of this bug has been far from optimal, and really appreciate all effort the maintainer(s) spend on this project, a turnaround time of 3 days for such a critical issue is not acceptable and currently forcing us to use the Java SSL implementation and BIO/NIO connectors.

Would it, for future reference, be possible to document the Windows library build process better? We too have been struggling (and failing) to build the library ourselves, which has forced us to switch to the BIO connector due to the extreme severity of the issue.
Comment 5 Mike Noordermeer 2014-04-10 14:53:25 UTC
BTW, Luke, the intended release can be found at http://people.apache.org/~mturk/native/1.1.30/.
Comment 6 Jeffrey.Janner 2014-04-10 15:07:54 UTC
Thanks to Mladen for his effort on getting this out as fast as he has.
The ease with which this problem was addressable on the *NIX platforms leads me to ask if there is a better way of addressing the native libraries in the Windows world?  
The *NIX guys could just go get the latest OpenSSL version, install it, and restart their Tomcats, because the libraries are dynamically linked. However, the Windows version is statically linked, so we had to wait for Mladen to work his magic or try to do the build ourselves using minimal instructions.
My question is, what would be the downside to leaving the Windows version as a set of 2 or 3 dll files instead of statically linking them as they are now?
Comment 7 Konstantin Kolinko 2014-04-10 15:17:52 UTC
(In reply to Mike Noordermeer from comment #4)

1. The timing is unfortunate. There is a conference going on right now. Key people are there.
http://www.apachecon.com/

Also this bug is where I learned this issue from. Thank you Mike for filing it.

2. Nobody here works for Microsoft.

Providing windows binaries is a courtesy and may stop at any random moment.

> Would it, for future reference, be possible to document the Windows
> library build process better?

3. Documentation improvement patches are appreciated. You may submit one via bugzilla.

(In reply to Jeffrey.Janner from comment #6)
> However, the Windows version is statically linked, so we had to wait for
> Mladen to work his magic or try to do the build ourselves using minimal
> instructions.

4. a) That would be a mess.  b) Somebody would still need to build OpenSSL for you. http://openssl.org/ provides source code only. See 2.
Comment 8 Mike Noordermeer 2014-04-10 15:32:41 UTC
(In reply to Konstantin Kolinko from comment #7)
> 2. Nobody here works for Microsoft.
> 
> Providing windows binaries is a courtesy and may stop at any random moment.

That's good to know, and may be something to mention on the site, so people don't build production infrastructure on it.

> > Would it, for future reference, be possible to document the Windows
> > library build process better?
> 
> 3. Documentation improvement patches are appreciated. You may submit one via
> bugzilla.

I would love to improve the documentation, but at the moment it seems nobody knows how the Windows build works except for Mladen. See for instance this post of Mark Thomas: http://mail-archives.apache.org/mod_mbox/tomcat-dev/201205.mbox/%3C4FBA6DFC.4090502@apache.org%3E
Comment 9 Jeffrey.Janner 2014-04-10 15:48:22 UTC
(In reply to Konstantin Kolinko from comment #7)
> (In reply to Jeffrey.Janner from comment #6)
> > However, the Windows version is statically linked, so we had to wait for
> > Mladen to work his magic or try to do the build ourselves using minimal
> > instructions.
> 
> 4. a) That would be a mess.  b) Somebody would still need to build OpenSSL
> for you. http://openssl.org/ provides source code only. See 2.

Yep.  Luckily, that site has a link (http://www.openssl.org/related/binaries.html) to someone who provides binaries almost as fast as OpenSSL releases them.
That site is http://slproweb.com/products/Win32OpenSSL.html.
I think in this case he and Mladen had versions available at about the same time, though we're still waiting on an "approved" release from the ASF.
Overall, I'm generally OK getting the full suite from the ASF, its just would be nice to have alternatives.
Comment 10 Mladen Turk 2014-04-10 15:52:21 UTC
I'll update the BUILDING with windows section since everyone are so concerned of my health :)

It's very simple. The biggest problem is compiling apr and openssl. OpenSSL needs to be patched to allow /MD with static lib (something I tried to convince openssl developers for years without luck). We have the patch in the source dist. Then you have to put apr and openssl include and lib files to some directory 
C:\foo\bar\include
C:\foo\bar\lib

Then execute:
nmake -f NMAKEMakefile WITH_APR=C:\foo\bar WITH_OPENSSL=C:\foo\bar APR_DECLARE_STATIC=1 [ENABLE_OCSP=1]

When building APR I also modify apr.hw and set  APR_HAVE_IPV6 to 1.

That's it. 10 minutes for all architectures on an average box.
Note. Forget about IDE builds. They change the format of those .dsp, .vcproj whatever files with each version. Use nmake and command line. You'll need them to build openssl anyhow, so why not for all.
Comment 11 Gary Bollinger 2014-04-11 21:08:26 UTC
Is there a prediction on when this compile will be released?
Comment 12 Klemen Novak 2014-04-14 09:26:43 UTC
Any news on the build for windows?
Comment 13 Mladen Turk 2014-04-14 09:32:44 UTC
Vote passed, ANN will follow after mirror sync. cca 24 hrs max
Comment 14 Mladen Turk 2014-04-14 19:53:47 UTC
Binaries uploaded
http://apache.org/dist/tomcat/tomcat-connectors/native/1.1.30/
Took some time because of our own infrastructure Heartblead issues.

I'll create ANN message tomorrow after mirrors sync with Apache server.
Comment 15 Christopher Schultz 2014-04-15 01:09:47 UTC
Just a note: r1587378 in trunk now requires tcnative 1.1.30, so Tomcat 8.0.6 will require tcnative 1.1.30 to start.
Comment 16 Konstantin Kolinko 2014-04-16 11:05:55 UTC
(In reply to Mladen Turk from comment #14)
> Binaries uploaded
> http://apache.org/dist/tomcat/tomcat-connectors/native/1.1.30/

The correct download link is
http://tomcat.apache.org/download-native.cgi

(A policy is that we do not advertise direct links to the ASF server,
but suggest using the mirrors, as linked from the download page.)
Comment 17 Keith Davis 2014-04-16 15:18:26 UTC
(In reply to Christopher Schultz from comment #15)
> Just a note: r1587378 in trunk now requires tcnative 1.1.30, so Tomcat 8.0.6
> will require tcnative 1.1.30 to start.

Please forgive my lack of understanding, but what am we are supposed to do to patch our servers? Build our own and then use this connector? What is the connector for?
Comment 18 Mladen Turk 2014-04-16 15:43:47 UTC
(In reply to Keith Davis from comment #17)
> 
> Please forgive my lack of understanding, but what am we are supposed to do
> to patch our servers? Build our own and then use this connector? What is the
> connector for?

Bugzilla is not support forum. Please use tomcat users list instead
Comment 19 Doug 2014-06-19 02:14:50 UTC
As a result of the latest OpenSSL vulnerability in 1.0.1g, I've been trying the build described here. However, the resulting .dll is always 200-300 KB (the build from April is ~1.5 MB), depending on which version of vc++ I use, and Tomcat cannot use it. Would you be able to provide a build with the latest OpenSSL 1.0.1h? Or more explicit instructions on completing the build on Windows?

(In reply to Mladen Turk from comment #10)
> I'll update the BUILDING with windows section since everyone are so
> concerned of my health :)
> 
> It's very simple. The biggest problem is compiling apr and openssl. OpenSSL
> needs to be patched to allow /MD with static lib (something I tried to
> convince openssl developers for years without luck). We have the patch in
> the source dist. Then you have to put apr and openssl include and lib files
> to some directory 
> C:\foo\bar\include
> C:\foo\bar\lib
> 
> Then execute:
> nmake -f NMAKEMakefile WITH_APR=C:\foo\bar WITH_OPENSSL=C:\foo\bar
> APR_DECLARE_STATIC=1 [ENABLE_OCSP=1]
> 
> When building APR I also modify apr.hw and set  APR_HAVE_IPV6 to 1.
> 
> That's it. 10 minutes for all architectures on an average box.
> Note. Forget about IDE builds. They change the format of those .dsp, .vcproj
> whatever files with each version. Use nmake and command line. You'll need
> them to build openssl anyhow, so why not for all.
Comment 20 Konstantin Kolinko 2014-06-19 11:52:10 UTC
(In reply to Doug from comment #19)

See current thread "Building tcnative on win32",
"Building tcnative on win32 [progress!]" on dev mailing list.
http://tomcat.markmail.org/thread/ohhpa3k2kicyvjvy

Maybe somebody could help there.
Comment 21 Christopher Schultz 2014-06-19 12:45:25 UTC
(In reply to Doug from comment #19)
> As a result of the latest OpenSSL vulnerability in 1.0.1g, I've been trying
> the build described here. However, the resulting .dll is always 200-300 KB
> (the build from April is ~1.5 MB), depending on which version of vc++ I use,
> and Tomcat cannot use it. Would you be able to provide a build with the
> latest OpenSSL 1.0.1h? Or more explicit instructions on completing the build
> on Windows?

The problem is likely that you are not building tcnative statically, but instead as a dynamically-linked DLL. For convenience, the Tomcat team provides a statically-linked tcnative which means it contains OpenSSL, APR, and tcnative. Note that, if a dynamically-linked DLL were distributed, no rebuild would be necessary to upgrade OpenSSL.
Comment 22 Christopher Schultz 2014-06-19 12:45:52 UTC
(In reply to Konstantin Kolinko from comment #20)
> (In reply to Doug from comment #19)
> 
> See current thread "Building tcnative on win32",
> "Building tcnative on win32 [progress!]" on dev mailing list.
> http://tomcat.markmail.org/thread/ohhpa3k2kicyvjvy
> 
> Maybe somebody could help there.

I'll post my current build script which has a few pre-requisites.
Comment 23 Doug 2014-06-20 13:14:16 UTC
(In reply to Christopher Schultz from comment #22)
> (In reply to Konstantin Kolinko from comment #20)
> > (In reply to Doug from comment #19)
> > 
> > See current thread "Building tcnative on win32",
> > "Building tcnative on win32 [progress!]" on dev mailing list.
> > http://tomcat.markmail.org/thread/ohhpa3k2kicyvjvy
> > 
> > Maybe somebody could help there.
> 
> I'll post my current build script which has a few pre-requisites.

Thanks, I would really appreciate that!