Issue 91269 - Normsdist tail calculation different from Matlab/Octave/Excel/Gnumeric
Summary: Normsdist tail calculation different from Matlab/Octave/Excel/Gnumeric
Status: CLOSED DUPLICATE of issue 69069
Alias: None
Product: Calc
Classification: Application
Component: code (show other issues)
Version: OOo 1.0.0
Hardware: All All
: P3 Trivial (vote)
Target Milestone: ---
Assignee: spreadsheet
QA Contact: issues@sc
URL:
Keywords: oooqa
Depends on:
Blocks:
 
Reported: 2008-07-01 15:22 UTC by rmcd
Modified: 2008-07-15 22:13 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this issue.
Description rmcd 2008-07-01 15:22:26 UTC 
Calc's Normsdist function gives different extreme tail values than Matlab,
Excel, Octave, and Gnumeric. Here is how all five programs compute the
equivalent value:

Software	Function	Value	                % diff vs. Gnumeric
Gnumeric 1.6.3	Normsdist(-5.5)	1.89895624658877E-08	0.0000E+00
Matlab 2006b	Normcdf(-5.5)	1.89895624658877E-08	0.0000E+00
Excel 2007	Normsdist(-5.5)	1.89895624780334E-08	6.3960E-10
Octave 3.0.1	Normcdf(-5.5)	1.89895624780334E-08	6.3960E-10
Calc 2.4.1/3.0	Normsdist(-5.5)	1.89901047664698E-08	2.8558E-05

Notice that Gnumeric and Matlab agree exactly, and Excel and Octave agree
exactly. Calc is the outlier. In casual tests the differences become apparent
for arguments below -4.5. 

I have listed this as a defect rather than an enhancement because it will
prevent my recommending Calc as an Excel replacement for financial calculations.
(I encountered this issue in a real calculation and spent a lot of time trying
to decide whether I to accept the Excel or Calc result as more accurate.)
Comment 1 Regina Henschel 2008-07-01 17:51:33 UTC 
You are right, see issue 69069.

You get slightly better values with 0.5+0.5*ERF(-5.5/SQRT(2)) in this range, but
unfortunately not for smaller values.

*** This issue has been marked as a duplicate of 6906 ***
Comment 2 Regina Henschel 2008-07-01 17:52:35 UTC 
You are right, see issue 69069.

You get slightly better values with 0.5+0.5*ERF(-5.5/SQRT(2)) in this range, but
unfortunately not for smaller values.

*** This issue has been marked as a duplicate of 69069 ***
Comment 3 discoleo 2008-07-01 18:16:10 UTC 
This is what R says:

> print(pnorm(-5.5),digits=22)
[1] 1.898956246588772e-08
Comment 4 oc 2008-07-15 22:13:10 UTC 
closed as duplicate