Bug 39299 - Internal Server Error (500) on COPY
Summary: Internal Server Error (500) on COPY
Status: RESOLVED LATER
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_dav (show other bugs)
Version: 2.0.55
Hardware: Other Linux
: P2 normal (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL: http://www.webdav.org/neon/litmus/
Keywords: FixedInTrunk, MassUpdate, PatchAvailable
Depends on:
Blocks:
 
Reported: 2006-04-13 13:55 UTC by Julian Reschke
Modified: 2018-11-07 21:09 UTC (History)
2 users (show)



Attachments
Fix for wrong HTTP return code. (943 bytes, patch)
2007-02-13 14:50 UTC, Basant Kumar Kukreja
Details | Diff
Revised patch. (3.50 KB, patch)
2007-03-06 16:47 UTC, Basant Kumar Kukreja
Details | Diff
Revised Patch (1.54 KB, patch)
2007-03-06 16:54 UTC, Basant Kumar Kukreja
Details | Diff
revised patch without additional apr_stat (1.23 KB, patch)
2008-01-04 22:04 UTC, Michael Clark
Details | Diff
mobile device error report (6.04 KB, message/rfc822)
2011-02-05 20:39 UTC, lee
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Reschke 2006-04-13 13:55:13 UTC
Litmus test case copy_nodestcoll (a COPY request where the parent of the
destination resource does not exist) fails with status 500 (instead of 409).

The error log shows:

[Thu Apr 13 14:37:58 2006] [error] [client 192.168.1.45] Could not MOVE/COPY
/test/litmus/copysrc.  [500, #0]
[Thu Apr 13 14:37:58 2006] [error] [client 192.168.1.45] (2)No such file or
directory: Could not open file for writing  [500, #0]

Excerpt from debug.log created by Litmus:

******* Running test 5: copy_nodestcoll ********
ah_create, for WWW-Authenticate
Running pre_send hooks
Not handling session.
Sending request headers:
COPY /test/litmus/copysrc HTTP/1.1^M
Host: krusty^M
User-Agent: litmus/0.10.5 neon/0.25.4^M
Connection: TE^M
TE: trailers^M
Depth: 0^M
Destination: http://krusty/test/litmus/nonesuch/foo^M
Overwrite: F^M
X-Litmus: copymove: 5 (copy_nodestcoll)^M
^M
Sending request-line and headers:
Request sent; retry is 1.
[status-line] < HTTP/1.1 500 Internal Server Error^M
[hdr] Date: Thu, 13 Apr 2006 12:37:58 GMT^M
Header Name: [date], Value: [Thu, 13 Apr 2006 12:37:58 GMT]
[hdr] Server: Apache/2.0.55 (Unix) mod_ssl/2.0.55 OpenSSL/0.9.7b DAV/2 mod_jk/1.
2.15^M
Header Name: [server], Value: [Apache/2.0.55 (Unix) mod_ssl/2.0.55 OpenSSL/0.9.7
b DAV/2 mod_jk/1.2.15]
[hdr] Vary: accept-language,accept-charset^M
Header Name: [vary], Value: [accept-language,accept-charset]
[hdr] Accept-Ranges: bytes^M
Header Name: [accept-ranges], Value: [bytes]
[hdr] Connection: close^M
Header Name: [connection], Value: [close]
[hdr] Content-Type: text/html; charset=iso-8859-1^M
Header Name: [content-type], Value: [text/html; charset=iso-8859-1]
[hdr] Content-Language: en^M
Header Name: [content-language], Value: [en]
[hdr] ^M
End of headers.
Reading 8192 bytes of response body.
Got 1110 bytes.
Read block (1110 bytes):
...
Comment 1 Basant Kumar Kukreja 2007-02-13 10:24:16 UTC
I am able to reproduce the bug in 2.2.x branch
Comment 2 Basant Kumar Kukreja 2007-02-13 14:50:46 UTC
Created attachment 19581 [details]
Fix for wrong HTTP return code.

If we send the COPY HTTP request and the destination points to a file whose
directory doesn't exist then apache returned 500 error.

In this fix, we check the destination directory (do stat on it) and
if we failed to stat on the directory then we return 409.

Here is the request :
COPY /DAVtest/litmus/copysrc HTTP/1.1
Host: lbasantk3.red.iplanet.com:4004
User-Agent: litmus/0.11 neon/0.25.5
Connection: TE
TE: trailers
Depth: 0
Destination: http://hostname.domainname:4004/DAVtest/litmus/nonesuch/foo
Overwrite: F
X-Litmus: copymove: 5 (copy_nodestcoll)


New response :
--------------------------------------------------
HTTP/1.1 409 Conflict
Date: Tue, 13 Feb 2007 22:44:16 GMT
Server: Apache/2.2.5-dev (Unix) DAV/2 mod_perl/2.0.4-dev Perl/v5.8.8
Content-Length: 509
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>409 Conflict</title>
</head><body>
<h1>Conflict</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 you@example.com and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
--------------------------------------------------
Log message in apache log file appears as 
[Tue Feb 13 14:44:16 2007] [error] [client 192.18.120.216] Could not MOVE/COPY
/DAVtest/litmus/copysrc.  [409, #0]
[Tue Feb 13 14:44:16 2007] [error] [client 192.18.120.216] (2)No such file or
directory: /disk/apache/apache2/htdocs/DAVtest/litmus/nonesuch/  [409, #0]
--------------------------------------------------
Comment 3 Basant Kumar Kukreja 2007-02-13 14:52:14 UTC
Adding keyword PatchAvailable
Comment 4 Basant Kumar Kukreja 2007-03-02 10:53:03 UTC
Suggested fix is incomplete.

Here is the another issue (kind of extension to this bug ) related with this :

For configuration like :

DavLockDB /disk/apache/apache2/var/DAVLockFs
<Directory "/disk/apache/apache2/htdocs/DAVtest">
Options Indexes FollowSymLinks
AllowOverride None
order allow,deny
allow from all
AuthName "SMA Development server"
AuthType Basic
DAV On
</Directory>

If the request is :

COPY /DAVtest/litmus/copysrc HTTP/1.1
Host: myhostname.mydomain:4004
User-Agent: litmus/0.11 neon/0.25.5
Connection: TE
TE: trailers
Depth: 0
Destination: http://myhostname.mydomain:4004/DAVtest1/litmus/nonesuch/foo
Overwrite: F
X-Litmus: copymove: 5 (copy_nodestcoll)

then apache dies. Here is the stack trace :

(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1221944416 (LWP 12041)]
0x080bf0e0 in dav_get_resource (r=0x8635d18, label_allowed=0, use_checked_in=0,
res_p=0xb72a9224)
    at mod_dav.c:724
724         err = (*conf->provider->repos->get_resource)(r, conf->dir,
(gdb) where 10
#0  0x080bf0e0 in dav_get_resource (r=0x8635d18, label_allowed=0,
use_checked_in=0, res_p=0xb72a9224)
    at mod_dav.c:724
#1  0x080c3012 in dav_method_copymove (r=0x8631d08, is_move=0) at mod_dav.c:2642
#2  0x080c70f7 in dav_handler (r=0x8631d08) at mod_dav.c:4656
#3  0x08080e31 in ap_run_handler (r=0x8631d08) at config.c:157
#4  0x0808157b in ap_invoke_handler (r=0x8631d08) at config.c:372
#5  0x080b767c in ap_process_request (r=0x8631d08) at http_request.c:258
#6  0x080b41fe in ap_process_http_connection (c=0x84d3a40) at http_core.c:184
#7  0x08089373 in ap_run_process_connection (c=0x84d3a40) at connection.c:43
#8  0x08089786 in ap_process_connection (c=0x84d3a40, csd=0x84d3890) at
connection.c:178
#9  0x080f25fc in process_socket (p=0x84d3858, sock=0x84d3890, my_child_num=0,
my_thread_num=0,
    bucket_alloc=0x862fcc8) at worker.c:544
(More stack frames follow...)
(gdb) p conf
$1 = (dav_dir_conf *) 0x84396f8
(gdb) p conf->provider
$2 = (const dav_provider *) 0x0
(gdb)

The reason is :

    conf = ap_get_module_config(r->per_dir_config, &dav_module);
    /* assert: conf->provider != NULL */

    /* resolve the resource */
    err = (*conf->provider->repos->get_resource)(r, conf->dir,
                                                 label, use_checked_in,
                                                 res_p);

dav_get_resource doesn't check if there is no provider in conf and try to access
the resource. This case also needs to be handled.

Removing the keyword PatchAvailable so that we can provide the fix
for this issue too.

Comment 5 Basant Kumar Kukreja 2007-03-06 16:47:27 UTC
Created attachment 19673 [details]
Revised patch.

In summary there were two issues with COPY method.
(a) If Destination directory doesn't exist but it is still inside the DAV
directory, apache returned 500. In this fix if destination directory doesn't
exist, we return 409 Conflict.
(b) If Destination directory doesn't fall under a DAV provider, apache crashes.

In this fix if Destination directory doesn't exist then 403 forbidden is
returned.

Here are response code suggested by rfc 2518 :
http://asg.web.cmu.edu/rfc/rfc2518.html#sec-8.8


Here is how I did the testing :

my httpd.conf has following configuration :

<Directory "/disk/apache/apache2/htdocs/DAVtest">
Options Indexes FollowSymLinks
AllowOverride None
order allow,deny
allow from all
AuthName "SMA Development server"
AuthType Basic
DAV On
</Directory>

<Directory "/disk/apache/apache2/htdocs/DAVOtherTest">
Options Indexes FollowSymLinks
AllowOverride None
order allow,deny
allow from all
AuthName "SMA Development server"
AuthType Basic
DAV On
</Directory>


---------------------------------------
Test 1 : Destination is outside DAV collection.

COPY /DAVtest/litmus/copysrc HTTP/1.1
Host: localhost:4004
User-Agent: litmus/0.11 neon/0.25.5
Connection: TE
TE: trailers
Depth: 0
Destination: http://localhost:4004/DAVtest1/litmus/nonesuch/foo
Overwrite: F
X-Litmus: copymove: 5 (copy_nodestcoll)

Response :

Date: Wed, 07 Mar 2007 00:32:52 GMT
Server: Apache/2.2.5-dev (Unix) mod_ssl/2.2.5-dev OpenSSL/0.9.8a DAV/2
SVN/1.4.3 mod_perl/2.0.4-dev Perl/v5.8.8
Content-Length: 178
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>Destination URI had an error.</p>
</body></html>

---------------------------------------
Test 2 : Destination is inside DAV collection but it doesn't exists.

COPY /DAVtest/litmus/copysrc HTTP/1.1
Host: localhost:4004
User-Agent: litmus/0.11 neon/0.25.5
Connection: TE
TE: trailers
Depth: 0
Destination: http://localhost:4004/DAVtest/litmus/nonesuch/foo
Overwrite: F
X-Litmus: copymove: 5 (copy_nodestcoll)


Date: Wed, 07 Mar 2007 00:33:02 GMT
Server: Apache/2.2.5-dev (Unix) mod_ssl/2.2.5-dev OpenSSL/0.9.8a DAV/2
SVN/1.4.3 mod_perl/2.0.4-dev Perl/v5.8.8
Content-Length: 509
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>409 Conflict</title>
</head><body>
<h1>Conflict</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
 you@example.com and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
---------------------------------------
Test 3 : Destination is in other DAV collection

COPY /DAVtest/litmus/copysrc HTTP/1.1
Host: localhost:4004
User-Agent: litmus/0.11 neon/0.25.5
Connection: TE
TE: trailers
Depth: 0
Destination: http://localhost:4004/DAVOtherTest/foo
Overwrite: F
X-Litmus: copymove: 5 (copy_nodestcoll)

Date: Wed, 07 Mar 2007 00:40:18 GMT
Server: Apache/2.2.5-dev (Unix) mod_ssl/2.2.5-dev OpenSSL/0.9.8a DAV/2
SVN/1.4.3 mod_perl/2.0.4-dev Perl/v5.8.8
Location: http://localhost:4004/DAVOtherTest/foo
Content-Length: 192
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>201 Created</title>
</head><body>
<h1>Created</h1>
<p>Destination /DAVOtherTest/foo has been created.</p>
</body></html>
---------------------------------------
Comment 6 Basant Kumar Kukreja 2007-03-06 16:51:47 UTC
Comment on attachment 19673 [details]
Revised patch.

Index: modules/dav/fs/repos.c
===================================================================
--- modules/dav/fs/repos.c	(revision 500367)
+++ modules/dav/fs/repos.c	(working copy)
@@ -348,8 +348,19 @@
     status = apr_file_open(&outf, dst, APR_WRITE | APR_CREATE | APR_TRUNCATE
			    | APR_BINARY, perms, p);
     if (status != APR_SUCCESS) {
+	 char *dirpath = NULL;
+	 apr_finfo_t dir_finfo;
+	 apr_status_t rv;
	 apr_file_close(inf);

+	 /* check if destination directory exists if not then return
+	  * HTTP_CONFLICT
+	  */
+	 dirpath = ap_make_dirstr_parent(p, dst);
+	 rv = apr_stat(&dir_finfo, dirpath, APR_FINFO_NORM, p);
+	 if (rv != APR_SUCCESS)
+	     return dav_new_error(p, HTTP_CONFLICT, 0, dirpath);
+
	 return dav_new_error(p, MAP_IO2HTTP(status), 0,
			      "Could not open file for writing");
     }
Index: modules/dav/main/mod_dav.c
===================================================================
--- modules/dav/main/mod_dav.c	(revision 500367)
+++ modules/dav/main/mod_dav.c	(working copy)
@@ -2637,6 +2637,10 @@
	 return dav_error_response(r, lookup.rnew->status,
				   "Destination URI had an error.");
     }
+    if (!dav_get_provider(lookup.rnew)) {
+	 return dav_error_response(r, HTTP_FORBIDDEN,
+				   "Destination URI had an error.");
+    }

     /* Resolve destination resource */
     err = dav_get_resource(lookup.rnew, 0 /* label_allowed */,
Comment 7 Basant Kumar Kukreja 2007-03-06 16:54:37 UTC
Created attachment 19674 [details]
Revised Patch

By mistake I attached comments as patch previously. Sorry for that. (Is there a
way to delete it?).
Comment 8 Rafa&#322; Malinowski 2007-12-13 10:43:04 UTC
For me the 500 on COPY occurs when source file is not readable (-w--w---- for example) and the 
destination is OK:

[Thu Dec 13 19:20:50 2007] [error] [client 127.0.0.1] Could not 
MOVE/COPY /uploads/kermit-ui/dialog.diff.  [500, #0]
[Thu Dec 13 19:20:50 2007] [error] [client 127.0.0.1] (13)Permission denied: Could not open file for 
reading  [500, #0]


I think it should return 403 or something, but not 500.
Comment 9 Michael Clark 2008-01-04 21:48:07 UTC
There is no need to perform the additional stat in the ENOENT case as the
preceding apr_file_open has the APR_CREATE flag set. This means the ENOENT
status will mean that an intermediate directory does not exist (man 2 open).
Although it could also mean a dangling symlink in the intermediate path,
this is for all intents and purposes equivalent - it should still be a 409.

Attaching a simpler patch shortly.

Although the patch does not fix a second (independent) problem
(COPY/MOVE from a non readable source should not return 500) I don't see
why this is a reason why not to fix one of these bugs.

The original report for the litmus test is:
(COPY/MOVE to a destination collection where an intermediate does not exist
should return 403) which is what the patch fixes.

Perhaps a new bug should be created (COPY/MOVE from a non readable source
should not return 500) or this one left open and retitled - but can we
possbily apply a fix for one of these bugs.
Comment 10 Michael Clark 2008-01-04 22:04:02 UTC
Created attachment 21345 [details]
revised patch without additional apr_stat

reformulate previous patch removing redundant apr_stat call and providing a
more informative error message.

fixes the "COPY/MOVE to a destination collection where an intermediate
directory does not exist should return 409" case. verified with litmus that it
fixes the warning on the copy_nodestcoll test.

the mod_dav.c fragment fixes a related but distinct issue (COPY/MOVE to a
destination URI with no provider causes segfault) - not specifically tested but
limtus passes and so do general tests with OS X DAV client.

COPY/MOVE from a non readable source should not return 500 issue not addressed
in this patch although I believe this is a separate bug.
Comment 11 Stefan Fritsch 2009-11-09 07:29:21 UTC
The segfault part is already fixed in trunk and 2.2.x as PR 44734.

The 500 instead of 409 part is fixed in trunk as r834073 and r834107
Comment 12 lee 2011-02-05 20:39:20 UTC
Created attachment 26610 [details]
mobile device error report

error created by blackberry.com
Comment 13 Wim Lewis 2017-05-16 23:06:11 UTC
(In reply to Stefan Fritsch from comment #11)
> The segfault part is already fixed in trunk and 2.2.x as PR 44734.
> 
> The 500 instead of 409 part is fixed in trunk as r834073 and r834107

This bug still exists in 2.4.25 (on MacOSX):

% cadaver https://slenderpoke.local:8001/test2/
Authentication required for My Test Realm on server `slenderpoke.local':
Username: test2
Password: 
dav:/test2/> mkcol foo
Creating `foo': succeeded.
dav:/test2/> mkcol foo/bar
Creating `foo/bar': succeeded.
dav:/test2/> put httpd-bug39299.pl foo/bar/somefile
Uploading httpd-bug39299.pl to `/test2/foo/bar/somefile':
Progress: [=============================>] 100.0% of 1158 bytes succeeded.
dav:/test2/> mv foo/bar/somefile foo/baz/somefile
Moving `/test2/foo/bar/somefile' to `/test2/foo/baz/somefile':  failed:
500 Internal Server Error

The 2.4.x branch was branched from r1200449, so it should include r834073 and r834107.
Comment 14 Wim Lewis 2017-05-16 23:28:46 UTC
The bug also still exists in httpd built from the trunk (r1795372):


dav:/bugzilla39299/> mkcol fromcoll
Creating `fromcoll': succeeded.
dav:/bugzilla39299/> put testfile fromcoll/somefile
Uploading testfile to `/bugzilla39299/fromcoll/somefile':
Progress: [=============================>] 100.0% of 18 bytes succeeded.
dav:/bugzilla39299/> mv fromcoll/somefile tocoll/somefile
Moving `/bugzilla39299/fromcoll/somefile' to `/bugzilla39299/tocoll/somefile':  failed:
500 Internal Server Error
dav:/bugzilla39299/> mkcol tocoll
Creating `tocoll': succeeded.
dav:/bugzilla39299/> mv fromcoll/somefile tocoll/somefile
Moving `/bugzilla39299/fromcoll/somefile' to `/bugzilla39299/tocoll/somefile':  succeeded.


[Tue May 16 23:15:33.225431 2017] [dav:error] [pid 69020:tid 123145307131904] [client 127.0.0.1:65432] Could not MOVE/COPY /bugzilla39299/fromcoll/somefile.  [500, #0]
[Tue May 16 23:15:33.225463 2017] [dav:error] [pid 69020:tid 123145307131904] (2)No such file or directory: [client 127.0.0.1:65432] Could not rename resource.  [500, #0]
Comment 15 William A. Rowe Jr. 2018-11-07 21:09:15 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.