Bug 54794 - mod_wsgi|fcgid and subversion >= 1.7 issues
Summary: mod_wsgi|fcgid and subversion >= 1.7 issues
Status: RESOLVED LATER
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Core (show other bugs)
Version: 2.2.24
Hardware: PC FreeBSD
: P2 regression (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords: MassUpdate
Depends on:
Blocks:
 
Reported: 2013-04-03 12:02 UTC by Adrian Gschwend
Modified: 2018-11-07 21:09 UTC (History)
1 user (show)



Attachments
pcap with subversion 1.6 which works (15.94 KB, application/octet-stream)
2013-04-03 12:02 UTC, Adrian Gschwend
Details
pcap with subversion 1.7 which fails (3.87 KB, application/octet-stream)
2013-04-03 12:02 UTC, Adrian Gschwend
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Gschwend 2013-04-03 12:02:16 UTC
Created attachment 30140 [details]
pcap with subversion 1.6 which works

This is a bug I posted on both the mod_wsgi as well on the Apache user mailing list:

mod_wsgi:

http://groups.google.com/group/modwsgi/browse_thread/thread/f7171b7442965c78#

Apache:

http://thread.gmane.org/gmane.comp.apache.user/99368

I repeat the issue here with updated versions of all components involved:

I run trac and subversion in the same configuration and on the same vhost, see http://ktk.netlabs.org/misc/svn.netlabs.conf for my configuration.

The configuration starts with the SVN repositories residing on /repos and then configures trac with mod_wsgi. So far this used to work just fine for many years. As you can see on http://svn.netlabs.org I run a lot of TRAC repositories for several OSS projects.

A few weeks ago I upgraded subversion on the server to version 1.7, since then I can still checkout and commit to svn repositories with clients which are below version 1.7 (I tested 1.6.x), everything works as expected. However, any subversion client >= 1.7 cannot _commit_ anymore, checkout works as before though.

After dumping with tcpdump I could see the following packet (decoded TCP stream via wireshark, Subversion 1.7.x client):

-- 
POST /repos/sandbox/!svn/me HTTP/1.1
User-Agent: SVN/1.7.3 neon/0.29.6
Connection: TE
TE: trailers
Host: svn.netlabs.org
Content-Type: application/vnd.svn-skel
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops
Content-Length: 14
Accept-Encoding: gzip

( create-txn )HTTP/1.1 404 Not Found
Date: Thu, 22 Mar 2012 20:38:39 GMT
Server: Apache
Content-Length: 21
Content-Type: text/plain

Environment not found
-- 

full decoded TCP stream: http://pastebin.com/4bAT2TMA
tcpdump: http://ktk.netlabs.org/misc/subversion-commit_17.dmp

This "Environment not found" is definitely an error message from TRAC in case there is no trac environment found. So for some reason mod_wsgi hits in at this point instead of the DAV handler. Again, this works just fine with subversion 1.6.x (it does not do a POST there though):

full decoded TCP stream: http://pastebin.com/X9J3k7kE
tcpdump: http://ktk.netlabs.org/misc/subversion-commit_16.dmp

From the subversion changelog I can see that they changed the way subversion interacts with the server:

http://subversion.apache.org/docs/release-notes/1.7.html#httpv2

-- 
Subversion 1.7 offers a simpler HTTP protocol variant that can be used when connecting to supported servers. This simpler protocol (sometimes referred to as HTTPv2) requires fewer client-server round trips to establish a connection, making Subversion much more performant on high-latency network connections.

As mentioned in the compatibility table, Subversion 1.7 will only use HTTPv2 when connecting with a 1.7 (or greater) server, but will continue to use existing protocols when communicating with earlier versions of Subversion. Likewise, older clients will only use the old protocol when connecting to a Subversion server, regardless of version.
-- 

So the only explanation I have is that with 1.7 clients subversion tries to do HTTPv2 and for some reason mod_wsgi decides to take over even if it's not its job because the post goes to /repos, which should be the DAV handler. Note that I can "fix" the issue if I deactivate mod_wsgi or mod_fcgid with the current subversion configuration so the subversion module itself seems to work fine with new clients as well.

I first suspected a bug in mod_wsgi, for that reason I changed the configuration to mod_fcgid. Unfortunately it behaves exactly like that so my next guess (with the limited understanding of Apache internals I have) would be a bug in the internal processing of Apache.

Software versions (running on FreeBSD 9.1):
ap22-mod_wsgi-3.4 or ap22-mod_fcgid-2.3.6_1
apache22-2.2.24
py27-subversion-1.7.8
subversion-1.7.8
trac-0.12.3
python27-2.7.3_6

content of trac-netlabs.wsgi (looks a bit different for mod_fcgid but as
I said same bug):

-- 
#!/usr/local/bin/python2.6
# -*- coding: utf-8 -*-
import os
def application(environ, start_request):
  os.environ['TRAC_ENV_PARENT_DIR'] = '/data/trac/netlabs.org'
  os.environ['PYTHON_EGG_CACHE'] = '/tmp/egg-cache'
  from trac.web.main import dispatch_request
  environ['trac.locale'] = 'sv_SE.UTF-8'
  return dispatch_request(environ, start_request)
-- 

If someone wants to try committing, there is a sandbox repository which is world read/writable at:

http://svn.netlabs.org/repos/sandbox/

I'm more than happy to run more tests if someone has some ideas, I ran out of them.

thanks

Adrian
Comment 1 Adrian Gschwend 2013-04-03 12:02:42 UTC
Created attachment 30141 [details]
pcap with subversion 1.7 which fails
Comment 2 Rainer Jung 2013-04-03 17:19:55 UTC
Your configuration maps "/" via WSGIScriptAlias. This is documented in mod_wsgi to be problematic, because the result will depend on module load order.

Is there a chance to install TRAC in a way, such that the svn repos (/repos) and TRAC do not overlap in URI space (e.g. using /trac)?

Regards,

Rainer
Comment 3 Adrian Gschwend 2013-04-03 17:28:39 UTC
Hi Rainer,

Ah I see what you mean with module order. Yeah I could do something like that, I thought about moving TRAC to trac.netlabs.org and using svn.netlabs.org uniquely for SVN repos. With a rewrite rule I could rewrite the old URIs to the new one.

But thought I will report first, is there a chance this gets solved or is this hard to fix? In this case I would propose to close the issue then.
Comment 4 Adrian Gschwend 2013-04-03 18:34:45 UTC
One argument against module loading order issues is the fact that it works with 1.6 in this configuration.
Comment 5 William A. Rowe Jr. 2018-11-07 21:09:16 UTC
Please help us to refine our list of open and current defects; this is a mass update of old and inactive Bugzilla reports which reflect user error, already resolved defects, and still-existing defects in httpd.

As repeatedly announced, the Apache HTTP Server Project has discontinued all development and patch review of the 2.2.x series of releases. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases. All reports older than 2.4.x have been updated to status RESOLVED/LATER; no further action is expected unless the report still applies to a current version of httpd.

If your report represented a question or confusion about how to use an httpd feature, an unexpected server behavior, problems building or installing httpd, or working with an external component (a third party module, browser etc.) we ask you to start by bringing your question to the User Support and Discussion mailing list, see [https://httpd.apache.org/lists.html#http-users] for details. Include a link to this Bugzilla report for completeness with your question.

If your report was clearly a defect in httpd or a feature request, we ask that you retest using a modern httpd release (2.4.33 or later) released in the past year. If it can be reproduced, please reopen this bug and change the Version field above to the httpd version you have reconfirmed with.

Your help in identifying defects or enhancements still applicable to the current httpd server software release is greatly appreciated.