Bug 58789 - header set multiple htaccess
Summary: header set multiple htaccess
Status: RESOLVED FIXED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Documentation (show other bugs)
Version: 2.5-HEAD
Hardware: PC Windows NT
: P2 normal (vote)
Target Milestone: ---
Assignee: HTTP Server Documentation List
URL: ccc.hyundai-motor.com
Keywords: PortForward
Depends on:
Blocks:
 
Reported: 2016-01-03 00:20 UTC by gilperon
Modified: 2017-02-24 22:04 UTC (History)
2 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description gilperon 2016-01-03 00:20:17 UTC
In my CENTOS 7 server I have Apache 2.4 installed with mod_headers enabled.

I have a site at directory /aaa/bbb/ccc/index.php

I have an htaccess on every directory "/aaa", another in "/aaa/bbb" and another in "/aaa/bbb/ccc".

In the htaccess located at /aaa I have this

Header set AAA ZZZ

In the htaccess located at /aaa/bbb and /aaa/bbb/ccc I have this

Header set AAA XXXXXXXX

When I open the website using my browser the header AAA has the value ZZZ. For some reason the other htaccess files are not overriding the value of AAA. But what is really strange is that if I change the htaccess from /aaa/bbb and /aaa/bbb/ccc to the code below the header gets overrided! Why?

<FilesMatch "^.*$">

    Header set AAA XXXXXXXX

</FilesMatch>
Comment 1 William A. Rowe Jr. 2016-01-04 23:09:13 UTC
This resource is for reporting defects in the Apache httpd software, not for support questions.

https://httpd.apache.org/docs/2.4/sections.html

may help you better understand the order of operations.

http://httpd.apache.org/lists.html#http-users

The users list may be able to answer your further questions.
Comment 2 gilperon 2016-01-04 23:40:01 UTC
I think you didnt read what I asked. I am a programmer, I dont want support, I know very well what I am doing. On Apache 2.2 this bug didnt occur, it only occurs on Apache 2.4.

I am trying to help the community, not asking a support. You should be more polite and read the bug instead of complaning.
Comment 3 William A. Rowe Jr. 2016-01-05 00:34:27 UTC
First you are asking a user question;

"if I change the htaccess from /aaa/bbb and /aaa/bbb/ccc to the code below the header gets overrided! Why?"

It has always worked this way, <Files > blocks are and always have been applied last after <Directory > blocks (including .htaccess assignments).

Secondly, it is not clear whether you have the .htaccess directives correctly configured in <Directory > blocks in your httpd.conf (and attaching your httpd.conf is usually a dead giveaway that this is a user question).

http://httpd.apache.org/docs/2.4/howto/htaccess.html

explains the controls.  Deleting the .htaccess file in /aaa could quickly tell you whether the .htaccess files in /aaa/bbb and /aaa/bbb/ccc are entirely ignored.

I expect you will resolve your own question reviewing the documention, but will leave this as NEEDINFO for you to explain precisely what you tried and precisely what the result was.  Thus far it looks like it is behaving as designed.
Comment 4 William A. Rowe Jr. 2016-01-05 00:40:02 UTC
Sorry I mis-parsed your question; is the following correct?

/aaa/.htaccess:
  Header set AAA ZZZ
  
/aaa/bbb/.htaccess:
  Header set AAA XXXXXXXX

/aaa/bbb/ccc/.htaccess:
  Header set AAA XXXXXXXX

(To clarify, there is no Header set AAA anywhere in your httpd.conf or other config files - and none of the directives above are in <Section > blocks)

Request:
GET /aaa/bbb/ccc/index.php

Response:
AAA: ZZZ

Expected:
AAA: XXXXXXXX

"When I open the website using my browser the header AAA has the value ZZZ. For some reason the other htaccess files are not overriding the value of AAA."

Got it...

"But what is really strange is that if I change the htaccess from /aaa/bbb and /aaa/bbb/ccc to the code below the header gets overrided! Why?"

/aaa/bbb/ccc/.htaccess:
  <FilesMatch "^.*$">
    Header set AAA XXXXXXXX
  </FilesMatch>

Request:
GET /aaa/bbb/ccc/index.php

Response:
AAA: XXXXXXXX

So - you have documented that httpd is configured to parse /aaa/bbb/ccc/.htaccess, thank you for the clarity, sorry I misread your question.
Comment 5 gilperon 2016-01-05 01:43:38 UTC
@William A. Rowe Jr.  maybe I should have been more clear about my question. Your description of what is happening is precise. I am not "used" to report bug, actually this is my first time so I had no idea how to do this.

The entire description that you provided is exactly what I did, and the results are exactly as you pointed.

All my 3 htaccess files are being "parsed" by Apache, I am really sure of that: if I remove 2 htaccess and leave only one, the result (response) is exactly as expected. If I remove any other 2 htaccess and leave another one, the result (response) is exactly as expected.

The problem I am facing has nothing to do with <Directory> or <Files> blocks, I am not even using them in any of my htaccess files. I am also using the default httpd.conf file, it has no "Header set AAA" in any line. I tested this bug on a completely fresh new Apache 2.4 install on my Centos 7.1 server. This bug does not happen with Apache 2.2, I mean: with Apache 2.2 the header "AAA" is overrided by the htaccess of the directory /aaa/bbb/ccc/, exactly as expected.
 
The lines below, that you kindly described, are also exactly what I am getting (which proves the bug):

Request:
GET /aaa/bbb/ccc/index.php

Response:
AAA: ZZZ

Expected Response:
AAA: XXXXXXXX

If I need to provide any other information I would be glad to help! Hope you can keep me updated what are the next steps I should do to try to make this bug get fixed.
Comment 6 William A. Rowe Jr. 2016-01-05 23:20:27 UTC
I don't immediately see the cause.

The config merge logic is correct, the broader (global/vhost/<Directory />) scopes are added before sub-scopes (nested directories, Files etc).

It almost sounds like you used setifempty;

  "The request header is set, but only if there is no previous header with this name. Available in 2.4.7 and later."

because the first Header set in the list will cause later setifempty assignments to fail even if they are in a more precise scope.  We process the list sequentially.

The problem would also occur if you used 'early', e.g.

Header set AAA XXXXXXXX early

  "Because early directives are processed before the request path's configuration is traversed, early headers can only be set in a main server or virtual host context. Early directives cannot depend on a request path, so they will fail in contexts such as <Directory> or <Location>."

You didn't do either of these things, but they may provide a clue about where the bug may be hiding.
Comment 7 gilperon 2016-01-06 01:23:36 UTC
Indeed you are correct, I didnt use setifempty nor early. Unfortunatelly I am not an advanced htaccess user. I dont know how reporting a bug works, but should the person, who report the bug, propose solutions to fix it? Or should the community try to help solving the problem?

The only clue I have is that if I use <FilesMatch "^.*$"> then the logic works fine, I mean, the header gets overrided just fine.
Comment 8 William A. Rowe Jr. 2016-01-06 13:48:24 UTC
Fast question, exactly which version of Apache 2.4?  There has been a lot of activity on mod_headers.

You may want to obtain the latest source from http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/metadata/mod_headers.c and compile this using apxs.  Simply load the replacement module in place of the old module by stopping and then starting your server instance again, and see if your observed behavior persists.

http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/modules/metadata/mod_headers.c gives you an idea of what all has changed in that module over the past few years.
Comment 9 gilperon 2016-01-06 15:29:09 UTC
Hi!

Server version: Apache/2.4.6 (CentOS)

I am not used to compile modules, but I am sure I am using the latest version of mod_headers provided by Apache and "yum update".

What do you advise me? To report the bug in another bug tracker of the mod_headers specifically?
Comment 10 Eric Covener 2016-01-06 15:45:47 UTC
Does it happen with static files? Do you use Action to run your PHP code? The best way to get help is to be able to demonstrate the simplest self-contained testcase and show it along with the input and output verbatim.
Comment 11 gilperon 2016-01-06 15:55:46 UTC
@Eric Covener this problem happens with any file, static, dynamic... I can easily reproduce it on my server with any file. I know you requested me to demonstrate this bug but since it's a server bug I have no idea how to demosntrate it... should I give you my server credentials so you can see it? I am afraid I cannot provide that, for security reasons.

But as William A. Rowe Jr. said, this bug is easily reproduceable. Install a fresh Apache server and allow htaccess (AllowOverride) on subdiectories to be interpreted by Apache. After that create these files on you public_html directory:

/aaa/.htaccess:
  Header set AAA ZZZ
  
/aaa/bbb/.htaccess:
  Header set AAA XXXXXXXX

/aaa/bbb/ccc/.htaccess:
  Header set AAA XXXXXXXX

After that request any file on this public_html directory (put an index.php inside of it) using a GET request on your browser. The response will be

AAA: ZZZ

But it should be:

AAA: XXXXXXXX

I sincerally have no idea how to make this clearer... if you have any other question I will be glad to provide you any information I can get!
Comment 12 Eric Covener 2016-01-06 18:34:12 UTC
> But as William A. Rowe Jr. said, this bug is easily reproduceable. Install a
> fresh Apache server and allow htaccess (AllowOverride) on subdiectories to
> be interpreted by Apache. After that create these files on you public_html
> directory:

I didn't get that sentiment from Bill's updates.  Any way, it works just fine for me:

$ cat built/htdocs/.htaccess built/htdocs/1/.htaccess built/htdocs/1/2/.htaccess built/htdocs/1/2/3/.htaccess
HEADER SET FOO ROOT
HEADER SET FOO SUB1
HEADER SET FOO SUB2
HEADER SET FOO SUB3


$ wget -S http://localhost/1/2/3/index.html -O- 2>&1 |grep FOO:
  FOO: SUB3
$ wget -S http://localhost/1/2/index.html -O- 2>&1 |grep FOO:
  FOO: SUB2
Comment 13 William A. Rowe Jr. 2016-01-06 18:46:57 UTC
Sorry, "I am not used to compile modules, but I am sure I am using the latest version of mod_headers provided by Apache and "yum update"."

is anything but the latest we support. 2.4.6 was shipped in July 2013, that's ancient and we simply don't install old packages to work out what had been broken in the past, we invest our efforts in improving the current code (currently 2.4.17).

r1503324 was that 2.4.6 tag, so if you look at all activity since that revision;

http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/modules/metadata/mod_headers.c

you will see there were 5 commits to mod_headers.c in these intervening years.

https://httpd.apache.org/docs/2.4/programs/apxs.html tells you how to use apxs to compile the latest sources (link in my earlier comment) as a mod_headers.so, you likely need to install some httpd-devel or httpd-tools package to obtain it.

Will look more closely if you can confirm the defect is still present in 2.4.17 mod_headers.c.
Comment 14 gilperon 2016-01-08 22:06:53 UTC
Thank you for helping. I will do what you say but I need to know how to do this. I executed "yum updated on my server" but my mod_headers didnt get "2.4.17". Why? Is this version stable? If it is stable, why yum is not installing it?
Comment 15 gilperon 2016-01-14 13:01:17 UTC
I did what you said, I installed the latest apache and the bug still happens. SO WHAT SHOULD I DO???
Comment 16 Alexey Melezhik 2016-02-01 09:34:51 UTC
Hi! 

Maybe this help:

I can't reproduce this issue on my environment ( freshly installed from source apache2 )

( https://github.com/melezhik/apache-swat is easy way to verify some existed apache2 issues against your environment )


vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 58789/
/home/vagrant/.swat/.cache/9653/prove/58789/aaa/bbb/data.txt/00.GET.t ......
ok 1 - GET 127.0.0.1/58789/aaa/bbb/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/9653/prove/wSVzxXCHHO.hdr
# body saved to /home/vagrant/.swat/.cache/9653/prove/wSVzxXCHHO
ok 2 - output match '200 OK'
ok 3 - output match '[aaa/bbb data]'
ok 4 - output match /^AAA: XXXXXXXX/
1..4
ok
/home/vagrant/.swat/.cache/9653/prove/58789/aaa/data.txt/00.GET.t ..........
ok 1 - GET 127.0.0.1/58789/aaa/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/9653/prove/zx5J0141IJ.hdr
# body saved to /home/vagrant/.swat/.cache/9653/prove/zx5J0141IJ
ok 2 - output match '200 OK'
ok 3 - output match '[aaa data]'
ok 4 - output match /^AAA: ZZZ/
1..4
ok
/home/vagrant/.swat/.cache/9653/prove/58789/aaa/bbb/ccc/data.txt/00.GET.t ..
ok 1 - GET 127.0.0.1/58789/aaa/bbb/ccc/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/9653/prove/dhejgbs_3_.hdr
# body saved to /home/vagrant/.swat/.cache/9653/prove/dhejgbs_3_
ok 2 - output match '200 OK'
ok 3 - output match '[aaa/bbb/ccc data]'
ok 4 - output match /^AAA: XXXXXXXX/
1..4
ok
All tests successful.
Files=3, Tests=12,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.15 cusr  0.00 csys =  0.18 CPU)
Result: PASS


Apache compile info:


vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ sudo ~/apache/bin/apachectl -V
Server version: Apache/2.4.18 (Unix)
Server built:   Jan 30 2016 10:17:37
Server's Module Magic Number: 20120211:52
Server loaded:  APR 1.5.1, APR-UTIL 1.5.4
Compiled using: APR 1.5.1, APR-UTIL 1.5.4
Architecture:   64-bit
Server MPM:     event
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)
Server compiled with....
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=256
 -D HTTPD_ROOT="/home/vagrant/apache/"
 -D SUEXEC_BIN="/home/vagrant/apache//bin/suexec"
 -D DEFAULT_PIDLOG="logs/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"
Comment 17 gilperon 2016-02-01 12:00:25 UTC
Really strange cause this bug is easily reproduceble in my Centos 6.5 and Centos 7.1 with fresh install. It happens in AWS and also on Linode using their Centos images... Anyone could please check this bug on envieroment like mines?
Comment 18 Eric Covener 2016-02-01 12:41:16 UTC
Not reproducible with the information provided.
Comment 19 Alexey Melezhik 2016-02-01 14:26:14 UTC
Hi gilperon@gmail.com ( Sorry could not find your name at this issue )

You could try to repeat the issue on _your_ environment with swat test suite. It's easy to install and setup - follow https://github.com/melezhik/apache-swat

Once you install and configure tests suite just say:

swat -t 58789 

And provide an output , thanks.

At least this test tries to verify the situation described at this issue ...
Comment 20 gilperon 2016-02-01 16:32:08 UTC
Hey I discovered something that you didnt do on your test!!

Inside the file "/aaa/.htaccess" you cant just do this:

Header set AAA ZZZ

You have to do this:

<FilesMatch ".*">

    Header set AAA ZZZ

</FilesMatch>

And in the other htaccess files you dont use FilesMatch you just use

Header set AAA XXXXXXXX

Please please try this, I am sure you will be able to reproduce the bug. I am really sure! Could you do this favor, please?
Comment 21 Alexey Melezhik 2016-02-01 16:43:02 UTC
Ok, you are a right , after changing to htdocs/58789/aaa/.htaccess


<FilesMatch ".*">

    Header set AAA ZZZ

</FilesMatch>

Bug is reproduced ( I have rewrite a checking logic  abit to emphasize the issue ) and look like this:


vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 58789/
/home/vagrant/.swat/.cache/13389/prove/58789/aaa/data.txt/00.GET.t ..........
ok 1 - GET 127.0.0.1/58789/aaa/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/13389/prove/HSHCipQEVF.hdr
# body saved to /home/vagrant/.swat/.cache/13389/prove/HSHCipQEVF
ok 2 - output match '200 OK'
ok 3 - output match '[aaa data]'
ok 4 - output match /^AAA: ZZZ/
1..4
ok
/home/vagrant/.swat/.cache/13389/prove/58789/aaa/bbb/ccc/data.txt/00.GET.t ..
ok 1 - GET 127.0.0.1/58789/aaa/bbb/ccc/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/13389/prove/tpFu_FHhzp.hdr
# body saved to /home/vagrant/.swat/.cache/13389/prove/tpFu_FHhzp
ok 2 - output match '200 OK'
ok 3 - output match '[aaa/bbb/ccc data]'
ok 4 - output match /^AAA: \S+/
not ok 5 - 'AAA: ZZZ' match 'XXXXXXXX'

#   Failed test ''AAA: ZZZ' match 'XXXXXXXX''
#   at /usr/local/share/perl/5.20.2/swat.pm line 218.
1..5
# Looks like you failed 1 test of 5.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/5 subtests
/home/vagrant/.swat/.cache/13389/prove/58789/aaa/bbb/data.txt/00.GET.t ......
ok 1 - GET 127.0.0.1/58789/aaa/bbb/data.txt succeeded
# http headers saved to /home/vagrant/.swat/.cache/13389/prove/yQrvMI3ulJ.hdr
# body saved to /home/vagrant/.swat/.cache/13389/prove/yQrvMI3ulJ
ok 2 - output match '200 OK'
ok 3 - output match '[aaa/bbb data]'
ok 4 - output match /^AAA: \S+/
not ok 5 - 'AAA: ZZZ' match 'XXXXXXXX'

#   Failed test ''AAA: ZZZ' match 'XXXXXXXX''
#   at /usr/local/share/perl/5.20.2/swat.pm line 218.
1..5
# Looks like you failed 1 test of 5.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/5 subtests

Test Summary Report
-------------------
/home/vagrant/.swat/.cache/13389/prove/58789/aaa/bbb/ccc/data.txt/00.GET.t (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  5
  Non-zero exit status: 1
/home/vagrant/.swat/.cache/13389/prove/58789/aaa/bbb/data.txt/00.GET.t    (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  5
  Non-zero exit status: 1
Files=3, Tests=14,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.15 cusr  0.01 csys =  0.19 CPU)
Result: FAIL



So, both aaa/bbb and aaa/bbb/cc now returns unexpected(?) headers.

Now I address this to developers, as I am only acting here as QA automation support guy ... 

All recent changes for test suite are commuted - https://github.com/melezhik/apache-swat/commit/e8e2c810762d5a98d4735ba173f5c774273e6879
Comment 22 gilperon 2016-02-01 16:49:13 UTC
You are awesome man! Thank you so much for checking this bug! I was afraid you could not reproduce it for some specific reason that I am not aware, but I am really glad you reproduced it. Thank you so much! I hope now some developer comes with a clever solutino :)
Comment 23 Eric Covener 2016-02-01 17:19:39 UTC
This seems to be working as designed, but the docs could use some elaboration on a few points:

https://httpd.apache.org/docs/2.4/sections.html

1) htacess context is equivalent to Directory context.
2) The implications related to nesting of things like <Files> in Directory and the false assumption that deeper htaccess = higher precedence.

-/-

The manual clearly states that Files/FilesMatch has precedence. That precedence is not overridden because there is a more closely matched Directory section.

If you want the matching Directory to have precedence for overlapping directives, use <Files *> to make them all fit into the Files merging precedence

-/-

Reclassifying to docs so someone (elukey?) can take a crack at explaining it.
Comment 24 gilperon 2016-02-01 17:40:45 UTC
Nice to know that. I also had problems in the past with mod_rewrite cause even placing rules at the top of the file the filesmatch at the bottom was being "hit" first!

Is there any docs where I can understand what types of codes inside htaccess are executed first cause in my experience the order that some lines of code appears in the hattcess file are not always executed as expected.
Comment 25 Eric Covener 2016-02-01 17:41:54 UTC
(In reply to gilperon from comment #24)
> Nice to know that. I also had problems in the past with mod_rewrite cause
> even placing rules at the top of the file the filesmatch at the bottom was
> being "hit" first!
> 
> Is there any docs where I can understand what types of codes inside htaccess
> are executed first cause in my experience the order that some lines of code
> appears in the hattcess file are not always executed as expected.

The document I linked to in my previous comment talks about how modules' _configuration_ is read and merged.  Execution doesn't happen line by line, it happens after all of the reading and merging is done.
Comment 26 gilperon 2016-02-01 17:43:39 UTC
at what line is that?
Comment 27 Alexey Melezhik 2016-02-01 18:50:11 UTC
Ok, guys I hope now it is all getting cleared, I will revert the test suite, so it does not fail.
Comment 28 Luca Toscano 2016-02-08 22:57:21 UTC
I have a question for gilperon@ (sorry didn't see your name anywhere!):

https://httpd.apache.org/docs/2.4/sections.html might be improved with examples but I would also be interested in what documentation did you read before creating the bug. It would help us figure out what are the most common places in which the information is needed :)

Luca
Comment 29 gilperon 2016-02-10 18:27:58 UTC
Hi Mr. Luca,

I read only the Apache 2.2 documentation (but I am usinh 2.4, I didnt know 2.4 documentation was already available). I also tried to get help from stackoverflow community and they told me to report this as a bug, so I did!

I think apache should make it a lot clearer what rules from htaccess are processed first and what are later. Maybe apache should say what order the htaccess directives are processed. Mod_rewirte I know are processed way after <filesmatch> even if I put all mod_rewrite rules at the top of htaccess the <filesmatch> rules will be processed first. To me this should be a bug or a misconception of how it works, but Apache should  correct this bug OR make it a lot clearer which order things are executed by Apache cause I manage webserver for about 6 years and never had heard of the order of the apache directives.
Comment 30 Eric Covener 2016-02-10 18:36:29 UTC
(In reply to gilperon from comment #29)
> Hi Mr. Luca,
> 
> I read only the Apache 2.2 documentation (but I am usinh 2.4, I didnt know
> 2.4 documentation was already available). I also tried to get help from
> stackoverflow community and they told me to report this as a bug, so I did!
> 
> I think apache should make it a lot clearer what rules from htaccess are
> processed first and what are later. Maybe apache should say what order the
> htaccess directives are processed. Mod_rewirte I know are processed way
> after <filesmatch> even if I put all mod_rewrite rules at the top of
> htaccess the <filesmatch> rules will be processed first. To me this should
> be a bug or a misconception of how it works, but Apache should  correct this
> bug OR make it a lot clearer which order things are executed by Apache cause
> I manage webserver for about 6 years and never had heard of the order of the
> apache directives.

The misconception here, that the sections.html could address is the contrasting of how _configuration_ is merged (sections.html) separately from how modules operate on the net result of the merged configuration.

We also need an example that emphasizes that the current sections.html info results in a configuration of "foo2" for the module to eventually act upon (and to note htaccess/<directory> equivalence.

# foo is merged with foo3, resulting in foo3
# output of previous merge, foo3 is merged with foo2, resulting in foo2
<Directory />
DirectiveWhichIsOverriddenWhenReused foo
<files *>
DirectiveWhichIsOverriddenWhenReused foo2
</files>
</Directory>

<Directory /var/www>
DirectiveWhichIsOverriddenWhenReused foo3
</Directory>
Comment 31 Luca Toscano 2016-02-11 07:40:30 UTC
gilperon@: Thank you for the insight about the documentation that you followed!

Covener: Thanks for the example, I am going to try adding a new section/sub-section to the page explaining the concept. I am also convinced that we should build a list of "essentials" doc pages to read (if not already present) and link them right beside the httpd download link (and front page maybe) to help people in finding those docs..

Anyhow, I'll start with the sections.html :)

Luca
Comment 32 gilperon 2016-02-11 16:28:56 UTC
Yes, your ideia would be great! Grouping only the most important topics at one single page with links to others would be amazing.

Thank you so much and I would like to be informed when you finish the sections page cause I would like to learn the correct order apache processes directives inside htaccess :) Thank you!
Comment 33 Luca Toscano 2016-02-15 13:24:10 UTC
gilperon@: would http://httpd.apache.org/docs/current/ have helped if linked somewhere? It is a very good example of "essentials", but I am not sure where to link it to increase its visibility (download page? front page? etc..).

Covener: I am adding new comments to http://httpd.apache.org/docs/current/sections.html
Comment 34 gilperon 2016-02-15 13:27:00 UTC
http://httpd.apache.org/docs/current/ is a great link I was not aware of its existence... maybe google should index it with higher priority so people come to know this page when they are looking for apache stuff.

However I cant find which part of that link explains in which order apache processes rules inside htaccess. Should it be "Configuration Sections"? There is nothing related to this doubt I have in there...
Comment 35 Luca Toscano 2016-02-15 13:33:53 UTC
Yes exactly, sections.html explains how directives like Directory (and .htaccess), File, Location, etc.. are merged together. I'll add an example about your use case, but at the same time I'd like to point more people to the right direction straight away..
Comment 36 gilperon 2016-02-15 13:47:52 UTC
I appreciate your efforts into this. Sections has a good documentation but I think more examples should be provided, mixing the most common apache modules (like mod_rewrite) with filesmatch, location... apache directives. You could also add a HEADER example (like the "bug" I thought I had found) cause it's easier to end user debug the rules using browser tools like F12 on Chrome.
Comment 37 Luca Toscano 2016-02-16 12:34:59 UTC
First attempt to address the issue:

http://httpd.apache.org/docs/trunk/sections.html#merging

I added an example and more info about how sections are merged.
Comment 38 gilperon 2016-02-16 14:28:33 UTC
Great, really great :) I will surelly use it on future problems I have with htaccess.

As you didnt explicitly said, I assume mod_rewrite and other Apaache modules are interpreted always after "5. <If>", correct?
Comment 39 Luca Toscano 2016-02-18 08:04:16 UTC
Hi gilperon@!

After a chat with Eric I re-wrote the page adding a dedicated section to it:

https://httpd.apache.org/docs/trunk/sections.html#merging

Still under review so it might change a bit, please be patient I am learning with you  :)

The main gotcha is that configuration directives belonging to httpd's core (from Directory to If) work to create configurations for modules, that in turn use the information to do their work. 

So mod_rewrite's directives are not "evaluated" afterwards, but simply mod_rewrite leverages the configuration returned to it by the httpd core for a certain request. In the case that we were discussing, setting Header to the value of the filesmatch block (due to its importance in the merging order).

To be precise: 
1) each module decides what to do when a configuration merge is requested (directory->filesmatch etc..), the default is to override.
2) each module decides what to do with the configuration returned by httpd core. The configuration to set the Header AAA to ZZZ is available to mod_headers when it is invoked in the HTTP request processing workflow (like /aaa/bbb/ccc/index.php) and the module can do whatever it want with it. For example, it can discard the directive because deprecated! Usually of course the module does something meaningful with it, it was just to give you the idea :)

This is my understanding of the powerful configuration/running system of httpd! Let me know if it makes sense!

Luca
Comment 40 gilperon 2016-02-18 12:30:03 UTC
@Luca

You are awesome, now that section is great! I totally agree with what you said: those apache directives (Directory, Filesmatch...) are used to "filter" modules use cases. Sometimes we want some modules to act only when certain Directories conditions are applied. You are correct. But sometimes not. Check this example:



RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteRule ^.*$ https://%{HTTP_HOST}?redirected1 [R=301,NE,L]

<If "%{REQUEST_URI} =~ m#^$#">

	RewriteRule ^.*$ https://%{HTTP_HOST}?redirected2 [R=301,NE,L]

</If>


In the case above, if the user requests "www.xxx.com" I would like it first to be redirected to "https://%{HTTP_HOST}?redirected1" and after that to "https://%{HTTP_HOST}?redirected2".

But the "bug" that I reported in this thread makes it differente: Apache will actually first process "https://%{HTTP_HOST}?redirected2" and after that "https://%{HTTP_HOST}?redirected1".

To overcome this "bug" I have to use a "hack" (I know this word is not great here, but that's how I name something that I have to do in order to correct some unexpected or bad programmed feature in a language):


<If "%{REQUEST_URI} =~ m#^.*$#">

RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteRule ^.*$ https://%{HTTP_HOST}?redirected1 [R=301,NE,L]

</If>

<If "%{REQUEST_URI} =~ m#^$#">

	RewriteRule ^.*$ https://%{HTTP_HOST}?redirected2 [R=301,NE,L]

</If>


You see, I just had to create '<If "%{REQUEST_URI} =~ m#^.*$#">' to try to overcome this problem. I know, I really know, I could program this code in a much better and faster way, I know it. It's only an example of how apache's confusing processing rules order is not intuitive and may cause really unexpected results. This "bug" took me at least 8 months to try to get it understood/solved. I lived with this "Header" "bug" for a long time but since it was not too critical I didnt care, but latelly I started using a CDN and I really had to know how to fix this problem.



By the way, the section "Relationship between modules and configuration sections" is a m a z i n g! It will surelly clarify lots of misconception about apache rules order :)
Comment 41 Luca Toscano 2016-02-19 13:29:15 UTC
Glad that you liked it! Credits to Eric Covener for the patience in explaining the concept.

I backported it to:

http://httpd.apache.org/docs/2.2/sections.html#merging
http://httpd.apache.org/docs/2.4/sections.html#merging

I believe that his bug can be closed, but please re-open it if anything is missing!

Luca