Bug 42050 - proxying does not work
Summary: proxying does not work
Status: RESOLVED FIXED
Alias: None
Product: Lenya
Classification: Unclassified
Component: Miscellaneous (show other bugs)
Version: Trunk
Hardware: Other other
: P2 blocker
Target Milestone: 2.0
Assignee: Lenya Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-04-04 11:00 UTC by Richard Frovarp
Modified: 2007-07-16 04:03 UTC (History)
0 users



Attachments
the output of default/authoring/index.html with proxying enabled (16.51 KB, text/plain)
2007-05-07 05:15 UTC, J
Details
Add option for relative URLs to UuidToUrlTransformer (5.40 KB, patch)
2007-05-11 03:59 UTC, Andreas Hartmann
Details | Diff
Add config+param option for relative URLs to UuidToUrlTransformer (6.26 KB, patch)
2007-05-15 02:28 UTC, Andreas Hartmann
Details | Diff
Use lenya-document:uuid URLs for navigation elements in sitetree module (10.79 KB, patch)
2007-05-22 04:58 UTC, Andreas Hartmann
Details | Diff
proxyTransformer version 1 (8.88 KB, patch)
2007-05-22 07:46 UTC, Thorsten Scherler
Details | Diff
Aggregate patch from all other with version 2 of the proxyTransformer (28.98 KB, patch)
2007-05-23 03:35 UTC, Thorsten Scherler
Details | Diff
version 3 fixing error when no proxy config is used (29.29 KB, patch)
2007-05-24 05:52 UTC, Thorsten Scherler
Details | Diff
Patch for docu explaining the global rewrite url (1.76 KB, patch)
2007-05-28 01:02 UTC, Thorsten Scherler
Details | Diff
Version 4 of the all in one (52.65 KB, patch)
2007-05-28 09:06 UTC, Thorsten Scherler
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Frovarp 2007-04-04 11:00:22 UTC
The live tab under authoring doesn't honor proxy settings. This is causing mixed
content errors in browser due to the page being loaded from HTTPS and portions
of the content (css for example) being loaded from HTTP.
Comment 1 J 2007-05-07 05:02:40 UTC
proxying does not work at all for me.
many parts of the lenya pipelines do not honor the proxy settings - notably the
styling and javascript of the GUI.

i'm setting this to blocker, because SSL proxying support is mandatory for
professional deployments, and HTTP proxying is needed to clean up the URLs -
"<pubname>/live" is not acceptable for most real-world installations.
Comment 2 J 2007-05-07 05:15:20 UTC
Created attachment 20139 [details]
the output of default/authoring/index.html with proxying enabled

this is the output of the default publication with the following proxy
settings:
<proxies>
  <proxy area="live" ssl="true" 
url="https://stackingdwarves.net/cms/default/live"/>
  <proxy area="live" ssl="false" 
url="http://stackingdwarves.net/cms/default/live"/>
  <proxy area="authoring" ssl="true" 
url="https://stackingdwarves.net/cms/default/authoring"/>
  <proxy area="authoring" ssl="false" 
url="http://stackingdwarves.net/cms/default/authoring"/>
</proxies>

the request URL was "https://stackingdwarves.net/cms/default/index.html".

the following issues arise:

* the lenya GUI does not honor proxying (see stylesheet and javascript links in
header and tab links in <div id="lenya-menus"/>
* the language selector creates absolute URLs that are not properly rewritten
* same for the search module
* links in content area sometimes miss the "https" (but not always!)
Comment 3 Richard Frovarp 2007-05-07 09:38:48 UTC
Here's what I have, which is working with the exception of the live tab.
However, I don't have any other use for SSL on this host other than Lenya
editing. I tried to use a subdirectory under SSL so I could use it for other
things and that didn't work at all.

In the publication.xml files:

<proxy area="live" ssl="false" url="http://publication.us"/>
<proxy area="authoring" ssl="false" url="https://myhost.edu/williston/authoring"/>

The Apache config file for publication.us:

ProxyRequests Off 
RewriteEngine On
RewriteRule ^/edit https://myhost.edu/williston/ [R]
RewriteRule ^/$ index.html [R]  
RewriteRule ^/williston/live/(.*)$ $1 [R,L] 
RewriteRule ^/(.*) http://myhost.edu:8890/williston/live/$1 [P]
ProxyPassReverse / http://myhost.edu:8890/

In the Apache config file for SSL:

ProxyRequests Off
RewriteEngine On
RewriteLog logs/test.rewrite.log
RewriteLogLevel 0
RewriteRule ^/([^/\.]+)$ $1/ [R]
RewriteRule ^/([^/\.]+)/$ https://myhost.edu/$1/authoring/index.htm l [R,L]
RewriteRule ^/(.*) http://myhost.edu:8890/$1 [P,L]
ProxyPassReverse / http://myhost.edu:8890/
Comment 4 J 2007-05-09 09:27:38 UTC
(In reply to comment #3)
> <proxy area="authoring" ssl="false" url="https://myhost.edu/williston/authoring"/>

ssl=false and a https uri. it looks like you want to bypass something. but does
it make any difference?

regarding the live proxy, have you succeeded in hiding the <pubname>/live part?
for the visible urls, i can make it to work by rewriting ^/(.*) to
/<pubname>/live/$1, but i also need to match /lenya and /module first, otherwise
stuff will break...
Comment 5 Richard Frovarp 2007-05-09 09:57:02 UTC
(In reply to comment #4)
> regarding the live proxy, have you succeeded in hiding the <pubname>/live part?
> for the visible urls, i can make it to work by rewriting ^/(.*) to
> /<pubname>/live/$1, but i also need to match /lenya and /module first, otherwise
> stuff will break...
> 

I see what you're talking about now. We're not using the SVG module or the
language selector. Our publications at the moment are very simple and don't need
anything from /lenya or /module (in live). Those would have to be matched first. 

So yes, I've hidden <pubname>/live, but I'm only doing basic publications, which
does make it a lot easier.
Comment 6 Andreas Hartmann 2007-05-11 03:59:24 UTC
Created attachment 20170 [details]
Add option for relative URLs to UuidToUrlTransformer

To enable relative URLs, change the transformer configuration:

    <map:transformer name="uuid2url" ...>
      <urls type="relative"/>
    </map:transformer>
Comment 7 Andreas Hartmann 2007-05-15 01:48:44 UTC
(In reply to comment #6)
> Created an attachment (id=20170) [edit]
> Add option for relative URLs to UuidToUrlTransformer

Caution, relative URLs are not compatible with the WYSIWYG editors (URL/UUID
transformation). The transformer has to be parameterized.
Comment 8 Andreas Hartmann 2007-05-15 02:28:31 UTC
Created attachment 20201 [details]
Add config+param option for relative URLs to UuidToUrlTransformer
Comment 9 Thorsten Scherler 2007-05-16 07:13:20 UTC
First of all I am not using proxy ATM so I am shooting in the dark. 

(In reply to comment #2)
> Created an attachment (id=20139) [edit]
> the output of default/authoring/index.html with proxying enabled
> 
> this is the output of the default publication with the following proxy
> settings:
> <proxies>
>   <proxy area="live" ssl="true" 
> url="https://stackingdwarves.net/cms/default/live"/>
>   <proxy area="live" ssl="false" 
> url="http://stackingdwarves.net/cms/default/live"/>
>   <proxy area="authoring" ssl="true" 
> url="https://stackingdwarves.net/cms/default/authoring"/>
>   <proxy area="authoring" ssl="false" 
> url="http://stackingdwarves.net/cms/default/authoring"/>
> </proxies>
> 
> the request URL was "https://stackingdwarves.net/cms/default/index.html".
> 
> the following issues arise:
> 
> * the lenya GUI does not honor proxying (see stylesheet and javascript links in
> header and tab links in <div id="lenya-menus"/>

Regarding css and javascript did you follow
http://lenya.apache.org/docs/1_2_x/components/deployment/proxying.html

# RewriteRules for css and images
     RewriteRule      ^/lenya/sandbox/(.*)/images/(.*)$
http://cms.live.ch:8080/lenya/sandbox/$1/images/$2 [L]
     RewriteRule      ^/(.*\.css$) http://cms.live.ch:8080/$1  [L]

     RewriteRule      ^/(lenya/[^/]+)/(.*)$   http://cms.live.ch:8080/$1/live/$2
 [P,L]

That may bring some of the css back. I reckon js is quite the same.

The problem is all the stuff that is not pub specific as e.g.
/lenya/menu/images/lenya-logo.gif or /modules/languageselector/flag-es-13.png
does not get rewritten from the <proxy/> tags.

Adding a RewriteRules like
RewriteRule      ^/lenya/(.*)$ http://cms.live.ch:8080/lenya/$1
RewriteRule      ^/modules/(.*)$ http://cms.live.ch:8080/modules/$1
should bring back another big part that seems to be broken.

However still the question why sometimes the rewrite miss the "https".

Can somebody test whether the ^/lenya and ^/modules is bringing back some more
resources.
Comment 10 Thorsten Scherler 2007-05-22 01:25:28 UTC
I tried the patch from andreas and there are still some issues.

I am atm looking at the xsl:
<link rel="stylesheet"
href="{$contextPath}/{$pubId}/modules/homepage/css/homepage.css" type="text/css"/>

which did not get rewritten:
<link type="text/css" href="/default/modules/homepage/css/homepage.css"
rel="stylesheet" />

However we actually include it two times where the pagee2xhtml is using
 <link rel="stylesheet" href="{$root}/css/{$document-type}.css" type="text/css"/>

where <!-- {context-prefix}/{publication-id}/{area} -->
  <xsl:param name="root"/>

Meaning the proxy rewritting is working fine if we follow:
{context-prefix}/{publication-id}/{area}

It is indeed failing if not use as the above.
Comment 11 Thorsten Scherler 2007-05-22 02:43:46 UTC
I am ATM trying to fix each part of the aggregation and doing the search stuff.

To debug I added:
<map:match pattern="authoring/navigation-element/*/*/*/*/*/**">
        <map:generate src="cocoon://modules/sitetree/{1}/{2}/{3}/{4}/{5}/{6}.xml"/>
        <map:transform type="uuid2url"/>
        <map:serialize type="xml"/>
      </map:match>

I found out that in UuidToUrlTransformer the following prevents the rewrite.
Where I think the comment is not correct since e.g. forms @action would hardly
ever use a lenyadoc: in the link. Further why the comment about:
 // don't rewrite /{pub}/modules/... ¿?

 /*
                             * This is legacy code. It rewrites links to
                             * non-document images (in resources/shared). These
                             * images shouldn't be referenced in documents since
                             * this violates the separation between content and
                             * layout.
                             */
                            String prefix = "/" +
this.currentDocument.getPublication().getId()
                                    + "/";
                            if (linkUrl.startsWith(prefix)) {
                                String pubUrl = linkUrl.substring(prefix.length());
                                String area = pubUrl.split("/")[0];
                                
                                // don't rewrite /{pub}/modules/...
                                if (area.equals(Publication.AUTHORING_AREA)) {
                                    String areaUrl =
pubUrl.substring(area.length());
                                    String newUrl = this.contextPath + prefix
                                            + this.currentDocument.getArea() +
areaUrl;
                                    setAttribute(newAttrs, attributeNames[i],
newUrl);
                                }
                            }
Comment 12 Andreas Hartmann 2007-05-22 02:54:05 UTC
(In reply to comment #11)
> I am ATM trying to fix each part of the aggregation

I'd suggest that we don't include URLs in the navigation elements, but use
UUID-based links and leave the rewriting to the UuidToUrlTransformer at the end
of the presentation pipeline. I have a patch in my sandbox, but it needs a bit
more testing.
Comment 13 Thorsten Scherler 2007-05-22 03:06:31 UTC
I am ATM writing a simple ProxyTransformer for testing. 

IMO the whole issue is quite simple to solve if we assume that we need to
rewrite all links that starts with "/" to the proxy url. 

Furthermore the whole proxy per area is suboptimal. IMO the live area should be
not prefixed and the rest just will replace "/" with
"https://stackingdwarves.net/cms/{$pub}" if no pub is defined in the first part
after "/" but e.g. /lenya or /module it would be just rewritten with the first part.

Anyway I will attach what I come up with shortly.
Comment 14 Andreas Hartmann 2007-05-22 04:58:59 UTC
Created attachment 20240 [details]
Use lenya-document:uuid URLs for navigation elements in sitetree module

This patch changes the syntax of navigation element calls, so custom sitemaps
have to be updated.
Comment 15 Andreas Hartmann 2007-05-22 05:00:36 UTC
How about adding an option for relative URLs to the BaseUrlModule as well?
It could for instance return "../.." for the request URL "/foo/bar/baz".
Comment 16 Andreas Hartmann 2007-05-22 05:03:51 UTC
(In reply to comment #11)
> Further why the comment about:
>  // don't rewrite /{pub}/modules/... ¿?

That's because of bug 42140.
Comment 17 Andreas Hartmann 2007-05-22 05:42:39 UTC
(In reply to comment #14)
> Created an attachment (id=20240) [edit]
> Use lenya-document:uuid URLs for navigation elements in sitetree module
> 
> This patch changes the syntax of navigation element calls, so custom sitemaps
> have to be updated.

BTW, the patch attaches a uuid2url.extension parameter with the document's
native extension (e.g., .zip) to the lenya-document URL. You might want to
change this if you'd like the links to end with .html.
Comment 18 Thorsten Scherler 2007-05-22 07:46:34 UTC
Created attachment 20241 [details]
proxyTransformer version 1

This is the first version of the proxy transformer. 

The open issues are still the following cases:
a) <link type="text/css" href="/default/modules/homepage/css/homepage.css"
rel="stylesheet" /> - Using pub but not a valid area
b) <a href="/index.html"><span>Apache Lenya Homepage</span></a> - the menu
links which should point to the website IMO anyway
c) <img alt="he" src="/modules/languageselector/flag-he-13.png" /> and <script
src="/lenya/menu/menu.js" type="text/javascript"> </script> - point to the
absolute root.

a) and c) can be solved introducing a global proxy url. 
Meaning something like:
<proxies url="https://www.host.com/lenya/">

Will now test the patch from andreas.
Comment 19 Andreas Hartmann 2007-05-23 01:36:18 UTC
+    try {
+      area = factory.getFromURL(linkUrl).getArea();
+    } catch (RuntimeException e) {
+      getLogger().warn("global link");
+    }

We shouldn't use exceptions for flow control ... :)
You can use factory.isDocument(linkUrl) instead.
Comment 20 Thorsten Scherler 2007-05-23 03:35:34 UTC
Created attachment 20246 [details]
Aggregate patch from all other with version 2 of the proxyTransformer
Comment 21 Thorsten Scherler 2007-05-24 05:52:06 UTC
Created attachment 20268 [details]
version 3 fixing error when no proxy config is used
Comment 22 Andreas Hartmann 2007-05-25 01:03:11 UTC
For the sake of a static export, it should be possible to render all internal
links as relative URLs.
Comment 23 Andreas Hartmann 2007-05-25 01:54:27 UTC
(In reply to comment #18)

> The open issues are still the following cases:
> b) <a href="/index.html"><span>Apache Lenya Homepage</span></a> - the menu
> links which should point to the website IMO anyway

I changed the link to point to the website. The "About Lenya" page is now a
usecase and therefore independent from proxying.
Comment 24 Andreas Hartmann 2007-05-25 01:57:24 UTC
Maybe we should move the option for relative paths from the UuidToUrlTransformer
to the ProxyTransformer.
Comment 25 Andreas Hartmann 2007-05-25 02:00:15 UTC
BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We could

- transform the CSS to XML
- apply the ProxyTransformer (instead of the XSLT we use now)
- convert them back to CSS
Comment 26 J 2007-05-26 18:44:51 UTC
(In reply to comment #25)
> BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We
could
> 
> - transform the CSS to XML
> - apply the ProxyTransformer (instead of the XSLT we use now)
> - convert them back to CSS

i'd rather we mandate the use of <foo>.xml instead of foo.css in all cases. then
we can add the proxying stuff to the stylesheet that currently handles <root/>
and friends. i think all our users should use that mechanism. even if it is not
used, it won't be a performance issue, because both browsers and cocoon are very
good at caching those things that change rarely.
let's add some docs to the effect that "if you want to proxy, make sure you put
all your css through the xml grinder..."
Comment 27 Thorsten Scherler 2007-05-28 01:02:40 UTC
Created attachment 20280 [details]
Patch for docu explaining the global rewrite url

For all links that are not pointing to a document the global proxy rule 
<proxies ssl="false" root="http://www.host.com/no-ssl/">
is applied. 

For example a link such as /modules/myModule/somePath/some.thing will be
rewritten to "http://www.host.com/no-ssl/modules/myModule/somePath/some.thing".


In short "/" will be replaced with "{proxies/@root}".
Comment 28 Thorsten Scherler 2007-05-28 01:24:21 UTC
(In reply to comment #25)
> BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We
could
> 
> - transform the CSS to XML
> - apply the ProxyTransformer (instead of the XSLT we use now)
> - convert them back to CSS

yes.

We could copy the forrest approach for this
<map:transform type="pattern" src="chaperon/grammars/link.xlex"/>
<map:transform src="chaperon/stylesheets/pattern2link.xsl"/>

http://svn.apache.org/viewvc/forrest/trunk/main/webapp/resources/chaperon/

Where the first will search for patterns we want to replace. The link.xlex can
be directly reused. The second will need some changes. 

Need to look into this.
Comment 29 Thorsten Scherler 2007-05-28 09:06:53 UTC
Created attachment 20282 [details]
Version 4 of the all in one

The patch brings rewritting of the css via chaperon. Includes a fix in the
proxyTransformer if the resource is no document. 

One thing that I do not like is that one need to implement 
<map:resource name="proxy-css"> in all sitemaps that is calling the resource.
Seems that the map:resources are not inheritable. :(
Comment 30 Thorsten Scherler 2007-06-07 23:38:02 UTC
I commited the Version 4 of the patches WITHOUT 
- UuidToUrlTransformer.java
- sitetree2nav.xsl

Since they partly seems to be applied to trunk already.

Please test and close this bug if it is working.
Comment 31 Richard Frovarp 2007-06-11 07:28:16 UTC
localhost:8888/lenya/css/menu.css doesn't load in the default publication. Error
indicates it is treating /lenya as a publication:

The configuration file
[/home/rfrovarp/asf/lenya-1.4.x/build/lenya/webapp/lenya/pubs/lenya/config/publication.xml]
does not exist! 

localhost:8888/lenya/css/boxes.css also shows the same error. /modules seems to
be working.
Comment 32 Andreas Hartmann 2007-06-12 02:27:20 UTC
The proxy-css resource in lenya.xmap assumes that the call goes to a publication:

<map:parameter name="root"
value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>
Comment 33 Andreas Hartmann 2007-06-12 02:32:16 UTC
IIUC the ProxyTransformer doesn't handle non-publication URLs either ...
Comment 34 Thorsten Scherler 2007-06-12 05:38:08 UTC
(In reply to comment #32)
> The proxy-css resource in lenya.xmap assumes that the call goes to a publication:
> 
> <map:parameter name="root"
> value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>
> 

<map:transform src="fallback://lenya/resources/css/css.xsl">
          <map:parameter name="contextprefix" value="{request:contextPath}"/>
          <map:parameter name="root"
           
value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>
        </map:transform>

That is not proxy specific but css.xsl. Maybe the best if the base-url returns
"" instead of throwing an exception.

Comment 35 Thorsten Scherler 2007-06-12 05:39:35 UTC
(In reply to comment #33)
> IIUC the ProxyTransformer doesn't handle non-publication URLs either ...

It did! But ATM it is not anymore and I have no clue why 
Comment 36 Andreas Hartmann 2007-06-12 06:55:30 UTC
Strange, if I uncomment the root parameter in lenya.xmap, I end up with an empty
menu.css. The Chaperon parser doesn't produce anything.
Comment 37 Andreas Hartmann 2007-06-12 07:52:15 UTC
In my working copy the PatternTransformer seems to fail if the input is plain text.
Comment 38 Andreas Hartmann 2007-06-12 07:57:26 UTC
The css.xsl doesn't strip the <css> now, but wraps the CSS in chaperon text
elements (revision 546516).
Comment 39 Andreas Hartmann 2007-06-13 08:22:34 UTC
The uuid2url and proxy transformers are now complementing each other. I did
quite a lot of testing on Tomcat, but not with a real proxy setting.
Comment 40 Andreas Hartmann 2007-06-14 03:56:08 UTC
I have configured Apache 2.2.4 with mod_proxy on my local machine and did some
testing without SSL. I also added a little tutorial how to setup the test
environment.

With Jetty everything looks good. But when I use Tomcat, I get "The session
doesn't contain the identity" when I try to log in.
Comment 41 Andreas Hartmann 2007-06-15 02:46:00 UTC
A major problem with the "global proxy" approach are requests to
publication-based module URLs. If they need authorization, the requests hit the
login screen. This occurs for instance with the TinyMCE JavaScript requests like

http://lenya.example.com/default/modules/tinymce/javascript/tiny_config.js

Making these URLs accessible by the world is certainly out of the question.
Comment 42 Andreas Hartmann 2007-06-15 02:54:55 UTC
(In reply to comment #41)
> A major problem with the "global proxy" approach are requests to
> publication-based module URLs. If they need authorization, the requests hit the
> login screen.

A workaround is to use the same proxy server for both publication-specific and
global resources, but this would mask the "lenya/..." and "modules/..." URL spaces.

Comment 43 Andreas Hartmann 2007-06-25 06:12:05 UTC
Non-SSL proxying works quite well now. Has to be tested with SSL yet.
Comment 44 Andreas Hartmann 2007-07-16 04:03:17 UTC
I tested it with SSL and mod_proxy_ajp. If the problems still occur, please reopen.