Bug 42050

proxying does not work at all for me.
many parts of the lenya pipelines do not honor the proxy settings - notably the
styling and javascript of the GUI.

i'm setting this to blocker, because SSL proxying support is mandatory for
professional deployments, and HTTP proxying is needed to clean up the URLs -
"<pubname>/live" is not acceptable for most real-world installations.

Created attachment 20139 [details]
the output of default/authoring/index.html with proxying enabled

this is the output of the default publication with the following proxy
settings:
<proxies>
  <proxy area="live" ssl="true" 
url="https://stackingdwarves.net/cms/default/live"/>
  <proxy area="live" ssl="false" 
url="http://stackingdwarves.net/cms/default/live"/>
  <proxy area="authoring" ssl="true" 
url="https://stackingdwarves.net/cms/default/authoring"/>
  <proxy area="authoring" ssl="false" 
url="http://stackingdwarves.net/cms/default/authoring"/>
</proxies>

the request URL was "https://stackingdwarves.net/cms/default/index.html".

the following issues arise:

* the lenya GUI does not honor proxying (see stylesheet and javascript links in
header and tab links in <div id="lenya-menus"/>
* the language selector creates absolute URLs that are not properly rewritten
* same for the search module
* links in content area sometimes miss the "https" (but not always!)

Here's what I have, which is working with the exception of the live tab.
However, I don't have any other use for SSL on this host other than Lenya
editing. I tried to use a subdirectory under SSL so I could use it for other
things and that didn't work at all.

In the publication.xml files:

<proxy area="live" ssl="false" url="http://publication.us"/>
<proxy area="authoring" ssl="false" url="https://myhost.edu/williston/authoring"/>

The Apache config file for publication.us:

ProxyRequests Off 
RewriteEngine On
RewriteRule ^/edit https://myhost.edu/williston/ [R]
RewriteRule ^/$ index.html [R]  
RewriteRule ^/williston/live/(.*)$ $1 [R,L] 
RewriteRule ^/(.*) http://myhost.edu:8890/williston/live/$1 [P]
ProxyPassReverse / http://myhost.edu:8890/

In the Apache config file for SSL:

ProxyRequests Off
RewriteEngine On
RewriteLog logs/test.rewrite.log
RewriteLogLevel 0
RewriteRule ^/([^/\.]+)$ $1/ [R]
RewriteRule ^/([^/\.]+)/$ https://myhost.edu/$1/authoring/index.htm l [R,L]
RewriteRule ^/(.*) http://myhost.edu:8890/$1 [P,L]
ProxyPassReverse / http://myhost.edu:8890/

(In reply to comment #3)
> <proxy area="authoring" ssl="false" url="https://myhost.edu/williston/authoring"/>

ssl=false and a https uri. it looks like you want to bypass something. but does
it make any difference?

regarding the live proxy, have you succeeded in hiding the <pubname>/live part?
for the visible urls, i can make it to work by rewriting ^/(.*) to
/<pubname>/live/$1, but i also need to match /lenya and /module first, otherwise
stuff will break...

(In reply to comment #4)
> regarding the live proxy, have you succeeded in hiding the <pubname>/live part?
> for the visible urls, i can make it to work by rewriting ^/(.*) to
> /<pubname>/live/$1, but i also need to match /lenya and /module first, otherwise
> stuff will break...
> 

I see what you're talking about now. We're not using the SVG module or the
language selector. Our publications at the moment are very simple and don't need
anything from /lenya or /module (in live). Those would have to be matched first. 

So yes, I've hidden <pubname>/live, but I'm only doing basic publications, which
does make it a lot easier.

Created attachment 20170 [details]
Add option for relative URLs to UuidToUrlTransformer

To enable relative URLs, change the transformer configuration:

    <map:transformer name="uuid2url" ...>
      <urls type="relative"/>
    </map:transformer>

(In reply to comment #6)
> Created an attachment (id=20170) [edit]
> Add option for relative URLs to UuidToUrlTransformer

Caution, relative URLs are not compatible with the WYSIWYG editors (URL/UUID
transformation). The transformer has to be parameterized.

Created attachment 20201 [details]
Add config+param option for relative URLs to UuidToUrlTransformer

First of all I am not using proxy ATM so I am shooting in the dark. 

(In reply to comment #2)
> Created an attachment (id=20139) [edit]
> the output of default/authoring/index.html with proxying enabled
> 
> this is the output of the default publication with the following proxy
> settings:
> <proxies>
>   <proxy area="live" ssl="true" 
> url="https://stackingdwarves.net/cms/default/live"/>
>   <proxy area="live" ssl="false" 
> url="http://stackingdwarves.net/cms/default/live"/>
>   <proxy area="authoring" ssl="true" 
> url="https://stackingdwarves.net/cms/default/authoring"/>
>   <proxy area="authoring" ssl="false" 
> url="http://stackingdwarves.net/cms/default/authoring"/>
> </proxies>
> 
> the request URL was "https://stackingdwarves.net/cms/default/index.html".
> 
> the following issues arise:
> 
> * the lenya GUI does not honor proxying (see stylesheet and javascript links in
> header and tab links in <div id="lenya-menus"/>

Regarding css and javascript did you follow
http://lenya.apache.org/docs/1_2_x/components/deployment/proxying.html

# RewriteRules for css and images
     RewriteRule      ^/lenya/sandbox/(.*)/images/(.*)$
http://cms.live.ch:8080/lenya/sandbox/$1/images/$2 [L]
     RewriteRule      ^/(.*\.css$) http://cms.live.ch:8080/$1  [L]

     RewriteRule      ^/(lenya/[^/]+)/(.*)$   http://cms.live.ch:8080/$1/live/$2
 [P,L]

That may bring some of the css back. I reckon js is quite the same.

The problem is all the stuff that is not pub specific as e.g.
/lenya/menu/images/lenya-logo.gif or /modules/languageselector/flag-es-13.png
does not get rewritten from the <proxy/> tags.

Adding a RewriteRules like
RewriteRule      ^/lenya/(.*)$ http://cms.live.ch:8080/lenya/$1
RewriteRule      ^/modules/(.*)$ http://cms.live.ch:8080/modules/$1
should bring back another big part that seems to be broken.

However still the question why sometimes the rewrite miss the "https".

Can somebody test whether the ^/lenya and ^/modules is bringing back some more
resources.

I tried the patch from andreas and there are still some issues.

I am atm looking at the xsl:
<link rel="stylesheet"
href="{$contextPath}/{$pubId}/modules/homepage/css/homepage.css" type="text/css"/>

which did not get rewritten:
<link type="text/css" href="/default/modules/homepage/css/homepage.css"
rel="stylesheet" />

However we actually include it two times where the pagee2xhtml is using
 <link rel="stylesheet" href="{$root}/css/{$document-type}.css" type="text/css"/>

where <!-- {context-prefix}/{publication-id}/{area} -->
  <xsl:param name="root"/>

Meaning the proxy rewritting is working fine if we follow:
{context-prefix}/{publication-id}/{area}

It is indeed failing if not use as the above.

I am ATM trying to fix each part of the aggregation and doing the search stuff.

To debug I added:
<map:match pattern="authoring/navigation-element/*/*/*/*/*/**">
        <map:generate src="cocoon://modules/sitetree/{1}/{2}/{3}/{4}/{5}/{6}.xml"/>
        <map:transform type="uuid2url"/>
        <map:serialize type="xml"/>
      </map:match>

I found out that in UuidToUrlTransformer the following prevents the rewrite.
Where I think the comment is not correct since e.g. forms @action would hardly
ever use a lenyadoc: in the link. Further why the comment about:
 // don't rewrite /{pub}/modules/... ¿?

 /*
                             * This is legacy code. It rewrites links to
                             * non-document images (in resources/shared). These
                             * images shouldn't be referenced in documents since
                             * this violates the separation between content and
                             * layout.
                             */
                            String prefix = "/" +
this.currentDocument.getPublication().getId()
                                    + "/";
                            if (linkUrl.startsWith(prefix)) {
                                String pubUrl = linkUrl.substring(prefix.length());
                                String area = pubUrl.split("/")[0];
                                
                                // don't rewrite /{pub}/modules/...
                                if (area.equals(Publication.AUTHORING_AREA)) {
                                    String areaUrl =
pubUrl.substring(area.length());
                                    String newUrl = this.contextPath + prefix
                                            + this.currentDocument.getArea() +
areaUrl;
                                    setAttribute(newAttrs, attributeNames[i],
newUrl);
                                }
                            }

(In reply to comment #11)
> I am ATM trying to fix each part of the aggregation

I'd suggest that we don't include URLs in the navigation elements, but use
UUID-based links and leave the rewriting to the UuidToUrlTransformer at the end
of the presentation pipeline. I have a patch in my sandbox, but it needs a bit
more testing.

I am ATM writing a simple ProxyTransformer for testing. 

IMO the whole issue is quite simple to solve if we assume that we need to
rewrite all links that starts with "/" to the proxy url. 

Furthermore the whole proxy per area is suboptimal. IMO the live area should be
not prefixed and the rest just will replace "/" with
"https://stackingdwarves.net/cms/{$pub}" if no pub is defined in the first part
after "/" but e.g. /lenya or /module it would be just rewritten with the first part.

Anyway I will attach what I come up with shortly.

Created attachment 20240 [details]
Use lenya-document:uuid URLs for navigation elements in sitetree module

This patch changes the syntax of navigation element calls, so custom sitemaps
have to be updated.

How about adding an option for relative URLs to the BaseUrlModule as well?
It could for instance return "../.." for the request URL "/foo/bar/baz".

(In reply to comment #11)
> Further why the comment about:
>  // don't rewrite /{pub}/modules/... ¿?

That's because of bug 42140.

(In reply to comment #14)
> Created an attachment (id=20240) [edit]
> Use lenya-document:uuid URLs for navigation elements in sitetree module
> 
> This patch changes the syntax of navigation element calls, so custom sitemaps
> have to be updated.

BTW, the patch attaches a uuid2url.extension parameter with the document's
native extension (e.g., .zip) to the lenya-document URL. You might want to
change this if you'd like the links to end with .html.

Created attachment 20241 [details]
proxyTransformer version 1

This is the first version of the proxy transformer. 

The open issues are still the following cases:
a) <link type="text/css" href="/default/modules/homepage/css/homepage.css"
rel="stylesheet" /> - Using pub but not a valid area
b) <a href="/index.html"><span>Apache Lenya Homepage</span></a> - the menu
links which should point to the website IMO anyway
c) <img alt="he" src="/modules/languageselector/flag-he-13.png" /> and <script
src="/lenya/menu/menu.js" type="text/javascript"> </script> - point to the
absolute root.

a) and c) can be solved introducing a global proxy url. 
Meaning something like:
<proxies url="https://www.host.com/lenya/">

Will now test the patch from andreas.

+    try {
+      area = factory.getFromURL(linkUrl).getArea();
+    } catch (RuntimeException e) {
+      getLogger().warn("global link");
+    }

We shouldn't use exceptions for flow control ... :)
You can use factory.isDocument(linkUrl) instead.

Created attachment 20246 [details]
Aggregate patch from all other with version 2 of the proxyTransformer

Created attachment 20268 [details]
version 3 fixing error when no proxy config is used

For the sake of a static export, it should be possible to render all internal
links as relative URLs.

(In reply to comment #18)

> The open issues are still the following cases:
> b) <a href="/index.html"><span>Apache Lenya Homepage</span></a> - the menu
> links which should point to the website IMO anyway

I changed the link to point to the website. The "About Lenya" page is now a
usecase and therefore independent from proxying.

Maybe we should move the option for relative paths from the UuidToUrlTransformer
to the ProxyTransformer.

BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We could

- transform the CSS to XML
- apply the ProxyTransformer (instead of the XSLT we use now)
- convert them back to CSS

(In reply to comment #25)
> BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We
could
> 
> - transform the CSS to XML
> - apply the ProxyTransformer (instead of the XSLT we use now)
> - convert them back to CSS

i'd rather we mandate the use of <foo>.xml instead of foo.css in all cases. then
we can add the proxying stuff to the stylesheet that currently handles <root/>
and friends. i think all our users should use that mechanism. even if it is not
used, it won't be a performance issue, because both browsers and cocoon are very
good at caching those things that change rarely.
let's add some docs to the effect that "if you want to proxy, make sure you put
all your css through the xml grinder..."

Created attachment 20280 [details]
Patch for docu explaining the global rewrite url

For all links that are not pointing to a document the global proxy rule 
<proxies ssl="false" root="http://www.host.com/no-ssl/">
is applied. 

For example a link such as /modules/myModule/somePath/some.thing will be
rewritten to "http://www.host.com/no-ssl/modules/myModule/somePath/some.thing".


In short "/" will be replaced with "{proxies/@root}".

(In reply to comment #25)
> BTW, a major problem is that the ProxyTransformer can't handle URLs in CSS. We
could
> 
> - transform the CSS to XML
> - apply the ProxyTransformer (instead of the XSLT we use now)
> - convert them back to CSS

yes.

We could copy the forrest approach for this
<map:transform type="pattern" src="chaperon/grammars/link.xlex"/>
<map:transform src="chaperon/stylesheets/pattern2link.xsl"/>

http://svn.apache.org/viewvc/forrest/trunk/main/webapp/resources/chaperon/

Where the first will search for patterns we want to replace. The link.xlex can
be directly reused. The second will need some changes. 

Need to look into this.

Created attachment 20282 [details]
Version 4 of the all in one

The patch brings rewritting of the css via chaperon. Includes a fix in the
proxyTransformer if the resource is no document. 

One thing that I do not like is that one need to implement 
<map:resource name="proxy-css"> in all sitemaps that is calling the resource.
Seems that the map:resources are not inheritable. :(

I commited the Version 4 of the patches WITHOUT 
- UuidToUrlTransformer.java
- sitetree2nav.xsl

Since they partly seems to be applied to trunk already.

Please test and close this bug if it is working.

localhost:8888/lenya/css/menu.css doesn't load in the default publication. Error
indicates it is treating /lenya as a publication:

The configuration file
[/home/rfrovarp/asf/lenya-1.4.x/build/lenya/webapp/lenya/pubs/lenya/config/publication.xml]
does not exist! 

localhost:8888/lenya/css/boxes.css also shows the same error. /modules seems to
be working.

The proxy-css resource in lenya.xmap assumes that the call goes to a publication:

<map:parameter name="root"
value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>

IIUC the ProxyTransformer doesn't handle non-publication URLs either ...

(In reply to comment #32)
> The proxy-css resource in lenya.xmap assumes that the call goes to a publication:
> 
> <map:parameter name="root"
> value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>
> 

<map:transform src="fallback://lenya/resources/css/css.xsl">
          <map:parameter name="contextprefix" value="{request:contextPath}"/>
          <map:parameter name="root"
           
value="{base-url:{page-envelope:publication-id}:{page-envelope:area}:false}"/>
        </map:transform>

That is not proxy specific but css.xsl. Maybe the best if the base-url returns
"" instead of throwing an exception.

(In reply to comment #33)
> IIUC the ProxyTransformer doesn't handle non-publication URLs either ...

It did! But ATM it is not anymore and I have no clue why 

Strange, if I uncomment the root parameter in lenya.xmap, I end up with an empty
menu.css. The Chaperon parser doesn't produce anything.

In my working copy the PatternTransformer seems to fail if the input is plain text.

The css.xsl doesn't strip the <css> now, but wraps the CSS in chaperon text
elements (revision 546516).

The uuid2url and proxy transformers are now complementing each other. I did
quite a lot of testing on Tomcat, but not with a real proxy setting.

I have configured Apache 2.2.4 with mod_proxy on my local machine and did some
testing without SSL. I also added a little tutorial how to setup the test
environment.

With Jetty everything looks good. But when I use Tomcat, I get "The session
doesn't contain the identity" when I try to log in.

A major problem with the "global proxy" approach are requests to
publication-based module URLs. If they need authorization, the requests hit the
login screen. This occurs for instance with the TinyMCE JavaScript requests like

http://lenya.example.com/default/modules/tinymce/javascript/tiny_config.js

Making these URLs accessible by the world is certainly out of the question.

(In reply to comment #41)
> A major problem with the "global proxy" approach are requests to
> publication-based module URLs. If they need authorization, the requests hit the
> login screen.

A workaround is to use the same proxy server for both publication-specific and
global resources, but this would mask the "lenya/..." and "modules/..." URL spaces.

Non-SSL proxying works quite well now. Has to be tested with SSL yet.

I tested it with SSL and mod_proxy_ajp. If the problems still occur, please reopen.