Recently I had to implement a portal that embedded contents from other web applications. I tried to build a framework for easily adding remote applications without any changes to the remote applications. The framework had following design:
- For different functionality and pages we stored feature-link and url in configuration, that configuration basically allowed us to find where to
get the HTML/XHTML render page or form. - However, instead of displaying raw HTML, I first converted it into valid XHTML using Tidy and then transformed such that all form submissions
go through our proxy server. As part of the transformation, I added original form URLs, methods and cookies as hidden fields. This allowed
me to keep the proxy server without any client state, which was nice for clustering. - When the form is submitted, the proxy server intercepts it and then makes the request to the remote application and sends back response. The proxy
server reads the original target URLs, method types, cookies from the form so that remote application can manage state if it’s using Sessions.
Following are a few simplified classes that I developed for the proxy framework:
Content Transformation
First, I developed HTML to XHTML conversion and transformation to modify forms. Following is the XSLT that I used:
1 <xsl:stylesheet 2 xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 3 version="1.0" 4 xmlns:xhtml="http://www.w3.org/1999/xhtml" 5 xmlns="http://www.w3.org/1999/xhtml" 6 exclude-result-prefixes="xhtml"> 7 8 <xsl:param name="callbackHandler"/> 9 <xsl:param name="callbackUser"/> 10 <xsl:param name="callbackState"/> 11 12 <xsl:template match="@* | node()"> 13 <xsl:copy> 14 <xsl:apply-templates select="@* | node()"/> 15 </xsl:copy> 16 </xsl:template> 17 18 <xsl:template match="form"> 19 <xsl:copy> 20 <xsl:apply-templates select="@*"/> 21 <xsl:attribute name="action" xmlns:java="http://xml.apache.org/xslt/java"> 22 <xsl:param name="callbackOriginalActionUrl" select="@action"/> 23 <xsl:text disable-output-escaping="yes">/web/myproxy?myarg=xxx&otherarg=2</xsl:text> 24 <xsl:value-of select="java:com.plexobject.transform.XslContentTransformer.setAction($callbackHandler, string(@action))" /> 25 </xsl:attribute> 26 <xsl:attribute name="method" xmlns:java="http://xml.apache.org/xslt/java"> 27 <xsl:param name="callbackOriginalMethodType" select="@method"/> 28 <xsl:text disable-output-escaping="yes">POST</xsl:text> 29 <xsl:value-of select="java:com.plexobject.transform.XslContentTransformer.setMethod($callbackHandler, string(@method))" /> 30 </xsl:attribute> 31 <xsl:attribute name="id">_Form</xsl:attribute> 32 <xsl:attribute name="name">_Form</xsl:attribute> 33 <input type="hidden" name="_user" value="{$callbackUser}"/> 34 <input type="hidden" name="_originalActionUrl" value="{@action}"/> 35 <input type="hidden" name="_orginalMethodType" value="{@method}"/> 36 <input type="hidden" name="_userState" value="{$callbackState}"/> 37 <xsl:apply-templates select="node()"/> 38 </xsl:copy> 39 </xsl:template> 40 41 42 <xsl:template match="title"/> 43 44 45 </xsl:stylesheet> 46 47 48
A few things note
- xsl:param allows passing parameters from the runtime (Java)
- xsl:template is matching for “form” tag and replaces action/method attributes and adds id/name attributes. It then adds a few input hidden fields
- Finally, I am removing title tag
ContentTransformer interface
1 package com.plexobject.transform; 2 3 import java.util.Map; 4 5 public interface ContentTransformer { 6 /** 7 * This method transforms given contents 8 * 9 * @param contents 10 * - input contents 11 * @param properties 12 * - input/output properties for transformation 13 * @return transformed contents 14 * @throws TransformationException 15 * - when error occurs while transforming content. 16 */ 17 public String transform(String contents, Map<String, String> properties) 18 throws TransformationException; 19 } 20 21
ContentTransformer implementation
A few things to note in following implementation:
- I use JTidy to convert HTML to XHTML
- I pass some of the parameters to the XSL stylesheet and I also read a few properties back. Though, reading properties back is a bit kludgy but it works.
1 package com.plexobject.transform; 2 3 import java.io.ByteArrayInputStream; 4 import java.io.ByteArrayOutputStream; 5 import java.io.InputStream; 6 import java.util.HashMap; 7 import java.util.Map; 8 9 import javax.xml.transform.Result; 10 import javax.xml.transform.Source; 11 import javax.xml.transform.Transformer; 12 import javax.xml.transform.TransformerException; 13 import javax.xml.transform.TransformerFactory; 14 import javax.xml.transform.stream.StreamResult; 15 import javax.xml.transform.stream.StreamSource; 16 17 import org.w3c.tidy.Tidy; 18 19 public class XslContentTransformer implements ContentTransformer { 20 public static final String ACTION = "form_action"; 21 public static final String METHOD = "form_method"; 22 23 private static final Map<String, Map<String, String>> xslProperties = new HashMap<String, Map<String, String>>(); 24 private volatile Transformer transformer; 25 private final String xslUri; 26 private final boolean useTidy; 27 28 public XslContentTransformer(final String xslUri, final boolean useTidy) { 29 this.xslUri = xslUri; 30 this.useTidy = useTidy; 31 } 32 33 public static final String setAction(final String callbackHandler, 34 final String action) { 35 getPropertiesForCallback(callbackHandler).put(ACTION, action); 36 return ""; 37 } 38 39 public static final String getAction(final String callbackHandler) { 40 return getPropertiesForCallback(callbackHandler).get(ACTION); 41 } 42 43 public static final String setMethod(final String callbackHandler, 44 final String method) { 45 getPropertiesForCallback(callbackHandler).put(METHOD, method); 46 return ""; 47 } 48 49 public static final String getMethod(final String callbackHandler) { 50 return getPropertiesForCallback(callbackHandler).get(METHOD); 51 } 52 53 /** 54 * This method transforms given contents 55 * 56 * @param contents 57 * - input contents 58 * @param properties 59 * - input/output properties for transformation 60 * @return transformed contents 61 * @throws TransformationException 62 * - when error occurs while transforming content. 63 */ 64 public String transform(String contents, Map<String, String> properties) 65 throws TransformationException { 66 initTransformer(); 67 final long started = System.currentTimeMillis(); 68 69 contents = contents.replaceAll("<!--.*?-->", ""); 70 InputStream in = new ByteArrayInputStream(contents.getBytes()); 71 if (useTidy) { 72 in = tidy(in, (int) contents.length()); 73 } 74 75 // 76 final Source xmlSource = new StreamSource(in); 77 final ByteArrayOutputStream out = new ByteArrayOutputStream( 78 (int) contents.length()); 79 80 final Result result = new StreamResult(out); 81 String callbackHandler = properties.get("callbackHandler"); 82 if (callbackHandler == null) { 83 callbackHandler = Thread.currentThread().getName(); 84 } 85 final Map<String, String> props = new HashMap<String, String>(); 86 xslProperties.put(callbackHandler, props); 87 transformer.setParameter("callbackHandler", callbackHandler); 88 for (Map.Entry<String, String> e : properties.entrySet()) { 89 transformer.setParameter(e.getKey(), e.getValue()); 90 } 91 try { 92 transformer.transform(xmlSource, result); 93 } catch (TransformerException e) { 94 throw new TransformationException("Failed to transform " + contents, e); 95 } 96 properties.put(ACTION, getAction(callbackHandler)); 97 properties.put(METHOD, getMethod(callbackHandler)); 98 xslProperties.remove(callbackHandler); 99 return new String(out.toByteArray()); 100 } 101 102 private static final Map<String, String> getPropertiesForCallback( 103 String callbackHandler) { 104 final Map<String, String> props = xslProperties.get(callbackHandler); 105 if (props == null) { 106 throw new NullPointerException( 107 "Failed to find properties for callback " + callbackHandler); 108 } 109 return props; 110 } 111 112 // no synchronization needed, multiple initialization is acceptable 113 private final void initTransformer() { 114 if (transformer == null) { 115 try { 116 TransformerFactory transFact = TransformerFactory.newInstance(); 117 InputStream in = getClass().getResourceAsStream(xslUri); 118 if (in == null) { 119 throw new TransformationException("failed to find xslt " 120 + xslUri); 121 } 122 Source xsltSource = new StreamSource(in); 123 transformer = transFact.newTransformer(xsltSource); 124 } catch (TransformationException e) { 125 throw e; 126 } catch (RuntimeException e) { 127 throw e; 128 } catch (Exception e) { 129 throw new TransformationException( 130 "Failed to initialize XSL transformer", e); 131 } 132 } 133 } 134 135 private final InputStream tidy(InputStream in, int length) { 136 ByteArrayOutputStream out = new ByteArrayOutputStream(length); 137 Tidy converter = new Tidy(); 138 converter.setTidyMark(false); 139 converter.setXmlOut(true); 140 converter.setXmlPi(true); 141 converter.setXmlPIs(true); 142 converter.setNumEntities(true); 143 converter.setDocType("omit"); 144 converter.setUpperCaseTags(false); 145 converter.setUpperCaseAttrs(false); 146 converter.setFixComments(true); 147 converter.parse(in, out); 148 return new ByteArrayInputStream(out.toByteArray()); 149 } 150 } 151 152
Proxy
Following interfaces and classes show how GET/POST requests are proxied:
Proxy Interface
1 package com.plexobject.web.proxy; 2 3 import java.io.IOException; 4 import java.util.Map; 5 6 public interface HttpProxy { 7 /** 8 * This method issues a GET or POST request based on method and URI URI specified in the ProxyState 9 * and adds given parameters to the request. 10 * 11 * @param state 12 * - proxy state 13 * @param params 14 * - name/value pairs of parameters that are sent to the get 15 * request 16 */ 17 public ProxyResponse request(ProxyState state, Map<String, String[]> params) 18 throws IOException; 19 } 20 21
Proxy Implementation
Following class implements HttpProxy interface using HTTPClient library:
1 package com.plexobject.web.proxy; 2 3 import java.io.IOException; 4 import java.util.ArrayList; 5 import java.util.List; 6 import java.util.Map; 7 8 import org.apache.commons.httpclient.Cookie; 9 import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler; 10 import org.apache.commons.httpclient.HttpClient; 11 import org.apache.commons.httpclient.HttpMethodBase; 12 import org.apache.commons.httpclient.HttpState; 13 import org.apache.commons.httpclient.NameValuePair; 14 import org.apache.commons.httpclient.cookie.CookiePolicy; 15 import org.apache.commons.httpclient.methods.GetMethod; 16 import org.apache.commons.httpclient.methods.PostMethod; 17 import org.apache.commons.httpclient.params.HttpMethodParams; 18 19 import com.plexobject.io.IoUtil; 20 21 public class HttpProxyImpl implements HttpProxy { 22 private static final int CONNECTION_TIMEOUT_MILLIS = 30000; 23 24 /** 25 * This method issues a GET or POST request based on method and URI URI specified in the ProxyState 26 * and adds given parameters to the request. 27 * 28 * @param state 29 * - proxy state 30 * @param params 31 * - name/value pairs of parameters that are sent to the get 32 * request 33 */ 34 public ProxyResponse request(ProxyState state, Map<String, String[]> params) 35 throws IOException { 36 if (state.getMethod() == MethodType.GET) { 37 return get(state, params); 38 } else { 39 return post(state, params); 40 } 41 } 42 43 44 /** 45 * This method issues a GET request on the URI specified in the ProxyState 46 * and adds given parameters to the request. 47 * 48 * @param state 49 * - proxy state 50 * @param params 51 * - name/value pairs of parameters that are sent to the get 52 * request 53 */ 54 private ProxyResponse get(ProxyState state, Map<String, String[]> params) 55 throws IOException { 56 GetMethod method = new GetMethod(state.getUri()); 57 method.setQueryString(toNameValues(params)); 58 return doRequest(state, params, method); 59 } 60 61 /** 62 * This method issues a POST request on the URI specified in the ProxyState 63 * and adds given parameters to the request. 64 * 65 * @param state 66 * - proxy state 67 * @param params 68 * - name/value pairs of parameters that are sent to the POST 69 * request 70 */ 71 private ProxyResponse post(ProxyState state, Map<String, String[]> params) 72 throws IOException { 73 PostMethod method = new PostMethod(state.getUri()); 74 method.setRequestBody(toNameValues(params)); 75 return doRequest(state, params, method); 76 } 77 78 private ProxyResponse doRequest(ProxyState proxyState, 79 Map<String, String[]> params, HttpMethodBase method) 80 throws IOException { 81 long started = System.currentTimeMillis(); 82 HttpClient client = new HttpClient(); 83 client.getHttpConnectionManager().getParams().setConnectionTimeout( 84 CONNECTION_TIMEOUT_MILLIS); 85 client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); 86 method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, 87 new DefaultHttpMethodRetryHandler(3, false)); 88 89 HttpState initialState = new HttpState(); 90 for (Cookie cookie : proxyState.getCookies()) { 91 initialState.addCookie(cookie); 92 } 93 client.setState(initialState); 94 95 try { 96 int statusCode = client.executeMethod(method); 97 String contents = IoUtil.read(method.getResponseBodyAsStream()); 98 // 99 Cookie[] cookies = client.getState().getCookies(); 100 for (Cookie cookie : cookies) { 101 proxyState.addCookie(cookie); 102 } 103 104 return new ProxyResponse(statusCode, contents, proxyState); 105 } catch (RuntimeException e) { 106 throw e; 107 } catch (IOException e) { 108 throw e; 109 } catch (Exception e) { 110 throw new IOException("failed to process request", e); 111 } finally { 112 method.releaseConnection(); 113 } 114 } 115 116 private NameValuePair[] toNameValues(Map<String, String[]> params) { 117 if (params == null || params.size() == 0) { 118 return new NameValuePair[0]; 119 } 120 List<NameValuePair> nvPairs = new ArrayList<NameValuePair>(); 121 for (Map.Entry<String, String[]> e : params.entrySet()) { 122 String[] values = e.getValue(); 123 for (int j = 0; j < values.length; j++) { 124 nvPairs.add(new NameValuePair(e.getKey(), values[j])); 125 } 126 } 127 return (NameValuePair[]) nvPairs.toArray(new NameValuePair[nvPairs 128 .size()]); 129 } 130 } 131 132
ProxyState
Following class maintains URL, cookies, headers, and other information related to web request:
1 package com.plexobject.web.proxy; 2 3 import java.io.Serializable; 4 import java.io.UnsupportedEncodingException; 5 import java.net.URLDecoder; 6 import java.net.URLEncoder; 7 import java.util.Collection; 8 import java.util.Date; 9 import java.util.HashMap; 10 import java.util.Map; 11 12 import org.apache.commons.httpclient.Cookie; 13 14 /** 15 * Class: ProxyState 16 * 17 * Description: This class stores state needed to make a proxy request including 18 * method type and cookies. 19 * 20 */ 21 public class ProxyState implements Serializable { 22 private static final long serialVersionUID = 1L; 23 private static final String DATA_DELIMITER = "\n"; 24 private static final String COOKIE_DELIMITER = ";"; 25 private static final String NULL = "null"; 26 27 private String uri; 28 private MethodType method; 29 private Map<String, Cookie> cookies; 30 31 /** 32 * Constructors for ProxyState 33 */ 34 public ProxyState(String uri, String method) { 35 this(uri, MethodType.valueOf(method)); 36 } 37 38 public ProxyState(String uri, MethodType method) { 39 this.uri = uri; 40 this.method = method; 41 this.cookies = new HashMap<String, Cookie>(); 42 } 43 44 /** 45 * @return uri 46 */ 47 public String getUri() { 48 return this.uri; 49 } 50 51 /** 52 * @return method 53 */ 54 public MethodType getMethod() { 55 return this.method; 56 } 57 58 /** 59 * @return cookies 60 */ 61 public Collection<Cookie> getCookies() { 62 return this.cookies.values(); 63 } 64 65 66 /** 67 * @param cookies 68 */ 69 public void addCookies(Collection<Cookie> cookies) { 70 for (Cookie cookie : cookies) { 71 addCookie(cookie); 72 } 73 } 74 75 /** 76 * @param cookie 77 * - to add 78 */ 79 public void addCookie(Cookie cookie) { 80 this.cookies.put(cookie.getName(), cookie); 81 } 82 83 public String getCookieString() { 84 StringBuilder sb = new StringBuilder(512); 85 for (Cookie cookie : cookies.values()) { 86 if (cookie.getDomain() != null) { 87 sb.append(cookie.getDomain()).append(COOKIE_DELIMITER); 88 } else { 89 sb.append(NULL).append(COOKIE_DELIMITER); 90 } 91 sb.append(cookie.getName()).append(COOKIE_DELIMITER).append( 92 cookie.getValue()).append(COOKIE_DELIMITER); 93 94 if (cookie.getPath() != null) { 95 sb.append(cookie.getPath()).append(COOKIE_DELIMITER); 96 } else { 97 sb.append(NULL).append(COOKIE_DELIMITER); 98 } 99 if (cookie.getExpiryDate() != null) { 100 sb.append(String.valueOf(cookie.getExpiryDate().getTime())) 101 .append(COOKIE_DELIMITER); 102 } else { 103 sb.append(NULL).append(COOKIE_DELIMITER); 104 } 105 sb.append(String.valueOf(cookie.getSecure())) 106 .append(DATA_DELIMITER); 107 } 108 return sb.toString(); 109 } 110 111 112 @Override 113 public String toString() { 114 StringBuilder sb = new StringBuilder(512); 115 sb.append(uri.toString()).append(DATA_DELIMITER); 116 sb.append(method.toString()).append(DATA_DELIMITER); 117 sb.append(getCookieString()); 118 return sb.toString(); 119 } 120 121 /** 122 * This method converts proxy state into string based serialized state 123 * 124 * @return string based serialized state 125 */ 126 public String toExternalFormat() { 127 try { 128 return URLEncoder.encode(toString(), "UTF8"); 129 } catch (UnsupportedEncodingException e) { 130 throw new IllegalStateException("failed to encode", e); 131 } 132 } 133 134 /** 135 * This method converts a string based serialized state into the proxy state 136 * 137 * @param ser 138 * - string based serialized state 139 * @return ProxyState 140 * @throws IllegalArgumentException 141 * - if serialized state is null or corrupted. 142 */ 143 public static ProxyState valueOf(String ser) { 144 if (ser == null) 145 throw new IllegalArgumentException("Null serialized object"); 146 String decoded; 147 try { 148 decoded = URLDecoder.decode(ser, "UTF8"); 149 } catch (UnsupportedEncodingException e) { 150 throw new IllegalArgumentException("Unsupported encoding " + ser, e); 151 } 152 String[] lines = decoded.split(DATA_DELIMITER); 153 if (lines.length < 2) 154 throw new IllegalArgumentException( 155 "Insufficient number of tokens in serialized object [" 156 + decoded + "]"); 157 ProxyState state = new ProxyState(lines[0], lines[1]); 158 for (int i = 2; i < lines.length; i++) { 159 String[] cookieFields = lines[i].split(COOKIE_DELIMITER); 160 if (cookieFields.length < 6) 161 throw new IllegalArgumentException( 162 "Insufficient number of tokens 6 in serialized cookies [" 163 + lines[i] + "]/[" + decoded + "]"); 164 String domain = cookieFields[0]; 165 if (NULL.equals(domain)) { 166 domain = null; 167 } 168 String name = cookieFields[1]; 169 String value = cookieFields[2]; 170 String path = cookieFields[3]; 171 if (NULL.equals(path)) { 172 path = null; 173 } 174 Date expires = null; 175 if (!NULL.equals(cookieFields[4])) { 176 expires = new Date(Long.parseLong(cookieFields[4])); 177 } 178 boolean secure = new Boolean(cookieFields[5]).booleanValue(); 179 Cookie cookie = new Cookie(domain, name, value, path, expires, 180 secure); 181 state.addCookie(cookie); 182 } 183 return state; 184 } 185 186 @Override 187 public boolean equals(Object o) { 188 if (this == o) 189 return true; 190 if (!(o instanceof ProxyState)) 191 return false; 192 final ProxyState other = (ProxyState) o; 193 if (uri != null ? !uri.equals(other.uri) : other.uri != null) 194 return false; 195 if (method != null ? !method.equals(other.method) 196 : other.method != null) 197 return false; 198 return true; 199 } 200 201 @Override 202 public int hashCode() { 203 int result; 204 result = (uri != null ? uri.hashCode() : 0); 205 result = 29 * result + (method != null ? method.hashCode() : 0); 206 return result; 207 } 208 } 209 210
ProxyResponse
Following class stores response from the HttpProxy interface:
1 package com.plexobject.web.proxy; 2 3 import java.io.Serializable; 4 5 6 /** 7 * Class: ProxyResponse 8 * 9 * Description: This class stores proxy state and response. 10 */ 11 public class ProxyResponse implements Serializable { 12 private static final long serialVersionUID = 1L; 13 private int responseCode; 14 private String contents; 15 private ProxyState state; 16 17 /** 18 * Constructor for ProxyResponse 19 */ 20 public ProxyResponse(int responseCode, String contents, ProxyState state) { 21 this.responseCode = responseCode; 22 this.contents = contents; 23 this.state = state; 24 } 25 26 /** 27 * @return http response code 28 */ 29 public int getResponseCode() { 30 return this.responseCode; 31 } 32 33 /** 34 * @return XHTML contents 35 */ 36 public String getContents() { 37 return this.contents; 38 } 39 40 /** 41 * @return state associated with the proxy web request 42 */ 43 public ProxyState getState() { 44 return this.state; 45 } 46 47 @Override 48 public String toString() { 49 return this.responseCode + "\n" + this.state + "\n" + this.contents; 50 } 51 } 52 53
MethodType
Following class defines enum for http method types:
1 package com.plexobject.web.proxy; 2 3 /** 4 * Class: MethodType 5 * 6 * Description: Defines supported method types for proxy request. 7 * 8 */ 9 public enum MethodType { 10 GET, POST; 11 } 12 13
Service Example
Following classes show how above HTTPProxy and ContentTransfomer interfaces can be used with Servlet/Portlet APIs:
ProxyService Interface
1 package com.plexobject.web.service; 2 import javax.servlet.http.*; 3 import java.io.*; 4 5 public interface ProxyService { 6 public void render(HttpServletRequest request, HttpServletResponse response) throws IOException ; 7 public void submit(HttpServletRequest request, HttpServletResponse response) throws IOException ; 8 } 9 0
ProxyService Implementation
1 package com.plexobject.web.service; 2 import com.plexobject.web.proxy.*; 3 import com.plexobject.transform.ContentTransformer; 4 import javax.servlet.http.*; 5 import java.io.*; 6 import java.util.*; 7 8 9 public class ProxyServiceImpl implements ProxyService { 10 private HttpProxy httpProxy; 11 private ContentTransformer contentTransformer; 12 public ProxyServiceImpl(HttpProxy httpProxy, ContentTransformer contentTransformer) { 13 this.httpProxy = httpProxy; 14 this.contentTransformer = contentTransformer; 15 } 16 17 public void render(HttpServletRequest request, HttpServletResponse response) throws IOException { 18 String url = "http://plexrails.plexobject.com/guest_book/sign"; 19 ProxyState state = new ProxyState(url, MethodType.GET); 20 String inputXhtml = httpProxy.request(state, null).getContents(); 21 Map<String, String> properties = new HashMap<String, String>(); 22 properties.put("callbackState", state.toExternalFormat()); 23 String transformedXhtml = contentTransformer.transform(inputXhtml, properties); 24 response.getWriter().println(transformedXhtml); 25 } 26 27 public void submit(HttpServletRequest request, HttpServletResponse response) throws IOException { 28 String originalActionUrl = request.getParameter("originalActionUrl"); 29 String orginalMethodType = request.getParameter("orginalMethodType"); 30 ProxyState userState = ProxyState.valueOf(request.getParameter("userState")); 31 Map<String, String[]> params = request.getParameterMap(); 32 ProxyState state = new ProxyState(originalActionUrl, orginalMethodType); 33 state.addCookies(userState.getCookies()); 34 ProxyResponse proxyResponse = httpProxy.request(state, params); 35 response.getWriter().println(proxyResponse.getContents()); 36 } 37 } 38 39
Download Code
You can download above code from here.
Acknowledgement
I would like to thank the folks at XSLT forum of Programmer-to-Programmer (http://p2p.wrox.com/forum.asp?FORUM_ID=79) for answering my XSLT questions.