We recently ran into an issue in a customer configuration where rewriting URLs using pass-through didn’t function as expected with OC4J-deployed applications. As it turned out, there’s a bug in the OC4J container and a relatively easy workaround for some.

The situation was this (names changed to protect the innocent):

  • An existing Java application deployment existed using JRun on Solaris. In that deployment, an application called “abc” would be called as “http://abcapp.corp.com/servlet/login”
  • Applications were to be migrated to Oracle Application Server 10.1.3.1.0.
  • Deployments on OAS were required to prefix the application with something and they used the application name. So, on the new site, the application would need to be called as “http://abcapp.corp.com/abc/servlet/login”. This was undesirable since bookmarks would have to be updated. While it could easily be handled with redirections, the desired behavior was to have all URLs match what they were on the old deployment.

On the surface, this seems like a relatively simple problem to solve using a RewriteRule with the [PT] option and few RewriteConds in the Apache configuration. That is, until you find the bug in OC4J that makes it impossible. First, let’s review the configuration parameters.

NameVirtualHost *:4430
<VirtualHost *:4430>
  ServerName abcapp.corp.com
  Port 443
  SimulateHttps On
  AddCertHeader HTTPS
  CustomLog "|c:\oracle\product\10.1.3\OracleAS_1\bin\rotatelogs logs/abcapp-80-http-access.log 86400 -300" common
  ErrorLog "|c:\oracle\product\10.1.3\OracleAS_1\bin\rotatelogs logs/abcapp-80-http-error.log 86400 -300"
  Alias /abc-images c:\abcapp\imagesRewriteEngine On
  RewriteRule ^/$ https://abcapp.corp.com/servlet/login [R,L]
  RewriteCond %{REQUEST_URI} !^/abc/
  RewriteCond %{REQUEST_URI} !^/abc-images/
  RewriteRule ^/(.*) /abc/$1 [PT]
</VirtualHost>

So, let’s go line by line. First, we’re using name-based virtual hosting here (NameVirtualHost), so we tell Apache to evaluate all incoming requests received on port 4430 by attempting to match the ServerName from one of the VirtualHost sections (for port 4430) to the Host header in the HTTP request. If it finds a match, it serves the request from that VirtualHost. Note that it only matches the Host header which contains the hostname, but not a port number. So, you can’t define the same name twice with two different port numbers unless they’re using different VirualHost port numbers. That is, you can have the same name for two name-based VirtualHosts only if one of them is <VirtualHost *:4430> and the other is <VirtualHost *:7777>.

ServerName was explained above. Port is the port number that the end client (the browser) used when referencing this VirtualHost. It is the port used by the Apache server when it creates self-referential URLs and redirects (when only a URI is provided). No matter what port is specified in the VirtualHost and NameVirtualHost directive, the Port directive should always be the port used by the browser to access the site. That port may be on a webcache server, a load balancer, or a firewall.

SimulateHttps and AddCertHeader are directives used when your Apache server is not terminating the SSL connection. That is, a configuration like this: browser > HTTPS > load balancer/webcache > HTTP > Apache HTTPD. In this case, SSL is “stripped” off by another device and only HTTP connections are handled by Apache HTTPD. So, you need to tell Apache that when it creates a self-referencing URL, it needs to use the https protocol instead of http in order for the client browser to connect properly. These parameters provide that directive. Note that in order to use the SimulateHttps directive, you’ll first have to add the “LoadModule certheaders_module libexec/mod_certheaders.so” (for UNIX) or “LoadModule certheaders_module modules/ApacheModuleCertHeaders.dll” and “AddModule mod_certheaders.c” to your httpd.conf file first. See Metalink Note 378003.1 for more information.

CustomLog and ErrorLog are included inside the VirtualHost to get specific logging on the requests and errors encountered by this VirtualHost. These logs often help with debugging and are also sometimes used to track requests to this VirtualHost for reporting purposes as the application manager may want to know how many hits are coming in to their application.

The Alias directive simply adds another virtual path to the virtual space Apache HTTPD will serve by mapping that virtual directory to a physical path on the local filesystem.

Before we get in to the directives about URL rewriting, I’d recommend reviewing the reference documentation on mod_rewrite and the URL Rewriting Guide from the Apache 1.3 documentation. Oracle HTTP Server is based on Apache 1.3, so that’s the relevant version for our work.

RewriteEngine enables the rewrite functionality. It is scoped within the VirutalHost, so even if it is defined elsewhere outside VirtualHosts, you’ll need to include it in the VirtualHost.

The first RewriteRule is a simple redirection to match the regular expression “^/$” which literally means there is a slash (/) at the beginning of the line (^) and with the end of the line ($) coming immediately after the slash. When that matches, we know that the request is for https://abcapp.corp.com/ and nothing else. So, we need to redirect (that’s the [R] at the end of the line) the browser to our application start page (https://abcapp.corp.com/servlet/login). The [R,L] are two separate arguments; the first argument (R) means that the response to the browser should be an “external” redirection and the browser will resubmit the reqeust to the target we send it, and the second argument (L) means that this is the last rule should be evaluated (if more rules are present). If this were the only rule here, we wouldn’t need the L, but since more rules follow, this tells the rule engine to stop evaluating rules if this one matches (default behavior is to continue reading rules).

The next few lines must be considered together. RewriteCond is a conditional statement that determines whether or not to evaluate the following RewriteRule. When more than one RewriteCond appears together, they are ANDed together by default. You can optionally OR them together–check the docs for syntax. In our case, we want to prepend the application deployment prefix (/abc) to all URLs that come through this VirtualHost unless they already have the prefix on the URL or if they’re for some static, non-Java content that will be served by Apache directly (instead of being sent to OC4J for service). In our case, that means that our conditions are that the URL does *not* (exclamation point is the negation operator) start with /abc/ (the trailing slash is very important so we don’t match all strings starting with /abc) and also any URL that does *not* start with /abc-images/ (which is a static images directory that is Alias’ed above). If the URL does *not* start with either one of those, then we’ll go to the RewriteRule. The RewriteRule matches anything that starts with a slash (which is all requests) and the parens are a grouping operator that group the pattern “.*”. In regular expressions, the period (.) is a wildcard that matches any single character and the asterisk (*) is an operator that says match 0 or more of the previous character. So, this “.*” pattern says to match everything. In this case, the word “character” means anything–not just letters. When that pattern matches (and the parens will create a reference-able group for that pattern), the rule is instructing Apache to translate that to the string /abc/$1 which means to start the string with /abc/ and then put the stuff from the parens (earlier on this same line) on the end. Basically, prepend all requests with /abc. For your reference, you can use multiple sets of parens on the same line and reference them (later on that same line) in order with $1, $2, etc. Finally, the [PT] at the end of this line tells Apache to apply this rule and then pass the resulting URL through to the rest of the Apache modules for handling. In our case, since /abc is an application prefix, the mod_oc4j module will identify /abc as an application and send the request to the proper OC4J for resolution.

Here’s the next problem, mod_oc4j does its job and sends the request to the proper OC4J for service. The OC4J, however, reviews the request by looking at the HTTP headers and sees that the client requested /servlet/login and since it doesn’t have an application deployed at the prefix /servlet, the OC4J container doesn’t know how to handle this request, so it sends the request to the default application as defined in the default-web-site.xml file. This is a bug in OC4J and is filed as Oracle bug 5726819. So, the request makes it to the proper OC4J, so mod_oc4j did its job. However, the container satisfied the request from the default web application instead of the correct web application. In our case, we have only one application deployed to each container, so our workaround was to change the default web application to be our custom application (/abc) instead of the application called “default” (which is automatically deployed in every OC4J container). That just involved changing these two lines in the %OH%\j2ee\OC4J_abc\config\default-web-site.xml file:

  <default-web-app application=”default” name=”defaultWebApp” root=”/j2ee” />
  <web-app application=”abc” name=”abcapp” load-on-startup=”true” root=”/abc” />

to these two lines instead (moving the “default-”):

  <web-app application=”default” name=”defaultWebApp” root=”/j2ee” />
  <default-web-app application=”abc” name=”abcapp” load-on-startup=”true” root=”/abc” />

In our case, on 10.1.3.1.0, that worked fine. We filed an SR anyway (we found this issue and resolution ourselves) to see if there was a patch or any unpublished workarounds that might be somehow better. What we learned was that there is another workaround that is available in 10.1.3.2.0 and 10.1.3.3.0 (not in our release of 10.1.3.1.0). That workaround is to put a new container environment variable in place on the start-parameters line in the opmn.xml. The variable is “-Dajp.use.virgin.uri=false”. If you attempt to put this in a 10.1.3.1.0 container, the container will fail to start, so it clearly was a customization added in 10.1.3.2.0 (apparently–we didn’t have luxury to test it on that version, but that’s the guidance from our Oracle Support SR).

Now, on to the next problem. Our application also used the Java session to store state information. That results in a session cookie (JSESSIONID) to be created and sent to the browser. All cookies contain a hostname and pathname that they should be used for. The client browser is required to only send cookies to the host and path that they have defined. That is, if the browser receives a cookie for abcapp.corp.com with path=/abc, it is not allowed to send that cookie to any URLs unless they are on the site abcapp.abc.com and the URI path starts with /abc. So, in our case, the OC4J (which creates the JSESSIONID cookie) thinks that the application path is /abc, so it creates the cookie with path=/abc. The browser receives it, stores it, but won’t send it back because the browser is accessing a URI like “/servlet/login” or some other URI that doesn’t start with /abc. In order to tell OC4J to override the cookie path, you must use the <session-tracking> element. This can be placed in the orion-web.xml file or the global-web-application.xml file, In our case, since we have just one application per OC4J, we defined it in the global-web-application.xml file. The <session-tracker> element must appear as a child of the <orion-web-app> element (which appears in both of the files I mentioned). In our global-web-application.xml file, we added this line just after the end of the opening <orion-web-app> tag (which is about 10 lines long) and before the <mime-mappings> tag:

  <session-tracking cookie-path=”/”>

Additionally, now that all applications use the path of “/”, if the same user wishes to access two different applications (which would be in two different OC4Js in our one-app-per-OC4J configuration), they will get a new JSESSIONID cookie that will collide with the previous one. If they use both applications simultaneously, their session information (and very likely their login session) on both applications will be affected. In order to have these applications share the JSESSIONID cookie, the OC4Js involved need to be instructed to look for a JSESSIONID cookie and use it for tracking the local session instead of creating its own JSESSIONID cookie. This is accomplished with another OC4J environment variable. By adding “-Doracle.useSessionIDFromCookie=true” to your start-options in the opmn.xml file for each OC4J, they will reuse the JSESSIONID cookie when it is found instead of creating a new one. See Metalink note 345167.1 for more on this configuration. Also note that while Metalink note 292972.1 has appropriate information describing the issue and a solution for the problem, that solution only applies to OAS versions 9.0.2 and 9.0.3 and *not* OAS 10g.

That’s it. Hopefully this will help someone resolve issues related to mod_oc4j, mod_rewrite, and java session management in such an environment.

A few more notes to mention:

  • This configuration was also load balanced with an F5 BIGIP and there are persistence settings needed on the BIGIP to ensure that the proper system is selected from the load balancer pools.
  • We did not replicate session information or cluster the OC4Js in this environment. That’s why it was critically important that the BIGIP persistence was working properly.
  • Ron

    Thanks for posting this, I found it quite helpful.

  • Dilshod

    Thanks! It helped very much!

    FYI: There is an updated metalink document(Note:468621.1)as of 3/13/2008.https://metalink.oracle.com/metalink/plsql/f?p=130:14:862239773707049499::::p14_database_id,p14_docid,p14_show_header,p14_show_help,p14_black_frame,p14_font:NOT,468621.1,1,1,1,helvetica

    The document has a link to an interim patch for version 10.1.3.1: Patch 5726819.

  • http://www.dannorris.com/ Dan

    @Dilshod Thanks for the tip! I’ll have to let the admins that originally hit this issue know that there’s a patch available now (though the workaround worked well for them).

  • Subbu

    Hi Dan,

    I have an exact similar issue, but not on oracle. The architecture is ——.
    SSL is terminated on the load balancer and apache HTTP 2.0.59 is running http only and not https. Application (running on tomcat), perhaps due to the
    use of absolute URLs when writing HTML, is not letting the content in general to work properly. We have mixed protocol references (https and http) due to the content. So would like to configure in apache’s httpd.conf something that spoofs https for the application. Something on the lines of Oracle’s mod_certheaders.so.

    LoadModule certheaders_module modules/mod_certheaders.so
    SimulateHttps on (inside the virtual host)

    Have tried mod_certheaders.so but apache fails to start saying: Cannot load /modules/mod_certheaders.so into server:
    /modules/mod_certheaders.so: undefined symbol: ap_ctx_set

    $ file mod_version.so mod_version.so: writable, executable, regular file, no
    read permission
    $ file mod_certheaders.so mod_certheaders.so: ELF 32-bit LSB shared object,
    Intel 80386, version 1 (SYSV), not stripped

    Any easy way to accomplish this on apache?

    (Reference:
    http://www.dannorris.com/2007/11/01/using-mod_rewrite-to-rewrite-oc4j-served-urls-a-complete-review/
    )
    (Reference:
    http://www.netscalerkb.com/netscaler_tricks_and_guides/howto_rewrite_html_body_urls_to_avoid_absolute_protocol_references-t218.0.html
    )

    Thanks,
    Subbu

  • jiffyjeff

    Fantastic — thanks for the insights… I was having a hard time with two web apps running in the same OC4J clobbering each other's JSESSIONIDs. By setting the line “-Doracle.useSessionIDFromCookie=true” in startup Java options (not OC4J option), the apps will use an existing JSESSION ID if it's set. Works great!

  • http://www.dannorris.com/ Dan Norris

    Great, thanks for the feedback and I'm glad it worked well for you!

  • http://femalenudity.blogspot.com female

    Thanks,very interesting and useful post

  • frank

    Hello, I am new with the SSL and Load Balancer. I am guessing how the flow works with a Big Ip (this works and ssl end termination). I send a https request (443) to the BigIp via and Internet Explorer.then the BigIp redirects the request to one web node , the node returns the response to the Big Ip, so the door is only the Big Ip.
    I don´t undestand very well the parameter Port 443 , suppose we have <VirtualHost *:4430> (same as your example) so the Big Ip convert the request to 443 to 4430
    I really apreciate you possible help.
    Thanks in advance

    Francisco Javier Muñoz Fdez
    Regards

  • http://www.gwu.edu/ Bill Zeledon

    Good information.
    Question: how was oc4j directives with dynamic mount points dealt with?
    An Metalink Doc Id: 739881.1 goes in detail, however I did see it this document.
    In version 10.1.3.4, this is an important part in restricting Web Application Access By VirtualHosting.

  • http://www.gwu.edu/ Bill Zeledon

    Good information.
    Question: how was oc4j directives with dynamic mount points dealt with?
    An Metalink Doc Id: 739881.1 goes in detail, however I did see it this document.
    In version 10.1.3.4, this is an important part in restricting Web Application Access By VirtualHosting.