Benchmarks Online

Skip Navigation Links


Page One

Campus Computing News

JAWS 8.0 Has Arrived!

Web Conferencing Tool on the Horizon: Teach & Meet Live Online

Getting WebCT Vista Help

Daylight Savings Time Issues and Concerns

EIS Campus Solutions 8.9 Upgrade Website

Today's Cartoon

RSS Matters

The Network Connection

Link of the Month

Helpdesk FYI

WWW@UNT.EDU

Short Courses

IRC News

Staff Activities

Subscribe to Benchmarks Online
    

WWW@UNT.EDU

ColdFusion7 Clustering Revisited

Note, last month's article, "Apache web farm configuration: Simpler than mod_macro?," originally appeared in the June 2006 issue of Benchmarks Online. - Ed.

By Shannon Eric Peevey, formerly employed with UNT Central Web Support

Introduction

I am still trying to get our ColdFusion 7 cluster to work correctly. Up until now, the cluster has not really worked like a cluster, in that:

  1. Configuration information is not shared between the clustered instances.

  2. The Apache connector was only configuring itself with one of the servers, and was unaware of the other cluster members.

  3. The ColdFusion packaged Verity license is only good for one ColdFusion instance.

Our Configuration:

Our current configuration consists of a load-balanced web farm, and a two-machine ColdFusion cluster. For example:

Apache2 - 192.168.1.30, 192.168.1.31, 192.168.1.32
Coldfusion7 - 192.168.1.40, 192.168.1.41

Configuration information is not shared between the clustered instances:

This was the first problem that we ran into. For some reason, configuration information for the ColdFusion instance is not shared between the nodes. For example, DSN "x" created on ColdFusion node1 was not replicated to Coldfusion node2, meaning that calls to node2 for DSN "x" would return a DSN not found error.

Since we are using a central storage area for our web roots, we were able to push our jrun4 directories into a central location:

user@mymachine:/remotestorage/jrun4$ ls /remotestorage/jrun4
cf_common_files lib cfinstance1 cfinstance2

As you can see, we have moved our jrun4 installation directory onto our remotestorage:

user@mymachine:/remotestorage/jrun4$ ls cfinstance1
bin                               hotfix               logs                                               pointbase           uninstall
C                                 jre                  Macromedia_ColdFusion_MX_7.0.2_InstallLog.log      PointBaseDebug.log  updater6-backup
cfmx_updater_installer_debug.txt  jre_installerbackup  Macromedia_ColdFusion_MX_7_InstallLog.log          Readme.htm          verity
connectors                        lib                  Macromedia_ColdFusion_MX_7_Updater_InstallLog.log  relnotes.htm
docs                              license.txt          Macromedia_JRun_4_Updater_6_InstallLog.log         servers
user@mymachine:/remotestorage/jrun4$ ls cfinstance2
bin                               docs    license.txt                                        Macromedia_JRun_4_Updater_6_InstallLog.log  servers
C                                 hotfix  logs                                               pointbase                                   uninstall
cfmx_updater_installer_debug.txt  jre     Macromedia_ColdFusion_MX_7_InstallLog.log          Readme.htm                                  updater6-backup
connectors                        lib     Macromedia_ColdFusion_MX_7_Updater_InstallLog.log  relnotes.htm                                verity

As well as our jrunserver.store, so that it is accessible to all of the Apache2 machines: 

user@mymachine:/remotestorage/jrun4$ ls lib/wsconfig/1/
ApacheBuildInstructions.txt  build_jrun20  jrunserver.store  jrunserver.store.bak  mod_jrun20.so

Next, we moved all files which are shared between the instances into the /remotestorage/jrun4/cf_common_files directory. These include:

.../myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/cfx

.../myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/CustomTags

Plus, the neo-* files and password.properties from:

.../myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/

user@mymachine:/remotestorage/jrun4$ ls cf_common_files/
cfx         CustomTags  lib         
user@mymachine:/remotestorage/jrun4$ ls cf_common_files/lib
neo-archivedeploy.xml  neo-debug.bak     neo-event.xml     neo-mail.bak    neo-query.bak     neo-runtime.xml   neo-security.xml  password.properties
neo-clientstore.xml    neo-debug.xml     neo-graphing.xml  neo-mail.xml    neo-query.xml     neo-search.bak    neo-verity.xml
neo-comobjmap.xml      neo-document.bak  neo_jaas.policy   neo-metric.xml  neo-registry.xml  neo-search.xml    neo-watch.xml
neo-cron.xml           neo-document.xml  neo-logging.xml   neo-probe.xml   neo-runtime.bak   neo-security.bak  neo-xmlrpc.xml

These are then symlinked back into the cfinstance directory trees:

user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/
lrwxrwxrwx  1 www-data www-data   38 2006-09-05 12:55 cfx -> /remotestorage/jrun4/cf_common_files/cfx/
lrwxrwxrwx  1 www-data www-data   45 2006-09-05 12:54 CustomTags -> /remotestorage/jrun4/cf_common_files/CustomTags/

user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib | grep neo 
lrwxrwxrwx  1 www-data www-data      59 2006-09-05 13:29 neo-archivedeploy.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-archivedeploy.xml
lrwxrwxrwx  1 www-data www-data      57 2006-09-05 13:29 neo-clientstore.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-clientstore.xml
lrwxrwxrwx  1 www-data www-data      55 2006-09-05 13:29 neo-comobjmap.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-comobjmap.xml
lrwxrwxrwx  1 www-data www-data      50 2006-09-05 13:29 neo-cron.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-cron.xml
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-debug.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-debug.bak
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-debug.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-debug.xml
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-document.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-document.bak
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-document.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-document.xml
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-event.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-event.xml
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-graphing.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-graphing.xml
lrwxrwxrwx  1 www-data www-data      53 2006-12-19 07:14 neo_jaas.policy -> /remotestorage/jrun4/cf_common_files/lib/neo_jaas.policy
lrwxrwxrwx  1 www-data www-data      53 2006-09-05 13:29 neo-logging.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-logging.xml
lrwxrwxrwx  1 www-data www-data      50 2006-09-05 13:29 neo-mail.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-mail.bak
lrwxrwxrwx  1 www-data www-data      50 2006-09-05 13:29 neo-mail.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-mail.xml
lrwxrwxrwx  1 www-data www-data      52 2006-09-05 13:29 ncoldfusion 7 clustereo-metric.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-metric.xml
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-probe.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-probe.xml
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-query.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-query.bak
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-query.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-query.xml
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-registry.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-registry.xml
lrwxrwxrwx  1 www-data www-data      53 2006-09-05 13:29 neo-runtime.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-runtime.bak
lrwxrwxrwx  1 www-data www-data      53 2006-09-05 13:29 neo-runtime.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-runtime.xml
lrwxrwxrwx  1 www-data www-data      52 2006-09-05 13:29 neo-search.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-search.bak
lrwxrwxrwx  1 www-data www-data      52 2006-09-05 13:29 neo-search.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-search.xml
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-security.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-security.bak
lrwxrwxrwx  1 www-data www-data      54 2006-09-05 13:29 neo-security.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-security.xml
lrwxrwxrwx  1 www-data www-data      52 2006-09-05 13:29 neo-verity.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-verity.xml
lrwxrwxrwx  1 www-data www-data      51 2006-09-05 13:29 neo-watch.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-watch.xml
lrwxrwxrwx  1 www-data www-data      52 2006-09-05 13:29 neo-xmlrpc.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-xmlrpc.xml
user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib | grep pass
lrwxrwxrwx  1 www-data www-data      57 2006-09-05 13:29 password.properties -> /remotestorage/jrun4/cf_common_files/lib/password.properties

Once the symlinks are in place, you can restart the ColdFusion server instances, and they will now share the configuration changes which take place in the ColdFusion administrator. To highlight a few files:

  1. neo-query.xml – contains your DSNs.

  2. password.properties – contains the ColdFusion administrator password

Any modified to the neo-* files will be automatically found by the ColdFusion instances and they will reload the appropriate service with a log entry like:

12/19 08:28:52 Information [scheduler-1] - neo-query.xml has been modified . Reloading the service coldfusion.sql.Executive@13b3b7e

The Apache connector was only configuring itself with one of the servers, and was unaware of the other cluster members:

Due to the lack of documentation, I have fumbled with configuring the Apache2/ColdFusion connector, which is essentially the Apache module which tells Apache how to communicate with the coldfusion server. I was under the impression that you set one ipaddress:port in:

JRunConfig Bootstrap 192.168.1.40:51020

and that the ColdFusion server would then tell the Apache about all of the instances in the cluster, which would then populate the proxyservers array in the jrunserver.store file. But, alas, that was an incorrect assumption. Instead, we need to use the Bootstrap directive to tell Apache2 about all of the instances in the ColdFusion cluster with a semi-colon delimited list of ipaddresses and ports:

JRunConfig Bootstrap 192.168.1.40:51020;192.168.1.41:51020

And then, when Apache2 starts, it tries to connect to all of the listed instances of coldfusion, then generates the jrunserver.store with the ipaddress:port of each server which Apache2 was able to successfully contact.

The ColdFusion packaged Verity license is only good for one ColdFusion instance:

Through research, and discussion with our Adobe vendor, we were able to ascertain, that the Verity, (now called Autonomy), license which is bundled with ColdFusion 7 may only be contacted by a single ColdFusion instance. Therefore, you will need to purchase a license from Autonomy for their enterprise license to use Verity searches in a clustered environment. We would not realize the benefits of this license, so decided to move our search indexing into the database instead.

FAQ:

  1. ColdFusion clusters must use J2EE sessions to replicate sessions between instances.

This is taken from the modify cluster page in the CF Administrator "Note: Use of J2EE Sessions is required for session replication or sticky sessions. All instances participating in the cluster must use J2EE sessions. Therefore, you will need to go to:

Server Settings ---> Memory Variables and check "Use J2EE session variables" for each cluster member."

  1. ColdFusion must have the webroot mounted locally if it is running on a separate machine from the web server(s).

    We have always used NFS to mount the web root locally to the ColdFusion instance. There have also been some mention of mirroring the web root to the ColdFusion machine, but that would only work if there was little development on a site, as there would be latency between files on the live site, and those on the ColdFusion server.

  1. Here is an example Apache2/Jrun configuration which sets some variables specific to clustering.

      # JRun Settings
    <IfModule mod_jrun20.c>
  JRunConfig Verbose false
        JRunConfig Apialloc false
        JRunConfig Ssl false
        JRunConfig Ignoresuffixmap false
        JRunConfig Serverstore /remotestorage/jrun4/lib/wsconfig/1/jrunserver.store
        JRunConfig Bootstrap 192.168.1.40:51020;192.168.1.41:51020
        JRunConfig ProxyRetryInterval 45
        JRunConfig ConnectTimeout 45
        JRunConfig RecvTimeout 45
        JRunConfig SendTimeout 45
        #JRunConfig Errorurl 
        AddHandler jrun-handler .cfm .cfml .cfc .jsp .jws
      AddType application/x-httpd-cfm .cfm .cfml .cfc
</IfModule>

These are somewhat documented here: http://livedocs.macromedia.com/coldfusion/7/htmldocs/wwhelp/wwhimpl/common/html/wwhelp.htm?context=ColdFusion_Documentation&file=00001756.htm

  1. Where can I view the value of the Proxy Port?

The Proxy Port, which is not the "Remote Port" mentioned in the ColdFusion Administrator, can only be viewed in the Jrun Administrator, which should running on your "master" Jrun4 server on port 8000. (ie http://my.master.node:8000)

  1. My browser won't show the Jrun4 administrator... :(

Check that you don't have a firewall blocking the port. Also, make sure that the admin instance is running on the server.

  1. I have forgotten my Jrun4 administrator password!

You can view your password in the:
/path/to/jrun4/servers/admin/SERVER-INF/jrun-users.xml

file. Look for a user with the jmcadmin element.

  1. I have forgotten my ColdFusion Administrator password!

You can get around this in two ways:

  1. Manually edit the password.properties file  http://kb.mediatemple.net/article.php?id=077
     

  2. Set <var name='admin.security.enabled'><boolean value='true'/></var> to “false” in the neo-security.xml file, restart coldfusion and visit the ColdFusion administrator to reset your admin password. (Don't forget to reset the admin.security.enabled value to “true” and restart ColdFusion, or else anybody would be able to access the administrator!!).

  1. How are the JrunConfig Bootstrap and proxyserver array (in jrunserver.store) related?

    As apache starts up, the jrun4 connector splits the array of proxy servers, and attempts to contact the coldfusion server instance on each ipaddress:port. (Remember the port is the Proxy Port). If the ColdFusion server instance does not respond, it is not added to the proxyserver array in the jrunserver.store.

  1. I have setup my Bootstrap with all of the correct ipaddresses and ports, but I still only have one member in the proxyserver array in the jrunserver.store file...

    Lower your firewall, and make sure that the firewall on either the client or server is not blocking communication. Also, startup try restarting your server instances to see if something has gone awry with them.
     

  2. Will my ColdFusion instances automatically detach modifications to shared configuration files?

Any modified to the neo-* files will be automatically found by the ColdFusion instances and they will reload the appropriate service with a log entry like:

12/19 08:28:52 Information [scheduler-1] - neo-query.xml has been modified . Reloading the service coldfusion.sql.Executive@13b3b7e

  1. How are changes made to the neo-* and password.properties files?

The neo-* and password.properties files are modified through the ColdFusion administrator. (They may be hand-edited as well, but not for the faint-at-heart).


Have you registered your website yet?

 

 

Please note that information published in Benchmarks Online is likely to degrade over time, especially links to various Websites. To make sure you have the most current information on a specific topic, it may be best to search the UNT Website - http://www.unt.edu . You can also search Benchmarks Online - http://www.unt.edu/benchmarks/archives/back.htm as well as consult the UNT Helpdesk - http://www.unt.edu/helpdesk/ Questions and comments should be directed to benchmarks@unt.edu

 

 

Return to top