Home for 5pm workshop at CFCamp

As previously announced, I'll be giving a session at CFCamp on "Home for 5pm", which is an updated version of the talk I gave at Scotch on the Rocks in 2011.

Additionally, I'll also be giving a half days Hands On Training (HOT) on the same topic.

The HOT will take the productivity features mentioned in the one hour presentation and put them into practice. There's no requirement to attend the session to get the most out of the training, but if you're looking to increase your productivity by streamlining your development through task focused development I'd strongly recommend you come along to one or the other (or why not both).

You'll learn about using Mylyn, and it's commercial big brother Tasktop to enable your task focused development. By minimising distractions, unnecessary noise and ultimately focusing on the task at hand, you'll see just how easy it is to be Home for 5pm.

The training session is €300+vat.

There's a host of other HOTs running as well on topics covering; Railo, jQuery, Regular Expressions and more so check them out.

Hopefully we'll see you in Munich.

CFCamp, Munich

This October we're heading to Munich to take part in CFCamp conference.

In addition to sponsoring the event, I'll also be presenting. I had the pleasure of speaking at the first CFCamp back in 2008, and I'm delighted to get the chance to present once more.

More details on my session will be available nearer the time, as will our overall plans for the conference.

So, if your food of choice is CFML, or you want to taste CFML for the first time, why not join us in Munich.

ColdFusion 10 instance creation path issue

When ColdFusion 7 was released back in 2005, the ColdFusion team gave us the ability to create new instances (assuming we were running in multi-server mode) directly from the ColdFusion Administrator of the default 'cfusion' instance.

There were a number of quirks with the instance creation process, which resulted in some of us sticking with the old manual process as used in CF6.1

One of the instance creation issues related to new instances inheriting paths pointing to the cfusion instance rather than the path for the newly created instance.

This bug has found it's way into ColdFusion 10. Thankfully it's incredibly easy to work around.

Let me demonstrate the issue, and how to fix it.

To create additional instances with ColdFusion 10, you need to be running ColdFusion Enterprise, or be running in trial or developer mode. If you log into the default installation instance, called 'cfusion', you'll have the ability to create additional instances via the Enterprise Manager option.

What actually happens when you create a new instance, is that this cfusion instance is effectively cloned, thereby inheriting settings you've configured. There's obviously pros and cons to this.

One of the cons is that with pages such as Charting or Logging Settings, you simply need to 'touch' the pages (i.e. click submit without making any changes), and the default path of /ColdFusion10/cfusion/logs will find it's way into new instances you create.

if you don't 'touch' the page then the correct path of /ColdFusion10/<newinstance>/logs will correctly be set.

Thankfully you can just manually correct the paths if required.

I logged this as a bug back in 2005/6, and have done so again for ColdFusion 10. The bug number is 3208658 and can be viewed at https://bugbase.adobe.com/index.cfm?event=bug&id=3208658

Tomcat User Security

In my previous post on Different JVM settings for different Tomcat instances, I briefly touched on using tomcat-users.xml to allow access to the the Server Status page. This page, along with the Tomcat Manager application (amongst others), are installed by default when you deploy Tomcat. Access to them is controlled via the tomcat-users.xml file.

As we saw in the previous article, we can enable access by defining a role, and subsequently a user who is part of that role.

Why doesn't Tomcat ship with a default account?: Simple. It's a security issue. Instead of providing that default account, Tomcat is locked down completely, and it's up to us to enable access.

Let's try accessing the Tomcat Manager without having any users defined in tomcat-users.xml.

Tomcat Manager

As you can see, access is restricted. However, what it does do is tell us (some of) the types of roles available to us.

Let's look at our tomcat-users.xml file again. Now, based on what you see in the file, and what we added in the previous article, I'm hoping most (if not all) of you are a little concerned by the fact passwords are stored in plain text.

Thankfully this file isn't directly accessible from a web browser, however it can be read by anyone who has access to the server.

Tomcat Manager roles

Tomcat 7 introduced granular control over the roles available to us for accessing the Tomcat Manager, Host Manager and Server status pages. (Technically it was already there in Tomcat 6, but you had to do a bunch of manual work).

Previously, there was a "manager' role that you would add users to, to give them access to the Server Status page. The downside of this approach was that by assigning the user to the "manager" role, you also gave them access to the Tomcat manager, which allows tasks such as deploying/undeploying apps, expiring sessions, etc.

Not really ideal.

Let me, or should I say, Tomcat 7 introduce four delegation roles:

manager-gui

Full access to the Tomcat Manager.

manager-script

Provides the same access as manager-gui, but from a text interface. Command-line savvy administrators can utilise this role to do everything that manager-gui does, but they can programatically control actions via scripts written using perl, python, etc.

manager-jmx

Provides access to the jmxproxy, which exposes information for monitoring purposes.

manager-status

Access to the Server Status page.

All manager-* roles can access the Server Status page.

As an aside, the "admin" role has also been split into admin-gui and admin-script. These do exactly what they say, and provide access to the Host Manager application.

Realms

Tomcat Realms are simply a way of "storing" usernames, passwords and roles. The default out of the box realm used is "UserDatabaseRealm". This particular realm reads clear text passwords out of tomcat-users.xml

We're going to look at a couple of alternative realms, namely: MemoryRealm and JDBCRealm. There are other realms available to you, and if your interested, you can read up on them at: http://tomcat.apache.org/tomcat-7.0-doc/realm-howto.html

MemoryRealm

What we're going to do is change the realm to "MemoryRealm", which whilst still reading the user details out of tomcat-users.xml, will now do so using a specified encrypted format: SHA, MD2 or MD5.

The algorithm must be supported by the java.security.MessageDigest class

Open up CATALINA_HOME/instance/conf/server.xml

Find "UserDatabaseRealm", as below, and comment it out.

<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>

Next, add the following Realm inside the Host block, which is just below the section you commented out.

<Realm className="org.apache.catalina.realm.MemoryRealm"
digest="SHA" />

Save the file, and exit.

We now need to create a SHA encrypted version of the password you want to use. Tomcat is useful here as it provides a script to do just that. Inside CATALINA_HOME/bin, run the following:

digest.sh -a SHA <yourpassword>
<yourpassword>:a_whole_bunch_of_hex

As a proper example:

digest.sh -a -SHA admin
admin:d033e22ae348aeb5660fc2140aec35850c4da997

You now want to replace your password (in tomcat-users.xml) with the encrypted version.

<role rolename="manager-gui"/>
<user username="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" roles="manager-gui"/>
</tomcat-users>

Now restart Tomcat and you should be able to login.

As another little aside, let's briefly mention Realms and how they apply.

    • This Realm is shared across all web applications, on all virtual hosts, unless overridden by a or element.
    • This Realm is shared across all web applications for this virtual host, unless overridden by a element.
    • This Realm will only be used for this application.

JDBCRealm

To configure the JDBCRealm, we need to do a couple of things first:

  1. Create tables and columns in your database of choice (we're going to use MySQL in this scenario)
  2. Configure a 'tomcat' user to allow Tomcat to talk to MySQL
  3. Drop the MySQL connector JAR file into place
  4. Set up our Realm

Use the following scripts to create your tables:

(I've created a dedicated database called TomcatAuthentication)

create table users (
user_name varchar(15) not null primary key,
user_pass varchar(50) not null
);
create table user_roles (
user_name varchar(15) not null,
role_name varchar(15) not null,
primary key (user_name, role_name) );

We need to populate our tables with data, so now run the following:

insert into users values('admin','d033e22ae348aeb5660fc2140aec35850c4da997');
insert into user_roles values('admin','manager-gui');

We now need to grant a 'tomcat' user read access to these tables, so run the following from your favourite MySQL client tool:

grant select on TomcatAuthentication.* to 'tomcat'@'localhost' identified by 'acRs7mLLbA';
flush privileges;

The next thing we need to do is get a hold of the MySQL JDBC driver, which we can download from: http://dev.mysql.com/downloads/connector/j/

Download the file, extract the contents, and then copy the JAR file (in my case mysql-connector-java-5.1.20-bin.jar) to CATALINA_HOME/lib

Now we want to add a Realm to our servers.xml file to support JDBC.

Add the following to our definition (replacing the MemoryRealm):

<Realm className="org.apache.catalina.realm.JDBCRealm"
driverName="com.mysql.jdbc.Driver"
connectionURL="jdbc:mysql://localhost/TomcatAuthentication?user=tomcat&password=acRs7mLLbA"
userTable="users" userNameCol="user_name" userCredCol="user_pass"
userRoleTable="user_roles" roleNameCol="role_name"
digest="SHA"/>

I've already replace the appropriate entries above to those we're using in this article.

Now you can restart Tomcat, and when you try to access the Tomcat Manager or Service Status page, authentication will now take place against MySQL.

You can obviously harden your password further by using SSL, and locking down access to internal IPs or even just localhost, and we'll look at this in a future article.

Apache and SSL Keys

SSL certificates expire and need renewed regularly, which involves at least replacing the certificate file and possibly the key and chain certificate files too, running "apachectl configtest" to make sure everything is properly in place and then restarting. Quick, simple, and therefore quite surprising when Apache dies immediately afterwards.

SSL certificates and keys need to be matched - the key must be the one used to generate the signing request for the certificate. As wonderful as it otherwise is, "apachectl configtest" does not test for this! If for whatever reason they don't match then Apache can't handle SSL requests and will refuse to start and you'll see this in the error logs :

[Fri Feb 17 21:58:56 2012] [error] Unable to configure RSA server private key
[Fri Feb 17 21:58:56 2012] [error] SSL Library Error: 185073780 error:0B080074:x509 
certificate routines:X509_check_private_key:key values mismatch

This shouldn't happen if you have been careful with your key, request and certificate generation, but sometimes life isn't straightforward - security considerations can mean it's impossible to request SSL certificates on behalf of customers. Non-technical customers can sometimes get mixed up during the process and use the wrong keys and CSRs which means you end up with a mismatch.

Now you have to track down the correct pair of key and certificate files - and quickly, as at this point Apache is down and will not come back up until you fix it or disable SSL. Luckily the OpenSSL command line tool can help.

The first step is to find the correct certificate file. OpenSSL will tell you everything you need to know to track down the right one. For each of your possible certificate files you can run the following, replacing MYDOMAIN.crt with your certificate file :

openssl x509 -noout -text -in MYDOMAIN.crt

This will print a lot of detailed information, of which we are only interested in the first few lines :

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 198312 (0x306a8)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=GB, O=Fuzzy Orange, CN=Quick Example SSL CA
        Validity
            Not Before: Nov  8 12:32:56 2011 GMT
            Not After : Aug 27 04:55:46 2012 GMT
        Subject: serialNumber=1pfX346-sw4eFZmll0aBd3dWkwKKjLDx, C=GB, 
O=WWW.MYDOMAIN.COM, OU=FO3269373, OU=Fuzzy Orange, CN=WWW.MYDOMAIN.COM

The key elements here are the validity dates and the O and CN elements of the subject - these let you know what domain this certificate is for and the dates it can be used between. When you've found the certificate that you should be using, you then need to know it's modulus to compare with your keys. Once again OpenSSL can do this for you - simply run the following, once again replacing MYDOMAIN.crt with your certificate file :

openssl x509 -noout -modulus -in MYDOMAIN.crt

This will print a large string of characters :

Modulus=F06D8B592B348B8D6704B05496CC3E094F875E6A6C5219DA33A...

This modulus string will only match up with the correct key for this certificate, so all we need to do now is check the modulus of your possible key files. Run this to see it, replacing MYDOMAIN.key with your key file :

openssl rsa -noout -modulus -in MYDOMAIN.key

Once again you'll get a large string of characters, this time of the key's modulus. Compare it to the certificate's modulus as extracted before - when you find a match then you have found your key and certificate pair.

Now quickly copy these matching certificate and key files to the location where Apache expects them and start it up again - this time it should run as normal and not report any errors. And next time around, use OpenSSL to verify the keys and certificates match in advance!

Different JVM settings for different Tomcat instances

My previous post about Running Multiple Tomcat Instances resulted in both instances using identical JVM settings.

There will be plenty of scenarios where that is fine, but for the times when it's not, it's really straight forward to change your JVM settings, including: the JVM being used, your memory settings, the garbage collector, and so on.

Let's take a simple scenario. Instance1 can use the default Tomcat JVM settings, but instance2 requires the following changes:

  • Max Heap size of 1GB
  • Permanent Generation to be sized to 192m
  • Different JVM

Open up instance2-startup.sh in your favourite text editor, and add the following:

(We created instance2-startup.sh in the previous article, and it can be found in your CATALINA_HOME/bin directory)

export JAVA_HOME=/usr/java/jdk1.6.0_32
export CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=192m"

The JAVA_HOME path obviously needs changed depending on the alternative JVM you want to use (and you'll need to install it of course).

Your startup file should now look (somewhat) similar to:

export JAVA_HOME=/usr/java/jdk1.6.0_32
export CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=192m"
export CATALINA_BASE=/Applications/ApacheTomcat/apache-tomcat-7.0.27/instance1
./startup.sh

Save your file. Job done.

To confirm that your instance is now using the custom JVM configuration, we're going to take a look at the Tomcat Server Status page. Before we can do that, we need to enable access.

Tomcat stores user account data in the tomcat-users.xml file that can be found in your instances conf folder, e.g.

/opt/ApacheTomcat/apache-tomcat-7.0.27/instance2/conf/tomcat-users.xml

Add the following to the file:

<role rolename="manager-gui"/>
<user username="admin" password="admin" roles="manager-gui"/>
</tomcat-users>

The above creates a role called 'manager-gui' which basically allows access to all the Tomcat management pages, as well as a new user who is assigned that role.

Clearly you don't want to use admin/admin for your username and password in production but it will suffice for our current needs.

Save the file, and then stop/start your Tomcat instance.

Fire up your Tomcat dashboard in your favourite browser; http://localhost:8282 (remember and use the port number associated with your instance).

You can now click on the 'Server Status' button that is shown on the right hand side of the page, enter admin/admin for the username/password, and you'll see a bunch of information including; JVM memory settings, the JVM being used and more.

If you fire up your other instance, the one using the default settings (remember to edit it's tomcat-users.xml file to allow access to the Server Status page), and then view its server status, you'll see the contrasting information, which proves your utilising different JVM configurations for your Tomcat instances.

Running Multiple Tomcat Instances

We've been doing a fair bit of work with Tomcat for a while now. It was partly instigated by the knowledge that ColdFusion 10 was going to use Tomcat under the hood (in replace of JRun), but we've also been doing work deploying Railo as well as some Java apps (deploying as WAR files).

One of the things we're so used to is the ability to run multiple instances of ColdFusion on JRun, and we wanted to do the same with Tomcat. As such, this entry is a generic walkthrough to get up and running with multiple instances of Tomcat.

The first thing you need to do is get a hold of Tomcat. Go grab the most recent version from http://tomcat.apache.org (v7.0.27 as of writing).

We like to install Tomcat versions inside a master ApacheTomcat folder, e.g. /opt/ApacheTomcat/apache-tomcat-7.0.27

Once you've extracted Tomcat, take a look at the contents of the folder. You'll find some text files, and more importantly a bunch of folders:

  • bin
    • contains all the binaries and scripts for running Tomcat
  • conf
    • configuration files including server.xml and web.xml
  • lib
    • the libraries that make Tomcat work
  • logs
    • funnily enough, logs
  • temp
    • used for temporary files
  • webapps
    • this is where we put our apps
  • work
    • folder used by Tomcat to write out files needed during runtime, including generated code for JSPs, class files, serialised sessions

An important part of any Tomcat installation is environment variables. There are 5 different variables that interact with Tomcat, two of which are mandatory:

Mandatory

  • CATALINA_HOME
  • JAVA_HOME

Optional

  • CATALINA_BASE
  • CATALINA_TMPDIR
  • CLASSPATH

All three optional variables can be calculated using CATALINA_HOME

The usual way

Under a normal install, we really only care about CATALINA_HOME. By default, this variable points to the Tomcat root folder, so using our set up, it points to /opt/ApacheTomcat/apache-tomcat-7.0.27

This gives access to the /bin and /lib folders, and allows all other environment variables to be set when we run startup.sh (found in the /bin folder).

And this is the important part. Instead of allowing CATALINA_BASE to be set automatically, we want to explicitly set it. That's the key to multiple instances.

Configuring Multiple Instances

Create two new folders inside your CATALINA_HOME folder, e.g.

/opt/ApacheTomcat/apache-tomcat-7.0.27/instance1
/opt/ApacheTomcat/apache-tomcat-7.0.27/instance2

Copy the following folders (and their contents) from CATALINA_HOME, and paste them into the two directories you created above:

  • /conf
  • /logs
  • /temp
  • /webapps
  • /work

(You might want to delete the contents of your copied log folders).

Next we need to configure the 3 different ports that each Tomcat instance will use. These are the connector port, the AJP port, and the shutdown port.

  • Connector port
    • What Tomcat uses to communicate with the outside world. Default: 8080
  • AJP port
    • If you're going to be connecting to Apache (or IIS), then you'll be looking to use AJP rather than the "basic" connector. AJP is a binary protocol and is therefore faster than the default connector. Default: 8009
  • Shutdown port
    • Tomcat uses a port as part of it's shutdown process. It communicates with this port to shutdown cleanly, and therefore it has to be unique per instance. Default: 8005

Let's start by opening and editing /instance1/conf/server.xml. Find the blocks similar to below for the three connector ports

<server port="8005" shutdown="SHUTDOWN">
.....
<connector connectiontimeout="20000" port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" redirectport="8443">

<connector port="8009" protocol="AJP/1.3" redirectport="8443">
</connector></connector></server>

Change the ports to something like:

<server port="8105" shutdown="SHUTDOWN">
.....
<connector connectiontimeout="20000" port="8181" protocol="org.apache.coyote.http11.Http11NioProtocol" redirectport="8443">
<connector port="8109" protocol="AJP/1.3" redirectport="8443">
</connector></connector></server>

Do the same for the second instance, providing different port numbers again.

Let's now create some startup/shutdown scripts for our instances. Create a new file called instance1-startup.sh in CATALINA_HOME/bin

Add the following to the script:

export CATALINA_BASE=/opt/ApacheTomcat/apache-tomcat-7.0.27/instance1
cd $CATALINA_HOME/bin
./startup.sh

Create another new file called instance1-shutdown.sh in CATALINA_HOME/bin, and add the following:

export CATALINA_BASE=/opt/ApacheTomcat/apache-tomcat-7.0.27/instance1
cd $CATALINA_HOME/bin
./shutdown.sh

Do the same for instance2, being careful to change the paths.

You can now run ./instance1-startup.sh and ./instance1-shutdown.sh to start/stop your first instance, as well as your other scripts to control your other instances.

Congratulations, you're now running multiple instances of Tomcat, which can be accessed via (depending on the port numbers you chose above)
http://localhost:8181
http://localhost:8282

You can now deploy your ColdFusion and Railo WAR files directly into an instances webapps folders, e.g.
/opt/ApacheTomcat/apache-tomcat-7.0.27/instance1/webapps

Adobe ColdFusion 10 available now

With the release of ColdFusion 10, Adobe's Enterprise application server for Rapid Application Development, there's a lot to get excited about.

HTML5 features heavily in the new release, with support for dynamic and interactive charting, web sockets, and video. RESTful web services are delivered out of the box. Improved Java integration as well the the move from JRun to Tomcat as the underlying JEE servlet container are just some of the new features.

You can find a full list of what's new, on the ColdFusion 10 Family section at http://www.adobe.com/uk/products/coldfusion-family.html

ColdFusion 10 is available now, and as an authorised reseller, Fuzzy Orange can help you with all your licensing requirements. Contact us for a quote, or for more information on how we can help you get the best out of ColdFusion 10.

Matt Gifford shortlisted for Developer of the Year

As previously blogged, our very own Matt Gifford was nominated in this years .NET Awards for Developer of the Year.

We're delighted to announce that Matt has been shortlisted for the award, along with Paul Irish and Lorna Mitchell.

Congrats Matt and best of luck!

.net Awards - Developer of the Year

The annual .net Awards, organised by .net magazine, is one of the Web industry's leading ceremonies recognising some of the best talent the world of Web Design and Development has to offer.

For 2011, our very own Matt Gifford has been nominated in the "Developer of the Year" category.

Matt has worked like a demon, not just for us or our our clients, but also for the ColdFusion community as a whole, releasing a huge number of open source applications, including the much loved monkehTweets. His hugely popular "Object Oriented Programming in ColdFusion" book, released Autumn 2010, has been a massive success, and he's authored numerous magazine articles covering (to name but a few): Building Mobile Application with Adobe AIR, ColdFusion ORM, HTML5, Application Security, and speaking at Conferences.

We know just how good Matt's work is, as do our clients, and it's great to see that work be recognised globally.

You too can vote for Matt. Just head to the .net Awards website.

More Entries

Downloads

Downloadable PDFs, video case studies, podcasts and more.