Volume Number: 19 (2003)
Issue Number: 8
Column Tag: Untangling the Web

Untangling the Web

Server Side Includes with Apache

by Kevin Hemenway

In our last column, we chatted about turning on our web server, getting more information concerning how it's been installed (like where the log and configuration files are), and then took a quick look at editing the primary control file, /etc/httpd/httpd.conf. After we made our changes (intended to circumvent an Evil ISP's port filter), we learned of an alternate route to restarting Apache by using the command line utility apachectl.

All relatively Duplo. Let's break out the Legos.

By running a web server, you're inviting anyone on the 'net to stop by your computer and access files you've deemed worthy. This should be a scary thought: what about all the files you DON'T deem worthy, like development versions of your software, database files that contain customer information, or the source code to any web scripts you may be running? Even if you've got a dedicated machine solely for your web site, you've still got to wear another hat: that of security princess ("I feel preetTty, OoOh soOO PretTtYy!").

This isn't security like in software downloads, where you concern yourself with viruses or trojans or pirated registrations. With web server security, you've got one wide-open front door, accessible to anyone who deems to visit. Any script you run, any software you install, any service you turn on - all are more points of access for a disgruntled cracker.

While we'll talk about security when necessary, there are two books that will put more diamonds in your tiara than I ever could. Both were sent as review copies, and both have since earned welcome places on my bookshelf.

The first, Mac OS X Maximum Security from John Ray and William C. Ray, covers every aspect of hardening your Mac OS X installation. Broken up into three primary sections, it gets you into the mindset of thinking secure, follows up with different ways people get into systems, and then instructs on how to actually batten down the hatches. Encompassing far more than just Apache, I'd recommend it to all readers, not just those serving web pages.

The second is much more specific to our topic: Maximum Apache Security by Anonymous. Psychotically comprehensive, it covers exactly (with source code) how Apache handles various bits of its own logic, as well as how it interacts with third party software like databases, scripting languages, and modules. Unlike Mac OS X Maximum Security, it assumes you already have a strong knowledge of how Apache works. If you do, and you're reading my columns solely for their rhythm, this is a book you should consider adding to your collection.

Server Side Includes (SSI) are an Apache built-in that, at its simplest, allow you to include one bit of content within another. If you're thinking variable interpolation, you've nailed it on the head. For web page design, this becomes very helpful in regards to navigation bars, headers, footers, copyright statements, or anything intended for every page. I've designed entire sites using SSI, with the content files being nothing more than semantic headers and paragraphs, and the pretty "shell" being included by Apache upon request. When a redesign occurs, modify two or three files and I'm done - the bulk of the site, containing hundreds of pages of content, remains unmodified.

That's not all SSIs can do, however. With a dash of conditionals and a smidgen of regular expressions, you've got a feature set that can quickly perform some interesting tricks for when you don't need (or want) the power of PHP or Perl (which we'll cover in future columns).

Even though they're built into Apache, SSI's aren't enabled by default. If you recall from the last column, the easiest way to learn about a feature in Apache is to just search for it in /etc/httpd/httpd.conf. Open said puppy up in your favorite authenticating text editor (BBEdit for me) and do a search for the word "includes". Your first match should be:

LoadModule includes_module    libexec/httpd/mod_include.so

Most of the features within Apache are controlled by modules, which can best be described as a "plugin" to the core web server code. A decent amount of modules ship with Apache already - you'll see them above and below our first search result. Here, we're loading a module named includes_module, which is located on the file system at /usr/libexec/httpd/mod_include.so (/usr being the Apache root directory via HTTPD_ROOT, see last column). When a module is enabled (indicated by no preceding comment character, #), it's ready to be configured for use.

For every LoadModule, there's a matching AddModule shortly after. To correctly enable a module within Apache, both lines need to exist uncommented. The match to the above LoadModule is AddModule mod_include.c, which you'll see in another fifty lines or so.

Any programmer can write a module to Apache, and a healthy list of third party enhancements is available at http://modules.apache.org/. Most modules are named mod_SOMETHING, like mod_include, mod_php, mod_access, etc, although there are occasional exceptions. If you're interested in exploring module creation for Apache, check out Writing Apache Modules with Perl and C (http://www.oreilly.com/catalog/wrapmod/).

Our next search result for "includes", of which I've snippeted only the relevant, is below. It contains the configuration for a specific directory on our machine, namely /Library/WebServer/Documents. Of more importance, as a concept, is the "block" or "container" - the directives within <Directory> only apply to the location specified. Apache has a number of block directives, which you'll see more of as the columns progress.

<Directory "/Library/WebServer/Documents">
    # This may also be "None", "All", or any combination of 
    # "Indexes", "Includes", "FollowSymLinks", "ExecCGI",   
    # or "MultiViews".
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

The configured directory is also Apache's DocumentRoot - the default location from which files will be served. http://127.0.0.1/demo/lition.html, for instance, would reference the file located at /Library/WebServer/Documents/demo/lition.html. While we won't be getting into the details of the other directives above (yet), we need to add Includes to the Options line, like so:

Options Indexes FollowSymLinks MultiViews Includes

By adding Includes, we're instructing Apache to allow SSI's within that directory and all it's children. If we wanted to support SSI's in only a certain subdirectory, we'd need to add a new <Directory> block entirely. Take the example below, which ensures that only /testbed/ (and it's children) are privileged. We don't have to specify the other directives, like AllowOverride, Order, and Allow, as those are inherited from the parent.

<Directory "/Library/WebServer/Documents">
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>
<Directory "/Library/WebServer/Documents/testbed">
    Options Includes 
</Directory>

We've still got a little more to go before we're up and running. Let's move on.

As we'll see in the second half of our article, using an SSI is a simple matter of including a special bit of code in your normal HTML. This code has to be interpreted by the SSI module, and the final bit of HTML (sans those special codes) is spit out to the browser. To interpret these codes, we need to tell Apache to associate certain files with the SSI module. This is where the last part of our configuration lies, and is our next relevant search result:

# If you want to use server side includes, or CGI outside
# ScriptAliased directories, uncomment the following lines.
#
# To use CGI scripts:
#
# AddHandler cgi-script .cgi
#
# To use server-parsed HTML files
#
# AddType text/html .shtml
# AddHandler server-parsed .shtml

We'll cover CGI next month, so concern yourself only with the last two lines. Both are commented (indicated by the # character that precedes them), and both help the final part of our configuration. The first command tells Apache that when a file with the extension shtml has been requested, the server should send a MIME type of text/html. This, in effect, treats all shtml files as if they were normal html (which, after parsing, they will be).

The next line is what actually associates shtml files with the SSI module. When someone requests one of these files, it will be "handled" by the server-parsed extension of Apache. When the handler is done, the completed results will be sent as a text/html file to the requesting user-agent (ie. your visitor's browser).

To finally enable SSIs, uncomment these last two lines, than restart the Apache web server (either through the Sharing System Preference, or with sudo apachectl restart). Once Apache has restarted successfully, we can finally move on to something demonstrative.

Playing With Server Side Includes: Our First Attempt

We're now going to create two files. The first will be the "shell" that includes some outside data, and the second file will be the outside data itself. Open your favorite text editor, add the following into a document called index.shtml, and save that file into /Library/WebServer/Documents:

<html>
<head>
 <title>Quote Selector</title>
</head>
<body>
<h1>Quote Selector</h1>
<blockquote>
  <!--#include virtual="quote.shtml" -->
</blockquote>
</body>
</html>

Note that our filename has an extension of shtml (index.shtml), which was what we configured our SSI handler for. If we had ended the file with an html extension (index.html), our special codes would be heartlessly ignored. Speaking of special codes, our newly created file also contains our first introduction. Let's dissect what we see:

• This SSI statement and, in fact, all SSI statements, are encased in an HTML comment tag. If you accidentally include one in a file that is not handled by the SSI module, you can always "view HTML source" in your browser and see the statements unchanged. This becomes an important barometer: if your SSIs aren't working, then "view source" and see if they're being interpreted at all. If they are, they'll disappear from the final browser output - if they're not, you'll see them as normal HTML comments.

• After the opening comment tag, a # immediately appears. No spaces should exist between the two - if some do, then your SSI statement won't be interpreted correctly.

• We virtually include a file, quote.shtml, which doesn't yet exist. As specified, this file will be in the current directory (being /Library/WebServer/Documents). We can also use the standard .. shortcut to point to files outside the current location.

Before we create our second file, load http://127.0.0.1/index.shtml in your browser. You'll be greeted by the error message in Figure 1. The cause of the error should be obvious: since quote.shtml doesn't exist, our first attempt at an SSI directive failed miserably.

A rarely used feature of SSI, however, is the ability to customize this error message. Too many times have I visited sites and seen this error chuckling at the ineptitude of the web master. It's well-founded mirth, especially when the fix is mighty mindless - as we can see below, we can customize the error message (with or without embedded HTML) for as many different uses as we need.

<html>
<head>
 <title>Quote Selector</title>
</head>
<body>
<h1>Quote Selector</h1>
<blockquote>
  <!--#config errmsg="Bah! Quote.shtml does not exist!"-->
  <!--#include virtual="quote.shtml" -->
  <!--#config errmsg="<br>Ouch! Nor does quote2.shtml!"-->
  <!--#include virtual="quote2.shtml" -->
</blockquote>
</body>
</html>

An example of this output is shown in Figure 2.

Let's create our quote.shtml, saving into /Library/WebServer/Documents:

Toynbee Idea
<br>In Kubrick's 2001
<br>Resurrects Dead
<br>On Planet Jupiter

With the new file in place, reloading our browser shows us Figure 3. Note that we didn't actually need to name our data file with the shtml extension (like quote.shtml) - only the file that contains actual SSI statements need follow that restriction (so quote.txt, quote.ssi and quote.include would all have been viable alternatives). We'll be expanding quote.shtml with it's own SSI's shortly.

Now, for literal purposes, this demonstration is winning no awards. The raw capability of SSI's, as we've mentioned, works best when combined with navigational elements, copyright statements, etc. Let's continue on with something a little more complicated.

Playing With Server Side Includes: Conditionals and Queries

We've named our example "Quote Selector" for a reason: we'd like to offer a few different quotes that people can click on, but we don't want to have one page for each quote (like we would with quote1.html, quote2.html, quote3.html, etc.). We can do this easily enough with SSI conditionals and GET queries.

When you're submitting data through a web browser (as in typing your address into a form at Amazon, or choosing different result sets from a database listing), you're transmitting the data in one of two ways: GET or POST. GET's are for general-purpose forms, and are often used when you're simply requesting information: a search result from Google or a query match from a database. You can always tell when you've just used a GET form, because the resultant URL will contain the information you submitted. For instance, searching for "biozombie soda" creates a URL like http://google.com/search?q=biozombie+soda, where a value of biozombie+soda was assigned to a variable named q.

The prime benefit of GET is that you can bookmark the above URL and revisit it at a later date. POST's, on the other hand, are generally used when the site is requesting a lot of data (like a cut-and-paste of your high-school transcript). Unlike GET, the information submitted with POST is not transmitted in the URL, so it's not something readily recreated.

We're going to edit our two files so that they'll return different quotes depending on the contents of a GET query. Replace your existing index.shtml with the following, which we've added a selectable list of quotes to. Each quote is linked to the current document (since there's no filename specified), and passes a certain value through a GET query (either q01 or q02 - even though they don't have values, they'll be passed in our query string).

<html>
<head>
 <title>Quote Selector</title>
</head>
<body>
<h1>Quote Selector</h1>
<h2>Choose a quote:</h2>
<ul>
 <li><a href="?q01">On pavement.</a></li>
 <li><a href="?q02">On papyrus.</a></li>
</ul>
<blockquote>
  <!--#include virtual="quote.shtml" -->
</blockquote>
</body>
</html>

And replace your existing quote.shtml with this next iteration, which contains a couple of noticeable additions, most prominent of which is a set of conditionals for testing against the value of $QUERY_STRING. Conveniently enough, the $QUERY_STRING is the entire value of the GET that would be submitted from our newly revised index.shtml. If neither quote was chosen (ie. this was our reader's first visit to the page), then we spit out a quick warning that no quote has been chosen.

<!--#if expr="\"$QUERY_STRING\" = \"q01\"" -->
  Toynbee Idea
  <br>In Kubrick's 2001
  <br>Resurrects Dead
  <br>On Planet Jupiter
<!--#elif expr="\"$QUERY_STRING\" = \"q02\"" -->
  That is not dead
  <br>which can eternal lie
  <br>And with strange aeons
  <br>even death may die
<!--#elif expr="\"$QUERY_STRING\" = \"\"" -->
  No quote has been selected!
<!--#endif --> 

Figure 4 shows the second quote having being selected, and the generated URL.

There's still more useful things you can do with this sort of structure: I've written before on using this technique to allow end users to customize positioning, specific stylesheets (which could radically change the layout, colors, fonts, etc.) and so forth, all without requiring the need of a cookie, and all choices being bookmarkable from computer to computer. Read more at "Allowing Simplistic User Preferences with SSI" (http://hacks.oreilly.com/pub/h/226).

I've also recently used this technique in my header files to change the logo that appears based on the specific URL (ie. example.com/category1/would have a different logo than example.com/category2/). To implement this, you'd use a few other tricks of SSI, namely the $DOCUMENT_URI variable, which contains the currently requested URL, as well as SSI's own getter (echo) and setter (set) methods. A quick example is below:

<!--#if expr="$DOCUMENT_URI = /books_and_related/" -->
   <!--#set var="header" value="header_books.gif"-->
<!--#elif expr="$DOCUMENT_URI = /comics_and_zines/" -->
   <!--#set var="header" value="header_comics.gif"-->
<!--#else -->
   <!--#set var="header" value="header_main.gif"-->
<!--#endif -->
<img src="/images/<!--#echo var="header"-->" 
   width="445" height="90" align="middle" hspace="5">

Seeing that I'm running out of space, it's best for me to link to some other SSI related hackery I've written, all of which demonstrate certain functionalities I've deigned to ignore here. "More Server Side Trickery" (http://hacks.oreilly.com/pub/h/222) covers cheap username and password authentication, different images or greetings depending on the current time, and server side hit counters and the powers of exec. A "Search Engine Friendly Image Gallery" (http://hacks.oreilly.com/pub/h/225), however, is a "full application", much like the quote selector above. It demonstrates how to use one SSI file to showcase an infinite number of images, with error correction, file size, modification times, and more.

You may have noticed that we've been referring to http://127.0.0.1/index.shtml instead of the cleaner http://127.0.0.1/. Depending on whether you deleted the default web server files or not, you may have received an error message or directory listing when you requested the shorter URL. Why doesn't index.shtml display when we don't specifically request a document (as in http://127.0.0.1/ or http://127.0.0.1/testbed/)? The answer, in a word: DirectoryIndex.

Apache's DirectoryIndex controls which files it should consider the default document for a directory - in other words, what should be served when no other file has been requested. By default, this is normally just index.html, but you can include as many fallbacks as you wish, as per the following example:

DirectoryIndex index.shtml index.html index.cgi default.htm

When you're editing this line in /etc/httpd/httpd.conf, be sure to list the file names in order of preference and usage: if you're going to be using default.htm more often than index.cgi, move that to earlier in the list. You'll get small performance gains by sorting correctly like this as Apache won't have to look through the entire list for each request.

In our next column, we'll chat about CGI: what it is, how to enable it, how to code for it, how to implement scripts you find on the 'net, and all the rigmarole and hilarity that ensues. If we have room, we'll also talk about how to remove the need for file extensions, definitely an important step to good URL design (see our first column). For now, students may contact the teacher at morbus@disobey.com.

• "I pity any" what "who isn't me tonight"?

• What other inventive things can SSI be used for?

• The quotes in our examples: where'd they come from?

• Ever seen Biozombie? Any similar suggestions?

  ---

  Kevin Hemenway, coauthor of Mac OS X Hacks, is better known as Morbus Iff, the creator of disobey.com, which bills itself as "content for the discontented." Publisher and developer of more home cooking than you could ever imagine (like the popular open-sourced aggregator AmphetaDesk, the best-kept gaming secret Gamegrene.com, articles for Apple's Internet Developer and the O'Reilly Network, etc.), he went out twice this summer, only to scurry back inside like a disgruntled cockroach. Contact him at morbus@disobey.com.