TweetFollow Us on Twitter

Enabling CGI Scripts, The Second

Volume Number: 19 (2003)
Issue Number: 10
Column Tag: Programming

Untangling the Web

Enabling CGI Scripts, The Second

by Kevin Hemenway

You've enabled CGI, but how do you know it's good?

In the last issue, we learned about CGI scripts: what they are, what they can do, how they're already enabled within Apache, and how to tweak that configuration to be more URL friendly. What we didn't do is teach you anything for the future: at most, we brought a wide-eyed wonder-boy to a patch of poison ivy, and backed away slowly. Will he rub it on his skinned knee? Pin it to little Susie's dress as a token of his affection? Roll around in it like catnip? Where is the inbred fear necessary for every child's survival?

Insert transitional one-liner here!

Dissection--Similarities

Before we can understand, be aware, and watch for the security ramifications of running CGI scripts from unknown and untrusted third parties, we need to see how they're coded, how poorly written ones can ruin our mornings, and how to look for some semblance of quality. The quickest way to get a general feel is with the two sample scripts already installed with Apache: /Library/WebServer/CGI-Executables/printenv and /Library/WebServer/CGI-Executables/printenv/test-cgi. If you looked at their source code last month, you may have noticed they're written in two different languages.

The smaller of the two scripts, test-cgi, starts with #!/bin/sh, whereas printenv instead uses #!/usr/bin/perl -T. These lines, specifically the #! prefix, are often called the "shebang", and tell us which interpreter will execute the programming instructions that follow. The interpreter located at /bin/sh, rarely seen in production CGI, indicates that the rest of the code is written in the shell scripting language. Any CGI script you deploy will need to have some sort of shebang--whether it's /bin/sh, /usr/bin/perl, /usr/bin/python or something else entirely, it's absolutely required. Not only is it necessary, it also has to be accurate: if your only Perl is /sw/bin/perl, then the shebang should point there instead. Shebangs can also contain command line arguments: in printenv, -T is passed directly to the /usr/bin/perl interpreter (where it means something we'll cover a bit later).

Another similar difference between our two scripts is the printing of something called a Content-type (Listing 1), which tell the requesting user-agent (your visitor's browser) what sort of data it's about to receive (an image to render, text to display, XML to parse, etc.). The Content-type will never actually be shown in your final output--it's hidden pixie dust for the browser's benefit only (if you're curious, Mozilla allows you to view the Content-type by getting the "Page Info" of the current URL). Without this crucial bit of contextual magic (and the two required newlines), Apache will fail your CGI scripts with an "Internal Server Error". This error is never a satisfying explanation--you'll need to check Apache's /var/log/httpd/error_log for the exact reasoning.

Listing 1: Printing the Content-type in Shell and Perl

From the sample CGI scripts printenv and test-cgi

# content type display from test-cgi
# note that echo spits out a newline,
# 2 echo's for the 2 required newlines.
echo Content-type: text/plain
echo
# and the similar entry from printenv
print "Content-type: text/html\n\n";

The values of our Content-types (text/plain and text/html) didn't just appear out of thin air--they're MIME types, and most any file you've ever worked with has one. You can find a large listing of MIME types, based on their common file extensions, by perusing the /etc/httpd/mime.types file. For example, the matching MIME types for JPEG, XHTML, Quicktime, and Microsoft Word files are:

image/jpeg                      jpeg jpg jpe
application/xhtml+xml           xhtml xht
video/quicktime                 qt mov
application/msword              doc

If you can't find the matching MIME type for the data you're interested in serving (either because it's not in the mime.types file or Google has spurned your search request), you can use the "some sort of data" MIME type of application/octet-stream. This has already been explicitly assigned to a number of files, including Apple disk images:

application/octet-stream   dms lha lzh exe class so dll dmg

Dissection--Why The Perl Script Is Arguably Stronger

All CGI scripts, regardless of what they're programmed in, can be run from the command line--whether they actually do anything useful is a case-by-case basis. This is a surprisingly useful bit of information: since troubleshooting and debugging happens best when unfrilled by complication, removing Apache from the process can prove helpful. Running your CGI scripts on the command line can preemptively weed out problems like missing Content-type's, file permission errors, invalid syntax problems, missing language extensions, and so forth.

Both the test-cgi and printenv scripts run "successfully" at the command line, although only the first gives any useful output (Figure 1). Compare this to the regular browser-based output we demonstrated in the last MacTech (or simply re-access http://127.0.0.1/cgi-bin/test-cgi). The first line is that dastardly Content-type and, as mentioned before, is normally processed by the browser and removed from the final display. Since we're running the script without the benefit of a web server or browser, the Content-type is viewable without extra effort. This becomes a handy barometer: if you run your CGI script from the command line and there's no Content-type, it'll never run correctly under Apache.


Figure 1: The slightly undefined test-cgi, when run in the Terminal

But wait... there's no Content-type if we try to run printenv (in fact, there's nothing at all), so why does it work when we access it by URL (http://127.0.0.1/cgi-bin/printenv)? In actuality, this is one of the "strengths" of the Perl version. If you check the source code, the next line after our required shebang (ignoring comments) is:

exit unless ($ENV{'REQUEST_METHOD'} eq "GET");

This terminates the script unless it was invoked via a GET request. Generically speaking, unless it is a POST, every request a web browser makes is a GET with or without key/value pairs. Since the shell isn't a web browser, no GET is issued and the script terminates. If we wanted to get fancy, we could fake the required method by running setenv REQUEST_METHOD GET && ./printenv (if you're using the tsch shell; REQUEST_METHOD=GET ./printenv if you prefer bash). As a result, we get a Terminal full of HTML listing the environment variables. We can redirect this mass of HTML to a file by adding > output.html to our previous command line; Figure 2 shows the generated file.


Figure 2: Shell output of our tricked printenv script

Figure 2 also gives us another reason why the Perl script is stronger: it doesn't pretend to know what the environment is going to look like. test-cgi, hard-coded to display the values of known variables (SERVER_SOFTWARE, SERVER_NAME, GATEWAY_INTERFACE, etc.), shows nothing but undefined values when run from the Terminal (Figure 1), where those specific entries don't normally exist.

Three Ways Perl CGI Scripts Can Be Improved

The bulk of the code within the printenv script caters to creating a pretty HTML page, something not important to the true purpose of generating a list of the current environment. To make our upcoming improvements more clearly, we'll base our changes on the Perl script shown in Listing 2, which does the exact same thing as printenv, only without the HTML. For all intents and purposes, this is a working CGI script: it's got the shebang pointing to the correct Perl interpreter, and it prints a plain-text Content-type before any other data.

Note that even though we're talking specifically about CGI scripts, the following improvements can, and should, be made in most any Perl script, especially those to be used in production environments. Security should never be a feature.

Listing 2: Printing the environment more simply

Our base.pl script could use some improvements.

#!/usr/bin/perl
print "Content-type: text/plain\n\n";
foreach $var (keys %ENV) {
   print "$var = $ENV{$var}\n";
}

Save this file as base.pl and run it from the command line; my output is in Figure 3. None of our upcoming improvements will change this display and, as you can see by comparing it to Figure 2, it's identical save for the loss of HTML (and the differences between Safari and the Terminal's interpretation of TERMCAP).


Figure 3: Our rewritten script's (base.pl) output

Our improvements to the script are quite minimal additions, but they ensure that user data has been properly checked for dangerous input, warnings have been enabled for common mistakes or typos that don't necessarily stop a script from running, and a stricter development environment has been used to encourage stronger coding and careful variable declaration. The revised script is shown in Listing 3.

Listing 3: Printing the environment more strongly

Our revised script is three times stronger than before.

#!/usr/bin/perl -wT
use strict;
print "Content-type: text/plain\n\n";
foreach my $var (keys %ENV) {
   print "$var = $ENV{$var}\n";
}
  • Use warnings: The first change, adding -w to the shebang, turns on Perl's warnings pragma, which spits a list of optional, non-fatal warnings to STDERR (which becomes Apache's error_log when run as a CGI). Technically, you don't have to address any of the messages since the script will continue on regardless, but they'll alert you to typos, uninitialized values, deprecated functions, and a slew of other mishaps that can eventually escalate into full-blown bugs. Typically, the messages are terse enough to be useful for seasoned Perl programmers, but you can increase their verbosity by adding use diagnostics; within the body of your code.

  • Use strict: Our third and fourth changes complement our warnings. Perl's strict pragma should be used in any script that is more than "casual", and ensures that every variable is pre-declared and localized, and that other "unsafe constructs" are detected and addressed. Unlike warnings, any error that triggers strict will stop your script from continuing further. You'll notice that we've localized our $var variable with the my() function. The first time you use strict, it'll feel like an unwieldy and overly doting mother, but scripts that compile cleanly benefit from an attention to detail that strengthens their quality immensely.

  • Use taint: Even though it is "strongly recommended", very few Perl or CGI scripts use taint mode, which is what the -T on the shebang enables. Under this mode, any outside data received by your code is considered highly dangerous, and will cause script errors until it has been checked for safety. These safety checks can be as simple as ensuring that a command line argument only contains alphanumerics, or that the process you're spawning isn't being handed potentially damaging shell metacharacters. While taint mode will force you to focus more strongly about the evils of the outside world and exactly what data you expect, programmers who misunderstand how to "untaint" data may inadvertently do so incorrectly, creating a false sense of security.

These programming additions aren't the ultimately panacea, but merely a placebo. Yes, your code will be stronger with them, but that doesn't mean crucial bugs won't creep in and ruin your day. Serious coders and sysadmins should take a look at the following sampling of Perl and CGI security links:

  • The Perl Security manpage, accessible by typing man perlsec in your Terminal, can also be read online at http://www.perldoc.com/perl5.6.1/pod/perlsec.html

  • "Avoiding security holes when developing an application", a six part series from LinuxFocus.org: http://www.linuxfocus.org/English/November2001/article203.shtml

  • SecureProgramming (http://www.secureprogramming.com/) offers a huge collection of links to over 50 articles, books, recipes to learn from and adapt, and more.

  • RFP's "Perl CGI problems", which appeared in an old issue of the seminal Phrack magazine, still remains relevant: http://www.wiretrip.net/rfp/txt/phrack55.txt

  • CERT's "How To Remove Meta-characters From User-Supplied Data In CGI Scripts", in both Perl and C: http://www.cert.org/tech_tips/cgi_metacharacters.html. Handy for when you're looking to untaint some data.

  • The "Securing Programming for Linux and Unix HOWTO", available from http://en.tldp.org/HOWTO/Secure-Programs-HOWTO/. Similar articles like "The Hack FAQ" (http://www.nmrc.org/pub/faq/hackfaq/index.html), and the "WWW Security FAQ" (http://www.w3.org/Security/Faq/www-security-faq.html) will also prove insightful.

    Choosing a CGI Script for Deployment

    The above programming suggestions are fine if you're solely looking at the code quality of a potential CGI script, but there are few more areas to investigate before you can consider a program worthy of being installed on your server:

    • Check the Bugtraq archives (http://securityfocus.com/archive/1). Anyone interested in security should be reading Bugtraq, where a large community of hackers, white hats, sysadmins, and professionals regularly post bugs, exploits, and warnings for insecure products. Occasionally, you'll also see new whitepapers concerning various aspects of security and programming. Before installing new scripts, comb the archives to see if any advisories have been posted. If so, ensure they've been fixed before using the code.

    • Googling for problems can prove illuminating, as you'll often find common tech support problems, heaps of praise or scorn for the code or author, and occasionally, other web hosts who offer the script for their own customer base.

    • Check the dates: When was the script last updated? Is it so long ago that no one will give a darn if you have a problem? Just because a script doesn't have any reported problems in Bugtraq doesn't mean that it isn't susceptible to relatively new exploits like cross-site scripting attacks (http://www.cgisecurity.com/articles/xss-faq.shtml). Code that has been updated recently has a better chance of good turnaround time for crucial fixes, updates, and support.

    • Got logfiles? Most CGI scripts don't have any logging capability, primarily because they only do one small thing (like email forms, add one to a number, display a calendar, etc.) Some complicated scripts, however, can benefit from logging, especially those with built-in user authentication ("who is using my site?") or flaw tracking ("a bug occurred at [time], and things turned awry [like this]"). Scripts can use their own logfiles or Perl's Sys::Syslog module to log directly to /var/log/system.log.

      Homework Malignments

      In our next column, we'll move on to configuring PHP, as well as explain the up- and downsides between forking processes (like CGI) and embedded modules (like mod_php). We'll explore the default configuration of PHP, the non-existent configuration file (php.ini) and, if we have time, how to install MySQL and do a few integration tests. For now, students may contact the teacher at morbus@disobey.com.

    • Besides -w, you can also enable Perl's warning pragma with use warnings; (similar to use strict;). Subtle differences exist between the two--research them and find out which satisfies your programming needs better.

    • Any Perl script with logging may eventually run up against a perceived "buffering" problem, the sordid details of which are explained in Mark Jason Dominus' "Suffering from Buffering?" (http://perl.plover.com/FAQs/Buffering.html).

    • If you're looking to brush up on your Perl knowledge, you can't go wrong with O'Reilly's Learning Perl, The Perl Cookbook (which just received an impressive Second Edition update), and the recent Learning Perl Objects, References, & Modules. You can read sample chapters from all the books at http://www.oreilly.com/.


      Kevin Hemenway, coauthor of Mac OS X Hacks and Spidering Hacks, is better known as Morbus Iff, the creator of disobey.com, which bills itself as "content for the discontented." Publisher and developer of more home cooking than you could ever imagine (like the popular open-sourced aggregator AmphetaDesk, the best-kept gaming secret Gamegrene.com, the ever ignorable Nonsense Network), he's twirling his hair and trying not to cheerlead. Contact him at morbus@disobey.com.

  •  
    AAPL
    $100.75
    Apple Inc.
    +0.00
    MSFT
    $46.36
    Microsoft Corpora
    +0.00
    GOOG
    $577.36
    Google Inc.
    +0.00

    MacTech Search:
    Community Search:

    Software Updates via MacUpdate

    QuickBooks 2015 16.0.0.1352 R1 - Financi...
    QuickBooks 2015 helps you manage your business easily and efficiently. Organize your finances all in one place, track money going in and out of your business, and spot areas where you can save.... Read more
    Mac DVDRipper Pro 5.0.1 - Copy, backup,...
    Mac DVDRipper Pro is the DVD backup solution that lets you protect your DVDs from scratches, save your batteries by reading your movies from your hard disk, manage your collection with just a few... Read more
    Apple OS X bash Update 1.0 - Fix for sec...
    The OS X bash Update fixes a security flaw in the bash UNIX shell on OS X 10.9.5 (also on OS X 10.8 and 10.7 [see Related Links below]). OS X 10.9.5 or later Downloads for OS X 10.8 and OS X 10.7 in... Read more
    SyncTwoFolders 2.0.5 - Syncs two user-sp...
    SyncTwoFolders simply synchronizes two folders. It supports synchronization across mounted network drives and it is a possibility to run a simulation showing in a log what will be done. Please visit... Read more
    FinderPop 2.5.7 - Classic Mac utility, n...
    FinderPop is a Universal preference pane that extends OS X's contextual menus using a FinderPop Items folder much as the Apple Menu Items folder used to do for the Apple menu. It has other features... Read more
    VueScan 9.4.45 - Scanner software with a...
    VueScan is a scanning program that works with most high-quality flatbed and film scanners to produce scans that have excellent color fidelity and color balance. VueScan is easy to use, and has... Read more
    LibreOffice 4.3.2.2 - Free Open Source o...
    LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
    calibre 2.4 - Complete e-library managem...
    Calibre is a complete e-book library manager. Organize your collection, convert your books to multiple formats, and sync with all of your devices. Let Calibre be your multi-tasking digital... Read more
    Default Folder X 4.6.9b1 - Enhances Open...
    Default Folder X attaches a toolbar to the right side of the Open and Save dialogs in any OS X-native application. The toolbar gives you fast access to various folders and commands. You just click... Read more
    Beamer 2.0.4 - Stream any movie file fro...
    Beamer streams to your Apple TV.... Plays any movie file - Just like the popular desktop movie players, Beamer accepts all common formats, codecs and resolutions. AVI, MKV, MOV, MP4, WMV, FLV. To... Read more

    Latest Forum Discussions

    See All

    Fold the Adventure Review
    Fold the Adventure Review By Jennifer Allen on October 1st, 2014 Our Rating: :: AWKWARD FOLDSUniversal App - Designed for iPhone and iPad Fold pieces of paper to create platforms for a princely rabbit in this puzzle game; something... | Read more »
    WW2: Sandbox. Strategy & Tactics (G...
    WW2: Sandbox. Strategy & Tactics 1.0.0 Device: iOS Universal Category: Games Price: $9.99, Version: 1.0.0 (iTunes) Description: Sandbox is the unlimited sequel to our Strategy & Tactics: World War II. We've abandoned... | Read more »
    apeFilter (Music)
    apeFilter 1.0 Device: iOS Universal Category: Music Price: $6.99, Version: 1.0 (iTunes) Description: | Read more »
    Shred It! Review
    Shred It! Review By Jennifer Allen on September 30th, 2014 Our Rating: :: GORGEOUS BUT BASICUniversal App - Designed for iPhone and iPad It might look lovely, but Shred It! is a pretty shallow endless runner/snowboarding game.   | Read more »
    Check Out the New Teaser Trailer forGAME...
    Check Out the New Teaser Trailer forGAMEVIL’s Darkness Reborn Posted by Jessica Fisher on September 30th, 2014 [ permalink ] Darkness Reborn, by GAMEVIL< | Read more »
    It Came From Canada: Angry Birds: Transf...
    Anyone afraid that throwing Transformers into the Angry Birds mix would result in a Michael Bay-level of childhood pillaging can rest easy. While Rovio’s famous fowls may be a 21st century staple, Angry Birds: Transformers wears its affection for... | Read more »
    Anomaly Defenders Review
    Anomaly Defenders Review By Jennifer Allen on September 30th, 2014 Our Rating: :: STRONG DEFENSESUniversal App - Designed for iPhone and iPad Wrapping up the successful Tower Defense series, Anomaly Defenders is a blast.   | Read more »
    Double the Dragons, Double the Fun – The...
    Double the Dragons, Double the Fun – The Majesty Bundle is Now Available on the App Store Posted by Jessica Fisher on September 30th, 2014 [ permalink ] Majesty, by | Read more »
    Galaxy Trucker Review
    Galaxy Trucker Review By Rob Thomas on September 30th, 2014 Our Rating: :: IN FOR THE LONG HAULiPad Only App - Designed for the iPad We got a great big convoy, rockin’ through the night… the endless night of outer space, that is.... | Read more »
    Moon and Sun – children’s book Review
    Moon and Sun – children’s book Review By Amy Solomon on September 30th, 2014 Our Rating: Moon and Sun is a fable-like children’s book with lush illustrations.   Developer: Marcio Monteiro Price: $3.99 Version: 1.0 App Reviewed on... | Read more »

    Price Scanner via MacPrices.net

    Roundup of Apple Mac and iPad Education disco...
    Purchase a new Mac or iPad at The Apple Store for Education and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free,... Read more
    Apple Boycotts German Magazine Computer Bild...
    Apple has revoked its PR accreditation of Germany’s Computer Bild, Europe’s best-selling PC magazine, in reaction to Bild’s posting of a “#Bentgate” YouTube video. Axel Telzerow, editor in chief of... Read more
    iPhone 6 & iPhone 6 Plus Available in Chi...
    Apple has announced that iPhone 6 and iPhone 6 Plus will be available in China beginning Friday, October 17 from the Apple Online Store (http://www.apple.com), Apple’s retail stores, and an expansive... Read more
    MacBook Airs on sale for $100 off MSRP, start...
    Best Buy has the new 2014 MacBook Airs on sale for $100 off MSRP on their online store. Choose free home shipping or free local store pickup (if available). Prices valid for online orders only, in-... Read more
    Apple Releases OS X Mavericks bash Update 1.0...
    Apple has released a patch update for OS X Mavericks users to address the recently-detected “Shellshock” security bug in BSD UNIX’s bash shell. Apple says only a few Mac users who had manually... Read more
    Pivotal Payments Ready for Apple Pay – FlexPo...
    Pivotal Payments, a provider of merchant services and global payment processing solutions, has announced its proprietary FlexPoint platform will support credit and debit transactions through Apple’s... Read more
    iStabilizer Announces Tabarm — First Friction...
    iStabilizer, a specialist in universal lightweight compact tripods, steady cams, dollies, mounts, and remotes for smartphones, tablets, and cameras, announced today the iStabilizer tabArm, the first... Read more
    IStabilizer Flex Smartphone Tripod Wins Usa T...
    iStabilizer, a specialist in universal lightweight compact tripods, steady cams, and other products for smartphones, tablets, and cameras, has announced today that its iStabilizer Flex smartphone... Read more
    13-inch 2.8GHz Retina MacBook Pro on sale for...
    B&H Photo has the new 2014 13″ 2.8GHz Retina MacBook Pro on sale for $1699.99 including free shipping plus NY sales tax only. They’ll also include free copies of Parallels Desktop and LoJack for... Read more
    15-inch Retina MacBook Pros on sale for up to...
    B&H Photo has the new 2014 15″ Retina MacBook Pros on sale for up to $150 off MSRP. Shipping is free, and B&H charges NY sales tax only. They’ll also include free copies of Parallels Desktop... Read more

    Jobs Board

    *Apple* Retail - Multiple Positions (US) - A...
    Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
    *Apple* Retail - Multiple Positions (US) - A...
    Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
    Senior Event Manager, *Apple* Retail Market...
    …This senior level position is responsible for leading and imagining the Apple Retail Team's global event strategy. Delivering an overarching brand story; in-store, Read more
    *Apple* Solutions Consultant (ASC) - Apple (...
    **Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
    Project Manager / Business Analyst, WW *Appl...
    …a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
    All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.