Volume Number: 18 (2002)
Issue Number: 9
Column Tag: Mac OS X

Introduction to Perl for Mac OS X

There's more than one way to do it.

by Joe Zobkiw

Perl is short for "Practical Extraction and Report Language." However, Perl really doesn't sound too interesting when you put it in those terms, so, now that you know that much, forget everything that you've learned up to this point.

Perl is an advanced, cross-platform programming language created by Larry Wall. It's strength lies in the fact that it can perform extremely complex tasks easily while not being too big or bulky for the simpler tasks. It is an expert at manipulating text, strings, numbers, streams of data, files and directories. It can just as easily massage massive amounts of data on your local computer as it can connect to a remote server, across an ocean, and feed it in streams. As long as you don't need a fancy GUI, Perl may just have an answer for you. In a word, Perl is elegant.

Perl (verion 5.6 as of this writing) comes pre-installed with Mac OS X so there really isn't anything to install or configure. You can create a Perl script with any text editor. I use BBEdit from Bare Bones Software since it automatically colors the syntax of your Perl script and is simply the best programmer's text editor available for Mac OS X. Because Perl is based in the command line, you primarily run Perl scripts using the Terminal application that comes with Mac OS X. Don't let this scare you though, as you'll soon learn, the Terminal isn't so scary.

As you can see in Figure 1, we ran Perl from within the Terminal passing the -v command. This tells Perl to spit back its version information. In this example we are using Perl 5.6.0 specifically built for Mac OS X. We then called Perl passing the -e command which tells Perl that we are including actual Perl code between the single quotes and expect it to be interpreted, executed and the results displayed in the Terminal. Perl performed the task flawlessly, as it told us Hello!

Most of the time you will write your Perl scripts and store them in a text file. These files end in .pl and should be saved with UNIX linefeeds, otherwise Perl will get confused and return confusing errors when it attempts to run your scripts. Remember, with Mac OS X you are really running the UNIX operating system underneath a Macintosh user experience! Assume we have the following text stored in a file named myfirstscript.pl:

#!/usr/bin/perl
print "Hello from Perl on Mac OS X!\n";

Note that that the very first line of the script points to Perl. This is how all Perl scripts must start - with the path to Perl. The line may be slightly different depending on your operating system but this is what you will usually see with Mac OS X. The # symbol is actually a comment designator. Normally whenever you see this symbol, anything to the right of it is a comment. The next line gives Perl the command to print to the Terminal. Looks just like what we typed on the command line earlier! To run this script using Terminal you first open Terminal, use the cd command to point to the directory which contains the script and then type perl myfirstscript.pl. Upon pressing return your script will execute and you'll see the "hello..." message printed on the screen! Now all that's left to do is write a script that actually does something.

One thing to remember is that Perl is a complex programming language that entire books were written about. It is impossible to go into all of the intricacies of the language in this short article. As you continue to read you will learn some of the basics and some great places to turn to learn the language itself. So don't turn away just yet...half a million programmers can't be wrong!

Now that you know the basics of how Perl fits into Mac OS X and how you create and execute scripts, what is it actually good for?

Let's say you have a bunch of ASCII text files that you need to scan for certain characters, change them to something else, then make a copy of the altered file and the original. If this was a one-time occurrence you might just spend 5 hours and do it by hand. However, if this is part of a process that you have to perform every day or even every month, why not the let the computer - with the help of Perl - do it for you? You could literally write such a program in Perl in less than 20 lines of code. Adding a few more lines of code would be all you would need to make it email or page you when it was finished processing. I know people in the corporate world who have their entire jobs automated - they can start a process and go biking for the rest of the day - if there is a problem (or upon success) their Perl scripts page them!

As you may already know most everything you send or receive via the Internet is text-based. The HTTP protocol that web servers use is a completely text-based protocol. Your web browser sends a text request to the server, which sends a text response in return. Your e-mail program works the same way. News readers work the same way. As do many of the simpler, more behind-the-scenes protocols such as Ping, Telnet and Finger.

Given this, you can relatively easily write an anti-spam Perl script that logs into your email server and deletes any email that contains the words "GET RICH" in the subject - before you ever get a chance to see them. How about a script that pulls down the latest weather information from a web site and emails you when an advisory is posted for your area. Consider a script that watches a newsgroup for job postings of interest and automatically emails them to you. The possibilities are literally endless when you look at all the data available on the net!

I recently wrote a Perl script that reads data from a GPS that is connected via a Keyspan USB PDA adapter locally to my computer. The script (available online at http://homepage.mac.com/zobkiw/) opens the Keyspan driver and begins reading and parsing position data being sent by the GPS. Once parsed, I can easily display detailed, self-updating maps (read from the Internet) showing my position. Given this technique, you can hook up any RS232-type device to your Mac and communicate with it via Perl. This would include personal weather stations, amateur radios, musical instruments, custom hardware and many other little gadgets.

Now that you have an idea of some of the things you can do with Perl, let's specifically take a look at a more advanced example. The example that we will walk through pulls the current page displayed at cnn.com (although you can easily change it to support any web site) and reports if certain words are "in the news." Take a look at the code and then we will explain it in detail.

#!/usr/bin/perl
use strict;
my @a = ("Clinton", "Gore", "Bush", "Cheney");
my $url = "http://www.cnn.com";
my $sysstring = "curl -s $url";
my $count = 1;
print "Opening...\n";
open(FOO, "$sysstring|");
print "Searching...\n";
while (<FOO>){
   my $lineout = $_;
   foreach my $search (@a) {
      if ($lineout =~ /$search/){
         print "$count. $search found.\n";
         $count++;
      }
   }
}
close FOO;
print "Complete.\n";

This is a pretty good example of Perl performing a complex task - easily. If you think about what is going on here in detail: your computer has to use DNS to resolve the cnn.com web site name to the proper IP address; connect to it; create the proper http request to obtain the contents of the web page; then search the web page for particular text and display the results. If you were to write this code by hand, following the multiple protocols (DNS and http), it might take you days - if not longer. Let's look at the code!

We've already discussed the very first line, which points to Perl itself so we will begin with use strict. The Perl keyword use is used like the keyword include in C. Whenever you need to let Perl know you will be making use of a module, or in this case, the services of Perl itself, you use the word use. Specifically, use strict tells Perl that you want it to be a bit stricter than it would otherwise be as it interprets your code. In Perl, there are more ways to perform a task than in C - hence the Perl motto "There's more than on way to do it.". It is very easy to make a mistake. Enabling strict helps to catch common problems before they cause you to lose your hair.

The next four lines declare some variables. The Perl keyword my is used to explicitly declare variables. In reality you don't have to declare variables in Perl, it will automatically create any variable you attempt to use. However, remember use strict. One of the features of strict is that Perl requires us to employ my to declare the variables we use before we use them. This can help with the hair loss mentioned earlier.

The first variable, a, is an array of strings. We use the @ symbol to signify an array of items. The items in the array follow in parenthesis and quotes. $url is a variable named url. Most variable names in Perl begin with a $. I say most because although a is a variable, it is also an array, so it starts with an @ symbol. It may seem confusing now but as you work with Perl you will begin to appreciate the power that Perl offers as it confuses you.

$sysstring is a variable that contains a command line command. The curl program is one that you can execute from the command line, that is, the Terminal. Curl is a client program that retrieves data from (and sends data to) various servers. It supports numerous protocols including http, https, FTP, GOPHER, DICT, TELNET, LDAP, and FILE. Type man curl in the Terminal for complete details. The important thing to come away with here is that you can execute command line commands and then process the results all from within your Perl script! Note the substitution of the $url variable in the string assignment of the $sysstring variable. The $sysstring variable ends up as 'curl -s http://www.cnn.com/' after this substitution.

Next we declare a $count variable to number the items we find and then use the print command to write some text to the Terminal so anyone running our script can follow along as it executes.

The open command opens the $sysstring variable for reading (hence the | symbol following $sysstring). Open can be used to open files too but in this case it is smart enough to execute the curl command line embedded in the $sysstring variable. Once executed, we can read the data returned by curl by referencing the FILEHANDLE named FOO. A FILEHANDLE, in this case, is an I/O (input/output) connection between the Perl script and the output of curl. Once we have the output available in FOO, we can use a standard while loop to examine each line. The $lineout variable is assigned the $_ variable, which is a special variable that while returns as it extracts each line - $_ contains the last line extracted. If you were to add print "$lineout"; at this point you would see each line in the Terminal.

Next we search the line for each item in our array of search strings. The foreach statement does just what it says. For each item in the array @a, we are going to place it into a variable named $search. At this point we use the =~ matching operator to search $lineout for the $search string. If the text is found, the print command is executed and we display what we found. We happen to be doing a case sensitive search here but you can change that if you like. Don't be afraid of the =~ stuff, this is just a fancy way to say "find this". If you are interested in researching this, it's all part of a topic larger than what can be covered in this article: regular expressions.

Once we've looped through each line $lineout and searched for each $search string within them, we close the FILEHANDLE FOO and the program is complete. Not too bad, huh? Here are some exercises for the reader: make the search case-insensitive; make the program return an individual total count of each search term found (ie: 2 Clinton, 5 Bush, 3 Cheney, 1 Gore); search an array of news sites for an array of search terms.

Thus far we've discussed using Perl in scripts that run locally on your computer to perform some task. However, one very popular use of Perl is as an Apache CGI. Apache is one of the most popular web servers in use today, and it comes pre-installed with Mac OS X. A CGI is a program (written in Perl, C, C++, PHP, ASP, etc.) that runs on a web server. You create a web-based form that allows a user to interact with the CGI. Examples of CGIs include search engines, guest books, shopping carts, etc. Most any time you fill out a form on a web page and press the "Submit" button, you are calling a CGI.

Behind the scenes, the web server receives the information from the web page and passes all of the fields to the CGI for processing. The CGI might verify the data and then send an email, write the information to a database, or send an order to the shipping department so you can receive your new toy via FedEx overnight delivery. Most of the time the CGI will then return a web page saying "thank you: order processed!"

Apache alone isn't too smart; it knows how to serve a file to a client but not much more. By adding a CGI you can extend Apache in any way you desire to perform tasks that the developers of Apache could never have imagined you would need to perform. Perl is the perfect language to write your CGI as thousands of examples are available throughout the net. For more information on Apache, you can visit http://www.apache.org/. For more information on specifically using Perl with Apache, you can visit http://perl.apache.org/.

We discussed how Perl can be used to extend Apache, but what about extending Perl itself? The standard installation of Perl comes with hundreds of "built-in" functions that will meet many of your needs, however it also supports something called modules for those times when you need a little more. A module is like a library in C. Modules may be written in Perl or in C or C++ but that is transparent to the user of the module. A module usually concentrates on the support of a particular task. There are modules that support Internet protocols, encryption schemes, scientific and mathematical algorithms, image manipulation, audio, and much more.

You can create your own modules or you can download thousands (yes, thousands!) of them that currently exist from CPAN, the Comprehensive Perl Archive Network (at http://cpan.org/). CPAN contains not only modules but many sample Perl scripts and the Perl distributions themselves. It covers many too many things to discuss in this article, so you should visit the web site to explore for yourself.

Hopefully this article has given you a spark to go pursue Perl on your own. There are some excellent web sites to help you learn the intricacies of Perl. Make sure to visit http://www.perl.com/, which is a great place to read articles and learn more about Perl. All of the documentation is available online for you to read. Also be sure to visit http://www.perl.org/ and http://learn.perl.org/. There are also plenty of books available as well. Now that you have a short introduction to Perl, look for more articles in these pages to introduce you to complete real-world examples of what you can do with Perl under Mac OS X!

Joe Zobkiw is a software developer, musician and author living in Raleigh, NC. He has been a Macintosh user since 1986 and has owned no less than a baker's dozen Macintosh computers. He is currently keeping busy on a PowerBook G4 running OS X and rediscovering the command line. You can email Joe at zobkiw@triplesoft.com between 9am and 5pm ET M-F.