Volume Number: 24
Issue Number: 12
Column Tag: Mac in the Shell

Mac in the Shell: Learning Python on the Mac-Part 2

All about strings

by Edward Marczak

Last month, we began a journey to learn the Python programming language on the Mac. Although I'm assuming little to no programming experience, it should certainly enable experienced programmers to get up to speed quickly in Python as well. Last month started with the absolute basics: variables, objects, the interactive interpreter and the inevitable "Hello, World!" program. There was also a small homework assignment to keep your brain engaged with Python. Let's pick up where we left off.

Last month, I ended the column asking you to "write a program that creates two integer variables,"start"and "end" and one string, "text". Have the program print a slice of the string using the variables and a print statement that precedes the string with "The slice is: ". Here's a script that will do just that:

#!/usr/bin/env python
start = 8
end = 12
text = "This is some text"
print "The slice is:",text[start:end]

Simple, no? Well, this is certainly one way to handle it. "One way?" you ask. "How many ways can there be to write this basic code?" you may think. Well, that's why this entire column will focus on strings in Python, string manipulation and other string subtleties.

Before we continue, I'd like to make a distinction in my writing of this topic. When I refer to Python as "Python"-Capital "P"-I'm referring to Python in general: conceptually. When I use "python", I'm specifically referring to the Python runtime engine and language specification. If you notice this shift in the case throughout the article, that's the reason behind it.

OK. Onward.

Like many other scripting languages, you may find that there are several ways to accomplish the same goal in Python. Sometimes the route you choose is purely stylistic. Sometimes the choice is "Pythonic"-instead of brute forcing a solution, Python may include some elegant, built-in way of dealing with your issue. Finally, there are just times where certain styles lend themselves to a particular situation better than other styles, so, you'll find yourself switching styles as needed.

Understanding strings in Python is important, as they are part of the collections class, and therefore respond to anything that a collection class will, as we saw with slices.

Strings in Python do take a little getting used to if you've used other scripting languages in the past. Let's get the basics out of the way.

Strings are formed using single, double or triple quotes. Single quotes and double quotes behave the same:

'This is a string'

and

"This is a string"

are treated the same. Both single and double quotes require escape sequences to represent special characters. Like any regular expression, Perl or PHP, special characters that you want represented literally, require a backslash escape. Examples include a quote within the outer quotes:

'Bill, it\'s time to go!'

and general special characters, such as newline:

print "Don't follow me too closely\n\n\n"

print "Is that far enough?"

Triple quotes-either ''' or """-have some special properties. Firstly, they can be multiline. Secondly, quotes are passed literally, but special characters are still recognized. The following will illustrate these points:

print """What on Earth is going on?
How am I spanning multiple lines? You think
this is funny?\n\nIt's "everyone's" opinion that
it isn't.
"""

This outputs:

What on Earth is going on?
How am I spanning multiple lines? You think
this is funny?
It's "everyone's" opinion that
it isn't.

Notice that the newline characters were honored, but there was no need to escape the inner quotes.

Now, triple quotes are useful, but more often than not, you're going to need to intersperse the contents of variables or manipulate strings for output. There are a few ways to handle these scenarios. Let's start with the most simple: automatic string concatenation. To illustrate, we need to look at the print function. The Python print function automatically outputs a newline after printing its entire string of data. The commands:

print "Don't follow me too closely."
print "Is that far enough?"

will display:

Don't follow me too closely.
Is that far enough?

No explicit newline character needed. That's pretty straightforward. One way to get rid of the newline character is to use Python's automatic string concatenation feature. Python doesn't require any specific character to concatenate adjacent strings:

print "String 1" "String 2"

prints "String 1String 2". Easy enough, right? Using parentheses, you can extend this to work with multi-line code:

print ("Don't follow me too closely."
       "Is that far enough?")

A comma, which also concatenates strings, will suppress a newline character. It's also a way to add in variables. You may have noticed the homework assignment string used a comma:

print "The slice is:",text[start:end]

What you may not have noticed is that the comma also inserts a space. (There will be a space between the colon and the text in this example). Another way to concatenate strings is the addition symbol, which is overridden to work with text. This method does not insert a space:

print "The slice is: " + text[start:end]

Notice that we added a space ourselves before the final quote mark.

More useful in many ways is the format string. C programmers will recognize this immediately: Python uses the same string formatters as the printf functions in C. The easy introduction is this: Python's print function will substitute the contents of a variable into a string where it finds the string format specifier %s. Time for an example:

username = "bill"
print "Hello, %s" % username

This trivial example will print, "Hello, bill". In a larger program, we'd likely be fetching the username from some other location, such as a database. This style keeps code much more readable, especially when more variables are substituted in a single string. For example:

print "%s, you have %s credit remaining, " \
      "out of %s total." % (user['first'],
      user['credit'],
      user['total'])

There are a few techniques pointed out here. First, you can use the backslash character to continue long lines. Use it where it makes your code more readable. The parentheses form a tuple-something we'll cover in detail next month. The values in the tuple are substituted into the string in order.

For the sake of completeness, there's one last form that is very useful, but won't make sense until we cover dictionaries. A dictionary is a Python data structure that offers a mapping of keys to values. Keys in a dictionary are unique. Due to this, a print format string will also accept a dictionary to map to:

print "%(first)s, you have %(credit)s credit remaining, " \
      "out of %(total)s total." % user

In this example, user is the dictionary-as in the previous example-and each format specifier contains the key in that dictionary to substitute the value of. This will be covered in future columns.

In order to bring this full circle, this shows another way we could have written the print statement in the homework assigment:

print "The slice is: %s" % text[start:end]

One thing I've glossed over a bit here: we're coercing all values into strings. To illustrate, look at this example:

foo=52
print "The value is %s." % foo

Here, foo is an integer value, but we're placing it into a string. The Python interpreter will happily do the right thing here. But there are situations where you'll need to be a little more precise. In fact, the right way to handle this is to specify that the variable being substituted is an integer. Instead of playing fast and loose and treating all values as a string, "%s", you can specify integers with "%d":

print "The value is %d." % foo

Why does this eally matter, you may ask? Last month, we covered Python's various data types. String, Unicode, integer and float are all used for different purposes. When substituting a float value, you may want to specify the number of decimal places. Try this example:

myfloat = 5.9872345234
print "The number is %s." % myfloat
print "Reduced to 2 decimal places,",\
      "and rounded, it\'s %0.2f" % myfloat

The Python documentation contains a full list of format specifiers. Find it at: http://www.python.org/doc/current/lib/

If you install the documentation as instructed in the next section, this can be found locally at file:///Library/ Frameworks/Python.framework/Versions/2.5/Resources/English.lproj/Documentation/lib/typesseq-strings.html.

One thing that I feel is important to teach is showing ways in which you can help yourself. Python was designed with this in mind, and supplies a built-in help system. In the interactive python shell, just type help(), or, look for help on a particular word:

>>> help('print')

Sorry, topic and keyword documentation is not available 
because the Python HTML documentation files could not be found. 
If you have installed them, please set the environment 
variable PYTHONDOCS to indicate their location.

Ah, yes...you'll run into this under OS X. The documentation is, for some reason, not installed with the out-of-the-box Leopard installation. This is easily remedied, however. Download the PythonMac 2.5 distribution (http://pythonmac.org/packages/py25-fat/dmg/python-2.5-macosx.dmg) and mount the image, but do not run the installer. The installer on the image is a metapackage. Control-click on the included mpkg installer, and choose "Show Package Contents" from the resulting Finder menu. Navigate to Contents/Packages and double-click on PythonDocumentation-2.5.pkg to install the documentation. The docs are dropped onto the boot volume, but buried at /Library/Frameworks/Python.framework/Versions/2.5/Resources/English.lproj/Documentation. Even though they're present on disk, python will still not know where to locate them. Let's fix that. Of course, there needs to be a slight explanation first.

Python uses an environment variable named PYTHONDOCS to locate its documentation. As you may expect, PYTHONDOCS specifies a path in the same format as the shell PATH variable: an absolute path on the filesystem. To enable python to locate the documentation, we can create the PYTHONDOCS variable and add the OS X-specific location. In your shell, type and enter:

export PYTHONDOCS=/Library/Frameworks/Python.framework/Versions/2.5/Resources/English.lproj/Documentation

You need to export the variable so the subshell running the python interpreter inherits the value. Ideally, you should add this to a startup file so it's available each time you start a shell. Dropping it in your ~/.bash_profile file is recommended. There are also two very OS X-ish ways of dealing with this for those of you in a GUI environment. One is to simply open the GUI application from the shell that has the exported variable:

open /Users/Shared/Applications/TextWrangler.app

Just supply open with the full path to the application in question. This will allow the application access to any exported environment variables in that shell session. Since I'm in a shell pretty much full-time, this is my preferred method. Create an alias for the command you use if you do this often.

The second way is to add the environment variable to the user-global environment.plist, or, directly to a particular application's plist. Apple has excellent documentation available on just this topic, so there's no need for me to repeat it. The environment variable developer docs can be found at http://developer.apple.com/documentation/MacOSX/Conceptual/BPRuntimeConfig/Articles/EnvironmentVars.html,. (If Apple ever moves this doc, hit up Google for "User Session Environment Variables").

If you're using TextMate, you can define shell variables in Preferences > Advanced > Shell Variables. Just add a variable named "PYTHONDOCS" with the path listed above, and you can avoid the entire edit-the-plist dance.

Once your environment variable is set, enter python again and ask for help:

>>> help()

Welcome to Python 2.5!  This is the online help utility.
(...output shortened for space considerations...)

Once in the help system, the python prompt will change to help>. You can look for general help on modules, topics or keywords just by typing "modules", "topics" or "keywords" (clever, eh?):

help> modules
Please wait a moment while I gather a list of all available modules...

If you know the module name or keyword, you can just bring up the information directly:

help> print
  ------------
  
  6.6 The print statement
...

Finally, there's no need to enter the help system at all. You can call help with your query as an argument. For example:

>>> help('print')

To exit the help system, press ctrl-d, just like you're leaving the interactive python shell.

For pedantic among you, there's a small distinction to make here. The python interpreter initializes its own environment from the PYTHONDOCS environment variable. Once python is running, changing this variable will have no effect. Conversely, you can look up the current PYTHONDOCS path. From the interactive shell, type the following:

>>> import pydoc
>>> pydoc.help.docdir

python should spit back to you the current path inherited from PYTHONDOCS.

Between last month and this month, you now know everything practical about strings in Python. While there's a good road left to travel in Python itself, the good news is that so much of the work you'll typically do involves strings and collections that this is a great topic to understand. In next column, we'll tackle more. For now, practice with what you have learned so far.

Media of the month: Neuromancer by William Gibson. Every techie needs some William Gibson in their library. If you've already read Neuromancer, but haven't explored more, use this as an opportunity to pick up something more recent (Count Zero and Pattern Recognition come to mind).

Next month is Macworld. Please stop by the MacTech booth and say hello! Until then, keep scripting!

Ed Marczak knew even from the punch card and TTY days that technology was in his future. He finds all technology interesting, but chooses OS X when possible. When not computing, he spends time with his wife and two daughters.