TweetFollow Us on Twitter

Mac in the Shell: Debugging Python

Volume Number: 25
Issue Number: 11
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python

Stepping through code with pdb

by Edward Marczak

Welcome

We've been covering Python in this column for the last few months. We've gone from the basics, such as built in data types, variable assignment and so on, through more advanced concepts such as creating classes and integrating with Cocoa via PyObj-C. The intent was never to imagine that this column alone would turn you into a master Python programmer, but to give you the tools and direction to do so. One tool that you will need on that journey, though, is a decent debugger. While it's less common in scripting languages like Python and Ruby to use a debugger, when something is just not working out as expected and you can't figure out why, a peek at the code while it's running is invaluable. This month, I'll show you how to do that in Python using the python debugger ("pdb").

Do The Needful

Using a Shell

The instructions in this column always try to respect the way people are used to working. However, debuggers are interactive and grew up in a shell environment. While there may be interaction with certain editors, that will be outside of the scope for this article. Edit in whatever editor you like, but we're going to run and debug from a shell. (I think the general unease with the shell is lessening in the Mac community...right?). So, fire up Terminal.app (or iTerm, Terminator, etc.) and we'll get started.

Learning Your History

A debugger is itself a program that lets you examine another running program. You can use a debugger to step through the running code of the target program one line at a time, examine the values of variables at a given point in the code, run up until a certain breakpoint and examine a program crash or exception. One of the more well-known multi-language debuggers is the GNU Debugger, or "gdb." While you could use gdb to debug Python, there happens to exist a Python-specific clone of GDB called pdb, or, the Python Debugger.

In many scripting languages, programs are typically relatively short, and debuggers are often unnecessary. Many scripters are accustomed to sprinkling 'print' or logging statements through their code that reveals the value of particular variables at a particular point in the program's execution. However, you may have realized that Python is a bit more grown up than many traditional scripting languages. There are many fairly large systems written in Python. As an application gets larger and contains more dependencies, a dedicated debugger becomes not only useful, but necessary.

Getting Started

We'll start off with some simple code as an example of basic debugging. You write the code in listing 1 in the hopes finding prime numbers through and including 10.

Listing 1: prime_debug.py - sample code for debugging.

#!/usr/bin/python
for n in range(2, 10):
  for x in range(2, n):
    if n % x == 0:
      print n, 'equals', x, '*', n/x
      break
    else:
      print n, 'is a prime number'

Of course, you run this code and see something a little different than you expected—there are two problems in this code. A basic reason for debugging! (Kudos if you already see the errors).

The Python debugger is implemented as a module, so, like other modules, you need to import it. Add the following import after the shebang line:

import pdb

You'll also need to pick a point where you want to start tracing. Since this is an example, we'll start right at the top. So also add the set_trace method immediately following the import statement:

pdb.set_trace()

Now you can just run the program (mark it as executable first with a chmod 770 prime_debug.py or simlar). However, when you run the program this time, you're looking at something different. Something like this:

$ ./prime_debug.py > /Users/marczak/dev/py/prime_debug.py(7)<module>() -> for n in range(2, 10): (Pdb)

What you are looking at is the pdb interactive debugger waiting for your command. You'll see this when the pdb.set_trace() method is called. At this point, pdb stops all execution, displays the statement that it's waiting to execute next and displays its prompt. For our purposes, we want to execute this line (for n in range(2, 10)), so, we enter n, for "next." After pressing return, we're greeted with new information and a new prompt:

-> for x in range(2, n):
(Pdb)

Ah! We've moved on to the next line of the program, and are looking at the next statement to execute. To do so, you can simply press return, as pdb will repeat the last command you gave it by pressing return. Keep doing this a few times until you're comfortable with the display and what you're looking at.

Just so we can get back in sync, quit the debugger and we'll start again. To quit pdb at any time, issue a q command. You'll see a diagnostic "bdb.BdbQuit" line printed and find yourself back at a shell prompt.

Run your program again and let it drop into the debugger, and let's do something a little more useful this time. Tracing program flow is useful, but just as useful is being able to examine the value of variables. You're now essentially waiting for the first line of the program to execute: "for n in range(2, 10)." If you try to examine the variable n right now, you'll receive an error, since this line hasn't yet executed and n isn't yet defined.

First, execute this first line by entering n for "next," then enter p n, which stands for "print the contents of n." You can display the contents of any variable with the p ("print") command. In our example, the output should look like this:

-> for x in range(2, n):
(Pdb) p n
2

This is completely in line with our expectations: n is 2, right at the beginning of its range. (Note that the displayed line is the next line, not the one we're examining the variable of).

Finding our problem

Let's go off and find our problem, which is actually two-fold. The output currently looks like this:

3 is a prime number
4 equals 2 * 2
5 is a prime number
5 is a prime number
5 is a prime number
6 equals 2 * 3
7 is a prime number
7 is a prime number
7 is a prime number
7 is a prime number
7 is a prime number
8 equals 2 * 4
9 is a prime number
9 equals 3 * 3

This is all technically correct, but ugly. What's with the repeating lines? Also, we wanted to find values through 10, not 9. Since the first time we see the repeating lines is when n is equal to 5, let's find that point. Run the program, step through each line using the n command until you see the first output of "5 is a prime number." It will look like this:

> /Users/marczak/dev/py/prime_debug.py (9)<module>()
-> if n % x == 0:
(Pdb) 
> /Users/marczak/dev/py/prime_debug.py (14)<module>()
-> print n, 'is a prime number'
(Pdb) 
5 is a prime number
> /Users/marczak/dev/py/prime_debug.py (8)<module>()
-> for x in range(2, n):
(Pdb) 

Now, let's pay attention as we continue to step through. After a few iterations (or sooner), it should become clear: our if statement is not True, which is fine, and the else clause is running our print statement, which isn't fine. We really only want to print that notification on the way out of the loop when it fails to find a factor. So, our logic error is simple: we have the wrong level of indentation on the else statement. It should be un-indented one level, to be a part of the for loop. The entire loop should look like this:

for n in range(2, 10):
  for x in range(2, n):
    if n % x == 0:
      print n, 'equals', x, '*', n/x
      break
  else:
    print n, 'is a prime number'

Again, notice the subtle difference in indentation for the else portion – it's really a part of the for loop. If you're 'too close' to your code, that's an easy one to miss. However, debugging can be similar to explaining your code to a rubber duck—you know how it's supposed to work, but you only have the 'a ha!' as you step through it.

Make it Easier

We found our major error, but now have another: we want to print primes up through and including the number 10. If you're like me, you need a refresher at this point as to where you are in the code. Issuing an l (not "one," but "ell," for "list") will do just that:

(Pdb) l
  3     import pdb
  4     
  5     pdb.set_trace()
  6     
  7     for n in range(2, 10):
  8  ->     for x in range(2, n):
  9         if n % x == 0:
 10           print n, 'equals', x, '*', n/x
 11           break
 12         else:
 13           # loop fell through without finding a factor

Ah! Now I know where I am. All we're really interested in from this point on is the value of n. Stepping through the remainder of the code shows that the initial for loop exits after 9. Didn't we ask it to run until 10?

Yes we did, but that's our misconception. Looking at the Python documentation for range() shows that the range intentionally excludes the final number.

While this may not be a common mistake that you make, it turns out that this is still a useful exercise: you may not always be debugging your own code.

Dealing with Functions

There's a few more pdb commands to understand before you tackle larger python programs. Specifically, you'll want to know how to deal with functions. Take, for example, the code in listing 2.

Listing 2: dict_iterate.py

#!/usr/bin/python
import pdb
pdb.set_trace()
def _PrintDict(dict):
  """Recursively iterate over a dictionary, printing results
  Args:
    dict: The dictionary to print
  """
  pdb.set_trace()
  for item in dict:
    if type(dict[item]) == dict:
      _PrintDict(dict[item])
    else:
      print "%s: %s" % (dict[item], type(dict[item]))
def main():
  """Main routine"""
  aDict = {'color': 'blue',
           'count': 15,
           'cust_info': {'pid': '94758476', 'uid': '348576'},
           'style': 'fruit'}
  _PrintDict(aDict)
if __name__ == "__main__":
  main()

This should look vaguely familiar to anyone who read the previous column on Python. Start this program running and step through it with n—you'll see python touch each function name to create an object for it. If you keep tracing with n ("next"), this program will end very quickly. This is because when the n command reaches a function, it executes the entire function without entering that function. So, stop tracing with n when you arrive at the call to main():

-> if __name__ == "__main__":
(Pdb) n
> /Users/marczak/dev/py/dict_iterate.py(30)<module>()
-> main()

We want to step into main(), so go ahead and enter s (for "step"). You should be greeted with:

def main():

showing that you're now looking at the definition for main(). Keep stepping as we want to also step into the call to _PrintDict().

When you do arrive in the _PrintDict() function, there's a for loop. Once you've traversed that loop, you may no longer be deeply interested in it, but want to get back to where you were before entering this function. pdb has a solution for you: r, for "return." Essentially, "finish up this function and return."

Be aware! Stepping into functions sometimes will have an unintended consequence for you: stepping into an library that you've included. This is often not the code that you're interested in debugging, though it may be. If you accidentally step into a library function—PyObj-C code included—just remember the r command and return until you're back to where you expect.

More pdb Features

You now know the core of pdb and can actually do some serious debugging. However, pdb offers a lot more. Some of which we'll save for another column, but there are two more useful things to pass on.

The easy way out: c, for "continue." If at any point, you've traced through all you've wanted to trace, but don't want to crash the program with a quit (q) command, there is another option. The continue command picks up and runs the remainder of the program.

Even better, though, is this: pdb is letting you load and run your Python program in an interactive environment. You can alter variables just by assigning them:

(Pdb) x =7
(Pdb) p x
7

Imagine the simple code in listing 3.

Listing 3: math.py

#!/usr/bin/python
import pdb
pdb.set_trace()
x = 5
for i in range(1, 10):
  print i + x

At any time after x gets assigned, you can reassign it. Your debugging session can look like this:

$ ./math.py 
> /Users/marczak/dev/py /math.py(7)<module>()
-> x = 5
(Pdb) n
> /Users/marczak/dev/py /math.py(9)<module>()
-> for i in range(1, 10):
(Pdb) 
> /Users/marczak/dev/py /math.py(10)<module>()
-> print i + x
(Pdb) 
6
> /Users/marczak/dev/py /math.py(9)<module>()
-> for i in range(1, 10):
(Pdb) x = 20
(Pdb) n
> /Users/marczak/dev/py /math.py(10)<module>()
-> print i + x
(Pdb) 
22

This is fantastic news if you want to test your code for fragility around scenarios where variable reach certain values. Of course, as the code runs, if a variable gets reassigned in your program, you need to watch for that.

Reference Sheet

Here's a handy list of the pdb topics discussed in this article:

pdb library: import pdb
Start tracing: pdb.set_trace()
n: execute next command
s: step into a function
r: return from function
c: continue running program
q: quit pdb (and error out of program).

Conclusion

While you may focus on shorted programs now, as your skills improve, your programs should grow in complexity and size. At some point, you'll likely confound yourself, and a debugger comes in very handy during these times. Fortunately, Python has an available debugger that is friendly to use and easy to make available to your program.

Media of the month: I've been in a "back to school" kind of way, but really just studying on my own. There's too much to learn, right? Well, I've taken the school approach: one topic a day and rotate through them and study each night. Now, I'm not recommending anything this drastic, but, I'd bet there's one subject that you've wanted to learn. Now is the time. Hit up your local bookstore, University or Amazon.com, find a book and go. If you're still an actual student, well, keep going!

Until next month, keep scripting.


Ed Marczak is the Executive Editor for MacTech Magazine, and has written the Mac in the Shell column since 2004.

 
AAPL
$524.94
Apple Inc.
+5.93
MSFT
$40.01
Microsoft Corpora
-0.39
GOOG
$536.10
Google Inc.
-20.44

MacTech Search:
Community Search:

Software Updates via MacUpdate

VMware Fusion 6.0.3 - Run Windows apps a...
VMware Fusion allows you to create a Virtual Machine on your Mac and run Windows (including Windows 8.1) and Windows software on your Mac. Run your favorite Windows applications alongside Mac... Read more
Tweetbot 1.5.1 - Popular iOS twitter cli...
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Mac DVDRipper Pro 4.1.7 - Copy, backup,...
Mac DVDRipper Pro is the DVD backup solution that lets you protect your DVDs from scratches, save your batteries by reading your movies from your hard disk, manage your collection with just a few... Read more
PDFpenPro 6.2 - Advanced PDF toolkit for...
PDFpenPro allows users to edit PDF's easily. Add text, images and signatures. Fill out PDF forms. Merge or split PDF documents. Reorder and delete pages. Even correct text and edit graphics! Create... Read more
PDFpen 6.2 - Edit and annotate PDFs with...
PDFpen allows users to easily edit PDF's. Add text, images and signatures. Fill out PDF forms. Merge or split PDF documents. Reorder and delete pages. Even correct text and edit graphics! Features... Read more
Monolingual 1.5.9 - Remove unwanted OS X...
Monolingual is a program for removing unnecesary language resources from OS X, in order to reclaim several hundred megabytes of disk space. It requires a 64-bit capable Intel-based Mac and at least... Read more
Maya 2015 - Professional 3D modeling and...
Maya is an award-winning software and powerful, integrated 3D modeling, animation, visual effects, and rendering solution. Because Maya is based on an open architecture, all your work can be scripted... Read more
Starcraft II: Wings of Liberty 1.1.1.180...
Download the patch by launching the Starcraft II game and downloading it through the Battle.net connection within the app. Starcraft II: Wings of Liberty is a strategy game played in real-time. You... Read more
Sibelius 7.5.0 - Music notation solution...
Sibelius is the world's best-selling music notation software for Mac. It is as intuitive to use as a pen, yet so powerful that it does most things in less than the blink of an eye. The demo includes... Read more
Typinator 5.9 - Speedy and reliable text...
Typinator turbo-charges your typing productivity. Type a little. Typinator does the rest. We've all faced projects that require repetitive typing tasks. With Typinator, you can store commonly used... Read more

Latest Forum Discussions

See All

Have a Special Dead Trigger 2 Easter Bas...
Have a Special Dead Trigger 2 Easter Basket Full of Goodies, Courtesy of Madfinger Games Posted by Rob Rich on April 18th, 2014 [ permalink ] Dead Trigger 2 | Read more »
Almost All of Playdek’s Library is on Sa...
Almost All of Playdek’s Library is on Sale Right Now, and You Should Check it Out Posted by Rob Rich on April 18th, 2014 [ permalink ] Playdek has released quite a few great iOS ports of board and card games over the years, and now most of them... | Read more »
Zynga Launches Brand New Farmville Exper...
Zynga Launches Brand New Farmville Experience with Farmville 2: Country Escape Posted by Tre Lawrence on April 18th, 2014 [ permalink ] | Read more »
David. Review
David. Review By Cata Modorcea on April 18th, 2014 Our Rating: :: MINIMALISTIC IN A DIFFERENT WAYUniversal App - Designed for iPhone and iPad David is a minimalistic game wrapped inside of a soothing atmosphere in which the hero... | Read more »
Eyefi Unveils New Eyefi Cloud Service Th...
Eyefi Unveils New Eyefi Cloud Service That Allows Users to Share Media Across Personal Devices Posted by Tre Lawrence on April 18th, 2014 [ permalink ] | Read more »
Tales from the Dragon Mountain: The Lair...
Tales from the Dragon Mountain: The Lair Review By Jennifer Allen on April 18th, 2014 Our Rating: :: STEADY ADVENTURINGiPad Only App - Designed for the iPad Treading a safe path, Tales from the Dragon Mountain: The Lair is a... | Read more »
Yahoo Updates Flickr App with Advanced E...
Yahoo Updates Flickr App with Advanced Editing Features and More Posted by Tre Lawrence on April 18th, 2014 [ permalink ] | Read more »
My Incredible Body - A Kid's App to...
My Incredible Body - A Kid's App to Learn about the Human Body 1.1.00 Device: iOS Universal Category: Education Price: $2.99, Version: 1.1.00 (iTunes) Description: Wouldn’t it be cool to look inside yourself and see what was going on... | Read more »
Trials Frontier Review
Trials Frontier Review By Carter Dotson on April 18th, 2014 Our Rating: :: A ROUGH LANDINGUniversal App - Designed for iPhone and iPad Trials Frontier finally brings the famed stunt racing franchise to mobile, but how much does its... | Read more »
Evernote Business Notebook by Moleskin I...
Evernote Business Notebook by Moleskin Introduced – Support Available in Evernote for iOS Posted by Tre Lawrence on April 18th, 2014 [ permalink ] | Read more »

Price Scanner via MacPrices.net

Free HopTo 2.2 Helps Enhance Your Productivit...
The HopTo app helps you do more on your iPad by providing more and easier adaccess to files and documents. Version 2.2 adds Egnyte and HopTo’s Mac OSX File Connector. If you already have the hopTo... Read more
National Distracted Driving Awareness Month:...
As the country recognizes National Distracted Driving Awareness Month, Sprint is reminding wireless consumers to focus on driving while behind the wheel, to not text or email while driving, and to... Read more
13-inch 2.4GHz Retina MacBook Pro available f...
Abt has the 13″ 2.4GHz 128GB Retina MacBook Pro available for $1229 including free shipping. Their price is $70 off MSRP. Read more
iMacs on sale for up to $160 off MSRP this we...
Best Buy has iMacs on sale for up to $160 off MSRP for a limited time. Choose free home shipping or free instant local store pickup (if available). Prices are valid for online orders only, in-store... Read more
iPad Airs on sale this weekend for up to $100...
Best Buy has WiFi iPad Airs on sale for $50 off MSRP and WiFi + Cellular iPad Airs on sale for $100 off MSRP on their online store for a limited time, with prices now starting at $449. Choose free... Read more
Apple restocks refurbished Mac minis starting...
The Apple Store has restocked Apple Certified Refurbished Mac minis for up to $150 off the cost of new models. Apple’s one-year warranty is included with each mini, and shipping is free: - 2.5GHz Mac... Read more
Hyundai Brings Apple CarPlay To The 2015 Sona...
Hyundai Motor America has announced it will bring Apple CarPlay functionality to the 2015 Sonata. CarPlay is pitched as a smarter, safer and easier way to use iPhone in the car and gives iPhone users... Read more
Updated iPads Coming Sooner Than We Had Thoug...
MacRumors, cites KGI securities analyst Ming Chi Kuo, well-respected as an Apple product prognisticator, saying that Apple will introduce an upgraded iPad Air and iPad mini in 2014/Q3, meaning the... Read more
Toshiba Unveils New High And Low End Laptop M...
Toshiba has announced new laptop models covering both the high-end and low-end of the notebook computer spectrum. Toshiba 4K Ultra HD Laptop Toshiba’s new Satellite P55t features one of the world’s... Read more
Save up to $270 with Apple refurbished 13-inc...
The Apple Store has Apple Certified Refurbished October 2013 13″ Retina MacBook Pros available starting at $1099, with models up to $270 off MSRP. Apple’s one-year warranty is standard, and shipping... Read more

Jobs Board

*Apple* Automotive Parts Department position...
Apple Automotive is one of the fastest growing dealer…and it shows. Consider making the switch to the Apple Automotive Group today! At Apple Automotive, we Read more
*Apple* Solutions Consultant (ASC) - Apple (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
*Apple* Retail - Manager - Holyoke - Apple I...
Job Summary Keeping an Apple Store thriving requires a diverse set of leadership skills, and as a Manager, you’re a master of them all. In the store’s fast-paced, Read more
*Apple* Retail - Manager - Apple (United Sta...
Job SummaryKeeping an Apple Store thriving requires a diverse set of leadership skills, and as a Manager, you're a master of them all. In the store's fast-paced, dynamic Read more
*Apple* Solutions Consultant (ASC) - Apple (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.