TweetFollow Us on Twitter

Mac in the Shell: Debugging Python

Volume Number: 25
Issue Number: 11
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python

Stepping through code with pdb

by Edward Marczak

Welcome

We've been covering Python in this column for the last few months. We've gone from the basics, such as built in data types, variable assignment and so on, through more advanced concepts such as creating classes and integrating with Cocoa via PyObj-C. The intent was never to imagine that this column alone would turn you into a master Python programmer, but to give you the tools and direction to do so. One tool that you will need on that journey, though, is a decent debugger. While it's less common in scripting languages like Python and Ruby to use a debugger, when something is just not working out as expected and you can't figure out why, a peek at the code while it's running is invaluable. This month, I'll show you how to do that in Python using the python debugger ("pdb").

Do The Needful

Using a Shell

The instructions in this column always try to respect the way people are used to working. However, debuggers are interactive and grew up in a shell environment. While there may be interaction with certain editors, that will be outside of the scope for this article. Edit in whatever editor you like, but we're going to run and debug from a shell. (I think the general unease with the shell is lessening in the Mac community...right?). So, fire up Terminal.app (or iTerm, Terminator, etc.) and we'll get started.

Learning Your History

A debugger is itself a program that lets you examine another running program. You can use a debugger to step through the running code of the target program one line at a time, examine the values of variables at a given point in the code, run up until a certain breakpoint and examine a program crash or exception. One of the more well-known multi-language debuggers is the GNU Debugger, or "gdb." While you could use gdb to debug Python, there happens to exist a Python-specific clone of GDB called pdb, or, the Python Debugger.

In many scripting languages, programs are typically relatively short, and debuggers are often unnecessary. Many scripters are accustomed to sprinkling 'print' or logging statements through their code that reveals the value of particular variables at a particular point in the program's execution. However, you may have realized that Python is a bit more grown up than many traditional scripting languages. There are many fairly large systems written in Python. As an application gets larger and contains more dependencies, a dedicated debugger becomes not only useful, but necessary.

Getting Started

We'll start off with some simple code as an example of basic debugging. You write the code in listing 1 in the hopes finding prime numbers through and including 10.

Listing 1: prime_debug.py - sample code for debugging.

#!/usr/bin/python
for n in range(2, 10):
  for x in range(2, n):
    if n % x == 0:
      print n, 'equals', x, '*', n/x
      break
    else:
      print n, 'is a prime number'

Of course, you run this code and see something a little different than you expected—there are two problems in this code. A basic reason for debugging! (Kudos if you already see the errors).

The Python debugger is implemented as a module, so, like other modules, you need to import it. Add the following import after the shebang line:

import pdb

You'll also need to pick a point where you want to start tracing. Since this is an example, we'll start right at the top. So also add the set_trace method immediately following the import statement:

pdb.set_trace()

Now you can just run the program (mark it as executable first with a chmod 770 prime_debug.py or simlar). However, when you run the program this time, you're looking at something different. Something like this:

$ ./prime_debug.py > /Users/marczak/dev/py/prime_debug.py(7)<module>() -> for n in range(2, 10): (Pdb)

What you are looking at is the pdb interactive debugger waiting for your command. You'll see this when the pdb.set_trace() method is called. At this point, pdb stops all execution, displays the statement that it's waiting to execute next and displays its prompt. For our purposes, we want to execute this line (for n in range(2, 10)), so, we enter n, for "next." After pressing return, we're greeted with new information and a new prompt:

-> for x in range(2, n):
(Pdb)

Ah! We've moved on to the next line of the program, and are looking at the next statement to execute. To do so, you can simply press return, as pdb will repeat the last command you gave it by pressing return. Keep doing this a few times until you're comfortable with the display and what you're looking at.

Just so we can get back in sync, quit the debugger and we'll start again. To quit pdb at any time, issue a q command. You'll see a diagnostic "bdb.BdbQuit" line printed and find yourself back at a shell prompt.

Run your program again and let it drop into the debugger, and let's do something a little more useful this time. Tracing program flow is useful, but just as useful is being able to examine the value of variables. You're now essentially waiting for the first line of the program to execute: "for n in range(2, 10)." If you try to examine the variable n right now, you'll receive an error, since this line hasn't yet executed and n isn't yet defined.

First, execute this first line by entering n for "next," then enter p n, which stands for "print the contents of n." You can display the contents of any variable with the p ("print") command. In our example, the output should look like this:

-> for x in range(2, n):
(Pdb) p n
2

This is completely in line with our expectations: n is 2, right at the beginning of its range. (Note that the displayed line is the next line, not the one we're examining the variable of).

Finding our problem

Let's go off and find our problem, which is actually two-fold. The output currently looks like this:

3 is a prime number
4 equals 2 * 2
5 is a prime number
5 is a prime number
5 is a prime number
6 equals 2 * 3
7 is a prime number
7 is a prime number
7 is a prime number
7 is a prime number
7 is a prime number
8 equals 2 * 4
9 is a prime number
9 equals 3 * 3

This is all technically correct, but ugly. What's with the repeating lines? Also, we wanted to find values through 10, not 9. Since the first time we see the repeating lines is when n is equal to 5, let's find that point. Run the program, step through each line using the n command until you see the first output of "5 is a prime number." It will look like this:

> /Users/marczak/dev/py/prime_debug.py (9)<module>()
-> if n % x == 0:
(Pdb) 
> /Users/marczak/dev/py/prime_debug.py (14)<module>()
-> print n, 'is a prime number'
(Pdb) 
5 is a prime number
> /Users/marczak/dev/py/prime_debug.py (8)<module>()
-> for x in range(2, n):
(Pdb) 

Now, let's pay attention as we continue to step through. After a few iterations (or sooner), it should become clear: our if statement is not True, which is fine, and the else clause is running our print statement, which isn't fine. We really only want to print that notification on the way out of the loop when it fails to find a factor. So, our logic error is simple: we have the wrong level of indentation on the else statement. It should be un-indented one level, to be a part of the for loop. The entire loop should look like this:

for n in range(2, 10):
  for x in range(2, n):
    if n % x == 0:
      print n, 'equals', x, '*', n/x
      break
  else:
    print n, 'is a prime number'

Again, notice the subtle difference in indentation for the else portion – it's really a part of the for loop. If you're 'too close' to your code, that's an easy one to miss. However, debugging can be similar to explaining your code to a rubber duck—you know how it's supposed to work, but you only have the 'a ha!' as you step through it.

Make it Easier

We found our major error, but now have another: we want to print primes up through and including the number 10. If you're like me, you need a refresher at this point as to where you are in the code. Issuing an l (not "one," but "ell," for "list") will do just that:

(Pdb) l
  3     import pdb
  4     
  5     pdb.set_trace()
  6     
  7     for n in range(2, 10):
  8  ->     for x in range(2, n):
  9         if n % x == 0:
 10           print n, 'equals', x, '*', n/x
 11           break
 12         else:
 13           # loop fell through without finding a factor

Ah! Now I know where I am. All we're really interested in from this point on is the value of n. Stepping through the remainder of the code shows that the initial for loop exits after 9. Didn't we ask it to run until 10?

Yes we did, but that's our misconception. Looking at the Python documentation for range() shows that the range intentionally excludes the final number.

While this may not be a common mistake that you make, it turns out that this is still a useful exercise: you may not always be debugging your own code.

Dealing with Functions

There's a few more pdb commands to understand before you tackle larger python programs. Specifically, you'll want to know how to deal with functions. Take, for example, the code in listing 2.

Listing 2: dict_iterate.py

#!/usr/bin/python
import pdb
pdb.set_trace()
def _PrintDict(dict):
  """Recursively iterate over a dictionary, printing results
  Args:
    dict: The dictionary to print
  """
  pdb.set_trace()
  for item in dict:
    if type(dict[item]) == dict:
      _PrintDict(dict[item])
    else:
      print "%s: %s" % (dict[item], type(dict[item]))
def main():
  """Main routine"""
  aDict = {'color': 'blue',
           'count': 15,
           'cust_info': {'pid': '94758476', 'uid': '348576'},
           'style': 'fruit'}
  _PrintDict(aDict)
if __name__ == "__main__":
  main()

This should look vaguely familiar to anyone who read the previous column on Python. Start this program running and step through it with n—you'll see python touch each function name to create an object for it. If you keep tracing with n ("next"), this program will end very quickly. This is because when the n command reaches a function, it executes the entire function without entering that function. So, stop tracing with n when you arrive at the call to main():

-> if __name__ == "__main__":
(Pdb) n
> /Users/marczak/dev/py/dict_iterate.py(30)<module>()
-> main()

We want to step into main(), so go ahead and enter s (for "step"). You should be greeted with:

def main():

showing that you're now looking at the definition for main(). Keep stepping as we want to also step into the call to _PrintDict().

When you do arrive in the _PrintDict() function, there's a for loop. Once you've traversed that loop, you may no longer be deeply interested in it, but want to get back to where you were before entering this function. pdb has a solution for you: r, for "return." Essentially, "finish up this function and return."

Be aware! Stepping into functions sometimes will have an unintended consequence for you: stepping into an library that you've included. This is often not the code that you're interested in debugging, though it may be. If you accidentally step into a library function—PyObj-C code included—just remember the r command and return until you're back to where you expect.

More pdb Features

You now know the core of pdb and can actually do some serious debugging. However, pdb offers a lot more. Some of which we'll save for another column, but there are two more useful things to pass on.

The easy way out: c, for "continue." If at any point, you've traced through all you've wanted to trace, but don't want to crash the program with a quit (q) command, there is another option. The continue command picks up and runs the remainder of the program.

Even better, though, is this: pdb is letting you load and run your Python program in an interactive environment. You can alter variables just by assigning them:

(Pdb) x =7
(Pdb) p x
7

Imagine the simple code in listing 3.

Listing 3: math.py

#!/usr/bin/python
import pdb
pdb.set_trace()
x = 5
for i in range(1, 10):
  print i + x

At any time after x gets assigned, you can reassign it. Your debugging session can look like this:

$ ./math.py 
> /Users/marczak/dev/py /math.py(7)<module>()
-> x = 5
(Pdb) n
> /Users/marczak/dev/py /math.py(9)<module>()
-> for i in range(1, 10):
(Pdb) 
> /Users/marczak/dev/py /math.py(10)<module>()
-> print i + x
(Pdb) 
6
> /Users/marczak/dev/py /math.py(9)<module>()
-> for i in range(1, 10):
(Pdb) x = 20
(Pdb) n
> /Users/marczak/dev/py /math.py(10)<module>()
-> print i + x
(Pdb) 
22

This is fantastic news if you want to test your code for fragility around scenarios where variable reach certain values. Of course, as the code runs, if a variable gets reassigned in your program, you need to watch for that.

Reference Sheet

Here's a handy list of the pdb topics discussed in this article:

pdb library: import pdb
Start tracing: pdb.set_trace()
n: execute next command
s: step into a function
r: return from function
c: continue running program
q: quit pdb (and error out of program).

Conclusion

While you may focus on shorted programs now, as your skills improve, your programs should grow in complexity and size. At some point, you'll likely confound yourself, and a debugger comes in very handy during these times. Fortunately, Python has an available debugger that is friendly to use and easy to make available to your program.

Media of the month: I've been in a "back to school" kind of way, but really just studying on my own. There's too much to learn, right? Well, I've taken the school approach: one topic a day and rotate through them and study each night. Now, I'm not recommending anything this drastic, but, I'd bet there's one subject that you've wanted to learn. Now is the time. Hit up your local bookstore, University or Amazon.com, find a book and go. If you're still an actual student, well, keep going!

Until next month, keep scripting.


Ed Marczak is the Executive Editor for MacTech Magazine, and has written the Mac in the Shell column since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

pwSafe 4.1 - Secure password management...
pwSafe provides simple and secure password management across devices and computers. pwSafe uses iCloud to keep your password databases backed-up and synced between Macs and iOS devices. It is... Read more
Kodi 15.0.rc1 - Powerful media center to...
Kodi (was XBMC) is an award-winning free and open-source (GPL) software media player and entertainment hub that can be installed on Linux, OS X, Windows, iOS, and Android, featuring a 10-foot user... Read more
Coda 2.5.11 - One-window Web development...
Coda is a powerful Web editor that puts everything in one place. An editor. Terminal. CSS. Files. With Coda 2, we went beyond expectations. With loads of new, much-requested features, a few surprises... Read more
Bookends 12.5.7 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Access the power of Bookends directly from Mellel, Nisus Writer Pro, or MS Word (... Read more
Maya 2016 - Professional 3D modeling and...
Maya is an award-winning software and powerful, integrated 3D modeling, animation, visual effects, and rendering solution. Because Maya is based on an open architecture, all your work can be scripted... Read more
RapidWeaver 6.2.3 - Create template-base...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more
MacFamilyTree 7.5.2 - Create and explore...
MacFamilyTree gives genealogy a facelift: it's modern, interactive, incredibly fast, and easy to use. We're convinced that generations of chroniclers would have loved to trade in their genealogy... Read more
Paragraphs 1.0.1 - Writing tool just for...
Paragraphs is an app just for writers. It was built for one thing and one thing only: writing. It gives you everything you need to create brilliant prose and does away with the rest. Everything in... Read more
BlueStacks App Player 0.9.21 - Run Andro...
BlueStacks App Player lets you run your Android apps fast and fullscreen on your Mac. Version 0.9.21: Note: Now requires OS X 10.8 or later running on a 64-bit Intel processor. Initial stable... Read more
Tweetbot 2.0.2 - Popular Twitter client....
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more

Rage of Bahamut is Giving Almost All of...
The App Store isn't what it used to be back in 2012, so it's not unexpected to see some games changing their structures with the times. Now we can add Rage of Bahamut to that list with the recent announcement that the game is severely cutting back... | Read more »
Adventures of Pip (Games)
Adventures of Pip 1.0 Device: iOS iPhone Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: ** ONE WEEK ONLY — 66% OFF! *** “Adventures of Pip is a delightful little platformer full of charm, challenge and impeccable... | Read more »
Divide By Sheep - Tips, Tricks, and Stre...
Who would have thought splitting up sheep could be so involved? Anyone who’s played Divide by Sheep, that’s who! While we’re not about to give you complete solutions to everything (because that’s just cheating), we will happily give you some... | Read more »
NaturalMotion and Zynga Have Started Tea...
An official sequel to 2012's CSR Racing is officially on the way, with Zynga and NaturalMotion releasing a short teaser trailer to get everyone excited. Well, as excited as one can get from a trailer with no gameplay footage, anyway. [Read more] | Read more »
Grab a Friend and Pick up Overkill 3, Be...
Overkill 3 is a pretty enjoyable third-person shooter that was sort of begging for some online multiplayer. Fortunately the begging can stop, because its newest update has added an online co-op mode. [Read more] | Read more »
Scanner Pro's Newest Update Adds Au...
Scanner Pro is one of the most popular document scanning apps on iOS, thanks in no small part to its near-constant updates, I'm sure. Now we're up to update number six, and it adds some pretty handy new features. [Read more] | Read more »
Heroki (Games)
Heroki 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: CLEAR THE SKIES FOR A NEW HERO!The peaceful sky village of Levantia is in danger! The dastardly Dr. N. Forchin and his accomplice,... | Read more »
Wars of the Roses (Games)
Wars of the Roses 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: | Read more »
TapMon Battle (Games)
TapMon Battle 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: It's time to battle!Tap! Tap! Tap! Try tap a egg to hatch a Tapmon!Do a battle with another tapmons using your hatched tapmons! *... | Read more »
Alchemic Dungeons (Games)
Alchemic Dungeons 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: ### Release Event! ### 2.99$->0.99$ for limited time! ### Roguelike Role Playing Game! ### Alchemic Dungeons is roguelike... | Read more »

Price Scanner via MacPrices.net

Canon PIXMA MG3620 Wireless Inkjet All-in-One...
Canon U.S.A., Inc. has announced the PIXMA MG3620 Wireless (1) Inkjet All-in-One (AIO) printer for high-quality photo and document printing. Built with convenience in mind for the everyday home user... Read more
July 4th Holiday Weekend 13-inch MacBook Pro...
Save up to $150 on the purchase of a new 2015 13″ Retina MacBook Pro at the following resellers this weekend. Shipping is free with each model: 2.7GHz/128GB MSRP $1299 2.7GHz/... Read more
27-inch 3.5GHz 5K iMac on sale for $2149, sav...
Best Buy has the 27″ 3.5GHz 5K iMac on sale for $2149.99. Choose free shipping or free local store pickup (if available). Sale price for online orders only, in-store prices may vary. Their price is $... Read more
Apple now offering refurbished 2015 11-inch...
The Apple Store is now offering Apple Certified Refurbished 2015 11″ MacBook Airs as well as 13″ MacBook Airs (the latest models), available for up to $180 off the cost of new models. An Apple one-... Read more
15-inch 2.5GHz Retina MacBook Pro on sale for...
Amazon.com has the 15″ 2.5GHz Retina MacBook Pro on sale for $2274 including free shipping. Their price is $225 off MSRP, and it’s the lowest price available for this model. Read more
Finally Safe To Upgrade To Yosemite’?
The reason I’ve held back from upgrading my MacBook Air from OS X 10.9 Mavericks to 10.10 Yosemite for nearly a year isn’t just procrastination. Among other bugs reported, there have been persistent... Read more
Logo Pop Free Vector Logo Design App For OS X...
128bit Technologies has released of Logo Pop Free 1.2 for Mac OS X, a vector based, full-fledged, logo design app available exclusively on the Mac App Store for the agreeable price of absolutely free... Read more
21-inch 1.4GHz iMac on sale for $999, save $1...
B&H Photo has new 21″ 1.4GHz iMac on sale for $999 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Best Buy has the 21″ 1.4GHz iMac on sale for $999.99 on their... Read more
16GB iPad mini 3 on sale for $339, save $60
B&H Photo has the 16GB iPad mini 3 WiFi on sale for $339 including free shipping plus NY tax only. Their price is $60 off MSRP. Read more
Save up to $40 on iPad Air 2, NY tax only, fr...
B&H Photo has iPad Air 2s on sale for up to $40 off MSRP including free shipping plus NY sales tax only: - 16GB iPad Air 2 WiFi: $489 $10 off - 64GB iPad Air 2 WiFi: $559 $40 off - 128GB iPad Air... Read more

Jobs Board

*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
Program Manger, WW *Apple* Direct Fulfillme...
**Job Summary** We are seeking a business analyst to work within our Worldwide Apple Direct Fulfillment Operations team. This role will work closely with related program Read more
Project Manager, *Apple* Retail New Store O...
**Job Summary** An Apple Retail New Store Openings & Remodels Project Manager is responsible for successfully managing the openings, remodels, and small works of Read more
Technical Project Manager - *Apple* Pay - A...
**Job Summary** Apple Pay is seeking an experienced technical PM to…manage the on boarding of new merchants for the Apple Pay platform in the US Within this role you Read more
*Apple* Pay- Automation Test Engineer - Appl...
**Job Summary** At Apple , great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.