TweetFollow Us on Twitter

Mac in the Shell: Debugging Python with vim

Volume Number: 26
Issue Number: 01
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python with vim

IDE-like debugging without leaving vim

by Edward Marczak

Welcome

Mac in the Shell has been focusing on Python for the System Administrator over the last few months. Last month re-introduced the vi(m) text editor. This month, we're going to tie them together a bit. I've covered pdb - the python debugger - already. It's really handy to have a debugger at your disposal. However, pdb's output and display can get a little unwieldy. Never fret: Vim to the rescue! This article will show you how to get the latest MacVim and start using it to debug Python code.

Use the Source, Luke

First thing is first: this will all fail miserably with the version of vim that's built into the base OS. Best solution: use MacVim. MacVim comes built with Python, GUI support and more. There's a few ways to get MacVim, and I'll detail both.

First way is the easy way: download a binary release, typically of the latest snapshot. Do that directly from the MacVim project page, currently on Google Code:

http://code.google.com/p/macvim/

But wait, there's more! If you like to customize your experience, like I do, you'll want the source code. This is a multi-part step. First, you'll need to make sure that the (free) Apple developer tools are installed on your machine. (and honestly, what good is a machine without those?). You'll also need git installed to check out the MacVim source. If you're a fink or MacPorts person, you can use either of those systems to install git. If you're a pre-built person, or just need to be in this case, go pick up a pre-built git from:

http://code.google.com/p/git-osx-installer/

Once git is up and running, check out the MacVim source by changing to the destination directory and issuing a clone command:

git clone git://repo.or.cz/MacVim.git vim7

Once that completes, you'll need to configure and build MacVim. This is where you can customize the binary a bit. I want the whole experience, so, I change to the vim7 directory and run configure with my options:

./configure 
—enable-perlinterp —enable-pythoninterp 
—enable-rubyinterp 
—with-features=huge 
—enable-gui=macvim 
—with-tlib=ncurses 
—enable-multibyte —with-python-config-dir=
/System/Library/Frameworks/Python.framework/Versions/2.6//lib/python2.6/config/

No matter how you configure Vim, I'd highly recommend the -enable-gui option and, for this article, you must enable the python interpreter. Of course, you're reading an article about debugging python using Vim, so why wouldn't you enable this?

Once configure finishes its business (successfully), type make and...wait. Barring any errors, you'll find the MacVim application in the src/MacVim/build/Release directory. If you're following this method, when you want to 'upgrade' (read: compile a new version), change into the vim7 directory and ask git to pull down the latest changes:

$ git pull

followed by make. Go pick up your fresh binary in the Release directory as before.

You may also want to copy the mvim script from the src/MacVim subdirectory and place it somewhere in your $PATH. Running mvim from the command-line will try to do the right thing: if you're logged in via ssh, you'll use console-based vim. If you have a GUI, you'll run GUI-based MacVim. You can also create alias or functions that call mvim as a different name. This will cause Vim to run in different modes. I have the following in my ~/.bash_profile initialization script:

export VIM_APP_DIR=${HOME}/Applications/MacVim/
vi() { ${HOME}/Applications/MacVim/mvim ${@}; }
vim() { ${HOME}/Applications/MacVim/mvim ${@}; }
gvim () { ${HOME}/Applications/MacVim/mvim ${@}; }
vimdiff() { ${HOME}/Applications/MacVim/mvim -d ${@}; }
export -f vi
export -f vim
export -f gvim
export -f vimdiff

I use bash functions rather than aliases as there are several problems with using aliases in this case. Primarily, a bash alias will fork off a new shell and there's too much juggling around of variables. Bash aliases have been deprecated in favor of bash functions in any case.

In my example .bash_profile snippet, I explicitly define where MacVim lives as I keep it in my home directory. Then I define the four ways I may call vi(m). One of the most important to me is the definition for vimdiff. When integrating with version control programs, I want to make sure it doesn't fork off, and it should come back after an individual window is closed. Once these are defined, I separately do the same for gvim and git integration if I'm logged in to a GUI session with:

export GIT_EDITOR='gvim -f -c "au VimLeave * !open -a Terminal"'

The final group of exports causes functions to be exported to subshells.

One last note: no matter how you choose to call vim, ensure that your method gets found before the system vi(m). Keep it early in your path, or use my method above, as functions are always used before searching $PATH.

The Setup

The entire point of the earlier Vim exercise is to ensure that you are using a Python-enabled version of vim. The vim binary that ships with OS X is not compiled with the Python, or any scripting option. Once you've downloaded or compiled MacVim and have it in place, launch it and test for Python support. Enter ex mode by typing a colon (":") followed by py and a python command. Something like:

:py print 6*7

You'll see the result on the command line. Not exceptionally exiting, perhaps...not at the outset, anyway. However, having Python support built in means that Python can be used to script just about anything. That is exciting! This is also, not coincidentally, how we're going to be able to debug python code entirely within vim. If you receive an error from 'py' - specifically "E319: Sorry, the command is not available in this version" - ensure that you're running the updated version of vim that you downloaded (or built!) and not the system vi(m).

Configuring Vim for Debugging

There are plenty of hoops one can jump through to debug directly in vim. My favorite, though, is a freely downloadable plugin. Go grab vimpdb from its project page:

http://code.google.com/p/vimpdb/

From there, click on the download link. Or, you can just grab the source directly using subversion:

svn checkout http://vimpdb.googlecode.com/svn/trunk/ vimpdb

Pleasantly, subversion is built into OS X as of 10.5, so, you can just run that command with no fuss.

However you've obtained vimpdb, you need to drop VimPdb.py and VimPdb.vim into your vim plugin directory (~/.vim/plugin). Now you're ready...maybe.

By default, vimpdb uses function keys for instructing it on what action to take. If you're using a 'real' keyboard, you can likely just use the default key bindings. By 'real' keyboard, I'm essentially talking about any stand-alone keyboard. It's the keyboards that are built into the MacBooks and MacBook Pro machines that are the problem. If you're like me, you'll occasionally want to debug on the go. The trouble starts with the way Apple overloads the function keys. Despite a setting in System Preferences that allows you to "use all" function keys "as standard function keys," this doesn't do what you expect initially. You'll still need to undo those key settings in the "Keyboard Shortcuts" section of the Keyboard pref pane. It's about picking your battles, I suppose. However, those shortcuts are really nice to have. (I know, this is a fairly lame argument, but, hey - I'm the end-user here and I demand satisfaction). I've experimented a bit with creating a wrapper script for vim that swaps out the plist file that controls this (~/Library/Preferences/com.apple.symbolichotkeys.plist) with one that has no F-key definitions, but that causes other issues for me. Ultimately, I reached a compromise and only undo a few default keyboard shortcuts: gone are ^F2, ^F4 and ^F8. So, let's go set up our vim key mappings.


Figure 1 - Assigning - or unassigning - keyboard shortcuts.

Again, if you want to leave the default keys for vimpdb and disable the OS X integrated keys in the Keyboard pref pane, feel free. For this article, I'm going to make some very simple changes, but you can set up the keys as you like. Open up the vimpdb.vim file that you just placed in your ~/.vim/plugin directory (using Vim, of course!). Move to line 540 (in Vim, type "540 G"). I've altered these lines to map F3 and F4 to step into and step over, respectively. They now read:

   " Was F7 and F8
   map <buffer> <silent> <F3> :call PdbStepInto()<CR>
   map <buffer> <silent> <F4> :call PdbStepOver()<CR>

Also, I've needed to change out the places that F3 and F4 were already used. Scroll down to line 560 or so, and make these changes:

   " Was F4
   map <buffer> <silent> <F7> :call PdbEvalCurrentWord()<CR>
   map <buffer> <silent> <C-F7> :call PdbEvalCurrentWORD()<CR>
   " Was F3
   map <buffer> <silent> <F8> :call PdbEvalExpression()<CR>

Save the file, and you're ready.

Lock and Load

The default vimpdb key binding along with the changes we've made leave us with the following keys for controlling a debug session:

Start and stop debugging

- F5 - Start/continue debug session of current file.

- Ctrl-F5 - Start debugging and do not pause at first line

- Ctrl-Shift-F5 - Start debugging with a given list of parameters.

- Shift-F5 - Stop the current debug session.

- Ctrl-Alt-Shift-F5 - Restart the current debug session.

Breakpoints

- F2 - Toggle breakpoint.

- Ctrl-F2 - Toggle conditional breakpoint

- Shift-F2 - Toggle temporary breakpoint

- Ctrl-Shift-F2 - Clear all breakpoints in current file

- Ctrl-Alt-Shift-F2 - Clear all breakpoints in all files

- F11 - Print condition of conditional breakpoint under the cursor

Step/continue

- F3 - Step into

- F4 - Step over

- Ctrl-F4 - Continue running until reaching a return from function

Cursor movement

- F6 - Move cursor to currently debugged line.

- Ctrl-F6 - Change current debugged line to where the cursor is currently placed.

Stack

- F9 - Move up in stack frame.

- F10 - Move down in stack frame.

- F12 - Print stack trace

Evaluate/Execute

- F7 - Evaluate a given expression (in the current debug context)

- Ctrl-F7 - Exec a given statement (in the current debug context)

- F8 - Evaluate the current word under the cursor (in the current debug context)

- Ctrl-F8 - Evaluate the current WORD under the cursor (in the current debug context)

Again, depending on the keyboard you use and the setting you have in Keyboard Shortcuts, some of these key mappings may (will?) have conflicts.

So, what does all of this get you? Figure 2 should give you an idea.


Figure 2 - VimPDB in action.

Unlike the pure command-line environment of pdb, vimpdb lets you keep your code in view and step along with it as it runs. Visually. In figure 2, line 3 - highlighted in green - is the current line. Line 9 - highlighted in red - has a breakpoint set on it. You can use any Python code that you like. I'm going to use the code that I offered up in my article on PyObjC from August 2009 that reads the Apple Address Book.

Debugging

Once you're set up, the debugging is fairly easy. Load the python code that you'd like to debug into the current buffer and press F5. This begins the debugger, and vim should print a status message that says "Starting debugger." You should also now see your first executable line highlighted in green. Now, the "first executable" line is the first thing that the parser needs to look at, not necessarily what you'd expect as "executable."

To step through the program, you can press F4 to step 1 instruction. This will step over any intervening code between the current line and what is visible as next. The current line will execute, and the green line will advance. If the current line is a function call that you want to inspect, press F3 to step into the function. You'll need to do this at least once if you use the Python convention of creating a main() function and calling that conditionally:

if __name__ == '__main__':
  main()

Once you're inside a function, often, you'll get to the part you're interested in, and then not want to have to step through the remainder of the function. You can accomplish that by pressing control-F4.

The great thing about using a debugger is not only can you step through and watch the flow of your code, but you can inspect variables as they change, too. Vimpdb doesn't let us down here. Position the cursor over the variable name you want to inspect and press F7. The value of the variable is displayed in the output area at the bottom of the vim screen followed by a note to press any key to return to debugging.

For example, if I were to press F7 over the "abgroups" variable showing in Figure 2, I receive this output:

 (
    "ABGroup (0x10835cb40) {\n\tCreation     : 2009-07-14 15:13:36 
    -0400\n\tGroupName    : WS\n\tModification : 2009-10-27 19:24:14 
    -0400\n\tUnique ID    : 8BBAEDE9-0D31-4B48-A967-826523BF9DA7:ABGroup\n}",
    "ABGroup (0x1006f3ab0) {\n\tCreation     : 2009-01-14 15:13:36 
    -0400\n\tGroupName    : MacTech Editorial\n\tMembers    
    : 28 members\n\tModification : 2009-11-23 08:04:24 
    -0500\n\tUnique ID    : 5712EB9A-9743-4744-BA05-354726896614:ABGroup\n}"
)

Note that variables you inspect must be defined. If the current line contains a variable that hasn't been defined yet, you can't inspect that variable until the line has been executed. If you try, you'll receive a NameError:

NameError: Name 'blah' is not defined.

Just ensure that the variable you're trying to inspect is defined. If it's first defined on the current line, execute the current line first.

Let's look at a typical short debug session. First, you'd open vim and load your code into the buffer:

vim ~/dev/py/address_book.py

(or, open vim and type :e ~/dev/py/address_book.py).

Start the debug session by pressing F5 to initialize the debugger and start the session. You should note the "starting debugger" message. No other vimpdb keys will do anything until this step is taken.

You can navigate down to the call to main() and set a breakpoint - using F2 while the cursor is on the line with said call. Once set, press F5 to run the program as usual until it hits your break point. If you stopped on main(), press F4 to step into the main function. Execute a line that assigns a value to a variable, cursor up to the variable name and press F7 to inspect the value that was assigned; it'll appear at the bottom of the vim window. Press F3 to execute the current line.

Rinse, repeat.

One extra nice feature of VimPDB: the ability to save and load breakpoints. In a larger program, you may set several breakpoints in different locations that you want to recall after quitting vim and then trying to debug again. While in debug mode with all of the breakpoints set, type \s (that's backslash, and then s). You'll be prompted for a filename. Type one and press return. When you later come back to vim and load your project, start the debugger and type \l (backslash, then lowercase L, for "load").

In Closing

VimPDB provides the best of all worlds for debugging Python code. It doesn't make you change code to drop in a bunch of print or logging statements. It gives you a larger view of your code as it's running. Finally, it can easily be run remotely over an ssh session (win!).

Also, VimPDB is pretty easy in general, and a great addition to your toolkit. This article focused mainly on the preparation: install and introduction. Once you start using it, it becomes second nature. Unfortunately, unless you wildly remap the default keys, you'll need to choose between OS X functions on the F-keys and bindings in Vim (and yes, OS X "wins"). This is of particular annoyance on a MacBook/Pro, but not insurmountable.

Media of the month: No better time to recommend "Learning the vi and Vim Editors" by Arnold Robbins, Elbert Hannah, and Linda Lamb. This is a pretty comprehensive look at learning and customizing vi(m). Available in dead-tree and electronic versions.

Hope to see everyone in San Francisco for Macworld!


Ed Marczak is the Executive Editor of MacTech Magazine. He has written the Mac in the Shell column since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

How to become the ultimate robot warrior...
Chrono Strike is a delightfully immersive beat ‘em up with a sense of humor (any game with a good Sims reference gets points in my book). [Read more] | Read more »
Tips and tricks to get a higher score in...
Snow Roll is a devilish endless runner very much in the vein of Flappy Bird. It revels in its dastardly level of difficulty, and doesn’t really care how angry you get at it as it knows you’ll keep coming back for more. [Read more] | Read more »
How to win big in Slots Deluxe
Cheating while gambling is illegal and morally wrong, and in some parts of the world it leads to men with names like Vinnie "Six Knuckles" Manchenzo beating you to a pulp in a dark alley. [Read more] | Read more »
How to take over the world in Dictator 2
Running a country isn't easy - especially when you're a dictator who wants to take over the world and crush everyone in your path while you do it. [Read more] | Read more »
Tips and tricks to get a higher score in...
Tank.iois - you guessed it! - another multiplayer arena battler likeAgar.io and Slither.io. It does differentiate itself by putting you in a tiny tank though, so it's not exactly the same. To help you get that all-important high score, we've got a... | Read more »
How to unlock characters in One Tap Tenn...
As the title suggests, One Tap Tennis requires only a single tap to play its particular brand of tennis, and rewards you with a ton of unlockable characters if you perform well. Fortunately for you, we at 148Apps have got a few tips and tricks to... | Read more »
Grab it now: Game Craft’s Legend of War...
The real time strategy game is now available for you to sink your teeth into, through the App Store and Google Play. Combining elements of skill, strategy and empire building, Legend of War is a real gamers’ game. [Read more] | Read more »
Skateboard Party 3 ft. Greg Lutzka (Gam...
Skateboard Party 3 ft. Greg Lutzka 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Skateboard Party is back! This third edition of the popular sports franchise features professional skater... | Read more »
Cubious (Games)
Cubious 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: Cubious – How smart are you? How high is your IQube? Solve the impossible puzzles to find out, and help a lost little cube find his... | Read more »
Goat Simulator Waste of Space (Games)
Goat Simulator Waste of Space 1.1 Device: iOS Universal Category: Games Price: $4.99, Version: 1.1 (iTunes) Description: ** IMPORTANT - SUPPORTED DEVICESiPhone 4S, iPad 2, iPod Touch 5 or better.** | Read more »

Price Scanner via MacPrices.net

Enterprise Workers Pick Technology Over Perks...
New Adobe study shows surprising attitudes about office jobs and where the future of work is heading. Adobe has released survey findings revealing that a surprising 70 percent of U.S. office workers... Read more
Memorial Day Weekend Sale: $50-$100 off 11-in...
B&H Photo has 13″ and 11″ MacBook Airs with 256GB SSDs on sale for $50-$100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 11″ 1.6GHz/256GB MacBook Air: $999 $100 off MSRP... Read more
Memorial Day Weekend Sales: Apple MacBook Pro...
B&H Photo has 13″ and 15″ Retina MacBook Pros on sale for up to $210 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1799 $200 off MSRP - 15″ 2.5GHz... Read more
Memorial Day Weekend Sales: Apple iMacs and M...
Take up to $150 off the price of a new iMac or Mac mini at the following Apple resellers this Memorial Day weekend: iMacs: B&H Photo has 21″ and 27″ iMacs on sale for up to $150 off MSRP... Read more
Apple refurbished Retina MacBook Pros availab...
Apple has Certified Refurbished 2015 13″ and 15″ Retina MacBook Pros available for up to $380 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free... Read more
Apple refurbished 11-inch MacBook Airs availa...
Apple has Certified Refurbished 11″ MacBook Airs (the latest models), available for up to $170 off the cost of new models. An Apple one-year warranty is included with each MacBook, and shipping is... Read more
Goal Zero and OtterBox Partner to Expand iPh...
Goal Zero, specialists in portable power, have announced a partnership with OtterBox, brand smartphone case protection, to offer the Slide and Slide Plus Batteries as modules compatible with the new... Read more
15-inch Retina MacBook Pros on sale for up to...
B&H Photo has 15″ Retina MacBook Pros on sale for up to $210 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1799 $200 off MSRP - 15″ 2.5GHz Retina... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $250 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799... Read more
Apple refurbished Apple TVs available for up...
Apple has Certified Refurbished 32GB and 64GB Apple TVs available for up to $30 off the cost of new models. Apple’s standard one-year warranty is included with each model, and shipping is free: -... Read more

Jobs Board

Senior *Apple* Engineer - Signature Technol...
One-year contract for an Apple consultant. The resource should be an Apple Certified Technical Coordinator or get the certification within 60 days of starting Read more
*Apple* Architect - AECOM (United States)
**Requisition/Vacancy No.** 132759BR **Position Title** Apple Architect **Job Category** Information Technology **Business Line** Government **Country** United Read more
*Apple* Solutions Consultant - APPLE (United...
Job Summary As an Apple Solutions Consultant, you'll be the link between our future customers and our products. You'll showcase your entrepreneurial spirit as you Read more
*Apple* Project Engineer - Smart Source Inc...
SmartSource is in need of an Apple Project Engineer for a 12 month contract opportunity in Pittsburg, PA. Role: Apple Project Engineer Location: Pittsburg, PA Read more
Automotive Sales Consultant - Apple Ford Linc...
…you. The best candidates are smart, technologically savvy and are customer focused. Apple Ford Lincoln Apple Valley is different, because: $30,000 annual salary Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.