TweetFollow Us on Twitter

Mac in the Shell: Debugging Python with vim

Volume Number: 26
Issue Number: 01
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python with vim

IDE-like debugging without leaving vim

by Edward Marczak

Welcome

Mac in the Shell has been focusing on Python for the System Administrator over the last few months. Last month re-introduced the vi(m) text editor. This month, we're going to tie them together a bit. I've covered pdb - the python debugger - already. It's really handy to have a debugger at your disposal. However, pdb's output and display can get a little unwieldy. Never fret: Vim to the rescue! This article will show you how to get the latest MacVim and start using it to debug Python code.

Use the Source, Luke

First thing is first: this will all fail miserably with the version of vim that's built into the base OS. Best solution: use MacVim. MacVim comes built with Python, GUI support and more. There's a few ways to get MacVim, and I'll detail both.

First way is the easy way: download a binary release, typically of the latest snapshot. Do that directly from the MacVim project page, currently on Google Code:

http://code.google.com/p/macvim/

But wait, there's more! If you like to customize your experience, like I do, you'll want the source code. This is a multi-part step. First, you'll need to make sure that the (free) Apple developer tools are installed on your machine. (and honestly, what good is a machine without those?). You'll also need git installed to check out the MacVim source. If you're a fink or MacPorts person, you can use either of those systems to install git. If you're a pre-built person, or just need to be in this case, go pick up a pre-built git from:

http://code.google.com/p/git-osx-installer/

Once git is up and running, check out the MacVim source by changing to the destination directory and issuing a clone command:

git clone git://repo.or.cz/MacVim.git vim7

Once that completes, you'll need to configure and build MacVim. This is where you can customize the binary a bit. I want the whole experience, so, I change to the vim7 directory and run configure with my options:

./configure 
—enable-perlinterp —enable-pythoninterp 
—enable-rubyinterp 
—with-features=huge 
—enable-gui=macvim 
—with-tlib=ncurses 
—enable-multibyte —with-python-config-dir=
/System/Library/Frameworks/Python.framework/Versions/2.6//lib/python2.6/config/

No matter how you configure Vim, I'd highly recommend the -enable-gui option and, for this article, you must enable the python interpreter. Of course, you're reading an article about debugging python using Vim, so why wouldn't you enable this?

Once configure finishes its business (successfully), type make and...wait. Barring any errors, you'll find the MacVim application in the src/MacVim/build/Release directory. If you're following this method, when you want to 'upgrade' (read: compile a new version), change into the vim7 directory and ask git to pull down the latest changes:

$ git pull

followed by make. Go pick up your fresh binary in the Release directory as before.

You may also want to copy the mvim script from the src/MacVim subdirectory and place it somewhere in your $PATH. Running mvim from the command-line will try to do the right thing: if you're logged in via ssh, you'll use console-based vim. If you have a GUI, you'll run GUI-based MacVim. You can also create alias or functions that call mvim as a different name. This will cause Vim to run in different modes. I have the following in my ~/.bash_profile initialization script:

export VIM_APP_DIR=${HOME}/Applications/MacVim/
vi() { ${HOME}/Applications/MacVim/mvim ${@}; }
vim() { ${HOME}/Applications/MacVim/mvim ${@}; }
gvim () { ${HOME}/Applications/MacVim/mvim ${@}; }
vimdiff() { ${HOME}/Applications/MacVim/mvim -d ${@}; }
export -f vi
export -f vim
export -f gvim
export -f vimdiff

I use bash functions rather than aliases as there are several problems with using aliases in this case. Primarily, a bash alias will fork off a new shell and there's too much juggling around of variables. Bash aliases have been deprecated in favor of bash functions in any case.

In my example .bash_profile snippet, I explicitly define where MacVim lives as I keep it in my home directory. Then I define the four ways I may call vi(m). One of the most important to me is the definition for vimdiff. When integrating with version control programs, I want to make sure it doesn't fork off, and it should come back after an individual window is closed. Once these are defined, I separately do the same for gvim and git integration if I'm logged in to a GUI session with:

export GIT_EDITOR='gvim -f -c "au VimLeave * !open -a Terminal"'

The final group of exports causes functions to be exported to subshells.

One last note: no matter how you choose to call vim, ensure that your method gets found before the system vi(m). Keep it early in your path, or use my method above, as functions are always used before searching $PATH.

The Setup

The entire point of the earlier Vim exercise is to ensure that you are using a Python-enabled version of vim. The vim binary that ships with OS X is not compiled with the Python, or any scripting option. Once you've downloaded or compiled MacVim and have it in place, launch it and test for Python support. Enter ex mode by typing a colon (":") followed by py and a python command. Something like:

:py print 6*7

You'll see the result on the command line. Not exceptionally exiting, perhaps...not at the outset, anyway. However, having Python support built in means that Python can be used to script just about anything. That is exciting! This is also, not coincidentally, how we're going to be able to debug python code entirely within vim. If you receive an error from 'py' - specifically "E319: Sorry, the command is not available in this version" - ensure that you're running the updated version of vim that you downloaded (or built!) and not the system vi(m).

Configuring Vim for Debugging

There are plenty of hoops one can jump through to debug directly in vim. My favorite, though, is a freely downloadable plugin. Go grab vimpdb from its project page:

http://code.google.com/p/vimpdb/

From there, click on the download link. Or, you can just grab the source directly using subversion:

svn checkout http://vimpdb.googlecode.com/svn/trunk/ vimpdb

Pleasantly, subversion is built into OS X as of 10.5, so, you can just run that command with no fuss.

However you've obtained vimpdb, you need to drop VimPdb.py and VimPdb.vim into your vim plugin directory (~/.vim/plugin). Now you're ready...maybe.

By default, vimpdb uses function keys for instructing it on what action to take. If you're using a 'real' keyboard, you can likely just use the default key bindings. By 'real' keyboard, I'm essentially talking about any stand-alone keyboard. It's the keyboards that are built into the MacBooks and MacBook Pro machines that are the problem. If you're like me, you'll occasionally want to debug on the go. The trouble starts with the way Apple overloads the function keys. Despite a setting in System Preferences that allows you to "use all" function keys "as standard function keys," this doesn't do what you expect initially. You'll still need to undo those key settings in the "Keyboard Shortcuts" section of the Keyboard pref pane. It's about picking your battles, I suppose. However, those shortcuts are really nice to have. (I know, this is a fairly lame argument, but, hey - I'm the end-user here and I demand satisfaction). I've experimented a bit with creating a wrapper script for vim that swaps out the plist file that controls this (~/Library/Preferences/com.apple.symbolichotkeys.plist) with one that has no F-key definitions, but that causes other issues for me. Ultimately, I reached a compromise and only undo a few default keyboard shortcuts: gone are ^F2, ^F4 and ^F8. So, let's go set up our vim key mappings.


Figure 1 - Assigning - or unassigning - keyboard shortcuts.

Again, if you want to leave the default keys for vimpdb and disable the OS X integrated keys in the Keyboard pref pane, feel free. For this article, I'm going to make some very simple changes, but you can set up the keys as you like. Open up the vimpdb.vim file that you just placed in your ~/.vim/plugin directory (using Vim, of course!). Move to line 540 (in Vim, type "540 G"). I've altered these lines to map F3 and F4 to step into and step over, respectively. They now read:

   " Was F7 and F8
   map <buffer> <silent> <F3> :call PdbStepInto()<CR>
   map <buffer> <silent> <F4> :call PdbStepOver()<CR>

Also, I've needed to change out the places that F3 and F4 were already used. Scroll down to line 560 or so, and make these changes:

   " Was F4
   map <buffer> <silent> <F7> :call PdbEvalCurrentWord()<CR>
   map <buffer> <silent> <C-F7> :call PdbEvalCurrentWORD()<CR>
   " Was F3
   map <buffer> <silent> <F8> :call PdbEvalExpression()<CR>

Save the file, and you're ready.

Lock and Load

The default vimpdb key binding along with the changes we've made leave us with the following keys for controlling a debug session:

Start and stop debugging

- F5 - Start/continue debug session of current file.

- Ctrl-F5 - Start debugging and do not pause at first line

- Ctrl-Shift-F5 - Start debugging with a given list of parameters.

- Shift-F5 - Stop the current debug session.

- Ctrl-Alt-Shift-F5 - Restart the current debug session.

Breakpoints

- F2 - Toggle breakpoint.

- Ctrl-F2 - Toggle conditional breakpoint

- Shift-F2 - Toggle temporary breakpoint

- Ctrl-Shift-F2 - Clear all breakpoints in current file

- Ctrl-Alt-Shift-F2 - Clear all breakpoints in all files

- F11 - Print condition of conditional breakpoint under the cursor

Step/continue

- F3 - Step into

- F4 - Step over

- Ctrl-F4 - Continue running until reaching a return from function

Cursor movement

- F6 - Move cursor to currently debugged line.

- Ctrl-F6 - Change current debugged line to where the cursor is currently placed.

Stack

- F9 - Move up in stack frame.

- F10 - Move down in stack frame.

- F12 - Print stack trace

Evaluate/Execute

- F7 - Evaluate a given expression (in the current debug context)

- Ctrl-F7 - Exec a given statement (in the current debug context)

- F8 - Evaluate the current word under the cursor (in the current debug context)

- Ctrl-F8 - Evaluate the current WORD under the cursor (in the current debug context)

Again, depending on the keyboard you use and the setting you have in Keyboard Shortcuts, some of these key mappings may (will?) have conflicts.

So, what does all of this get you? Figure 2 should give you an idea.


Figure 2 - VimPDB in action.

Unlike the pure command-line environment of pdb, vimpdb lets you keep your code in view and step along with it as it runs. Visually. In figure 2, line 3 - highlighted in green - is the current line. Line 9 - highlighted in red - has a breakpoint set on it. You can use any Python code that you like. I'm going to use the code that I offered up in my article on PyObjC from August 2009 that reads the Apple Address Book.

Debugging

Once you're set up, the debugging is fairly easy. Load the python code that you'd like to debug into the current buffer and press F5. This begins the debugger, and vim should print a status message that says "Starting debugger." You should also now see your first executable line highlighted in green. Now, the "first executable" line is the first thing that the parser needs to look at, not necessarily what you'd expect as "executable."

To step through the program, you can press F4 to step 1 instruction. This will step over any intervening code between the current line and what is visible as next. The current line will execute, and the green line will advance. If the current line is a function call that you want to inspect, press F3 to step into the function. You'll need to do this at least once if you use the Python convention of creating a main() function and calling that conditionally:

if __name__ == '__main__':
  main()

Once you're inside a function, often, you'll get to the part you're interested in, and then not want to have to step through the remainder of the function. You can accomplish that by pressing control-F4.

The great thing about using a debugger is not only can you step through and watch the flow of your code, but you can inspect variables as they change, too. Vimpdb doesn't let us down here. Position the cursor over the variable name you want to inspect and press F7. The value of the variable is displayed in the output area at the bottom of the vim screen followed by a note to press any key to return to debugging.

For example, if I were to press F7 over the "abgroups" variable showing in Figure 2, I receive this output:

 (
    "ABGroup (0x10835cb40) {\n\tCreation     : 2009-07-14 15:13:36 
    -0400\n\tGroupName    : WS\n\tModification : 2009-10-27 19:24:14 
    -0400\n\tUnique ID    : 8BBAEDE9-0D31-4B48-A967-826523BF9DA7:ABGroup\n}",
    "ABGroup (0x1006f3ab0) {\n\tCreation     : 2009-01-14 15:13:36 
    -0400\n\tGroupName    : MacTech Editorial\n\tMembers    
    : 28 members\n\tModification : 2009-11-23 08:04:24 
    -0500\n\tUnique ID    : 5712EB9A-9743-4744-BA05-354726896614:ABGroup\n}"
)

Note that variables you inspect must be defined. If the current line contains a variable that hasn't been defined yet, you can't inspect that variable until the line has been executed. If you try, you'll receive a NameError:

NameError: Name 'blah' is not defined.

Just ensure that the variable you're trying to inspect is defined. If it's first defined on the current line, execute the current line first.

Let's look at a typical short debug session. First, you'd open vim and load your code into the buffer:

vim ~/dev/py/address_book.py

(or, open vim and type :e ~/dev/py/address_book.py).

Start the debug session by pressing F5 to initialize the debugger and start the session. You should note the "starting debugger" message. No other vimpdb keys will do anything until this step is taken.

You can navigate down to the call to main() and set a breakpoint - using F2 while the cursor is on the line with said call. Once set, press F5 to run the program as usual until it hits your break point. If you stopped on main(), press F4 to step into the main function. Execute a line that assigns a value to a variable, cursor up to the variable name and press F7 to inspect the value that was assigned; it'll appear at the bottom of the vim window. Press F3 to execute the current line.

Rinse, repeat.

One extra nice feature of VimPDB: the ability to save and load breakpoints. In a larger program, you may set several breakpoints in different locations that you want to recall after quitting vim and then trying to debug again. While in debug mode with all of the breakpoints set, type \s (that's backslash, and then s). You'll be prompted for a filename. Type one and press return. When you later come back to vim and load your project, start the debugger and type \l (backslash, then lowercase L, for "load").

In Closing

VimPDB provides the best of all worlds for debugging Python code. It doesn't make you change code to drop in a bunch of print or logging statements. It gives you a larger view of your code as it's running. Finally, it can easily be run remotely over an ssh session (win!).

Also, VimPDB is pretty easy in general, and a great addition to your toolkit. This article focused mainly on the preparation: install and introduction. Once you start using it, it becomes second nature. Unfortunately, unless you wildly remap the default keys, you'll need to choose between OS X functions on the F-keys and bindings in Vim (and yes, OS X "wins"). This is of particular annoyance on a MacBook/Pro, but not insurmountable.

Media of the month: No better time to recommend "Learning the vi and Vim Editors" by Arnold Robbins, Elbert Hannah, and Linda Lamb. This is a pretty comprehensive look at learning and customizing vi(m). Available in dead-tree and electronic versions.

Hope to see everyone in San Francisco for Macworld!


Ed Marczak is the Executive Editor of MacTech Magazine. He has written the Mac in the Shell column since 2004.

 
AAPL
$95.79
Apple Inc.
-2.36
MSFT
$43.28
Microsoft Corpora
-0.30
GOOG
$572.85
Google Inc.
-14.57

MacTech Search:
Community Search:

Software Updates via MacUpdate

Ember 1.8 - Versatile digital scrapbook....
Ember (formerly LittleSnapper) is your digital scrapbook of things that inspire you: websites, photos, apps or other things. Just drag in images that you want to keep, organize them into relevant... Read more
OmniPlan 2.3.6 - Robust project manageme...
With OmniPlan, you can create logical, manageable project plans with Gantt charts, schedules, summaries, milestones, and critical paths. Break down the tasks needed to make your project a success,... Read more
Command-C 1.1.1 - Clipboard sharing tool...
Command-C is a revolutionary app which makes easy to share your clipboard between iOS and OS X using your local WiFi network, even if the app is not currently opened. Copy anything (text, pictures,... Read more
Knock 1.1.7 - Unlock your Mac by knockin...
Knock is a faster, safer way to sign in. You keep your iPhone with you all the time. Now you can use it as a password. You never have to open the app -- just knock on your phone twice, even when it's... Read more
Mellel 3.3.6 - Powerful word processor w...
Mellel is the leading word processor for OS X and has been widely considered the industry standard since its inception. Mellel focuses on writers and scholars for technical writing and multilingual... Read more
LibreOffice 4.3.0.4 - Free Open Source o...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
Freeway Pro 7.0 - Drag-and-drop Web desi...
Freeway Pro lets you build websites with speed and precision... without writing a line of code! With it's user-oriented drag-and-drop interface, Freeway Pro helps you piece together the website of... Read more
Drive Genius 3.2.4 - Powerful system uti...
Drive Genius is an OS X utility designed to provide unsurpassed storage management. Featuring an easy-to-use interface, Drive Genius is packed with powerful tools such as a drive optimizer, a... Read more
Vitamin-R 2.15 - Personal productivity t...
Vitamin-R creates the optimal conditions for your brain to work at its best by structuring your work into short bursts of distraction-free, highly focused activity alternating with opportunities for... Read more
Toast Titanium 12.0 - The ultimate media...
Toast Titanium goes way beyond the very basic burning in the Mac OS and iLife software, and sets the standard for burning CDs, DVDs, and now Blu-ray discs on the Mac. Create superior sounding audio... Read more

Latest Forum Discussions

See All

Dawn of the Immortals Review
Dawn of the Immortals Review By Jennifer Allen on July 31st, 2014 Our Rating: :: RESPECTABLE EXPLORATIONUniversal App - Designed for iPhone and iPad Dawn of the Immortals might not re-invent the wheel, but it does tweak it a little... | Read more »
80 Days Review
80 Days Review By Jennifer Allen on July 31st, 2014 Our Rating: :: EPIC ADVENTUREUniversal App - Designed for iPhone and iPad A fantastic and fascinating re-envisioning of the classic novel by Jules Verne, 80 Days is a delightful... | Read more »
Battleheart Legacy Guide
The world of Battleheart Legacy is fun and deep; full of wizards, warriors, and witches. Here are some tips and tactics to help you get the most enjoyment out of this great game. | Read more »
Puzzle Roo Review
Puzzle Roo Review By Jennifer Allen on July 31st, 2014 Our Rating: :: PUZZLE-BASED TWISTUniversal App - Designed for iPhone and iPad A different take on the usual block dropping puzzle game, Puzzle Roo is quite pleasant.   | Read more »
Super Crossfire Re-Release Super Crossfi...
Super Crossfire Re-Release Super Crossfighter Coming Soon, Other Radiangames Titles Go 50% Off Posted by Ellis Spice on July 31st, 2014 [ | Read more »
Hexiled Review
Hexiled Review By Rob Thomas on July 31st, 2014 Our Rating: :: HEX SELLSUniversal App - Designed for iPhone and iPad In space, no one can hear you… spell? Hexiled is a neat concept for a word scramble puzzle, but it doesn’t go too... | Read more »
Summoners War: Sky Arena Passes 10 Milli...
Summoners War: Sky Arena Passes 10 Million Installs! Posted by Jessica Fisher on July 31st, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Deep Loot Review
Deep Loot Review By Jennifer Allen on July 31st, 2014 Our Rating: :: DIVE DEEPUniversal App - Designed for iPhone and iPad Dive deep in this fun explore-em-up that’s a little grind heavy but ultimately quite entertaining.   | Read more »
Despicable Me: Minion Rush is One Year O...
Despicable Me: Minion Rush is One Year Old, Gets its Biggest Update Yet Posted by Jennifer Allen on July 31st, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Fish & Shark Review
Fish & Shark Review By Jordan Minor on July 31st, 2014 Our Rating: :: FLAPPY FISHUniversal App - Designed for iPhone and iPad Fish & Shark’s beauty is only scale deep.   | Read more »

Price Scanner via MacPrices.net

Save up to $130 on an iPad mini with Apple re...
The Apple Store has Certified Refurbished 2nd generation iPad minis with Retina Displays available for up to $130 off the cost of new models, starting at $339. Apple’s one-year warranty is included... Read more
iPad Cannibalization Threat “Overblown”
Seeking Alpha’s Kevin Greenhalgh observes that while many commentators think Apple’s forthcoming 5.5-inch panel iPhone 6 will cannibalize iPad sales, in his estimation, these concerns are being... Read more
Primate Labs Releases July 2014 MacBook Pro P...
Primate Labs’ John Poole has posted Geekbench 3 results for most of the new MacBook Pro models that Apple released on Tuesday. Poole observes that overall performance improvements for the new MacBook... Read more
Apple Re-Releases Bugfixed MacBook Air EFI Fi...
Apple has posted a bugfixed version EFI Firmware Update 2.9 a for MacBook Air (Mid 2011) models. The update addresses an issue where systems may take longer to wake from sleep than expected, and... Read more
Save $50 on the 2.5GHz Mac mini, plus free sh...
B&H Photo has the 2.5GHz Mac mini on sale for $549.99 including free shipping. That’s $50 off MSRP, and B&H will also include a free copy of Parallels Desktop software. NY sales tax only. Read more
Save up to $140 on an iPad Air with Apple ref...
Apple is offering Certified Refurbished iPad Airs for up to $140 off MSRP. Apple’s one-year warranty is included with each model, and shipping is free. Stock tends to come and go with some of these... Read more
$250 price drop on leftover 15-inch Retina Ma...
B&H Photo has dropped prices on 2013 15″ Retina MacBook Pros by $250 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.3GHz Retina MacBook Pro: $2249, $250 off... Read more
More iPad Upgrade Musings – The ‘Book Mystiqu...
Much discussed recently, what with Apple reporting iPad sales shrinkage over two consecutive quarters, is that it had apparently been widely assumed that tablet users would follow a two-year hardware... Read more
13-inch 2.5GHz MacBook Pro on sale for $999,...
Best Buy has the 13″ 2.5GHz MacBook Pro available for $999.99 on their online store. Choose free shipping or free instant local store pickup (if available). Their price is $100 off MSRP. Price is... Read more
Save up to $300 on an iMac with Apple refurbi...
The Apple Store has Apple Certified Refurbished iMacs available for up to $300 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free. These are the best prices on... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Sr. Product Leader, *Apple* Store Apps - Ap...
**Job Summary** Imagine what you could do here. At Apple , great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring Read more
Sr Software Lead Engineer, *Apple* Online S...
Sr Software Lead Engineer, Apple Online Store Publishing Systems Keywords: Company: Apple Job Code: E3PCAK8MgYYkw Location (City or ZIP): Santa Clara Status: Full Read more
Sr Software Lead Engineer, *Apple* Online S...
Sr Software Lead Engineer, Apple Online Store Publishing Systems Keywords: Company: Apple Job Code: E3PCAK8MgYYkw Location (City or ZIP): Santa Clara Status: Full Read more
*Apple* Solutions Consultant (ASC) - Apple (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.