TweetFollow Us on Twitter

Mac in the Shell: Debugging Python with vim

Volume Number: 26
Issue Number: 01
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python with vim

IDE-like debugging without leaving vim

by Edward Marczak

Welcome

Mac in the Shell has been focusing on Python for the System Administrator over the last few months. Last month re-introduced the vi(m) text editor. This month, we're going to tie them together a bit. I've covered pdb - the python debugger - already. It's really handy to have a debugger at your disposal. However, pdb's output and display can get a little unwieldy. Never fret: Vim to the rescue! This article will show you how to get the latest MacVim and start using it to debug Python code.

Use the Source, Luke

First thing is first: this will all fail miserably with the version of vim that's built into the base OS. Best solution: use MacVim. MacVim comes built with Python, GUI support and more. There's a few ways to get MacVim, and I'll detail both.

First way is the easy way: download a binary release, typically of the latest snapshot. Do that directly from the MacVim project page, currently on Google Code:

http://code.google.com/p/macvim/

But wait, there's more! If you like to customize your experience, like I do, you'll want the source code. This is a multi-part step. First, you'll need to make sure that the (free) Apple developer tools are installed on your machine. (and honestly, what good is a machine without those?). You'll also need git installed to check out the MacVim source. If you're a fink or MacPorts person, you can use either of those systems to install git. If you're a pre-built person, or just need to be in this case, go pick up a pre-built git from:

http://code.google.com/p/git-osx-installer/

Once git is up and running, check out the MacVim source by changing to the destination directory and issuing a clone command:

git clone git://repo.or.cz/MacVim.git vim7

Once that completes, you'll need to configure and build MacVim. This is where you can customize the binary a bit. I want the whole experience, so, I change to the vim7 directory and run configure with my options:

./configure 
—enable-perlinterp —enable-pythoninterp 
—enable-rubyinterp 
—with-features=huge 
—enable-gui=macvim 
—with-tlib=ncurses 
—enable-multibyte —with-python-config-dir=
/System/Library/Frameworks/Python.framework/Versions/2.6//lib/python2.6/config/

No matter how you configure Vim, I'd highly recommend the -enable-gui option and, for this article, you must enable the python interpreter. Of course, you're reading an article about debugging python using Vim, so why wouldn't you enable this?

Once configure finishes its business (successfully), type make and...wait. Barring any errors, you'll find the MacVim application in the src/MacVim/build/Release directory. If you're following this method, when you want to 'upgrade' (read: compile a new version), change into the vim7 directory and ask git to pull down the latest changes:

$ git pull

followed by make. Go pick up your fresh binary in the Release directory as before.

You may also want to copy the mvim script from the src/MacVim subdirectory and place it somewhere in your $PATH. Running mvim from the command-line will try to do the right thing: if you're logged in via ssh, you'll use console-based vim. If you have a GUI, you'll run GUI-based MacVim. You can also create alias or functions that call mvim as a different name. This will cause Vim to run in different modes. I have the following in my ~/.bash_profile initialization script:

export VIM_APP_DIR=${HOME}/Applications/MacVim/
vi() { ${HOME}/Applications/MacVim/mvim ${@}; }
vim() { ${HOME}/Applications/MacVim/mvim ${@}; }
gvim () { ${HOME}/Applications/MacVim/mvim ${@}; }
vimdiff() { ${HOME}/Applications/MacVim/mvim -d ${@}; }
export -f vi
export -f vim
export -f gvim
export -f vimdiff

I use bash functions rather than aliases as there are several problems with using aliases in this case. Primarily, a bash alias will fork off a new shell and there's too much juggling around of variables. Bash aliases have been deprecated in favor of bash functions in any case.

In my example .bash_profile snippet, I explicitly define where MacVim lives as I keep it in my home directory. Then I define the four ways I may call vi(m). One of the most important to me is the definition for vimdiff. When integrating with version control programs, I want to make sure it doesn't fork off, and it should come back after an individual window is closed. Once these are defined, I separately do the same for gvim and git integration if I'm logged in to a GUI session with:

export GIT_EDITOR='gvim -f -c "au VimLeave * !open -a Terminal"'

The final group of exports causes functions to be exported to subshells.

One last note: no matter how you choose to call vim, ensure that your method gets found before the system vi(m). Keep it early in your path, or use my method above, as functions are always used before searching $PATH.

The Setup

The entire point of the earlier Vim exercise is to ensure that you are using a Python-enabled version of vim. The vim binary that ships with OS X is not compiled with the Python, or any scripting option. Once you've downloaded or compiled MacVim and have it in place, launch it and test for Python support. Enter ex mode by typing a colon (":") followed by py and a python command. Something like:

:py print 6*7

You'll see the result on the command line. Not exceptionally exiting, perhaps...not at the outset, anyway. However, having Python support built in means that Python can be used to script just about anything. That is exciting! This is also, not coincidentally, how we're going to be able to debug python code entirely within vim. If you receive an error from 'py' - specifically "E319: Sorry, the command is not available in this version" - ensure that you're running the updated version of vim that you downloaded (or built!) and not the system vi(m).

Configuring Vim for Debugging

There are plenty of hoops one can jump through to debug directly in vim. My favorite, though, is a freely downloadable plugin. Go grab vimpdb from its project page:

http://code.google.com/p/vimpdb/

From there, click on the download link. Or, you can just grab the source directly using subversion:

svn checkout http://vimpdb.googlecode.com/svn/trunk/ vimpdb

Pleasantly, subversion is built into OS X as of 10.5, so, you can just run that command with no fuss.

However you've obtained vimpdb, you need to drop VimPdb.py and VimPdb.vim into your vim plugin directory (~/.vim/plugin). Now you're ready...maybe.

By default, vimpdb uses function keys for instructing it on what action to take. If you're using a 'real' keyboard, you can likely just use the default key bindings. By 'real' keyboard, I'm essentially talking about any stand-alone keyboard. It's the keyboards that are built into the MacBooks and MacBook Pro machines that are the problem. If you're like me, you'll occasionally want to debug on the go. The trouble starts with the way Apple overloads the function keys. Despite a setting in System Preferences that allows you to "use all" function keys "as standard function keys," this doesn't do what you expect initially. You'll still need to undo those key settings in the "Keyboard Shortcuts" section of the Keyboard pref pane. It's about picking your battles, I suppose. However, those shortcuts are really nice to have. (I know, this is a fairly lame argument, but, hey - I'm the end-user here and I demand satisfaction). I've experimented a bit with creating a wrapper script for vim that swaps out the plist file that controls this (~/Library/Preferences/com.apple.symbolichotkeys.plist) with one that has no F-key definitions, but that causes other issues for me. Ultimately, I reached a compromise and only undo a few default keyboard shortcuts: gone are ^F2, ^F4 and ^F8. So, let's go set up our vim key mappings.


Figure 1 - Assigning - or unassigning - keyboard shortcuts.

Again, if you want to leave the default keys for vimpdb and disable the OS X integrated keys in the Keyboard pref pane, feel free. For this article, I'm going to make some very simple changes, but you can set up the keys as you like. Open up the vimpdb.vim file that you just placed in your ~/.vim/plugin directory (using Vim, of course!). Move to line 540 (in Vim, type "540 G"). I've altered these lines to map F3 and F4 to step into and step over, respectively. They now read:

   " Was F7 and F8
   map <buffer> <silent> <F3> :call PdbStepInto()<CR>
   map <buffer> <silent> <F4> :call PdbStepOver()<CR>

Also, I've needed to change out the places that F3 and F4 were already used. Scroll down to line 560 or so, and make these changes:

   " Was F4
   map <buffer> <silent> <F7> :call PdbEvalCurrentWord()<CR>
   map <buffer> <silent> <C-F7> :call PdbEvalCurrentWORD()<CR>
   " Was F3
   map <buffer> <silent> <F8> :call PdbEvalExpression()<CR>

Save the file, and you're ready.

Lock and Load

The default vimpdb key binding along with the changes we've made leave us with the following keys for controlling a debug session:

Start and stop debugging

- F5 - Start/continue debug session of current file.

- Ctrl-F5 - Start debugging and do not pause at first line

- Ctrl-Shift-F5 - Start debugging with a given list of parameters.

- Shift-F5 - Stop the current debug session.

- Ctrl-Alt-Shift-F5 - Restart the current debug session.

Breakpoints

- F2 - Toggle breakpoint.

- Ctrl-F2 - Toggle conditional breakpoint

- Shift-F2 - Toggle temporary breakpoint

- Ctrl-Shift-F2 - Clear all breakpoints in current file

- Ctrl-Alt-Shift-F2 - Clear all breakpoints in all files

- F11 - Print condition of conditional breakpoint under the cursor

Step/continue

- F3 - Step into

- F4 - Step over

- Ctrl-F4 - Continue running until reaching a return from function

Cursor movement

- F6 - Move cursor to currently debugged line.

- Ctrl-F6 - Change current debugged line to where the cursor is currently placed.

Stack

- F9 - Move up in stack frame.

- F10 - Move down in stack frame.

- F12 - Print stack trace

Evaluate/Execute

- F7 - Evaluate a given expression (in the current debug context)

- Ctrl-F7 - Exec a given statement (in the current debug context)

- F8 - Evaluate the current word under the cursor (in the current debug context)

- Ctrl-F8 - Evaluate the current WORD under the cursor (in the current debug context)

Again, depending on the keyboard you use and the setting you have in Keyboard Shortcuts, some of these key mappings may (will?) have conflicts.

So, what does all of this get you? Figure 2 should give you an idea.


Figure 2 - VimPDB in action.

Unlike the pure command-line environment of pdb, vimpdb lets you keep your code in view and step along with it as it runs. Visually. In figure 2, line 3 - highlighted in green - is the current line. Line 9 - highlighted in red - has a breakpoint set on it. You can use any Python code that you like. I'm going to use the code that I offered up in my article on PyObjC from August 2009 that reads the Apple Address Book.

Debugging

Once you're set up, the debugging is fairly easy. Load the python code that you'd like to debug into the current buffer and press F5. This begins the debugger, and vim should print a status message that says "Starting debugger." You should also now see your first executable line highlighted in green. Now, the "first executable" line is the first thing that the parser needs to look at, not necessarily what you'd expect as "executable."

To step through the program, you can press F4 to step 1 instruction. This will step over any intervening code between the current line and what is visible as next. The current line will execute, and the green line will advance. If the current line is a function call that you want to inspect, press F3 to step into the function. You'll need to do this at least once if you use the Python convention of creating a main() function and calling that conditionally:

if __name__ == '__main__':
  main()

Once you're inside a function, often, you'll get to the part you're interested in, and then not want to have to step through the remainder of the function. You can accomplish that by pressing control-F4.

The great thing about using a debugger is not only can you step through and watch the flow of your code, but you can inspect variables as they change, too. Vimpdb doesn't let us down here. Position the cursor over the variable name you want to inspect and press F7. The value of the variable is displayed in the output area at the bottom of the vim screen followed by a note to press any key to return to debugging.

For example, if I were to press F7 over the "abgroups" variable showing in Figure 2, I receive this output:

 (
    "ABGroup (0x10835cb40) {\n\tCreation     : 2009-07-14 15:13:36 
    -0400\n\tGroupName    : WS\n\tModification : 2009-10-27 19:24:14 
    -0400\n\tUnique ID    : 8BBAEDE9-0D31-4B48-A967-826523BF9DA7:ABGroup\n}",
    "ABGroup (0x1006f3ab0) {\n\tCreation     : 2009-01-14 15:13:36 
    -0400\n\tGroupName    : MacTech Editorial\n\tMembers    
    : 28 members\n\tModification : 2009-11-23 08:04:24 
    -0500\n\tUnique ID    : 5712EB9A-9743-4744-BA05-354726896614:ABGroup\n}"
)

Note that variables you inspect must be defined. If the current line contains a variable that hasn't been defined yet, you can't inspect that variable until the line has been executed. If you try, you'll receive a NameError:

NameError: Name 'blah' is not defined.

Just ensure that the variable you're trying to inspect is defined. If it's first defined on the current line, execute the current line first.

Let's look at a typical short debug session. First, you'd open vim and load your code into the buffer:

vim ~/dev/py/address_book.py

(or, open vim and type :e ~/dev/py/address_book.py).

Start the debug session by pressing F5 to initialize the debugger and start the session. You should note the "starting debugger" message. No other vimpdb keys will do anything until this step is taken.

You can navigate down to the call to main() and set a breakpoint - using F2 while the cursor is on the line with said call. Once set, press F5 to run the program as usual until it hits your break point. If you stopped on main(), press F4 to step into the main function. Execute a line that assigns a value to a variable, cursor up to the variable name and press F7 to inspect the value that was assigned; it'll appear at the bottom of the vim window. Press F3 to execute the current line.

Rinse, repeat.

One extra nice feature of VimPDB: the ability to save and load breakpoints. In a larger program, you may set several breakpoints in different locations that you want to recall after quitting vim and then trying to debug again. While in debug mode with all of the breakpoints set, type \s (that's backslash, and then s). You'll be prompted for a filename. Type one and press return. When you later come back to vim and load your project, start the debugger and type \l (backslash, then lowercase L, for "load").

In Closing

VimPDB provides the best of all worlds for debugging Python code. It doesn't make you change code to drop in a bunch of print or logging statements. It gives you a larger view of your code as it's running. Finally, it can easily be run remotely over an ssh session (win!).

Also, VimPDB is pretty easy in general, and a great addition to your toolkit. This article focused mainly on the preparation: install and introduction. Once you start using it, it becomes second nature. Unfortunately, unless you wildly remap the default keys, you'll need to choose between OS X functions on the F-keys and bindings in Vim (and yes, OS X "wins"). This is of particular annoyance on a MacBook/Pro, but not insurmountable.

Media of the month: No better time to recommend "Learning the vi and Vim Editors" by Arnold Robbins, Elbert Hannah, and Linda Lamb. This is a pretty comprehensive look at learning and customizing vi(m). Available in dead-tree and electronic versions.

Hope to see everyone in San Francisco for Macworld!


Ed Marczak is the Executive Editor of MacTech Magazine. He has written the Mac in the Shell column since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Fantastical 2.4.1 - Create calendar even...
Fantastical 2 is the Mac calendar you'll actually enjoy using. Creating an event with Fantastical is quick, easy, and fun: Open Fantastical with a single click or keystroke Type in your event... Read more
Live Home 3D Pro 3.2.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
Live Home 3D Pro 3.2.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
FileZilla 3.27.0.1 - Fast and reliable F...
FileZilla (ported from Windows) is a fast and reliable FTP client and server with lots of useful features and an intuitive interface. Version 3.27.0.1: MSW: Add misssing file to .zip binary package... Read more
Spotify 1.0.59.395. - Stream music, crea...
Spotify is a streaming music service that gives you on-demand access to millions of songs. Whether you like driving rock, silky R&B, or grandiose classical music, Spotify's massive catalogue puts... Read more
Sierra Cache Cleaner 11.0.6 - Clear cach...
Sierra Cache Cleaner is an award-winning general purpose tool for macOS X. SCC makes system maintenance simple with an easy point-and-click interface to many macOS X functions. Novice and expert... Read more
DiskCatalogMaker 7.1.2 - Catalog your di...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast Finder-like intuitive look and feel Super-fast search algorithm Can compress catalog data for... Read more
Live Home 3D Pro 3.1.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
Deeper 2.2.1 - Enable hidden features in...
Deeper is a personalization utility for macOS which allows you to enable and disable the hidden functions of the Finder, Dock, QuickTime, Safari, iTunes, login window, Spotlight, and many of Apple's... Read more
Pinegrow 3.04 - Mockup and design webpag...
Pinegrow (was Pinegrow Web Designer) is desktop app that lets you mockup and design webpages faster with multi-page editing, CSS and LESS styling, and smart components for Bootstrap, Foundation,... Read more

Latest Forum Discussions

See All

The best deals on the App Store this wee...
There are quite a few truly superb games on sale on the App Store this week. If you haven't played some of these, many of which are true classics, now's the time to jump on the bandwagon. Here are the deals you need to know about. [Read more] | Read more »
Realpolitiks Mobile (Games)
Realpolitiks Mobile 1.0 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0 (iTunes) Description: PLEASE NOTE: The game might not work properly on discontinued 1GB of RAM devices (iPhone 5s, iPhone 6, iPhone 6 Plus, iPad... | Read more »
Layton’s Mystery Journey (Games)
Layton’s Mystery Journey 1.0.0 Device: iOS Universal Category: Games Price: $15.99, Version: 1.0.0 (iTunes) Description: THE MUCH-LOVED LAYTON SERIES IS BACK WITH A 10TH ANNIVERSARY INSTALLMENT! Developed by LEVEL-5, LAYTON’S... | Read more »
Full Throttle Remastered (Games)
Full Throttle Remastered 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Originally released by LucasArts in 1995, Full Throttle is a classic graphic adventure game from industry legend Tim... | Read more »
Stunning shooter Morphite gets a new tra...
Morphite is officially landing on iOS in September. The game looks like the space shooter we've been needing on mobile, and we're going to see if it fits the bill quite shortly. The game's a collaborative effort between Blowfish Studios, We're Five... | Read more »
Layton's Mystery Journey arrives to...
As you might recall, Layton's Mystery Journey is headed to iOS and Android -- tomorrow! To celebrate the impending launch, Level-5's released a new trailer, complete with an adorable hamster. [Read more] | Read more »
Sidewords (Games)
Sidewords 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: Grab a cup of coffee and relax with Sidewords. Sidewords is part logic puzzle, part word game, all original. No timers. No... | Read more »
Noodlecake Games' 'Leap On!...
Noodlecake Games is always good for some light-hearted arcade fun, and its latest project, Leap On! could carry on that tradition. It's a bit like high stakes tetherball in a way. Your job is to guide a cute little blob around a series of floating... | Read more »
RuneScape goes mobile later this year
Yes, RuneScape still exists. In fact, it's coming to iOS and Android in just a few short months. Jagex, creators of the hit fantasy MMORPG of yesteryear, is releasing RuneScape Mobile and Old School RuneScape for mobile devices, complete with... | Read more »
Crash of Cars wants you to capture the c...
Crash of Cars is going full on medieval in its latest update, introducing castles and all manner of new cars and skins fresh from the Dark Ages. The update introduces a new castle-themed map (complete with catapults) and a gladiator-style battle... | Read more »

Price Scanner via MacPrices.net

13″ 2.3GHz/128GB Space Gray MacBook Pro on sa...
MacMall has the 13″ 2.3GHz/128GB Space Gray MacBook Pro (MPXQ2LL/A) on sale for $1219 including free shipping. Their price is $80 off MSRP. Read more
Clearance 2016 12-inch Retina MacBooks, Apple...
Apple recently dropped prices on Certified Refurbished 2016 12″ Retina MacBooks, with models now available starting at $1019. Apple will include a standard one-year warranty with each MacBook, and... Read more
Save or Share
FotoJet Designer, is a simple but powerful new graphic design apps available on both Mac and Windows. With FotoJet Designer’s 900+ templates, thousands of resources, and powerful editing tools you... Read more
Logo Maker Shop iOS App Lets Businesses Get C...
A newly released app is designed to help business owners to get creative with their branding by designing their own logos. With more than 1,000 editable templates, Logo Maker Shop 1.0 provides the... Read more
Sale! New 15-inch MacBook Pros for up to $150...
Amazon has the new 2017 15″ MacBook Pros on sale for up to $150 off MSRP including free shipping: – 15″ 2.8GHz MacBook Pro Space Gray: $2249 $150 off MSRP – 15″ 2.89Hz MacBook Pro Space Gray: $2779 $... Read more
DEVONthink To Go 2.1.7 For iOS Brings Usabili...
DEVONtechnologies has updated DEVONthink To Go, the iOS companion to DEVONthink for Mac, with enhancements and bug fixes. Version 2.1.7 adds an option to clear the Global Inbox and makes the grid... Read more
15-inch 2.2GHz Retina MacBook Pro, Apple refu...
Apple has Certified Refurbished 2015 15″ 2.2GHz Retina MacBook Pros available for $1699. That’s $300 off MSRP, and it’s the lowest price available for a 15″ MacBook Pro. An Apple one-year warranty is... Read more
13-inch 2.3GHz Silver MacBook Pro on sale for...
B&H Photo has the new 2017 13″ 2.3GHz/256GB Silver MacBook Pro (MPXU2LL/A) on sale for $1399 including free shipping plus NY & NJ sales tax only. Their price is $100 off MSRP. Read more
Apple Tackles Distracted Driving With iOS 11...
One of the most important new features coming in iOS 11 is Do Not Disturb while driving, intended to help drivers stay more focused on the road. With Do Not Disturb while driving, your iPhone can... Read more
iMazing Mini for Mac: Free Automatic and Priv...
Geneva, Switzerland-based indie developer DigiDNA has released iMazing Mini, their free macOS utility designed to automatically back up iOS devices over any local Wi-Fi network. The app offers users... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple...
SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions - Apple...
SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Senior Payments Architect - *Apple* Pay - A...
Changing the world is all in a day's work at Apple . If you love innovation, here's your chance to make a career of it. You'll work hard. But the job comes with more Read more
Frameworks Engineering Manager, *Apple* Wat...
Frameworks Engineering Manager, Apple Watch Job Number: 41632321 Santa Clara Valley, California, United States Posted: Jun. 15, 2017 Weekly Hours: 40.00 Job Summary Read more
Manager, *Apple* Media Products - Apple Inc...
Job Summary The Apple Media Products Discovery, Fraud and Abuse team is responsible for protecting the integrity of Apple services. As a manager of the team, you Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.