TweetFollow Us on Twitter

Mac in the Shell: Debugging Python with vim

Volume Number: 26
Issue Number: 01
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python with vim

IDE-like debugging without leaving vim

by Edward Marczak

Welcome

Mac in the Shell has been focusing on Python for the System Administrator over the last few months. Last month re-introduced the vi(m) text editor. This month, we're going to tie them together a bit. I've covered pdb - the python debugger - already. It's really handy to have a debugger at your disposal. However, pdb's output and display can get a little unwieldy. Never fret: Vim to the rescue! This article will show you how to get the latest MacVim and start using it to debug Python code.

Use the Source, Luke

First thing is first: this will all fail miserably with the version of vim that's built into the base OS. Best solution: use MacVim. MacVim comes built with Python, GUI support and more. There's a few ways to get MacVim, and I'll detail both.

First way is the easy way: download a binary release, typically of the latest snapshot. Do that directly from the MacVim project page, currently on Google Code:

http://code.google.com/p/macvim/

But wait, there's more! If you like to customize your experience, like I do, you'll want the source code. This is a multi-part step. First, you'll need to make sure that the (free) Apple developer tools are installed on your machine. (and honestly, what good is a machine without those?). You'll also need git installed to check out the MacVim source. If you're a fink or MacPorts person, you can use either of those systems to install git. If you're a pre-built person, or just need to be in this case, go pick up a pre-built git from:

http://code.google.com/p/git-osx-installer/

Once git is up and running, check out the MacVim source by changing to the destination directory and issuing a clone command:

git clone git://repo.or.cz/MacVim.git vim7

Once that completes, you'll need to configure and build MacVim. This is where you can customize the binary a bit. I want the whole experience, so, I change to the vim7 directory and run configure with my options:

./configure 
—enable-perlinterp —enable-pythoninterp 
—enable-rubyinterp 
—with-features=huge 
—enable-gui=macvim 
—with-tlib=ncurses 
—enable-multibyte —with-python-config-dir=
/System/Library/Frameworks/Python.framework/Versions/2.6//lib/python2.6/config/

No matter how you configure Vim, I'd highly recommend the -enable-gui option and, for this article, you must enable the python interpreter. Of course, you're reading an article about debugging python using Vim, so why wouldn't you enable this?

Once configure finishes its business (successfully), type make and...wait. Barring any errors, you'll find the MacVim application in the src/MacVim/build/Release directory. If you're following this method, when you want to 'upgrade' (read: compile a new version), change into the vim7 directory and ask git to pull down the latest changes:

$ git pull

followed by make. Go pick up your fresh binary in the Release directory as before.

You may also want to copy the mvim script from the src/MacVim subdirectory and place it somewhere in your $PATH. Running mvim from the command-line will try to do the right thing: if you're logged in via ssh, you'll use console-based vim. If you have a GUI, you'll run GUI-based MacVim. You can also create alias or functions that call mvim as a different name. This will cause Vim to run in different modes. I have the following in my ~/.bash_profile initialization script:

export VIM_APP_DIR=${HOME}/Applications/MacVim/
vi() { ${HOME}/Applications/MacVim/mvim ${@}; }
vim() { ${HOME}/Applications/MacVim/mvim ${@}; }
gvim () { ${HOME}/Applications/MacVim/mvim ${@}; }
vimdiff() { ${HOME}/Applications/MacVim/mvim -d ${@}; }
export -f vi
export -f vim
export -f gvim
export -f vimdiff

I use bash functions rather than aliases as there are several problems with using aliases in this case. Primarily, a bash alias will fork off a new shell and there's too much juggling around of variables. Bash aliases have been deprecated in favor of bash functions in any case.

In my example .bash_profile snippet, I explicitly define where MacVim lives as I keep it in my home directory. Then I define the four ways I may call vi(m). One of the most important to me is the definition for vimdiff. When integrating with version control programs, I want to make sure it doesn't fork off, and it should come back after an individual window is closed. Once these are defined, I separately do the same for gvim and git integration if I'm logged in to a GUI session with:

export GIT_EDITOR='gvim -f -c "au VimLeave * !open -a Terminal"'

The final group of exports causes functions to be exported to subshells.

One last note: no matter how you choose to call vim, ensure that your method gets found before the system vi(m). Keep it early in your path, or use my method above, as functions are always used before searching $PATH.

The Setup

The entire point of the earlier Vim exercise is to ensure that you are using a Python-enabled version of vim. The vim binary that ships with OS X is not compiled with the Python, or any scripting option. Once you've downloaded or compiled MacVim and have it in place, launch it and test for Python support. Enter ex mode by typing a colon (":") followed by py and a python command. Something like:

:py print 6*7

You'll see the result on the command line. Not exceptionally exiting, perhaps...not at the outset, anyway. However, having Python support built in means that Python can be used to script just about anything. That is exciting! This is also, not coincidentally, how we're going to be able to debug python code entirely within vim. If you receive an error from 'py' - specifically "E319: Sorry, the command is not available in this version" - ensure that you're running the updated version of vim that you downloaded (or built!) and not the system vi(m).

Configuring Vim for Debugging

There are plenty of hoops one can jump through to debug directly in vim. My favorite, though, is a freely downloadable plugin. Go grab vimpdb from its project page:

http://code.google.com/p/vimpdb/

From there, click on the download link. Or, you can just grab the source directly using subversion:

svn checkout http://vimpdb.googlecode.com/svn/trunk/ vimpdb

Pleasantly, subversion is built into OS X as of 10.5, so, you can just run that command with no fuss.

However you've obtained vimpdb, you need to drop VimPdb.py and VimPdb.vim into your vim plugin directory (~/.vim/plugin). Now you're ready...maybe.

By default, vimpdb uses function keys for instructing it on what action to take. If you're using a 'real' keyboard, you can likely just use the default key bindings. By 'real' keyboard, I'm essentially talking about any stand-alone keyboard. It's the keyboards that are built into the MacBooks and MacBook Pro machines that are the problem. If you're like me, you'll occasionally want to debug on the go. The trouble starts with the way Apple overloads the function keys. Despite a setting in System Preferences that allows you to "use all" function keys "as standard function keys," this doesn't do what you expect initially. You'll still need to undo those key settings in the "Keyboard Shortcuts" section of the Keyboard pref pane. It's about picking your battles, I suppose. However, those shortcuts are really nice to have. (I know, this is a fairly lame argument, but, hey - I'm the end-user here and I demand satisfaction). I've experimented a bit with creating a wrapper script for vim that swaps out the plist file that controls this (~/Library/Preferences/com.apple.symbolichotkeys.plist) with one that has no F-key definitions, but that causes other issues for me. Ultimately, I reached a compromise and only undo a few default keyboard shortcuts: gone are ^F2, ^F4 and ^F8. So, let's go set up our vim key mappings.


Figure 1 - Assigning - or unassigning - keyboard shortcuts.

Again, if you want to leave the default keys for vimpdb and disable the OS X integrated keys in the Keyboard pref pane, feel free. For this article, I'm going to make some very simple changes, but you can set up the keys as you like. Open up the vimpdb.vim file that you just placed in your ~/.vim/plugin directory (using Vim, of course!). Move to line 540 (in Vim, type "540 G"). I've altered these lines to map F3 and F4 to step into and step over, respectively. They now read:

   " Was F7 and F8
   map <buffer> <silent> <F3> :call PdbStepInto()<CR>
   map <buffer> <silent> <F4> :call PdbStepOver()<CR>

Also, I've needed to change out the places that F3 and F4 were already used. Scroll down to line 560 or so, and make these changes:

   " Was F4
   map <buffer> <silent> <F7> :call PdbEvalCurrentWord()<CR>
   map <buffer> <silent> <C-F7> :call PdbEvalCurrentWORD()<CR>
   " Was F3
   map <buffer> <silent> <F8> :call PdbEvalExpression()<CR>

Save the file, and you're ready.

Lock and Load

The default vimpdb key binding along with the changes we've made leave us with the following keys for controlling a debug session:

Start and stop debugging

- F5 - Start/continue debug session of current file.

- Ctrl-F5 - Start debugging and do not pause at first line

- Ctrl-Shift-F5 - Start debugging with a given list of parameters.

- Shift-F5 - Stop the current debug session.

- Ctrl-Alt-Shift-F5 - Restart the current debug session.

Breakpoints

- F2 - Toggle breakpoint.

- Ctrl-F2 - Toggle conditional breakpoint

- Shift-F2 - Toggle temporary breakpoint

- Ctrl-Shift-F2 - Clear all breakpoints in current file

- Ctrl-Alt-Shift-F2 - Clear all breakpoints in all files

- F11 - Print condition of conditional breakpoint under the cursor

Step/continue

- F3 - Step into

- F4 - Step over

- Ctrl-F4 - Continue running until reaching a return from function

Cursor movement

- F6 - Move cursor to currently debugged line.

- Ctrl-F6 - Change current debugged line to where the cursor is currently placed.

Stack

- F9 - Move up in stack frame.

- F10 - Move down in stack frame.

- F12 - Print stack trace

Evaluate/Execute

- F7 - Evaluate a given expression (in the current debug context)

- Ctrl-F7 - Exec a given statement (in the current debug context)

- F8 - Evaluate the current word under the cursor (in the current debug context)

- Ctrl-F8 - Evaluate the current WORD under the cursor (in the current debug context)

Again, depending on the keyboard you use and the setting you have in Keyboard Shortcuts, some of these key mappings may (will?) have conflicts.

So, what does all of this get you? Figure 2 should give you an idea.


Figure 2 - VimPDB in action.

Unlike the pure command-line environment of pdb, vimpdb lets you keep your code in view and step along with it as it runs. Visually. In figure 2, line 3 - highlighted in green - is the current line. Line 9 - highlighted in red - has a breakpoint set on it. You can use any Python code that you like. I'm going to use the code that I offered up in my article on PyObjC from August 2009 that reads the Apple Address Book.

Debugging

Once you're set up, the debugging is fairly easy. Load the python code that you'd like to debug into the current buffer and press F5. This begins the debugger, and vim should print a status message that says "Starting debugger." You should also now see your first executable line highlighted in green. Now, the "first executable" line is the first thing that the parser needs to look at, not necessarily what you'd expect as "executable."

To step through the program, you can press F4 to step 1 instruction. This will step over any intervening code between the current line and what is visible as next. The current line will execute, and the green line will advance. If the current line is a function call that you want to inspect, press F3 to step into the function. You'll need to do this at least once if you use the Python convention of creating a main() function and calling that conditionally:

if __name__ == '__main__':
  main()

Once you're inside a function, often, you'll get to the part you're interested in, and then not want to have to step through the remainder of the function. You can accomplish that by pressing control-F4.

The great thing about using a debugger is not only can you step through and watch the flow of your code, but you can inspect variables as they change, too. Vimpdb doesn't let us down here. Position the cursor over the variable name you want to inspect and press F7. The value of the variable is displayed in the output area at the bottom of the vim screen followed by a note to press any key to return to debugging.

For example, if I were to press F7 over the "abgroups" variable showing in Figure 2, I receive this output:

 (
    "ABGroup (0x10835cb40) {\n\tCreation     : 2009-07-14 15:13:36 
    -0400\n\tGroupName    : WS\n\tModification : 2009-10-27 19:24:14 
    -0400\n\tUnique ID    : 8BBAEDE9-0D31-4B48-A967-826523BF9DA7:ABGroup\n}",
    "ABGroup (0x1006f3ab0) {\n\tCreation     : 2009-01-14 15:13:36 
    -0400\n\tGroupName    : MacTech Editorial\n\tMembers    
    : 28 members\n\tModification : 2009-11-23 08:04:24 
    -0500\n\tUnique ID    : 5712EB9A-9743-4744-BA05-354726896614:ABGroup\n}"
)

Note that variables you inspect must be defined. If the current line contains a variable that hasn't been defined yet, you can't inspect that variable until the line has been executed. If you try, you'll receive a NameError:

NameError: Name 'blah' is not defined.

Just ensure that the variable you're trying to inspect is defined. If it's first defined on the current line, execute the current line first.

Let's look at a typical short debug session. First, you'd open vim and load your code into the buffer:

vim ~/dev/py/address_book.py

(or, open vim and type :e ~/dev/py/address_book.py).

Start the debug session by pressing F5 to initialize the debugger and start the session. You should note the "starting debugger" message. No other vimpdb keys will do anything until this step is taken.

You can navigate down to the call to main() and set a breakpoint - using F2 while the cursor is on the line with said call. Once set, press F5 to run the program as usual until it hits your break point. If you stopped on main(), press F4 to step into the main function. Execute a line that assigns a value to a variable, cursor up to the variable name and press F7 to inspect the value that was assigned; it'll appear at the bottom of the vim window. Press F3 to execute the current line.

Rinse, repeat.

One extra nice feature of VimPDB: the ability to save and load breakpoints. In a larger program, you may set several breakpoints in different locations that you want to recall after quitting vim and then trying to debug again. While in debug mode with all of the breakpoints set, type \s (that's backslash, and then s). You'll be prompted for a filename. Type one and press return. When you later come back to vim and load your project, start the debugger and type \l (backslash, then lowercase L, for "load").

In Closing

VimPDB provides the best of all worlds for debugging Python code. It doesn't make you change code to drop in a bunch of print or logging statements. It gives you a larger view of your code as it's running. Finally, it can easily be run remotely over an ssh session (win!).

Also, VimPDB is pretty easy in general, and a great addition to your toolkit. This article focused mainly on the preparation: install and introduction. Once you start using it, it becomes second nature. Unfortunately, unless you wildly remap the default keys, you'll need to choose between OS X functions on the F-keys and bindings in Vim (and yes, OS X "wins"). This is of particular annoyance on a MacBook/Pro, but not insurmountable.

Media of the month: No better time to recommend "Learning the vi and Vim Editors" by Arnold Robbins, Elbert Hannah, and Linda Lamb. This is a pretty comprehensive look at learning and customizing vi(m). Available in dead-tree and electronic versions.

Hope to see everyone in San Francisco for Macworld!


Ed Marczak is the Executive Editor of MacTech Magazine. He has written the Mac in the Shell column since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

NTFS 14.3.318 - $19.95
This latest version supports the new macOS 10.12 Sierra! NTFS breaks down the barriers between Windows and OS X. Paragon NTFS effectively solves the communication problems between the Mac system and... Read more
iFFmpeg 6.2.2 - Convert multimedia files...
iFFmpeg is a comprehensive media tool to convert movie, audio and media files between formats. The FFmpeg command line instructions can be very hard to master/understand, so iFFmpeg does all the hard... Read more
ForeverSave 2.1.6 - Universal auto-save...
ForeverSave auto-saves all documents you're working on while simultaneously doing backup versioning in the background. Lost data can be quickly restored at any time. Define your preferred time... Read more
BetterTouchTool 1.961 - Customize Multi-...
BetterTouchTool adds many new, fully customizable gestures to the Magic Mouse, Multi-Touch MacBook trackpad, and Magic Trackpad. These gestures are customizable: Magic Mouse: Pinch in / out (zoom... Read more
EtreCheck 3.1.4 - For troubleshooting yo...
EtreCheck is an app that displays the important details of your system configuration and allow you to copy that information to the Clipboard. It is meant to be used with Apple Support Communities to... Read more
Together 3.7 - Store and organize all of...
Together helps you organize your Mac, giving you the ability to store, edit and preview your files in a single clean, uncluttered interface. Features Smart storage. With simple drag-and-drop... Read more
Together 3.7 - Store and organize all of...
Together helps you organize your Mac, giving you the ability to store, edit and preview your files in a single clean, uncluttered interface. Features Smart storage. With simple drag-and-drop... Read more
EtreCheck 3.1.4 - For troubleshooting yo...
EtreCheck is an app that displays the important details of your system configuration and allow you to copy that information to the Clipboard. It is meant to be used with Apple Support Communities to... Read more
Postbox 5.0.9 - Powerful and flexible em...
Postbox is a new email application that helps you organize your work life and get stuff done. It has all the elegance and simplicity of Apple Mail, but with more power and flexibility to manage even... Read more
DiskCatalogMaker 6.5.16 - Catalog your d...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast. Finder-like intuitive look and feel. Super-fast search algorithm. Can compress catalog data... Read more

Latest Forum Discussions

See All

Clash of Clans is getting its own animat...
Riding on its unending wave of fame and success, Clash of Clans is getting an animated web series based on its Clash-A-Rama animated shorts.As opposed to the current shorts' 60 second run time, the new and improved Clash-A-Rama will be comprised of... | Read more »
Leaks hint at Pokémon GO and Starbucks C...
Leaked images from a hub for Starbucks employees suggests that a big Pokémon GO event with the coffee giant could begin this very week. The images appeared on Reddit and hint at some exciting new things to come for Niantic's smash hit game. | Read more »
Silent Depth Submarine Simulation (Game...
Silent Depth Submarine Simulation 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: | Read more »
Enneas Saga lets you lead your own demon...
Defend the land of Enneas Continent from the forces of evil in the new fantasy MMORPG from Lyto Mobi: Enneas Saga. Can’t wait? No problem. It’s available to download now on Android devices. | Read more »
Great zombie games in the spirit of Dead...
Dead Rising 4 arrives tomorrow, giving enthusiasts a fresh chance to take selfies with zombies and get up to other ridiculous end-of-the-world shenanigans. To really get into the spirit of things, we've gone and gathered the best zombie games that... | Read more »
Amateur Surgeon 4 Guide: Advanced tips a...
Amateur Surgeon 4 is still tackling the competition at the top of the App Store charts, so if you haven't tried it out yet, you should probably do that right away. If you've been at it for a while, though, perhaps you're ready to start expanding... | Read more »
Amateur Surgeon 4 Guide: Become the worl...
It's time to wield your trusty pizza cutter again, as Amateur Surgeon has returned with a whole fresh set of challenges (and some old, familiar ones, too). Starting anew isn't easy, especially when all you have at your disposal is a lighter, the... | Read more »
Le Parker: Sous Chef Extraordinaire (Ga...
Le Parker: Sous Chef Extraordinaire 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: | Read more »
Telltale Games really is working on a Gu...
Telltale Games' next episodic adventure is indeed Guardians of the Galaxy. A document tied to the voice actors strike suggested that the project was in the work, but now we have direct confirmation following an announcement at the Game Awards that... | Read more »
Amateur Surgeon returns to iOS and Andro...
Amateur Surgeon and its two sequels disappeared from the App Store some time and it was sad days for all. But now, just in time for the holidays, the Adult Swim favorite makes its joyous return in the shape of Amateur Surgeon 4, a remake with... | Read more »

Price Scanner via MacPrices.net

Back in stock! 13-inch 2.7GHz Retina MacBook...
Apple has Apple Certified Refurbished 2015 13″ 2.7GHz/128GB Retina MacBook Pros (MF839LL/A) available again for $1099 including free shipping. That’s $200 off MSRP, and it’s the lowest price... Read more
Slate Tablet Market Share to Fall Below 75% i...
After two years of decline, the tablet market is showing signs of new life as productivity trumps entertainment, and tablets become cost-effective computing devices for both entertainment and... Read more
Yostand Launches Indigogo campaign for iStand...
China-based startup Yostand (meaning ‘your stand’), has announced the launch of its Indigogo campaign for their newly awaited iStand7. This product is a one of a kind iPhone battery case that offers... Read more
Green App – Budget Forecasting Now Available...
Indianapolis, Indiana based CoopToons has announced the release of Green – Budget Forecasting 1.5, an update to their personal budgeting app developed exclusively for iOS devices. Green aims to be a... Read more
New 2016 13-inch 2.0GHz MacBook Pros in stock...
Overstock.com has the non-Touch Bar 13″ MacBook Pros in stock today for $150 off MSRP. Shipping is free: - 13″ 2.0GHz MacBook Pro Space Gray (MLL42LL/A): $1349.99 $150 off MSRP - 13″ 2.0GHz MacBook... Read more
15-inch 2.6GHz Silver Touch Bar MacBook Pro o...
Adorama has the new 2016 15″ 2.6GHz Silver Touch Bar MacBook Pro (MLW72LL/A) in stock and available for $2349 including free shipping. Adorama charges sales tax in NY & NJ only. Their price is $... Read more
13-inch MacBook Airs on sale for up to $180 o...
Overstock.com has 13″ MacBook Airs on sale for up to $180 off MSRP including free shipping: - 13″ 1.6GHz/128GB MacBook Air (MMGF2LL/A): $869.99 $130 off MSRP - 13″ 1.6GHz/256GB MacBook Air (sku... Read more
13-inch 2.5GHz MacBook Pro (Apple refurbished...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros (MD101LL/A) available for $829, or $270 off original MSRP. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook... Read more
Monday roundup of Holiday Mac sales: Up to $3...
Take up to $300 off MSRP on the price of a new Apple Mac at B&H Photo today as part of their Holiday sale. Shipping is free, and B&H charges NY sales tax only. Touch Bar MacBook Pros are in... Read more
12-inch WiFi Apple iPad Pros on sale for up t...
B&H Photo has 12″ WiFi Apple iPad Pros on sale for up to $50 off MSRP, each including free shipping. B&H charges sales tax in NY only: - 12″ Space Gray 32GB WiFi iPad Pro: $749 $50 off MSRP... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Lead *Apple* Solutions Consultant - Apple (...
# Lead Apple Solutions Consultant Job Number: 53586123 Pittsburgh, Pennsylvania, United States Posted: Nov. 28, 2016 Weekly Hours: 40.00 **Job Summary** The Lead ASC Read more
*Apple* Retail - Multiple Positions- Plano,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions- Kansas...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions- Chicago...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.