Volume Number: 25
Issue Number: 09
Column Tag: Mac in the Shell

Mac in the Shell: Reading and Writing plist files with Python

Tame those pesky plists

by Edwarcd Marczak

Property list files, also known as 'plists,' are pervasive in OS X. This article teaches you the basic inner-workings of the plist format, system level methods of working with plist files and how to interact with these files using Python under OS X.

Plist files are structured XML (eXtensible Markup Language) files and easily understandable. Essentially, a plist file is a way to store standard types of data. By "standard," I mean string, integer, Boolean and so on, although there are ways to store arbitrary data as well. A plist file can easily be read into and written out from an NSDictionary object. Thanks to PyObj-C, an NSDictionary can be mapped onto and manipulated with a Python-based dictionary object.

Given the following dictionary:

{
    color:'blue',
    count:15,
    style:'fruit'
}

the plist in Listing 1 would be created

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Let's take a closer look at this plist. The header declares this file as an XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Ultimately, this header isn't up to you. In this article, you'll see that Apple's Cocoa APIs will properly generate this upon writing a plist. For more information about XML, see the specification page as http://xml.org, or the Wikipedia entry at http://en.wikipedia.org/wiki/Xml.

The plist tag wraps the entire file:

<plist version="1.0">

Again, Apple's APIs will write this out as appropriate. Next, we find a dictionary tag:

<dict>

As I mentioned earlier, the structure we wrote out was a dictionary. In fact, that's all you'll ever really do with plist files: read a plist into a dictionary or create one from scratch, and then let Apple's APIs write it out.

Wrapped in the dictionary are its values:

<key>color</key>
<string>blue</string>
<key>count</key>
<integer>15</integer>
<key>style</key>
<string>fruit</string>
Following this, the tags are closed and the file ends:
</dict>
</plist>

Each tag should lead to a new level of indentation. It's easy to see the structure here. Best of all, it's easily human-readable.

However, beginning with OS X 10.5, the bulk of plist files found on the system are stored in a binary format, not plain text. While this does have the effect of using less space on disk and faster load times, it takes the human-readable part out of the picture. Of course, there are ways to deal with that.

There are several ways to work with plist files, both graphically and from the command line. Apple's Property List Editor is installed as part of the free developer tools suite (Xcode et al). In a standard install, it is found at /Developer/Applications/Utilities/Property List Editor.app. This is the easiest way to visualize a plist. It's also useful for creating a plist from scratch. Property List Editor can also edit entries in a plist file.

Figure 1-Property List Editor.app displaying the hierarchy of a plist file.

While Property List Editor is fine for one-off plist work, it doesn't really scale too well. That it doesn't have a dictionary to use with AppleScript is just one example. What if you need to modify a plist on thousands of machines? (Or even 15 machines-it's a pain to walk around to each machine and potentially interrupt people's work. You may even want to update them after hours, while you're home). Once again, it's scripting to the rescue.

There are several utilities for standard shell scripting or ad-hoc use. plutil, defaults and PlistBuddy all have different purposes and capabilities.

plutil is the most basic and utilitarian of the three. plutil, the plist utility, converts plist files between text (xml) and binary formats and can also verify the structure of a plist. An example is in order. If you want to view the contents of a binary plist-com.apple.nat.plist, for example-but don't care to open it in Property List Editor you can run this:

plutil -convert xml1 -o - /Library/Preferences/com.apple.nat.plist

(This makes a very nice alias: alias viewplist="plutil -convert xml1 -o - $1". Keep that in your .bash_profile). Running this command tells plutil to convert the plist to text ("xml1") and send the output ("-o") to standard out. You could certainly write the output to another file on disk if you choose.

plutil can also lint a file; that is, check it for consistency and basic errors. What it cannot do is verify that your key-names and data are correct. Running a lint check is as simple as passing in the -lint switch:

$ plutil -lint /Library/Preferences/com.apple.loginwindow.plist 
/Library/Preferences/com.apple.loginwindow.plist: OK

If the lint process encounters an error (or errors, perhaps), you're told the error and on which line:

$ plutil -lint someplist 
someplist: Encountered unknown tag stringblue</string on line 6

The defaults command gives you access to the user defaults system. The "user defaults system" is a fancy way of saying "preferences," which, you'll probably recognize as data stored in a plist file. The name is derived from the Cocoa API that performs the same task: NSUserDefaults. The defaults utility allows for reading and writing individual keys and their data to and from a plist file, reading a plist in whole and more.

Perhaps the simplest use of the defaults command is reading an entire plist file. This is equivalent to the plutil command given earlier:

$ defaults read /Library/Preferences/com.apple.nat
{
    NatPortMapDisabled = 0;
}

The defaults command reads plist files of either xml or binary. However, it will only write a plist out in the binary variety. It will even go so far as to convert an xml plist into binary if used to update a value in that plist. Do note that the target plist is specified without the .plist extension.

The defaults command, however, is not exactly a general-purpose plist utility like plutil or Propery List Editor.app. As mentioned, it works within the bounds of the user defaults system. The upshot of this is that it expects plists to reside in specific places: one of the Library/Preferences directories on the system. Do not rely on the defaults command to read and write arbitrary plists. (In 10.5 and 10.6, accessing arbitrary plist files is possible, however, that functionality is said to be going away. Plus, you're reading this article and will be learning better ways of handling this). One other small problem with defaults: it's virtually impossible to work with values in nested dictionaries. Which brings us to PlistBuddy.

PlistBuddy started off as a utility that was only found embedded into packages for Apple updates. Clearly, Apple realized they needed a utility like this and developed it for their own use. As of Leopard, though, it's a real part of the OS: it is found at /usr/libexec/PlistBuddy and even has a man page. While the defaults command can handle most tasks, PlistBuddy excels at editing keys and values in a nested dictionary.

Let's imagine our example plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>cust_info</key>
   <dict>
      <key>pid</key>
      <string>98234573</string>
      <key>uid</key>
      <string>348576</string>
   </dict>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Notice that the key, "cust_info" is a dictionary, rather than a simple, single value. PlistBuddy can easily update the values in this nested dictionary. PlistBuddy can work interactively, which I will not cover, but can also pass in all commands using the "-c" switch. To set the value of a key, you need the path to the key and the set command. The path to the key starts with a colon (":") and uses a colon as the separator for each level in the hierarchy. Here's how to change ("set") the value of the existing "pid" key to 94758476, in the plist, "com.mactech.example.plist":

/usr/libexec/PlistBuddy -c "Set :cust_info:pid 94758476" ./com.mactech.example.plist

(This is running the command in the same directory as the target plist. Otherwise, you'd need to specify the full path to the plist to edit). See the PlistBuddy man page (note the capitalization!) for more information on the utility. PlistBuddy is capable of much, much more, including copying values and merging plist files.

From time to time, as a system administrator, you'll find yourself in a position where you'd like a script to store its own preferences. Or, simply have a script analyze a plist and act on the contents in some manner. In many cases, bash scripting that uses the commands already presented (plutil, PlistBuddy and, particularly, defaults) will be perfectly acceptable. However, for anything with a little more complexity, you may already be scripting in Python (or perl, or Ruby, etc.). Since Mac in the Shell has been focusing on Python for the last several columns, we'll use it here as well.

Python, with PyObj-C, makes this trivial. More interestingly, you get the best of both worlds: Apple's APIs along with Python's ease of use and the speed of the edit and run cycle (skipping the compile step of C-based languages). To see this in action, let's start with nearly the most simple example possible. Listing 2 contains write_plist.py, which demonstrates creating a dictionary that gets written to a plist.

#!/usr/bin/python2.5
from Foundation import NSMutableDictionary
my_dict = NSMutableDictionary.dictionary()
my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'
success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)
if not success:
  print "plist failed to write!"
  sys.exit(1)

Upon running this program, com.mactech.example.plist will be created in the same working directory as the program itself. The plist file will match the output that is shown in Listing 1. Let's examine this line-by-line to see how it works.

The very first line-#!/usr/bin/python2.5-is a good reminder that Python version 2.5 or higher is required for PyObj-C integration. This will not work on Tiger systems out of the box.

from Foundation import NSMutableDictionary

This import is responsible for all of the magic here. While we could import all of Foundation, we'll just import the portion we need: NSMutableDictionary.

my_dict = NSMutableDictionary.dictionary()

Typically, creating a dictionary in Python would use curly braces, like this:

new_dict = {}

or, you can even fill it on creation:

new_dict = {'color':'blue', 'count':15, 'style':'fruit'}

However, we need to create a real Cocoa NSMutableDictionary object, so that's what we've done. Nicely, we can no go on and treat that just like a Python dictionary:

my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'

You can use the Cocoa API for adding entries to a dictionary as well:

my_dict.setValue_forKey_('stop', 'state')

This would set the key 'state' to store the value 'stop', and add the following to the plist once written out:

<key>state</key>
<string>stop</string>

But, really... if you're using Python, take advantage of it where you can! (I suggest using the Python method). You will need to use the Cocoa API to write the dictionary out to disk as a plist file:

success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)

The Cocoa writeToFile:atomically: method of NSDictionary (and, by extension, NSMutableDictionary) writes a property list representation of the contents of the dictionary to the path given.

if not success:
  print "plist failed to write!"
  sys.exit(1)

This final conditional tests to see if the writeToFile:atomically: method returned a True ("success") or False ("failure") value. While not strictly necessary for this program to run, checking these values is a good habit to get into.

Just as a reminder, once you create the NSMutableDictionary, you can use standard Python mechanisms to manipulate and traverse it. Adding a key with a dictionary as its value is as simpe as you'd expect. Just create the dictionary and then assign it to the parent dictionary. For example, to recreate the com.mactech.example.plist shown earlier, we would add the following to our program, after creating the initial dictionary:

sub_dict = {}
sub_dict['uid'] = '348576'
sub_dict['pid'] = '98234573'
my_dict['cust_info'] = sub_dict

Also, as shown earlier, you can also use all of the Cocoa APIs available to you to manipulate the dictionary as well. The style you choose may be situation dependent. Some situations may call for using the Cocoa-way, while others may favor more Pythonic writing. When working with any Cocoa API, though, as always, you'll want to keep the documentation handy.

Use It or Lose It

This was an incredibly fun article to write. The topic is incredibly practical for everyday use. The plist format is pervasive throughout OS X. Every technical person should have a familiarity with it, and System Admins should be even more deeply involved. While many cases can simply be solved with a single command-line call to defaults or PlistBuddy, anything with deeper involvement should use a scripting language like Python. The nice thing about the scripting solution is that once you build up your library of routines, they're written and ready for re-use. Reading about it here only gets you so far. Go write a script and run it on a test system so you're ready for the real thing when the opportunity arrives.

Media of the month: Kick it old-school with vintage computer brochures and manuals at http://assemblyman-eph.blogspot.com/2009/04/vintage-computer-brochures.html. Full PDFs of how it used to be. I remember when taking one of my first computer courses, the teacher launching into the history of computing. Naturally, I just wanted to get into sitting at a computer and coding. But now, perhaps more than ever before, it's really useful to be able to frame our current experience with that which it was built on and evolved from.

Next month, we'll be covering Snow Leopard related topics! It'll be two issues before we get back to Python and scripting in general. Until then, keep practicing.

"About Property Lists": https://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/AboutPropertyLists/AboutPropertyLists.html#/apple_ref/doc/uid/20001010-46719

"Understanding XML Property Lists": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

"Introduction to Property List Programming Topics for Core Foundation": http://developer.apple.com/iphone/library/documentation/CoreFoundation/Conceptual/CFPropertyLists/CFPropertyLists.html

"Introduction to User Defaults": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

Ed Marczak is the Executive Editor of MacTech Magazine. He has written for MacTech since 2004.