TweetFollow Us on Twitter

Mac in the Shell: Reading and Writing plist files with Python

Volume Number: 25
Issue Number: 09
Column Tag: Mac in the Shell

Mac in the Shell: Reading and Writing plist files with Python

Tame those pesky plists

by Edwarcd Marczak

Welcome

Property list files, also known as 'plists,' are pervasive in OS X. This article teaches you the basic inner-workings of the plist format, system level methods of working with plist files and how to interact with these files using Python under OS X.

Anatomy

Plist files are structured XML (eXtensible Markup Language) files and easily understandable. Essentially, a plist file is a way to store standard types of data. By "standard," I mean string, integer, Boolean and so on, although there are ways to store arbitrary data as well. A plist file can easily be read into and written out from an NSDictionary object. Thanks to PyObj-C, an NSDictionary can be mapped onto and manipulated with a Python-based dictionary object.

Given the following dictionary:

{
    color:'blue',
    count:15,
    style:'fruit'
}

the plist in Listing 1 would be created

Listing 1-example plist file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Let's take a closer look at this plist. The header declares this file as an XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Ultimately, this header isn't up to you. In this article, you'll see that Apple's Cocoa APIs will properly generate this upon writing a plist. For more information about XML, see the specification page as http://xml.org, or the Wikipedia entry at http://en.wikipedia.org/wiki/Xml.

The plist tag wraps the entire file:

<plist version="1.0">

Again, Apple's APIs will write this out as appropriate. Next, we find a dictionary tag:

<dict>

As I mentioned earlier, the structure we wrote out was a dictionary. In fact, that's all you'll ever really do with plist files: read a plist into a dictionary or create one from scratch, and then let Apple's APIs write it out.

Wrapped in the dictionary are its values:

<key>color</key>
<string>blue</string>
<key>count</key>
<integer>15</integer>
<key>style</key>
<string>fruit</string>
Following this, the tags are closed and the file ends:
</dict>
</plist>

Each tag should lead to a new level of indentation. It's easy to see the structure here. Best of all, it's easily human-readable.

However, beginning with OS X 10.5, the bulk of plist files found on the system are stored in a binary format, not plain text. While this does have the effect of using less space on disk and faster load times, it takes the human-readable part out of the picture. Of course, there are ways to deal with that.

System Tools

There are several ways to work with plist files, both graphically and from the command line. Apple's Property List Editor is installed as part of the free developer tools suite (Xcode et al). In a standard install, it is found at /Developer/Applications/Utilities/Property List Editor.app. This is the easiest way to visualize a plist. It's also useful for creating a plist from scratch. Property List Editor can also edit entries in a plist file.


Figure 1-Property List Editor.app displaying the hierarchy of a plist file.

While Property List Editor is fine for one-off plist work, it doesn't really scale too well. That it doesn't have a dictionary to use with AppleScript is just one example. What if you need to modify a plist on thousands of machines? (Or even 15 machines-it's a pain to walk around to each machine and potentially interrupt people's work. You may even want to update them after hours, while you're home). Once again, it's scripting to the rescue.

There are several utilities for standard shell scripting or ad-hoc use. plutil, defaults and PlistBuddy all have different purposes and capabilities.

plutil is the most basic and utilitarian of the three. plutil, the plist utility, converts plist files between text (xml) and binary formats and can also verify the structure of a plist. An example is in order. If you want to view the contents of a binary plist-com.apple.nat.plist, for example-but don't care to open it in Property List Editor you can run this:

plutil -convert xml1 -o - /Library/Preferences/com.apple.nat.plist

(This makes a very nice alias: alias viewplist="plutil -convert xml1 -o - $1". Keep that in your .bash_profile). Running this command tells plutil to convert the plist to text ("xml1") and send the output ("-o") to standard out. You could certainly write the output to another file on disk if you choose.

plutil can also lint a file; that is, check it for consistency and basic errors. What it cannot do is verify that your key-names and data are correct. Running a lint check is as simple as passing in the -lint switch:

$ plutil -lint /Library/Preferences/com.apple.loginwindow.plist 
/Library/Preferences/com.apple.loginwindow.plist: OK

If the lint process encounters an error (or errors, perhaps), you're told the error and on which line:

$ plutil -lint someplist 
someplist: Encountered unknown tag stringblue</string on line 6

The defaults command gives you access to the user defaults system. The "user defaults system" is a fancy way of saying "preferences," which, you'll probably recognize as data stored in a plist file. The name is derived from the Cocoa API that performs the same task: NSUserDefaults. The defaults utility allows for reading and writing individual keys and their data to and from a plist file, reading a plist in whole and more.

Perhaps the simplest use of the defaults command is reading an entire plist file. This is equivalent to the plutil command given earlier:

$ defaults read /Library/Preferences/com.apple.nat
{
    NatPortMapDisabled = 0;
}

The defaults command reads plist files of either xml or binary. However, it will only write a plist out in the binary variety. It will even go so far as to convert an xml plist into binary if used to update a value in that plist. Do note that the target plist is specified without the .plist extension.

The defaults command, however, is not exactly a general-purpose plist utility like plutil or Propery List Editor.app. As mentioned, it works within the bounds of the user defaults system. The upshot of this is that it expects plists to reside in specific places: one of the Library/Preferences directories on the system. Do not rely on the defaults command to read and write arbitrary plists. (In 10.5 and 10.6, accessing arbitrary plist files is possible, however, that functionality is said to be going away. Plus, you're reading this article and will be learning better ways of handling this). One other small problem with defaults: it's virtually impossible to work with values in nested dictionaries. Which brings us to PlistBuddy.

PlistBuddy started off as a utility that was only found embedded into packages for Apple updates. Clearly, Apple realized they needed a utility like this and developed it for their own use. As of Leopard, though, it's a real part of the OS: it is found at /usr/libexec/PlistBuddy and even has a man page. While the defaults command can handle most tasks, PlistBuddy excels at editing keys and values in a nested dictionary.

Let's imagine our example plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>cust_info</key>
   <dict>
      <key>pid</key>
      <string>98234573</string>
      <key>uid</key>
      <string>348576</string>
   </dict>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Notice that the key, "cust_info" is a dictionary, rather than a simple, single value. PlistBuddy can easily update the values in this nested dictionary. PlistBuddy can work interactively, which I will not cover, but can also pass in all commands using the "-c" switch. To set the value of a key, you need the path to the key and the set command. The path to the key starts with a colon (":") and uses a colon as the separator for each level in the hierarchy. Here's how to change ("set") the value of the existing "pid" key to 94758476, in the plist, "com.mactech.example.plist":

/usr/libexec/PlistBuddy -c "Set :cust_info:pid 94758476" ./com.mactech.example.plist

(This is running the command in the same directory as the target plist. Otherwise, you'd need to specify the full path to the plist to edit). See the PlistBuddy man page (note the capitalization!) for more information on the utility. PlistBuddy is capable of much, much more, including copying values and merging plist files.

Accessing plists Via Python

From time to time, as a system administrator, you'll find yourself in a position where you'd like a script to store its own preferences. Or, simply have a script analyze a plist and act on the contents in some manner. In many cases, bash scripting that uses the commands already presented (plutil, PlistBuddy and, particularly, defaults) will be perfectly acceptable. However, for anything with a little more complexity, you may already be scripting in Python (or perl, or Ruby, etc.). Since Mac in the Shell has been focusing on Python for the last several columns, we'll use it here as well.

Python, with PyObj-C, makes this trivial. More interestingly, you get the best of both worlds: Apple's APIs along with Python's ease of use and the speed of the edit and run cycle (skipping the compile step of C-based languages). To see this in action, let's start with nearly the most simple example possible. Listing 2 contains write_plist.py, which demonstrates creating a dictionary that gets written to a plist.

Listing 2-write_plist.py

#!/usr/bin/python2.5
from Foundation import NSMutableDictionary
my_dict = NSMutableDictionary.dictionary()
my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'
success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)
if not success:
  print "plist failed to write!"
  sys.exit(1)

Upon running this program, com.mactech.example.plist will be created in the same working directory as the program itself. The plist file will match the output that is shown in Listing 1. Let's examine this line-by-line to see how it works.

The very first line-#!/usr/bin/python2.5-is a good reminder that Python version 2.5 or higher is required for PyObj-C integration. This will not work on Tiger systems out of the box.

from Foundation import NSMutableDictionary

This import is responsible for all of the magic here. While we could import all of Foundation, we'll just import the portion we need: NSMutableDictionary.

my_dict = NSMutableDictionary.dictionary()
-

Typically, creating a dictionary in Python would use curly braces, like this:

new_dict = {}

or, you can even fill it on creation:

new_dict = {'color':'blue', 'count':15, 'style':'fruit'}

However, we need to create a real Cocoa NSMutableDictionary object, so that's what we've done. Nicely, we can no go on and treat that just like a Python dictionary:

my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'

You can use the Cocoa API for adding entries to a dictionary as well:

my_dict.setValue_forKey_('stop', 'state')

This would set the key 'state' to store the value 'stop', and add the following to the plist once written out:

<key>state</key>
<string>stop</string>

But, really... if you're using Python, take advantage of it where you can! (I suggest using the Python method). You will need to use the Cocoa API to write the dictionary out to disk as a plist file:

success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)

The Cocoa writeToFile:atomically: method of NSDictionary (and, by extension, NSMutableDictionary) writes a property list representation of the contents of the dictionary to the path given.

if not success:
  print "plist failed to write!"
  sys.exit(1)

This final conditional tests to see if the writeToFile:atomically: method returned a True ("success") or False ("failure") value. While not strictly necessary for this program to run, checking these values is a good habit to get into.

Python Ease

Just as a reminder, once you create the NSMutableDictionary, you can use standard Python mechanisms to manipulate and traverse it. Adding a key with a dictionary as its value is as simpe as you'd expect. Just create the dictionary and then assign it to the parent dictionary. For example, to recreate the com.mactech.example.plist shown earlier, we would add the following to our program, after creating the initial dictionary:

sub_dict = {}
sub_dict['uid'] = '348576'
sub_dict['pid'] = '98234573'
my_dict['cust_info'] = sub_dict

Also, as shown earlier, you can also use all of the Cocoa APIs available to you to manipulate the dictionary as well. The style you choose may be situation dependent. Some situations may call for using the Cocoa-way, while others may favor more Pythonic writing. When working with any Cocoa API, though, as always, you'll want to keep the documentation handy.

Use It or Lose It

This was an incredibly fun article to write. The topic is incredibly practical for everyday use. The plist format is pervasive throughout OS X. Every technical person should have a familiarity with it, and System Admins should be even more deeply involved. While many cases can simply be solved with a single command-line call to defaults or PlistBuddy, anything with deeper involvement should use a scripting language like Python. The nice thing about the scripting solution is that once you build up your library of routines, they're written and ready for re-use. Reading about it here only gets you so far. Go write a script and run it on a test system so you're ready for the real thing when the opportunity arrives.

Media of the month: Kick it old-school with vintage computer brochures and manuals at http://assemblyman-eph.blogspot.com/2009/04/vintage-computer-brochures.html. Full PDFs of how it used to be. I remember when taking one of my first computer courses, the teacher launching into the history of computing. Naturally, I just wanted to get into sitting at a computer and coding. But now, perhaps more than ever before, it's really useful to be able to frame our current experience with that which it was built on and evolved from.

Next month, we'll be covering Snow Leopard related topics! It'll be two issues before we get back to Python and scripting in general. Until then, keep practicing.

References

"About Property Lists": https://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/AboutPropertyLists/AboutPropertyLists.html#/apple_ref/doc/uid/20001010-46719

"Understanding XML Property Lists": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

"Introduction to Property List Programming Topics for Core Foundation": http://developer.apple.com/iphone/library/documentation/CoreFoundation/Conceptual/CFPropertyLists/CFPropertyLists.html

"Introduction to User Defaults": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1


Ed Marczak is the Executive Editor of MacTech Magazine. He has written for MacTech since 2004.

 
AAPL
$109.41
Apple Inc.
+2.67
MSFT
$45.74
Microsoft Corpora
+0.58
GOOG
$504.89
Google Inc.
+9.50

MacTech Search:
Community Search:

Software Updates via MacUpdate

Command-C 1.1.7 - Clipboard sharing tool...
Command-C is a revolutionary app which makes easy to share your clipboard between iOS and OS X using your local WiFi network, even if the app is not currently opened. Copy anything (text, pictures,... Read more
Tidy Up 4.0.2 - Find duplicate files and...
Tidy Up is a complete duplicate finder and disk-tidiness utility. With Tidy Up you can search for duplicate files and packages by the owner application, content, type, creator, extension, time... Read more
Typinator 6.3 - Speedy and reliable text...
Typinator turbo-charges your typing productivity. Type a little. Typinator does the rest. We've all faced projects that require repetitive typing tasks. With Typinator, you can store commonly used... Read more
GraphicConverter 9.5 - Graphics editor w...
GraphicConverter is an all-purpose image-editing program that can import 200 different graphic-based formats, edit the image, and export it to any of 80 available file formats. The high-end editing... Read more
Toast Titanium 12.0.1 - The ultimate med...
Toast Titanium goes way beyond the very basic burning in the Mac OS and iLife software, and sets the standard for burning CDs, DVDs, and now Blu-ray discs on the Mac. Create superior sounding audio... Read more
QuickBooks 2015 16.0.2.1422 R3 - Financi...
Save 20% on QuickBooks Pro for Mac today through this special discount link QuickBooks Pro 2013 helps you manage your business easily and efficiently. Organize your finances all in one place, track... Read more
Remotix 3.0.6 - Access all your computer...
Remotix is a fast and powerful application to easily access multiple Macs (and PCs) from your own Mac. Features: Complete Apple Screen Sharing support - including Mac OS X login, clipboard... Read more
BetterZip Quick Look Generator 1.5 - Let...
BetterZip Quick Look Generator lets you view the contents of compressed archives through OS X's Quick Look functions. Simply select an archive in the Finder, in Mail, or Spotlight and press the... Read more
Sandvox 2.9.3 - Easily build eye-catchin...
Sandvox is for Mac users who want to create a professional looking website quickly and easily. With Sandvox, you don't need to be a Web genius to build a stylish, feature-rich, standards-compliant... Read more
Pixelmator 3.3.1 - Powerful layer-based...
Pixelmator is a beautifully designed, easy-to-use, fast, and powerful image editor for OS X. It has everything you need to create, edit, and enhance your images. Pixelmator is a layer-based image... Read more

Latest Forum Discussions

See All

Pentaction: Medieval (Games)
Pentaction: Medieval 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Pentaction: Medieval is a turn-based strategy board-game about chance and skill on the battlefield. Take control of your... | Read more »
Hipstify Review
Hipstify Review By Jennifer Allen on December 17th, 2014 Our Rating: :: COOL FILTERSUniversal App - Designed for iPhone and iPad Add filters, quotes, and fancy frames to your images, thanks to Hipstify.   | Read more »
Mighty Smighties Gets Evolve Cards and N...
Mighty Smighties Gets Evolve Cards and New Worlds Posted by Jessica Fisher on December 17th, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Duckie Deck Card Wars Review
Duckie Deck Card Wars Review By Amy Solomon on December 17th, 2014 Our Rating: :: STYLISH GAME OF CARDSUniversal App - Designed for iPhone and iPad Duckie Deck Card Wars adapts the classic card game War for use on devices, complete... | Read more »
PDF Office Review
PDF Office Review By Jennifer Allen on December 17th, 2014 Our Rating: :: CONVENIENT PDF EDITINGiPad Only App - Designed for the iPad Want to create your own PDF files? Import them from elsewhere? Adapt a web page into a PDF? PDF... | Read more »
The Out There: Ω Edition Update will be...
The Out There: Ω Edition Update will be Releasing in 2015, Bringing Better Graphics and Additional Content Posted by Jessica Fisher on December 17th, 2014 [ permalink ] | Read more »
Pinball Planet Pro (Games)
Pinball Planet Pro 1.02 Device: iOS Universal Category: Games Price: $2.99, Version: 1.02 (iTunes) Description: - Our missionWe've always loved to play pinball games but we noticed most modern pinball games are simulators for the... | Read more »
Ultrakam 4k. The Professional Camera App...
Ultrakam 4k. The Professional Camera App. 3.0 Device: iOS Universal Category: Photography Price: $9.99, Version: 3.0 (iTunes) Description: In March 2014, Ultrakam brought Film Resolution for iPhone for the first time ever and now is... | Read more »
Email+ (Business)
Email+ 1.0 Device: iOS Universal Category: Business Price: $2.99, Version: 1.0 (iTunes) Description: Send email, fast! | Read more »
Mayor it up in SimCity BuildIt, Out Now
Mayor it up in SimCity BuildIt, Out Now Posted by Jessica Fisher on December 16th, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »

Price Scanner via MacPrices.net

Holiday sales continue: MacBook Pros for up t...
 B&H Photo has new MacBook Pros on sale for up to $300 off MSRP as part of their Holiday pricing. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $1699... Read more
Google Search App For iOS Gets A Major Makeov...
Google has given iOS users an early Christmas present with a substantial update of it’s not-very-often-upgraded Google Search app. Google Search has been my go-to tool for Web searches since it was... Read more
ShopKeep Apple Pay And Chip Card Reader Avail...
ShopKeep, a cloud-based technology provider to more than 10,000 small business owners to manage retail shops and restaurants with iPads, has released its new Apple Pay and chip card reader. This... Read more
Holiday sale! 27-inch 5K iMac for $2299, save...
 B&H Photo has the 27″ 3.5GHz 5K iMac in stock today and on sale for $2299 including free shipping plus NY sales tax only. Their price is $200 off MSRP, and it’s the lowest price available for... Read more
Holiday Sale! 3.7GHz Quad Core Mac Pro availa...
 B&H Photo has the 3.7GHz Quad Core Mac Pro on sale for $2599 including free shipping plus NY sales tax only. Their price is $400 off MSRP, and it’s the lowest price for this model from any... Read more
iPhone 6 Number 3 Canadian Google Search Of 2...
CTVNews.ca reports that Apple’s iPhone 6 was the third highest-trending Google Canada search topic of 2014, exceeded only by Robin Williams largely after his death by suicide in August, and the FIFA... Read more
New iPad mini 3 Counter-Top & Wall Mount...
newMacgadgets has announced new secure all-acrylic displays for the iPad mini 3 (also works fine with the mini 2, last year’s iPad mini With Retina Display, and the original iPad mini). The new iPad... Read more
Holiday sales continue, MacBook Airs for up t...
B&H Photo has 2014 MacBook Airs on sale for up to $120 off MSRP, for a limited time, for the Thanksgiving/Christmas Holiday shopping season. Shipping is free, and B&H charges NY sales tax... Read more
B&H lowers price on 27-inch 3.2GHz iMac t...
B&H Photo has lowered their price on the 27″ 3.2GHz iMac, now on sale for $1629 including free shipping plus NY sales tax only. Their price is $170 off MSRP, and it’s the lowest price for this... Read more
15-inch 2.0GHz Retina MacBook Pro available f...
B&H Photo has lowered their price on leftover 2013 15″ 2.0GHz Retina MacBook Pros to $1499 including free shipping plus NY sales tax only. Their price is $500 off original MSRP. Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Project Manager / Business Analyst, WW *Appl...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.