TweetFollow Us on Twitter

Mac in the Shell: Reading and Writing plist files with Python

Volume Number: 25
Issue Number: 09
Column Tag: Mac in the Shell

Mac in the Shell: Reading and Writing plist files with Python

Tame those pesky plists

by Edwarcd Marczak

Welcome

Property list files, also known as 'plists,' are pervasive in OS X. This article teaches you the basic inner-workings of the plist format, system level methods of working with plist files and how to interact with these files using Python under OS X.

Anatomy

Plist files are structured XML (eXtensible Markup Language) files and easily understandable. Essentially, a plist file is a way to store standard types of data. By "standard," I mean string, integer, Boolean and so on, although there are ways to store arbitrary data as well. A plist file can easily be read into and written out from an NSDictionary object. Thanks to PyObj-C, an NSDictionary can be mapped onto and manipulated with a Python-based dictionary object.

Given the following dictionary:

{
    color:'blue',
    count:15,
    style:'fruit'
}

the plist in Listing 1 would be created

Listing 1-example plist file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Let's take a closer look at this plist. The header declares this file as an XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Ultimately, this header isn't up to you. In this article, you'll see that Apple's Cocoa APIs will properly generate this upon writing a plist. For more information about XML, see the specification page as http://xml.org, or the Wikipedia entry at http://en.wikipedia.org/wiki/Xml.

The plist tag wraps the entire file:

<plist version="1.0">

Again, Apple's APIs will write this out as appropriate. Next, we find a dictionary tag:

<dict>

As I mentioned earlier, the structure we wrote out was a dictionary. In fact, that's all you'll ever really do with plist files: read a plist into a dictionary or create one from scratch, and then let Apple's APIs write it out.

Wrapped in the dictionary are its values:

<key>color</key>
<string>blue</string>
<key>count</key>
<integer>15</integer>
<key>style</key>
<string>fruit</string>
Following this, the tags are closed and the file ends:
</dict>
</plist>

Each tag should lead to a new level of indentation. It's easy to see the structure here. Best of all, it's easily human-readable.

However, beginning with OS X 10.5, the bulk of plist files found on the system are stored in a binary format, not plain text. While this does have the effect of using less space on disk and faster load times, it takes the human-readable part out of the picture. Of course, there are ways to deal with that.

System Tools

There are several ways to work with plist files, both graphically and from the command line. Apple's Property List Editor is installed as part of the free developer tools suite (Xcode et al). In a standard install, it is found at /Developer/Applications/Utilities/Property List Editor.app. This is the easiest way to visualize a plist. It's also useful for creating a plist from scratch. Property List Editor can also edit entries in a plist file.


Figure 1-Property List Editor.app displaying the hierarchy of a plist file.

While Property List Editor is fine for one-off plist work, it doesn't really scale too well. That it doesn't have a dictionary to use with AppleScript is just one example. What if you need to modify a plist on thousands of machines? (Or even 15 machines-it's a pain to walk around to each machine and potentially interrupt people's work. You may even want to update them after hours, while you're home). Once again, it's scripting to the rescue.

There are several utilities for standard shell scripting or ad-hoc use. plutil, defaults and PlistBuddy all have different purposes and capabilities.

plutil is the most basic and utilitarian of the three. plutil, the plist utility, converts plist files between text (xml) and binary formats and can also verify the structure of a plist. An example is in order. If you want to view the contents of a binary plist-com.apple.nat.plist, for example-but don't care to open it in Property List Editor you can run this:

plutil -convert xml1 -o - /Library/Preferences/com.apple.nat.plist

(This makes a very nice alias: alias viewplist="plutil -convert xml1 -o - $1". Keep that in your .bash_profile). Running this command tells plutil to convert the plist to text ("xml1") and send the output ("-o") to standard out. You could certainly write the output to another file on disk if you choose.

plutil can also lint a file; that is, check it for consistency and basic errors. What it cannot do is verify that your key-names and data are correct. Running a lint check is as simple as passing in the -lint switch:

$ plutil -lint /Library/Preferences/com.apple.loginwindow.plist 
/Library/Preferences/com.apple.loginwindow.plist: OK

If the lint process encounters an error (or errors, perhaps), you're told the error and on which line:

$ plutil -lint someplist 
someplist: Encountered unknown tag stringblue</string on line 6

The defaults command gives you access to the user defaults system. The "user defaults system" is a fancy way of saying "preferences," which, you'll probably recognize as data stored in a plist file. The name is derived from the Cocoa API that performs the same task: NSUserDefaults. The defaults utility allows for reading and writing individual keys and their data to and from a plist file, reading a plist in whole and more.

Perhaps the simplest use of the defaults command is reading an entire plist file. This is equivalent to the plutil command given earlier:

$ defaults read /Library/Preferences/com.apple.nat
{
    NatPortMapDisabled = 0;
}

The defaults command reads plist files of either xml or binary. However, it will only write a plist out in the binary variety. It will even go so far as to convert an xml plist into binary if used to update a value in that plist. Do note that the target plist is specified without the .plist extension.

The defaults command, however, is not exactly a general-purpose plist utility like plutil or Propery List Editor.app. As mentioned, it works within the bounds of the user defaults system. The upshot of this is that it expects plists to reside in specific places: one of the Library/Preferences directories on the system. Do not rely on the defaults command to read and write arbitrary plists. (In 10.5 and 10.6, accessing arbitrary plist files is possible, however, that functionality is said to be going away. Plus, you're reading this article and will be learning better ways of handling this). One other small problem with defaults: it's virtually impossible to work with values in nested dictionaries. Which brings us to PlistBuddy.

PlistBuddy started off as a utility that was only found embedded into packages for Apple updates. Clearly, Apple realized they needed a utility like this and developed it for their own use. As of Leopard, though, it's a real part of the OS: it is found at /usr/libexec/PlistBuddy and even has a man page. While the defaults command can handle most tasks, PlistBuddy excels at editing keys and values in a nested dictionary.

Let's imagine our example plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>cust_info</key>
   <dict>
      <key>pid</key>
      <string>98234573</string>
      <key>uid</key>
      <string>348576</string>
   </dict>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Notice that the key, "cust_info" is a dictionary, rather than a simple, single value. PlistBuddy can easily update the values in this nested dictionary. PlistBuddy can work interactively, which I will not cover, but can also pass in all commands using the "-c" switch. To set the value of a key, you need the path to the key and the set command. The path to the key starts with a colon (":") and uses a colon as the separator for each level in the hierarchy. Here's how to change ("set") the value of the existing "pid" key to 94758476, in the plist, "com.mactech.example.plist":

/usr/libexec/PlistBuddy -c "Set :cust_info:pid 94758476" ./com.mactech.example.plist

(This is running the command in the same directory as the target plist. Otherwise, you'd need to specify the full path to the plist to edit). See the PlistBuddy man page (note the capitalization!) for more information on the utility. PlistBuddy is capable of much, much more, including copying values and merging plist files.

Accessing plists Via Python

From time to time, as a system administrator, you'll find yourself in a position where you'd like a script to store its own preferences. Or, simply have a script analyze a plist and act on the contents in some manner. In many cases, bash scripting that uses the commands already presented (plutil, PlistBuddy and, particularly, defaults) will be perfectly acceptable. However, for anything with a little more complexity, you may already be scripting in Python (or perl, or Ruby, etc.). Since Mac in the Shell has been focusing on Python for the last several columns, we'll use it here as well.

Python, with PyObj-C, makes this trivial. More interestingly, you get the best of both worlds: Apple's APIs along with Python's ease of use and the speed of the edit and run cycle (skipping the compile step of C-based languages). To see this in action, let's start with nearly the most simple example possible. Listing 2 contains write_plist.py, which demonstrates creating a dictionary that gets written to a plist.

Listing 2-write_plist.py

#!/usr/bin/python2.5
from Foundation import NSMutableDictionary
my_dict = NSMutableDictionary.dictionary()
my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'
success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)
if not success:
  print "plist failed to write!"
  sys.exit(1)

Upon running this program, com.mactech.example.plist will be created in the same working directory as the program itself. The plist file will match the output that is shown in Listing 1. Let's examine this line-by-line to see how it works.

The very first line-#!/usr/bin/python2.5-is a good reminder that Python version 2.5 or higher is required for PyObj-C integration. This will not work on Tiger systems out of the box.

from Foundation import NSMutableDictionary

This import is responsible for all of the magic here. While we could import all of Foundation, we'll just import the portion we need: NSMutableDictionary.

my_dict = NSMutableDictionary.dictionary()
-

Typically, creating a dictionary in Python would use curly braces, like this:

new_dict = {}

or, you can even fill it on creation:

new_dict = {'color':'blue', 'count':15, 'style':'fruit'}

However, we need to create a real Cocoa NSMutableDictionary object, so that's what we've done. Nicely, we can no go on and treat that just like a Python dictionary:

my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'

You can use the Cocoa API for adding entries to a dictionary as well:

my_dict.setValue_forKey_('stop', 'state')

This would set the key 'state' to store the value 'stop', and add the following to the plist once written out:

<key>state</key>
<string>stop</string>

But, really... if you're using Python, take advantage of it where you can! (I suggest using the Python method). You will need to use the Cocoa API to write the dictionary out to disk as a plist file:

success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)

The Cocoa writeToFile:atomically: method of NSDictionary (and, by extension, NSMutableDictionary) writes a property list representation of the contents of the dictionary to the path given.

if not success:
  print "plist failed to write!"
  sys.exit(1)

This final conditional tests to see if the writeToFile:atomically: method returned a True ("success") or False ("failure") value. While not strictly necessary for this program to run, checking these values is a good habit to get into.

Python Ease

Just as a reminder, once you create the NSMutableDictionary, you can use standard Python mechanisms to manipulate and traverse it. Adding a key with a dictionary as its value is as simpe as you'd expect. Just create the dictionary and then assign it to the parent dictionary. For example, to recreate the com.mactech.example.plist shown earlier, we would add the following to our program, after creating the initial dictionary:

sub_dict = {}
sub_dict['uid'] = '348576'
sub_dict['pid'] = '98234573'
my_dict['cust_info'] = sub_dict

Also, as shown earlier, you can also use all of the Cocoa APIs available to you to manipulate the dictionary as well. The style you choose may be situation dependent. Some situations may call for using the Cocoa-way, while others may favor more Pythonic writing. When working with any Cocoa API, though, as always, you'll want to keep the documentation handy.

Use It or Lose It

This was an incredibly fun article to write. The topic is incredibly practical for everyday use. The plist format is pervasive throughout OS X. Every technical person should have a familiarity with it, and System Admins should be even more deeply involved. While many cases can simply be solved with a single command-line call to defaults or PlistBuddy, anything with deeper involvement should use a scripting language like Python. The nice thing about the scripting solution is that once you build up your library of routines, they're written and ready for re-use. Reading about it here only gets you so far. Go write a script and run it on a test system so you're ready for the real thing when the opportunity arrives.

Media of the month: Kick it old-school with vintage computer brochures and manuals at http://assemblyman-eph.blogspot.com/2009/04/vintage-computer-brochures.html. Full PDFs of how it used to be. I remember when taking one of my first computer courses, the teacher launching into the history of computing. Naturally, I just wanted to get into sitting at a computer and coding. But now, perhaps more than ever before, it's really useful to be able to frame our current experience with that which it was built on and evolved from.

Next month, we'll be covering Snow Leopard related topics! It'll be two issues before we get back to Python and scripting in general. Until then, keep practicing.

References

"About Property Lists": https://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/AboutPropertyLists/AboutPropertyLists.html#/apple_ref/doc/uid/20001010-46719

"Understanding XML Property Lists": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

"Introduction to Property List Programming Topics for Core Foundation": http://developer.apple.com/iphone/library/documentation/CoreFoundation/Conceptual/CFPropertyLists/CFPropertyLists.html

"Introduction to User Defaults": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1


Ed Marczak is the Executive Editor of MacTech Magazine. He has written for MacTech since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Vienna 3.0.6 :5eaf312: - RSS and Atom ne...
Vienna is a freeware and Open-Source RSS/Atom newsreader with article storage and management via a SQLite database, written in Objective-C and Cocoa, for the OS X operating system. It provides... Read more
Kodi 15.1.rc1 - Powerful media center to...
Kodi (was XBMC) is an award-winning free and open-source (GPL) software media player and entertainment hub that can be installed on Linux, OS X, Windows, iOS, and Android, featuring a 10-foot user... Read more
Bookends 12.5.8 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Access the power of Bookends directly from Mellel, Nisus Writer Pro, or MS Word (... Read more
Chromium 44.0.2403.125 - Fast and stable...
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all Internet users to experience the web. Version 44.0.2403.125: This release contains a number... Read more
iMazing 1.2.2 - Complete iOS device mana...
iMazing (was DiskAid) is the ultimate iOS device manager with capabilities far beyond what iTunes offers. With iMazing and your iOS device (iPhone, iPad, or iPod), you can: Copy music to and from... Read more
Audio Hijack 3.2.0 - Record and enhance...
Audio Hijack (was Audio Hijack Pro) drastically changes the way you use audio on your computer, giving you the freedom to listen to audio when you want and how you want. Record and enhance any audio... Read more
FontExplorer X Pro 5.0.1 - Font manageme...
FontExplorer X Pro is optimized for professional use; it's the solution that gives you the power you need to manage all your fonts. Now you can more easily manage, activate and organize your... Read more
Calcbot 1.0.2 - Intelligent calculator a...
Calcbot is an intelligent calculator and unit converter for the rest of us. Featuring an easy-to-read history tape, expression view, intuitive conversion, and much more! Features History Tape -... Read more
MTR 5.0.0.1 - The Mac's oldest and...
MTR (was MacTheRipper)--the Mac's oldest and smartest DVD-backup app--is now updated to version 5.001 MTR -- the complete toolbox, not a one-trick, point-and-click extractor. MTR is intended for... Read more
LibreOffice 4.4.5.2 - Free, open-source...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more

Card King: Dragon Wars - Tips, Tricks an...
[Read more] | Read more »
Pac-Man Championship Edition DX has brou...
Bandai Namco has released Pac-Man Championship Edition DX on iOS and Android, which features the classic arcade gameplay that we've all grown to love. Pac-Man Championship Edition DX can be enjoyed in much shorter bursts than the arcade versions... | Read more »
Cosmonautica (Games)
Cosmonautica 1.1 Device: iOS Universal Category: Games Price: $6.99, Version: 1.1 (iTunes) Description: Cast off! Are you ready for some hilarious adventures in outer space? | Read more »
Rescue humanity from a Demon horde in An...
Angel Stone is Fincon's follow up to the massively successful Hello Hero and is out now on iOS and Android. You play as a member of The Resistance, a group of mighty human warriors who have risen up in defiance of the Demon horde threatening to... | Read more »
Gallery Doctor (Photography)
Gallery Doctor 1.0 Device: iOS iPhone Category: Photography Price: $2.99, Version: 1.0 (iTunes) Description: Free up valuable iCloud and iPhone storage with Gallery Doctor, the only iPhone cleaner that automatically identifies the... | Read more »
You Against Me (Games)
You Against Me 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: A simple game… You. Me. Claim, steal, lock, score, win! | Read more »
Yep, it's True - Angry Birds 2 is O...
The not exactly rumors were true and the birds are back. Angry Birds 2 has come to the App Store and the world will... well I suppose it'll still be the same, but now we have more bird-flinging options! [Read more] | Read more »
You Could Design Your Own Card for Chain...
If you've ever wanted to create your own item, weapon, trap, or even monster for Chainsaw Warrior: Lords of the Night, this is your chance. Auroch Digital is currently holding a contest so that fans can fight to the death (not really) to see which... | Read more »
Bitcoin Billionaire is Going Back in Tim...
If you thought you managed to buy everything there is to buy in Bitcoin Billionaire and make all the money, well you though wrong. Those of you who made it far enough might remember investing in time travel - and it looks like that investment is... | Read more »
Domino Drop (Games)
Domino Drop 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Domino Drop is a delightful new puzzle game with dominos and gravity!Learn how to play it in a minute, master it day by day.Your... | Read more »

Price Scanner via MacPrices.net

Back-to-School with Tablet and Smartphone Acc...
Belkin helps you prepare for the coming school year with a wide variety of the latest mobile and tablet accessories to outfit both grade school and college students. The line-up includes charging... Read more
11-inch MacBook Airs on sale for $100 off MSR...
Best Buy has 11-inch MacBook Airs on sale for $100 off MSRP. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store prices may vary: - 11″ 1.6GHz... Read more
iPad Air 2 on sale for up to $100 off MSRP
Best Buy has iPad Air 2s on sale for up to $100 off MSRP on their online store for a limited time. Choose free shipping or free local store pickup (if available). Sale prices available for online... Read more
Sale! 13-inch MacBook Pros on sale for $100 o...
B&H Photo has 13″ MacBook Pros on sale for $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.5GHz/500GB MacBook Pro: $999.99 save $100 - 13″ 2.7GHz/128GB Retina... Read more
Sale! Save $100 on 13-inch MacBook Airs this...
B&H Photo has the 13″ 1.6GHz/128GB MacBook Air on sale for $899.99 including free shipping plus NY tax only. Their price is $100 off MSRP, and it’s the lowest price available for this model.... Read more
Worldwide Tablet Market Decline Continues, Ap...
The worldwide tablet market declined -7.0% year-over-year in the second quarter of 2015 (2Q15) with shipments totaling 44.7 million units according to preliminary data from the International Data... Read more
TP-LINK TL-PA8030P KIT Powerline Featuring Ho...
Consumer and business networking products provider TP-LINK is now shipping its TL-PA8030P KIT AV1200 3-Port Gigabit Passthrough Powerline Starter Kit that expands your home’s network over its... Read more
Apple refurbished iPad Air 2s available for u...
The Apple Store has Apple Certified Refurbished iPad Air 2s available for up to $140 off the price of new models. Apple’s one-year warranty is included with each model, and shipping is free: - 128GB... Read more
Updated Apple iPad Price Trackers
We’ve updated our iPad Air Price Tracker and our iPad mini Price Tracker with the latest information on prices and availability from Apple and other resellers. Read more
Apple refurbished 2014 13-inch 128GB MacBook...
The Apple Store has Apple Certified Refurbished 2014 13″ MacBook Airs available starting at $759. An Apple one-year warranty is included with each MacBook, and shipping is free: - 13″ 1.4GHz/128GB... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Infrastructure Engineer - *Apple* /Mac - Hil...
Infrastructure Engineer - Apple /Mac Job Code: 1608 # of openings: 1 Description Our fortune 500 client is looking to hire an experienced Infrastructure Engineer to join Read more
Executive Administrative Assistant, *Apple*...
…supporting presentation development for senior leadership. * User experience with Apple hardware and software is preferred. Additional Requirements The following list Read more
*Apple* Bus Company is now hirin - Apple Bus...
Apple Bus Company is now hiring school bus drivers in the Pettis County area. Class B CDL preferred. Free training provided. No nights or weekends required. Flexible Read more
*Apple* Certified Mac Technician - Updated 6...
…and friendly, hands-on technical support to customers troubleshooting and repairing Apple /Mac products with courtesy, speed and skill. Use your problem-solving skills Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.