TweetFollow Us on Twitter

Mac in the Shell: Reading and Writing plist files with Python

Volume Number: 25
Issue Number: 09
Column Tag: Mac in the Shell

Mac in the Shell: Reading and Writing plist files with Python

Tame those pesky plists

by Edwarcd Marczak

Welcome

Property list files, also known as 'plists,' are pervasive in OS X. This article teaches you the basic inner-workings of the plist format, system level methods of working with plist files and how to interact with these files using Python under OS X.

Anatomy

Plist files are structured XML (eXtensible Markup Language) files and easily understandable. Essentially, a plist file is a way to store standard types of data. By "standard," I mean string, integer, Boolean and so on, although there are ways to store arbitrary data as well. A plist file can easily be read into and written out from an NSDictionary object. Thanks to PyObj-C, an NSDictionary can be mapped onto and manipulated with a Python-based dictionary object.

Given the following dictionary:

{
    color:'blue',
    count:15,
    style:'fruit'
}

the plist in Listing 1 would be created

Listing 1-example plist file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Let's take a closer look at this plist. The header declares this file as an XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Ultimately, this header isn't up to you. In this article, you'll see that Apple's Cocoa APIs will properly generate this upon writing a plist. For more information about XML, see the specification page as http://xml.org, or the Wikipedia entry at http://en.wikipedia.org/wiki/Xml.

The plist tag wraps the entire file:

<plist version="1.0">

Again, Apple's APIs will write this out as appropriate. Next, we find a dictionary tag:

<dict>

As I mentioned earlier, the structure we wrote out was a dictionary. In fact, that's all you'll ever really do with plist files: read a plist into a dictionary or create one from scratch, and then let Apple's APIs write it out.

Wrapped in the dictionary are its values:

<key>color</key>
<string>blue</string>
<key>count</key>
<integer>15</integer>
<key>style</key>
<string>fruit</string>
Following this, the tags are closed and the file ends:
</dict>
</plist>

Each tag should lead to a new level of indentation. It's easy to see the structure here. Best of all, it's easily human-readable.

However, beginning with OS X 10.5, the bulk of plist files found on the system are stored in a binary format, not plain text. While this does have the effect of using less space on disk and faster load times, it takes the human-readable part out of the picture. Of course, there are ways to deal with that.

System Tools

There are several ways to work with plist files, both graphically and from the command line. Apple's Property List Editor is installed as part of the free developer tools suite (Xcode et al). In a standard install, it is found at /Developer/Applications/Utilities/Property List Editor.app. This is the easiest way to visualize a plist. It's also useful for creating a plist from scratch. Property List Editor can also edit entries in a plist file.


Figure 1-Property List Editor.app displaying the hierarchy of a plist file.

While Property List Editor is fine for one-off plist work, it doesn't really scale too well. That it doesn't have a dictionary to use with AppleScript is just one example. What if you need to modify a plist on thousands of machines? (Or even 15 machines-it's a pain to walk around to each machine and potentially interrupt people's work. You may even want to update them after hours, while you're home). Once again, it's scripting to the rescue.

There are several utilities for standard shell scripting or ad-hoc use. plutil, defaults and PlistBuddy all have different purposes and capabilities.

plutil is the most basic and utilitarian of the three. plutil, the plist utility, converts plist files between text (xml) and binary formats and can also verify the structure of a plist. An example is in order. If you want to view the contents of a binary plist-com.apple.nat.plist, for example-but don't care to open it in Property List Editor you can run this:

plutil -convert xml1 -o - /Library/Preferences/com.apple.nat.plist

(This makes a very nice alias: alias viewplist="plutil -convert xml1 -o - $1". Keep that in your .bash_profile). Running this command tells plutil to convert the plist to text ("xml1") and send the output ("-o") to standard out. You could certainly write the output to another file on disk if you choose.

plutil can also lint a file; that is, check it for consistency and basic errors. What it cannot do is verify that your key-names and data are correct. Running a lint check is as simple as passing in the -lint switch:

$ plutil -lint /Library/Preferences/com.apple.loginwindow.plist 
/Library/Preferences/com.apple.loginwindow.plist: OK

If the lint process encounters an error (or errors, perhaps), you're told the error and on which line:

$ plutil -lint someplist 
someplist: Encountered unknown tag stringblue</string on line 6

The defaults command gives you access to the user defaults system. The "user defaults system" is a fancy way of saying "preferences," which, you'll probably recognize as data stored in a plist file. The name is derived from the Cocoa API that performs the same task: NSUserDefaults. The defaults utility allows for reading and writing individual keys and their data to and from a plist file, reading a plist in whole and more.

Perhaps the simplest use of the defaults command is reading an entire plist file. This is equivalent to the plutil command given earlier:

$ defaults read /Library/Preferences/com.apple.nat
{
    NatPortMapDisabled = 0;
}

The defaults command reads plist files of either xml or binary. However, it will only write a plist out in the binary variety. It will even go so far as to convert an xml plist into binary if used to update a value in that plist. Do note that the target plist is specified without the .plist extension.

The defaults command, however, is not exactly a general-purpose plist utility like plutil or Propery List Editor.app. As mentioned, it works within the bounds of the user defaults system. The upshot of this is that it expects plists to reside in specific places: one of the Library/Preferences directories on the system. Do not rely on the defaults command to read and write arbitrary plists. (In 10.5 and 10.6, accessing arbitrary plist files is possible, however, that functionality is said to be going away. Plus, you're reading this article and will be learning better ways of handling this). One other small problem with defaults: it's virtually impossible to work with values in nested dictionaries. Which brings us to PlistBuddy.

PlistBuddy started off as a utility that was only found embedded into packages for Apple updates. Clearly, Apple realized they needed a utility like this and developed it for their own use. As of Leopard, though, it's a real part of the OS: it is found at /usr/libexec/PlistBuddy and even has a man page. While the defaults command can handle most tasks, PlistBuddy excels at editing keys and values in a nested dictionary.

Let's imagine our example plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>cust_info</key>
   <dict>
      <key>pid</key>
      <string>98234573</string>
      <key>uid</key>
      <string>348576</string>
   </dict>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Notice that the key, "cust_info" is a dictionary, rather than a simple, single value. PlistBuddy can easily update the values in this nested dictionary. PlistBuddy can work interactively, which I will not cover, but can also pass in all commands using the "-c" switch. To set the value of a key, you need the path to the key and the set command. The path to the key starts with a colon (":") and uses a colon as the separator for each level in the hierarchy. Here's how to change ("set") the value of the existing "pid" key to 94758476, in the plist, "com.mactech.example.plist":

/usr/libexec/PlistBuddy -c "Set :cust_info:pid 94758476" ./com.mactech.example.plist

(This is running the command in the same directory as the target plist. Otherwise, you'd need to specify the full path to the plist to edit). See the PlistBuddy man page (note the capitalization!) for more information on the utility. PlistBuddy is capable of much, much more, including copying values and merging plist files.

Accessing plists Via Python

From time to time, as a system administrator, you'll find yourself in a position where you'd like a script to store its own preferences. Or, simply have a script analyze a plist and act on the contents in some manner. In many cases, bash scripting that uses the commands already presented (plutil, PlistBuddy and, particularly, defaults) will be perfectly acceptable. However, for anything with a little more complexity, you may already be scripting in Python (or perl, or Ruby, etc.). Since Mac in the Shell has been focusing on Python for the last several columns, we'll use it here as well.

Python, with PyObj-C, makes this trivial. More interestingly, you get the best of both worlds: Apple's APIs along with Python's ease of use and the speed of the edit and run cycle (skipping the compile step of C-based languages). To see this in action, let's start with nearly the most simple example possible. Listing 2 contains write_plist.py, which demonstrates creating a dictionary that gets written to a plist.

Listing 2-write_plist.py

#!/usr/bin/python2.5
from Foundation import NSMutableDictionary
my_dict = NSMutableDictionary.dictionary()
my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'
success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)
if not success:
  print "plist failed to write!"
  sys.exit(1)

Upon running this program, com.mactech.example.plist will be created in the same working directory as the program itself. The plist file will match the output that is shown in Listing 1. Let's examine this line-by-line to see how it works.

The very first line-#!/usr/bin/python2.5-is a good reminder that Python version 2.5 or higher is required for PyObj-C integration. This will not work on Tiger systems out of the box.

from Foundation import NSMutableDictionary

This import is responsible for all of the magic here. While we could import all of Foundation, we'll just import the portion we need: NSMutableDictionary.

my_dict = NSMutableDictionary.dictionary()
-

Typically, creating a dictionary in Python would use curly braces, like this:

new_dict = {}

or, you can even fill it on creation:

new_dict = {'color':'blue', 'count':15, 'style':'fruit'}

However, we need to create a real Cocoa NSMutableDictionary object, so that's what we've done. Nicely, we can no go on and treat that just like a Python dictionary:

my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'

You can use the Cocoa API for adding entries to a dictionary as well:

my_dict.setValue_forKey_('stop', 'state')

This would set the key 'state' to store the value 'stop', and add the following to the plist once written out:

<key>state</key>
<string>stop</string>

But, really... if you're using Python, take advantage of it where you can! (I suggest using the Python method). You will need to use the Cocoa API to write the dictionary out to disk as a plist file:

success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)

The Cocoa writeToFile:atomically: method of NSDictionary (and, by extension, NSMutableDictionary) writes a property list representation of the contents of the dictionary to the path given.

if not success:
  print "plist failed to write!"
  sys.exit(1)

This final conditional tests to see if the writeToFile:atomically: method returned a True ("success") or False ("failure") value. While not strictly necessary for this program to run, checking these values is a good habit to get into.

Python Ease

Just as a reminder, once you create the NSMutableDictionary, you can use standard Python mechanisms to manipulate and traverse it. Adding a key with a dictionary as its value is as simpe as you'd expect. Just create the dictionary and then assign it to the parent dictionary. For example, to recreate the com.mactech.example.plist shown earlier, we would add the following to our program, after creating the initial dictionary:

sub_dict = {}
sub_dict['uid'] = '348576'
sub_dict['pid'] = '98234573'
my_dict['cust_info'] = sub_dict

Also, as shown earlier, you can also use all of the Cocoa APIs available to you to manipulate the dictionary as well. The style you choose may be situation dependent. Some situations may call for using the Cocoa-way, while others may favor more Pythonic writing. When working with any Cocoa API, though, as always, you'll want to keep the documentation handy.

Use It or Lose It

This was an incredibly fun article to write. The topic is incredibly practical for everyday use. The plist format is pervasive throughout OS X. Every technical person should have a familiarity with it, and System Admins should be even more deeply involved. While many cases can simply be solved with a single command-line call to defaults or PlistBuddy, anything with deeper involvement should use a scripting language like Python. The nice thing about the scripting solution is that once you build up your library of routines, they're written and ready for re-use. Reading about it here only gets you so far. Go write a script and run it on a test system so you're ready for the real thing when the opportunity arrives.

Media of the month: Kick it old-school with vintage computer brochures and manuals at http://assemblyman-eph.blogspot.com/2009/04/vintage-computer-brochures.html. Full PDFs of how it used to be. I remember when taking one of my first computer courses, the teacher launching into the history of computing. Naturally, I just wanted to get into sitting at a computer and coding. But now, perhaps more than ever before, it's really useful to be able to frame our current experience with that which it was built on and evolved from.

Next month, we'll be covering Snow Leopard related topics! It'll be two issues before we get back to Python and scripting in general. Until then, keep practicing.

References

"About Property Lists": https://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/AboutPropertyLists/AboutPropertyLists.html#/apple_ref/doc/uid/20001010-46719

"Understanding XML Property Lists": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

"Introduction to Property List Programming Topics for Core Foundation": http://developer.apple.com/iphone/library/documentation/CoreFoundation/Conceptual/CFPropertyLists/CFPropertyLists.html

"Introduction to User Defaults": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1


Ed Marczak is the Executive Editor of MacTech Magazine. He has written for MacTech since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Amazon Chime 4.3.5721 - Amazon-based com...
Amazon Chime is a communications service that transforms online meetings with a secure, easy-to-use application that you can trust. Amazon Chime works seamlessly across your devices so that you can... Read more
Posterino 3.3.7 - Create posters, collag...
Posterino offers enhanced customization and flexibility including a variety of new, stylish templates featuring grids of identical or odd-sized image boxes. You can customize the size and shape of... Read more
Airmail 3.2.9 - Powerful, minimal email...
Airmail is an mail client with fast performance and intuitive interaction. Support for iCloud, MS Exchange, Gmail, Google Apps, IMAP, POP3, Yahoo!, AOL, Outlook.com, Live.com. Airmail was designed... Read more
Arq 5.8.4 - Online backup to Google Driv...
Arq is super-easy online backup for Mac and Windows computers. Back up to your own cloud account (Amazon Cloud Drive, Google Drive, Dropbox, OneDrive, Google Cloud Storage, any S3-compatible server... Read more
Microsoft Remote Desktop 8.0.39 - Connec...
With Microsoft Remote Desktop, you can connect to a remote PC and your work resources from almost anywhere. Experience the power of Windows with RemoteFX in a Remote Desktop client designed to help... Read more
Arq 5.8.4 - Online backup to Google Driv...
Arq is super-easy online backup for Mac and Windows computers. Back up to your own cloud account (Amazon Cloud Drive, Google Drive, Dropbox, OneDrive, Google Cloud Storage, any S3-compatible server... Read more
Airmail 3.2.9 - Powerful, minimal email...
Airmail is an mail client with fast performance and intuitive interaction. Support for iCloud, MS Exchange, Gmail, Google Apps, IMAP, POP3, Yahoo!, AOL, Outlook.com, Live.com. Airmail was designed... Read more
Microsoft Remote Desktop 8.0.39 - Connec...
With Microsoft Remote Desktop, you can connect to a remote PC and your work resources from almost anywhere. Experience the power of Windows with RemoteFX in a Remote Desktop client designed to help... Read more
Posterino 3.3.7 - Create posters, collag...
Posterino offers enhanced customization and flexibility including a variety of new, stylish templates featuring grids of identical or odd-sized image boxes. You can customize the size and shape of... Read more
Airmail 3.2.8 - Powerful, minimal email...
Airmail is an mail client with fast performance and intuitive interaction. Support for iCloud, MS Exchange, Gmail, Google Apps, IMAP, POP3, Yahoo!, AOL, Outlook.com, Live.com. Airmail was designed... Read more

Latest Forum Discussions

See All

Pokémon GO might be getting legendaries...
The long-awaited legendary Pokémon may soon be coming to Pokémon GO at long last. Data miners have already discovered that the legendary birds, Articuno, Moltres, and Zapdos are already in the game, it’s just a matter of time. [Read more] | Read more »
The best deals on the App Store this wee...
If you’ve got the Monday blues we have just the thing to cheer you up. The week is shaping up to be a spectacular one for sales. We’ve got a bunch of well-loved indie games at discounted prices this week along with a few that are a little more... | Read more »
Honor 8 Pro, a great choice for gamers
Honor is making strides to bring its brand to the forefront of mobile gaming with its latest phone, the Honor 8 Pro. The Pro sets itself apart from its predecessor, the Honor 8, with a host of premium updates that boost the device’s graphical and... | Read more »
The 4 best outdoor adventure apps
Now that we're well into the pleasant, warmer months, it's time to start making the most of the great outdoors. Spring and summer are ideal times for a bit of trekking or exploration. You don't have to go it alone, though. There are plenty of... | Read more »
Things 3 (Productivity)
Things 3 3.0.1 Device: iOS iPhone Category: Productivity Price: $7.99, Version: 3.0.1 (iTunes) Description: Meet the all-new Things! A complete rethinking of the original, award-winning task manager – with a perfect balance between... | Read more »
Oddball mash-up Arkanoid vs Space Invade...
In a move no one was really expecting, Square Enix has put forth an Arkanoid/Space Invaders mash-up aptly titled Arkanoid vs Space Invaders. The game launched today on both iOS and Android and the reviews are actually quite good. [Read more] | Read more »
Arkanoid vs Space Invaders (Games)
Arkanoid vs Space Invaders 1.0 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0 (iTunes) Description: LAUNCH SALE: GET THE GAME AT 20% OFF! Two of the most iconic classic games ever made meet in Arkanoid vs Space... | Read more »
The best new games we played this week
Things got off to a bit of a slow start this week, but as we steadily creep towards Friday a bunch of great games have started cropping up. If you're looking for a quality new release to play this weekend, we've got you covered. Here's a handy... | Read more »
No Stick Shooter (Games)
No Stick Shooter 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Happy Robot presents a fresh take on the classic retro-arcade shooters we all know and love. Featuring 30 levels of non-stop... | Read more »
Chroma Squad (Games)
Chroma Squad 1.0.85 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.85 (iTunes) Description: | Read more »

Price Scanner via MacPrices.net

Five To Six Million 10.5-inch iPad Pro Tablet...
Digitimes’ Siu Han and Joseph Tsai report that upstream supply chain shipments for Apple’s new 10.5-inch iPad Pro have been increasing, with monthly shipment volume expected to hit 600,000 units by... Read more
Georgia Tech Students Win Toyota and Net Impa...
Earlier this year, a team of students at Georgia Tech realized that there was a critical gap in transportation services for people who use wheelchairs, and wondered if the solution could be in the... Read more
13-inch 2.0GHz Space Gray MacBook Pro on sale...
Amazon has the 13″ 2.0GHz Space Gray non-Touch Bar MacBook Pro (MLL42LL/A) on sale for $1299.99 including free shipping. Their price is $200 off MSRP, and it’s currently the lowest price available... Read more
Roundup of 15-inch MacBook Pro sale prices, m...
B&H Photo has the new 2016 15″ Apple Touch Bar MacBook Pros in stock today and on sale for up to $200 off MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: - 15″ 2.7GHz... Read more
15-inch 2.2GHz Retina MacBook Pro on sale for...
B&H Photo has the 2015 15″ 2.2GHz Retina MacBook Pro (MJLQ2LL/A) on sale for $1849 including free shipping plus NY & NJ sales tax only. Their price is $150 off MSRP. Read more
21-inch iMacs on sale for $100 off MSRP, save...
B&H Photo has 21″ iMacs on sale for $100 off MSRP, each including free shipping plus NY & NJ sales tax only: - 21″ 3.1GHz iMac 4K: $1399.99 $100 off MSRP - 21″ 2.8GHz iMac: $1199.99 $100 off... Read more
Groupon Survey Finds Families Will Spend One-...
With warmer weather finally arriving in most parts of the country and the school year winding down to a close, Groupon (http://www.groupon.com) asked 1,000 U.S. parents how much time their families... Read more
Panasonic Anounces Toughbook 33 2-in-1 Rugged...
Panasonic has announced the U.S. availability of the Panasonic Toughbook 33, claimed to be the first fully-rugged 2-in-1 detachable laptop that features a 3:2 display. Inspired by the hard-working... Read more
9-inch 32GB iPad Pros available for up to $10...
Amazon resellers are offering new 9″ 32GB iPad Pros for up to $101 off MSRP, starting at $498, including free shipping. Colors may be Space Gray, Silver, Gold, or Rose Gold, depending on the reseller... Read more
Back in stock: Apple refurbished Mac minis st...
Apple has Certified Refurbished Mac minis available starting at $419. Apple’s one-year warranty is included with each mini, and shipping is free: - 1.4GHz Mac mini: $419 $80 off MSRP - 2.6GHz Mac... Read more

Jobs Board

Director, *Apple* Program - ERP & Cloud...
…make a real difference. Come, shine with us! Astellas is announcing a Director, Apple Program - ERP & Cloud Platform Architecture Lead opportunity in Northbrook, IL. Read more
*Apple* Mobile Master - Best Buy (United Sta...
**508102BR** **Job Title:** Apple Mobile Master **Location Number:** 000511-Athens-Store **Job Description:** **What does a Best Buy Apple Mobile Master do?** At Read more
*Apple* Media Products - Commerce Engineerin...
Apple Media Products - Commerce Engineering Manager Job Number: 57037480 Santa Clara Valley, California, United States Posted: Apr. 18, 2017 Weekly Hours: 40.00 Job Read more
Geek Squad *Apple* Master Consultation Agen...
**500444BR** **Job Title:** Geek Squad Apple Master Consultation Agent **Location Number:** 000103-City Of Industry-Store **Job Description:** **What does a Geek Read more
Security Engineer, *Apple* Retail - Apple I...
Changing the world is all in a day's work at Apple . If you love innovation, here's your chance to make a career of it. You'll work hard. But the job comes with more Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.