TweetFollow Us on Twitter

Mac in the Shell-Learning Python on the Mac: Modules

Volume Number: 25
Issue Number: 03
Column Tag: Mac in the Shell

Learning Python on the Mac: Modules

Organize code into reusable entities

by Edward Marczak

Introduction

Python has an incredible amount to offer, and its concept of modules is near the top of the list of power. Modules bring a lot of functionality to the language, and offer the author the ability to reuse code in as many different applications as they wish. Since the start of our journey down the path of Python, we've covered basic data types, flow control and functions. Modules will be one of the last very high-level concepts we cover for a bit, but it's an important one. Like most Mac in the Shell columns, this one, too, is best read with a Terminal.app window open and your favorite text editor up and running. So, clear your brain, and let's dive in.

What is a Module?

Modules, really, are just files. Like many things in Python, some very powerful features come with little fanfare, and they 'just work.' By placing functions, variables and code into a standalone file, you create an organized, self-contained, reusable package known as a namespace. In the past two months, you've seen the import keyword, and just had to trust me that it did something useful.

In actuality, import reaches out into the file system, locates, reads and runs an external module. The imported file becomes an object in the importing script, and all definitions in the file become attributes of that object. For illustrative purposes, let's imagine this: top-level ("main") file a.py imports b.py. b.py also imports c.py and also imports some modules from the standard Python library of modules. This is shown in Figure 1.


Figure 1 - Python high-level program architecture.

In essence, the introductory Mac in the Shell articles introducing Python were incredibly simplified; it's rare that a Python program won't have the need to import a module. While we'll be concentrating on importing modules written in Python, do note that Python can actually import a variety of different module types.

How does python know where to find modules? The python interpreter walks a pre-defined set of locations to know where to look. After looking in the same directory as the importing program, an environment variable named PYTHONPATH is consulted. PYTHONPATH is a simple list of directories in which to search for modules. It is constructed just like the PYTHONDOCS variable, covered in the second article introducing Python (December 2008, issue 24.12). After searching PYTHONPATH, python will search among the standard library (location defined at compile time). Finally, python includes any paths listed in .pth files (a slightly advanced topic that we won't be covering this month).

Most people don't ever have to create a PYTHONPATH variable, as the default locations serve most needs. However, if you're sharing development among a few people and need to have a common location for specific modules, you'll need to point PYTHONPATH to your custom location. Like the PATH variable, PYTHONPATH is simply a colon-separated list of paths, searched in order (from left to right). For example, in the bash shell:

export PYTHONPATH="/usr/share/python/"

With this path set, python will additionally search for modules in the /usr/share/python directory.

All of this comes together for the python interpreter as sys.path. From a running program, or, via the interactive interpreter, import the built-in sys module and print sys.path. Here's an abridged listing:

>>> import sys
>>> print sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5']

Note that the path starts with an empty entry, which is how the program's directory is searched.

Let's start with a simple example to demonstrate modules.

Using Modules

Sticking with the a.py, b.py and c.py concept, let's define them for real. Follow along with these short listings.

Listing 1 - a.py

#!/usr/bin/env python
import b
if b.IsInAdmin('root'):
  print "root is in the admin group"
else:
  print "root is not in the admin group"

As stated earlier, the import statement reads a module file and executes it. So, the line import b reads the file b.py and executes it like any other Python program. The if statement tests the return value from an object imported from b. We can also say that the object is in b's namespace. Listing 2 reveals what b.py is doing.

Listing 2 - b.py

#!/usr/bin/env python
import c
def IsInAdmin(user):
  admins = c.GetAdminGroup();
  if user in admins:
    return True
  else:
    return False

b.py starts off with its own import: c.py. b.py defines one function, IsInAdmin(). Finally, look at c.py to see how it works its magic.

Listing 3 - c.py

#!/usr/bin/env python
import plistlib
def GetAdminGroup():
  admin_plist = plistlib.readPlist('/var/db/dslocal/nodes/Default/groups/admin.plist')
  return admin_plist['users']

c.py imports plistlib-something you don't have to define, as it's part of the standard modules shipped with OS X. plistlib allows the author an easy manner of reading and writing OS X's plist files (ASCII only, currently). Here, we'll directly grab the admin.plist file and return the contents from the users key.

Save all three of these files in the same directory, mark a.py executable (chmod 770 a.py), and run it via sudo (it needs root privileges as the admin.plist file is protected). The output looks simply like this:

$ sudo ./a.py 
root is in the admin group

Overall, that was fairly little work. Of course, in a real-world scenario, you wouldn't need to break it down as far as we did here. However, GetAdminGroup() might make a really nice addition to a larger OS X-specific library.

Imports and Namespaces

We've spoken a little about the concept of namespaces without formally defining it. A namespace is a mapping from names to objects. The important thing to realize is that there is no relation between names in different namespaces. Two modules can define x, and there will be no conflict. Internally to python, namespaces are implemented as dictionaries (remember those? Covered briefly in our introduction - November 2008, issue 24.11). To see this in action, run the interactive python shell (by typing python at the shell prompt) in the same directory as the modules we've been working on. A module's dictionary is hidden away in the __dict__ attribute. It responds to all standard dictionary messages. For example, to list the contents of the object, import our c.py module:

>>> import c
and then list the keys:
>>> c.__dict__.keys()
['GetAdminGroup', 'plistlib', '__builtins__', '__file__', '__name__', '__doc__']

This is the same list that you get already with the dir() function, covered in earlier columns. However, it does lead to some fairly (advanced) trickery and techniques.

The point of this is that an import statement will import objects into their own namespace. This name of a given namespace is derived from the filename. This means that module filenames must conform to good variable names. You can certainly create a module named while.py, but since "while" is also a built-in keyword, you won't be able to import it. Our c.py module has been imported into a namespace of 'c'.

This also means that you should be careful to name your module something useful, as you'll be referring to it in code... or will you? Well, that depends. It depends somewhat on how you import a module, and somewhat on style. I'll say it now: my personal preference is that you always import as we've been showing-by using the import keyword-and referencing the namespace in code. Some people consider this bulky, though. Look at our plistlib usage:

plistlib.readPlist()

Instead of simply calling readPlist, we have to qualify it. This prevents it from conflicting with any other function (object, really) of that name. Importing plistlib created a namespace for that module. However, there is a way to import into the current namespace, and that relies on the from keyword variant.

To import an entire module, or just a select part of it into the current namespace, you tell python, "from [module] import [what]". The "what" can reference a specific object in the module, or use a wildcard. To import all of c.py directly into the current namespace, b.py can issue:

from c.py import *

b.py could then make the following change:

admins = GetAdminGroup();

See? No need to qualify the function call. Also, be aware that a subset of a module can be imported by specifying it explicitly:

from plistlib import readPlist

While all of this may seem to lessen the burden on the python interpreter, it doesn't in reality. The problem with this is much more from a maintenance-for-the-author perspective. This is especially true when multiple people are working on a project, each by contributing a module. As you may guess, conflicting object names simply get crushed, and the last name defined 'wins.' You, the author, lose, of course. Additionally, since python reads the entire module file in any case, or, has a pre-compiled byte-code version, there are no real performance/time savings by importing a single object. You may change a module that is imported in this manner and pollute the namespace of the importing application without realizing it. Using a standard import, there will never be conflicting objects.

To Be Executed

I mentioned above that when a module is imported, it is run like any other Python application. This is true; it's how objects get created, just like in the top-level file. We can prove this, too. Consider the following code:

Listing 4 - msgtest.py

#!/usr/bin/env python
print "Running top-level python file"
import msgmod
print "Finishing up top-level file"

Listing 5 - msgmod.py

#!/usr/bin/env python print "Running msgmod.py module" Running msgtest.py yields the following output: Running top-level python file Running msgmod.py module Finishing up top-level file

Not surprising if you've been following along! The same goes for any assignment statements in a module. Of course, functions are read and turned into an object, but the contents of which are not executed until explicitly called. Sometimes, however, a file has value as both a top-level file and as something to be imported. As a top-level file, we want it to execute, but as an imported file, we just want it imported, ready at our beck and call. There's a Python-ism that addresses this. A built-in variable, __name__, is assigned in each module at runtime. When the top-level application is run, __name__ is "__main__". Instead of just defining statements at the top indent-level, every statement in the top-level file is put into a function (or class) definition. The only top level statement tests __name__, and call your main function as appropriate. For example, our a.py program would be altered like this:

#!/usr/bin/env python
import b
def main():
  if b.IsInAdmin('root'):
    print "root is in the admin group"
  else:
    print "root is not in the admin group"
if __name__ == '__main__':
  main()

This way, if we had other useful functions in a.py, it could be run directly, or imported as a module.

I will be using this idiom going forward.

Fin

For this month, that's enough. Modules are such a core concept in Python, it's worth it to make sure it's understood before moving on. Next month, we'll talk a little more about style, and then cover classes.

As an aside, I met many people at Macworld who were really exited by this column turning its attention to Python. I'd love to hear ideas from all readers, so, letters@mactech.com and emarczak@mactech.com are ready and waiting for mail.

Media of the month: "Synthetic Worlds: The Business and Culture of Online Games," by Edward Castronova. Interesting work that covers, mainly, the economy of virtual worlds (ok, MMOs). The book is a touch dated at this point, but overall, it should make the reader think about how much more our technology is touching and influencing our 'non-technological' world.

See you next month!


Ed Marczak is the Executive Editor of MacTech Magazine. He lives in New York with his wife, two daughters and various pets. He has been involved with technology since Atari sucked him in, and has followed Apple since the Apple I days. He spends his days on the Mac team at Google, and free time with his family and/or playing music. Ed is the author of the Apple Training Series book, "Advanced System Administration v10.5," and has written for MacTech since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

djay Pro 1.1 - Transform your Mac into a...
djay Pro provides a complete toolkit for performing DJs. Its unique modern interface is built around a sophisticated integration with iTunes and Spotify, giving you instant access to millions of... Read more
Vivaldi 1.0.118.19 - Lightweight browser...
Vivaldi browser. In 1994, two programmers started working on a web browser. Our idea was to make a really fast browser, capable of running on limited hardware, keeping in mind that users are... Read more
Stacks 2.6.11 - New way to create pages...
Stacks is a new way to create pages in RapidWeaver. It's a plugin designed to combine drag-and-drop simplicity with the power of fluid layout. Features: Fluid Layout: Stacks lets you build pages... Read more
xScope 4.1.3 - Onscreen graphic measurem...
xScope is powerful set of tools that are ideal for measuring, inspecting, and testing on-screen graphics and layouts. Its tools float above your desktop windows and can be accessed via a toolbar,... Read more
Cyberduck 4.7 - FTP and SFTP browser. (F...
Cyberduck is a robust FTP/FTP-TLS/SFTP browser for the Mac whose lack of visual clutter and cleverly intuitive features make it easy to use. Support for external editors and system technologies such... Read more
Labels & Addresses 1.7 - Powerful la...
Labels & Addresses is a home and office tool for printing all sorts of labels, envelopes, inventory labels, and price tags. Merge-printing capability makes the program a great tool for holiday... Read more
teleport 1.2.1 - Use one mouse/keyboard...
teleport is a simple utility to let you use one single mouse and keyboard to control several of your Macs. Simply reach the edge of your screen, and your mouse teleports to your other Mac! The... Read more
Apple iMovie 10.0.8 - Edit personal vide...
With an all-new design, Apple iMovie lets you enjoy your videos like never before. Browse your clips more easily, instantly share your favorite moments, and create beautiful HD movies and Hollywood-... Read more
Box Sync 4.0.6233 - Online synchronizati...
Box Sync gives you a hard-drive in the Cloud for online storage. Note: You must first sign up to use Box. What if the files you need are on your laptop -- but you're on the road with your iPhone? No... Read more
Fantastical 2.0.3 - Create calendar even...
Fantastical 2 is the Mac calendar you'll actually enjoy using. Creating an event with Fantastical is quick, easy, and fun: Open Fantastical with a single click or keystroke Type in your event... Read more

SoundHound + LiveLyrics is Making its De...
SoundHound Inc. has announced that SoundHound + LiveLyrics, will be one of the first third-party apps to hit the Apple Watch. With  SoundHound you'll be able to tap on your watch and have the app recognize the music you are listening to, then have... | Read more »
Adobe Joins the Apple Watch Lineup With...
A whole tidal wave of apps are headed for the Apple Watch, and Adobe has joined in with 3 new ways to enhance your creativity and collaborate with others. The watch apps pair with iPad/iPhone apps to give you total control over your Adobe projects... | Read more »
Z Steel Soldiers, Sequel to Kavcom'...
Kavcom has released Z Steel Soldiers, which continues the story of the comedic RTS originally created by the Bitmap Brothers. [Read more] | Read more »
Seene Lets You Create 3D Images With You...
Seene, by Obvious Engineering, is a 3D capture app that's meant to allow you to create visually stunning 3D images with a tap of your finger, and then share them as a 3D photo, video or gif. [Read more] | Read more »
Lost Within - Tips, Tricks, and Strategi...
Have you just downloaded Lost Within and are you in need of a guiding hand? While it’s not the toughest of games out there you might still want some helpful tips to get you started. [Read more] | Read more »
Entertain Your Pet With Your Watch With...
The Petcube Camera is a device that lets you use live video to check in on your pet, talk to them, and play with them using a laser pointer - all while you're away. And the Petcube app is coming to the Apple Watch, so you'll be able to hang out with... | Read more »
Now You Can Manage Your Line2 Calls With...
You'll be able to get your Line2 cloud phone service on the Apple Watch very soon. The watch app can send and receive messages using hands-free voice dictation, or by selecting from a list of provided responses. [Read more] | Read more »
R.B.I. Baseball 15 (Games)
R.B.I. Baseball 15 1.01 Device: iOS Universal Category: Games Price: $4.99, Version: 1.01 (iTunes) Description: The legendary Major League Baseball franchise returns to the diamond. Make History. ** ALL iPOD Touch, the iPad 2 and the... | Read more »
Here's How You Can Tell if an App W...
The Apple Watch is pretty much here, and that means a whole lot of compatible apps and games are going to be updated or released onto the App Store. That's okay though, beacause Apple has quietly updated their app description pages to make things... | Read more »
Forgotten Memories : Alternate Realities...
Forgotten Memories : Alternate Realities 1.0.1 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.1 (iTunes) Description: + REDUCED PRICE ONLY THE LAUNCHING WEEK + "The most exciting horror game of 2015." - AppSpy... | Read more »

Price Scanner via MacPrices.net

Intel Compute Stick: A New Mini-Computing For...
The Intel Compute Stick, a new pocket-sized computer based on a quad-core Intel Atom processor running Windows 8.1 with Bing, is available now through Intel Authorized Dealers across much of the... Read more
Heal to Launch First One-Touch House Call Doc...
Santa Monica, California based Heal, a pioneer in on-demand personal health care services — will offer the first one-touch, on-demand house call doctor app for the Apple Watch. Heal’s Watch app,... Read more
Mac Notebooks: Avoiding MagSafe Power Adapter...
Apple Support says proper usage, care, and maintenance of Your Mac notebook’s MagSafe power adapter can substantially increase the the adapter’s service life. Of course, MagSafe itself is an Apple... Read more
12″ Retina MacBook In Shootout With Air And P...
BareFeats’ rob-ART morgan has posted another comparison of the 12″ MacBook with other Mac laptops, noting that the general goodness of all Mac laptops can make which one to purchase a tough decision... Read more
FileMaker Go for iPad and iPhone: Over 1.5 Mi...
FileMaker has announced that its FileMaker Go for iPad and iPhone app has surpassed 1.5 million downloads from the iTunes App Store. The milestone confirms the continued popularity of the FileMaker... Read more
Sale! 13-inch 2.7GHz Retina MacBook Pro for $...
 Best Buy has the new 2015 13″ 2.7GHz/128GB Retina MacBook Pro on sale for $1099 – $200 off MSRP. Choose free shipping or free local store pickup (if available). Price for online orders only, in-... Read more
Minimalist MacBook Confirms Death of Steve Jo...
ReadWrite’s Adriana Lee has posted a eulogy for the “Digital Hub” concept Steve Jobs first proposed back in 2001, declaring the new 12-inch MacBook with its single, over-subscribed USB-C port to be... Read more
13-inch 2.7GHz Retina MacBook Pro for $1234 w...
Adorama has the 13″ 2.7GHz/128GB Retina MacBook Pro in stock for $1234.99 ($65 off MSRP) including free shipping plus a free LG external DVD/CD optical drive. Adorama charges sales tax in NY & NJ... Read more
13-inch 2.5GHz MacBook Pro available for $999...
 Adorama has the 13-inch 2.5GHz MacBook Pro on sale for $999 including free shipping plus NY & NJ sales tax only. Their price is $100 off MSRP. Read more
Save up to $600 with Apple refurbished Mac Pr...
The Apple Store is offering Apple Certified Refurbished Mac Pros for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The... Read more

Jobs Board

*Apple* Support Technician IV - Jack Henry a...
Job Description Jack Henry & Associates is seeking an Apple Support Technician. This position while acting independently, ensures the proper day-to-day control of Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Service-Learning Counselor, *APPLE* Corps -...
…CONTRACT TITLE Higher Education Assistant FLSA Exempt CAMPUS SPECIFIC INFORMATION APPLE Corps (Academic Preparation Program for Law Enforcement), a partnership between Read more
*Apple* iOS Specialist - Kforce (United Stat...
Our client is seeking an Apple iOS Specialist to join their team in Quincy, Massachusetts (MA). Duties: * Responsible for configuration and distribution of desktop, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.