Volume Number: 26
Issue Number: 02
Column Tag: Installers

The Flat Package

Examining a newer package format

by José R.C. Cruz

In today's article, we will look at the new flat package format. First, we learn how this format compares with earlier bundle-style formats. Then we examine the structure of a typical flat package. Finally, we look at three useful tools with which to manage the package.

The article assumes that readers know how to use PackageMaker 3.0, as well as the Terminal utility. If this is your first time using PackageMaker 3.0, refer to my introductory article Packaging For Leopard for the necessary basics.

When PackageMaker starts a new project, it displays its sheet dialog Install Project (Figure 1). Use this dialog to supply some basic information about your project. For instance, in the Organization field, enter a unique ID for the package. Convention dictates that you use your product or company's URL as the ID string. Make sure to enter the string using reverse-domain syntax. If your product URL is www.foobar.com/foo, then type the string as com.foobar.foo.

Next, from the pop-up menu Minimum Target, choose which OS target you plan to support at least. This menu provides three choices: Panther (10.3), Tiger (10.4), and Leopard (10.5). By choosing an OS target, you set the format of your final installer package.

You can always change the format by choosing Install Properties from the Project menu, and then choosing a different OS target from the pop-up menu. Doing so, however, may corrupt your project file. A safer approach is to keep a separate project file for each OS target.

Figure 1. The Install Properties dialog.

The payload package

Assume you have chosen 10.3 as the minimum OS target. Also, assume your installer project has only one payload. If you choose Build from the Project menu, PackageMaker stores your payload inside a payload package, also known as a component package. This package is the simplest of four possible packages.

Figure 2 shows the structure of a typical payload package. As you can see from the diagram, the package is, in essence, a bundle. Its root directory, Contents, contains four items, which are listed as follows.

• Archive.bom

  - A binary file that serves as the bill of materials for the entire package. To view its data, use the command-line tool lsbom.

• Archive.pax.gz

  - The payload itself rendered as a pax archive and compressed with the gzip tool.

• Info.plist

  - An XML file containing the default settings for the entire package. To view its contents, use the Property List Editor tool. You can also view the file with a text editor, if you are familiar with XML.

• PkgInfo

  - A simple text file containing the file and creator type of the entire bundle.

• Resources

  - A directory of language bundles containing simple localization strings.

Figure 2. The payload package.

The meta-package

Assume again that your minimum OS target is 10.3. But this time, you have more than one payload in your installer project. Or you have customized the install panels using PackageMaker's Interface Editor. Or you used external scripts to customize the install session. When you choose Build from the Project menu, PackageMaker builds the installer package as a meta-package (Figure 3).

In a meta-package, each payload is stored as a payload package in Contents/Packages/. Any files used to localize the install panel go into the correct lproj bundle in Contents/Resoures/. External scripts that customize the install session also go into Contents/Resources/.

But the meta-package does not let you customize the Conclusion panel nor the payload choices. It does not support scripts written in Installer JavaScript. And it cannot display just the Easy Install or Custom Install panels -- it has to display both panels.

Figure 3. The meta-package.

The distribution package

Now assume you have 10.4 as your minimum target. In this case, when you choose Build from the Project menu, PackageMaker builds your installer package as a distribution package (Figure 4). At first, its structure appears similar to that of the meta-package. A distribution package, however, has several advantages over a meta-package.

First, it lets you use embedded text to customize each install panel. It also lets you customize all install panels, as well as the payload choices. Next, it supports scripts written in the new Installer JavaScript dialect. Plus, it can assume one of three install modes: Easy, Custom, or both. Finally, it keeps all its scripts, settings, and some localized text in a single XML file, distribution.dist. But it still keeps localized files inside an lproj bundle in Contents/Resources/.

Interestingly enough, the meta-package also has a distribution.dist file. This is likely an attempt to support that format on 10.5.

Figure 4. The distribution package.

The Flat Package

Finally, assume you selected 10.5 as your minimum OS target. (In Snow Leopard's PackageMaker v3.04, "10.5" remains the highest target OS). Now, when you choose Build from the Project menu, PackageMaker builds the package as a flat file, not as a bundle. As a result, you can no longer view the package's contents from the Finder using the Show Package Contents menu.

The flat package uses a structure different from the previous three formats (Figure 5). First, it does away with the Contents directory, and certain files like Info.plist and PkgInfo. Second, it keeps its main items like payloads and resources at the root level. Third, it renames the distribution.dist file as simply Distribution. It still, however, uses lproj bundles to store any localized files.

Figure 5. The flat package.

The flat package also uses a more compact format for its payloads. Each payload package now has three files, instead of the bevy of items from older formats (see Figure 4). The Bom file takes the place of Archive.bom. The Payload file replaces Archive.pax.gz, using gzipped cpio as an archive format. The PackageInfo file, on the other hand, is an XML rendition of PkgInfo.

Finally, the flat package stores its install scripts differently. A pre- or post-install action, for instance, becomes an action payload (Figure 6). Its payload name consists of the main package name and the action type.

Figure 6. A flat package with an action payload.

On the other hand, pre- and post-install scripts for each payload combine into a single Scripts file, which is then stored in each payload package (Figure 7). This Scripts file uses specialized tags to identify and separate each script. I have, of course, covered pre- and post-install actions and scripts in a previous MacTech article.

Figure 7. A flat package with a Scripts file.

Since the flat package is a single file, it is much easier to distribute over the network. Bundle packages, on the other hand, needs to be "repackaged" in another form like a disk image or a zip archive. But the flat package is supported only on OS X 10.5 or newer. It harder to customize after being built, and it does not support installer plug-ins.

PackageMaker 3.0 comes with three tools that you can use to work with flat packages. One tool, the Flat Package Editor, lets you view the package's contents, as well as make some simple changes. Obviously, you cannot use this tool to open one of the three bundle packages. Doing so will only result in an error.

There are two ways to start the editor. The first way is to choose Launch Flat Package Editor... from the Edit menu of PackageMaker. After the editor launches, choose Open from its File menu, and use the Open File dialog to select the flat package. Click the Open button to open the selected package.

The second way is to select the flat package with a Control-click. Then choose Flat Package Editor from the Open With contextual sub-menu. Both approaches will display the package contents as a hierarchical list (Figure 8). Bundles and directories inside the package appear with a triangle widget next to their icons. Clicking the triangle displays the contents of that bundle or directory.

Figure 8. The Flat Package Editor window.

The Flat Package Editor is not a "separate" tool. In fact, it resides inside the Contents/Resources directory of PackageMaker 3.0. You can, if you prefer, place the editor on the Dock. Just switch to the Finder and Control-click the PackageMaker tool. Choose Show Package Contents from the contextual menu, and open the Contents/Resources directory. Then drag Flat Package Editor from that directory onto the Dock.

Adding an item

Assume you want to add a new item to the flat package. To add a file, drag the file from the Finder to the editor window. Once the editor highlights the desired position in the package, drop the file at that position (Figure 9). The editor then copies the file into the package. If the position happens to be in a bundle or directory, the editor reveals its contents prior to the drop.

Figure 9. Adding a file.

To add a bundle or directory, use the same steps of selecting the item from the Finder and dragging it to the editor window. This time, when you drop the item, the editor copies not only item but its contents as well (Figure 10). It even includes any subdirectories or hidden files inside the item. At the time of writing, the editor does not provide any means to filter out unwanted items.

Figure 10. Adding a directory.

To add a new directory to the package, click the New Folder button from the editor toolbar. The editor adds the directory at the end of the list (Figure 11, left). You can then rename the new directory by double-clicking its name.

Figure 11. Adding a new directory.

To enclose a package item inside a new directory, first select the item from the window. Then click the New Folder button from the toolbar. This time, the editor creates the directory at the same position as the item. It then moves the item into said directory (Figure 11, right). But the editor will not enclose certain files (such as Distribution) with a new directory. Nor will it do the same for payload bundles or the Resources directory. So, if you do select any of these items, the editor will simply refuse to do the New Folder action.

Finally, to cancel any of the additions, choose Undo from the Edit menu. The editor then cancels each addition starting with the most recent one. To cancel all additions, choose Revert from the File menu. This one, however, leaves the editor in an unstable state, which can result in a crash.

Removing an item

Now assume you want to remove an item from the flat package. First, select the item from the editor window. Then click the Delete button on the window toolbar (Figure 12). If the selected item is a file, the editor marks it for deletion and then removes its icon from the window. If the item is a bundle or directory, the editor marks it and its contents for deletion. In either case, the item is still present in the flat package. To complete the removal, choose Save from the File menu. To cancel the removal, close the editor window; then click the Don't Save button from the ensuing dialog.

Figure 12. Removing an item.

This feature has some notable quirks. First, pressing the Delete or Backspace key does not remove the item; neither does choosing Delete from the Edit menu. Also, choosing Undo does not cancel the removal; nor does choosing Revert from the File menu. In fact, choosing Revert can make the editor unstable, prone to a later crash. This is especially true if the item being removed is either a bundle or a directory.

Editing an item

We can also edit the package items, although in a limited and restricted fashion. To start, select the item in question; then click the Info button from the editor toolbar. The editor responds by displaying an Info panel for the selected item (Figure 13).

Figure 13. Editing a package item.

Now the Info panel lets you change some aspects of the selected item. To change its name, enter the new name into the field provided. To change the compression scheme, choose the new scheme from the pop-up menu. At the time of writing, the editor provides three compression options: gzip, bzip2, and none. Future versions of the editor may present more options.

Close the Info panel after you made your changes. Then choose Save from the File menu to commit your changes to the package.

The edit feature has a couple of quirks. First, selecting a different package item does not update the Info panel. Instead, the panel continues to display information for the last selected item. You can, however, click the Info button and get another Info panel for the new selection. Second, changing the compression scheme sometimes does not affect the item's size. This may mean a number of things: the item is incompressible, the editor has a display bug, or the compression simply did not work. Whatever the cause is unconfirmed at this time.

Signing the package

Finally, you can use the editor to digitally sign a flat package. This feature helps ensure and protect a package's authenticity for your users. You can learn more about digital signatures, their benefits, and issues in MacTech's series on PKI (June 2009, through September 2009), or from the following Wikipaedia entry:

http://en.wikipedia.org/wiki/Digital_signature

First, you need a digital certificate. There are several ways to obtain one, but the simplest way is to use the Keychain Access utility. Go to /Applications/Utility, and locate and launch Keychain Access. Then go the Keychain Access menu, and choose Create a Certificate from the Certificate Assistant submenu. Follow the instructions on the assistant dialog (Figure 14) to create your own digital certificate. Keychain Access then adds the certificate to the My Certificates group.

Figure 14. Creating a digital certificate.

Next, switch to the Flat Package Editor and click the lock icon on the upper-right corner of the toolbar. The editor responds with a sheet dialog (Figure 15). To see more details of the certificate, select it and click the Show Certificate button. To use the certificate, click the Choose button. Then choose Save from the File menu to sign the package with that certificate. If the signing is successful, the editor will display a grey lock icon.

Figure 15. Selecting a digital certificate.

The pkgutil tool is another utiity that we can use when working with a flat package. Unlike the Flat Package Editor, pkgutil is a command-line tool. This means we can only use the tool within a Terminal window session or as part of a shell script.

The basic usage syntax of the pkgutil tool is as follows.

pkgutil command [package [directory]]

Here, the command flag sets the desired operation. The package argument is the path to the flat package, the directory argument the path to a working directory. There are but a handful of commands that apply a flat package. The rest of the commands only apply to the new receipts database, which is a topic for another time.

Examining a package

Assume you are working with the flat package Foobar.pkg. To get a list of payloads, use the command --payload-files. Then pass the package's name as follows:

pkgutil --payload-files Foobar.pkg

The tool lists each path to the payload relative to the package. If you find the list too long to read, pipe the tool's output to the less tool for paging (press the 'q' key to exit the less utility).

pkgutil --payload-files Foobar.pkg | less

To filter out specific payloads, pipe the output to the grep tool.

pkgutil --payload-files Foobar.pkg | grep "BBEdit"

In the above line, grep displays only the payloads whose paths contain the string "BBEdit". To save the list, redirect the output to the desired file.

pkgutil --payload-files Foobar.pkg > foobar.log

This line saves the payload list for Foobar.pkg into the file foobar.log.

Retrieving the Bom

Next, assume you want the bill of materials for Foobar.pkg. To get the Bom, use the --bom command. Again, pass the package's name as follows.

   pkgutil --bom Foobar.pkg

The pkgutil tool responds by extracting the Bom file inside Foobar.pkg. It then stores the file into /tmp using the following path pattern.

   /tmp/Foobar.pkg.boms.id/payload.pkg/BOM

Now in actual usage, the id part of the path is a six-character ID string. The payload part is the name of the payload package. Once you have extracted the Bom, use the lsbom tool to read the Bom data as follows.

   lsbom /tmp/Foobar.pkg.boms.id/payload.pkg/BOM

Suppose Foobar.pkg has two payloads: pictures.pkg and scripts.pkg. In this case, pkgutil extracts the Bom files from each payload and stores each file into /tmp at the following paths.

   /tmp/Foobar.pkg.boms.id/pictures.pkg/BOM

   /tmp/Foobar.pkg.boms.id/scripts.pkg/BOM

Each path differs in the value of its id and payload parts.

Assemble and dissamble

Finally, assume you want to change the contents of Foobar.pkg. To do so, you need to take apart the package, make your changes, and then put those pieces back together.

To disassemble Foobar.pkg, use the pkgutil tool with an --expand command. Pass the package's name and a target directory as follows.

   pkgutil --expand Foobar.pkg foobar

Here, the pkgutil tool extracts all the items in Foobar.pkg and stores those items into the foobar directory. The tool even arranges the items in the same order in the same order they appear in Foobar.pkg. If the foobar directory does not exist, pkgutil will create an empty directory.

To assemble a new flat package, use the tool with a --flatten command. Then pass the source directory and the package's name as follows.

   pkgutil --flatten foobar barfoo.pkg

In the above example, pkgutil uses the items stored in the foobar directory to create the new flat package named barfoo.pkg. Make sure, of course, to maintain the same directory structure as shown in Figure 5. Otherwise, the newly created flat package will be unusable.

Yet, another tool we can use with flat packages is the xar tool. This tool creates the archive format used by a flat package. The xar tool is a product of the defunct OpenDarwin project. It is a command-line tool, like pkgutil. But while pkgutil works on flat packages and the new receipts database, xar works exclusively on flat packages.

Below is the basic usage syntax for the tool.

xar -command [-options] -f package [operands]

The -command flag sets the desired operation, the -options flag the desired behavior. The -f flag specifies the package name, while operands specifies one or more files or directories.

Mac OS X 10.6.1 bundles version "1.6dev" of the xar tool. To find out what version of the tool you have, type xar --version at the Terminal prompt. To get its online manual, type info xar.

Also note that the ditto command can read and write cpio archives, and may be more convenient than using xar.

Tool operations

Assume you are still working with the package Foobar.pkg. To view its contents, use the xar tool with a -t command as follows.

   xar -t -f Foobar.pkg

The tool responds by listing the contents of Foobar.pkg on stdout. It even lists the contents of any bundles or directories in the package. To extract a specific item, use the -x command as follows.

   xar -x -f Foobar.pkg Distribution

Here, the tool extracts the Distribution file from Foobar.pkg. To extract all the items from Foobar.pkg, use the same command as follows.

   xar -x -f Foobar.pkg

At the time of writing, the tool stores the extracted files at the same location as Foobar.pkg. It will not let you choose a different location.

Next, assume you have your payloads, support files and bundles in the directory Foo. To store these items into a flat-package, first type cd Foo at the Terminal prompt. Then use the xar tool with a -c command as follows.

   xar -c -f Barfoo.pkg *

Here, the xar tool creates the package Barfoo.pkg using the items in the Foo directory. But creating a flat package with the xar tool presents some issues. First, the payloads must use the bundle format shown in Figure 5. This is difficult to do as we do not know the format of the Payload file.

Second, the tool does not allow new items added to the package. So, if you type the following line,

   xar -c -f Barfoo.pkg foo.pkg 

the tool will not add the payload foo.pkg to Barfoo.pkg. Instead, it will replace Barfoo.pkg with one that contains only the payload foo.pkg.

Third, the tool can only create the flat package in the same working directory as the items. If you type this line,

   xar -c -f Barfoo.pkg Foo/*

the tool first stores Foo at the package's root level, then each item into the Foo directory. In other words, the xar tool does not see Foo as the root level of the package.

Tool options

When the xar tool creates a package, it compresses each item with gzip. To use a different compression scheme, specify the scheme by name with the --compression option. For instance, to use bzip2 compression, type the xar statement as follows.

   xar --compression bzip2 -c -f Foobar.pkg *

To leave the package uncompressed, pass none to the option flag.

   xar --compression none -c -f Foobar.pkg *

At the time of writing, the tool supports three possible schemes: gzip, bzip2, and none. Future versions of the tool may support other schemes.

Next, xar excludes all invisible items as it creates a package. To exclude other items, add the exclude option to the tool. Then pass a regex pattern as the option's input. For instance, to exclude any file named "narf", type the xar statement as follows.

   xar --exclude "narf" -c -f Foobar.pkg *

Finally, the xar tool does its tasks quietly, displaying only error messages. For a more verbose operation, use the tool with a -v option flag.

   xar -v -c -f Foobar.pkg *

In the above example, the tool will list each file it adds to the package. And it will print any status messages for the task. Use this option flag to identify any problems when using the tool.

The flat package has several benefits not found in the older bundle-type packages. It is easier to distribute online due to being a single file. Its structure is simpler and more compact than either a meta- or distribution package. Also, it supports digital signatures.

Yet the flat package is harder to customize and fix once built. Its support tools are either unpolished or limited in features. It does not support plug-ins to enhance the install session. Also, it can only deliver its payloads to 10.5 or 10.6 targets.

So, if you still support 10.4 and older targets, best stay with either a meta- or distribution package. But if you plan to support only targets that are 10.5 or above, then give the flat package a try.

Apple Computers. "Packages". Software Delivery Guide. 2006 Jul 24. Copyright 2007. Apple Computers, Inc. Online:

http://developer.apple.com/documentation/DeveloperTools/Conceptual/SoftwareDistribution/Managed_Installs/chapter_5_section_2.html#//apple_ref/doc/uid/10000145i-CH6-SW11

Apple Computers. "pkgutil". BSD General Commands Manual. 2008 Apr 2. Copyright 2008. Apple Computers, Inc. Online:

http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/pkgutil.1.html

Apple Computers. "xar". User Commands. 2005 Aug 22. Copyright 2005-2008. Apple Computers, Inc. Online:

http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/xar.1.html

Rob Braun. Why xar is interesting. 2007 Jan 20. Accessed on 2008 Sep 21. Google Code. Online:

http://code.google.com/p/xar/wiki/whyxar

Jos é R.C. Cruz. Packaging For Leopard. MacTech Magazine. 2008 June.

JC is a freelance engineering writer from North Vancouver, British Columbia. He spends his time writing technical articles; tinkering with Cocoa, REALbasic, and Python; and visiting his foster nephew. He can be reached at anarakisware@gmail.com.