TweetFollow Us on Twitter

File System Metadata

Volume Number: 18 (2002)
Issue Number: 12
Column Tag: Section 7

What's in there, beside the data...

by Rich Morin

One of Apple's larger challenges, in merging Mac OS and BSD, has been the need to reconcile differences between their respective ways of handling files. Previous columns have touched on some of these issues; this month, we'll go into the topic in a bit more depth.

In order to keep both BSD and Mac OS program(mer)s happy, Apple has had to implement nearly every feature found in either HFS+ (Mac OS's "Extended Hierarchical File System") or UFS (BSD's "Unix File System"). This is a challenging task; the systems evolved independently and a variety of decisions were made in different manners.

A look at links

To take just one example, consider file links (BSD) and aliases (Mac OS). Conceptually, these are very similar: they provide ways to have multiple names for a single file. The devil, of course, lies in the details.

Hard links have been around since the earliest days of Unix. A hard link is a directory entry, within the same file system, to the same item. So, for example, the "." and ".." entries that you see in an "ls -al" directory listing are actually implemented as hard links.

Because all of BSD's file metadata is stored in the file's inode (information node), anything that affects any hard link to a file affects every hard link to the file. So, for example, you can't change the ownership or permissions on just one of a file's hard links.

Symbolic links (i.e., symlinks) were invented by the BSD developers, but other versions of Unix quickly picked up on the idea. A symlink is a specially-interpreted a file, containing the name of some other file, directory, etc. When the kernel is asked to access a symlink, it grabs the contained file name, then wanders off to resolve it. The referenced file may be anywhere on the system; in fact, it may not even exist!

Aliases are quite a bit like symlinks, but they differ in some subtle ways. For instance, Mac OS will try to find the file that an alias references, even if it has been moved or renamed. As long as the referenced file stays in the same file system, Mac OS will typically succeed in finding it. Symlinks, in contrast, either match completely or not at all.

These issues aside, symlinks act (mostly) like aliases, when viewed from the Finder. An alias, however, is unusable from the BSD command line; it simply appears to be a zero-length file. In short, if you need a link that is visible everywhere, use either a hard link or a symlink...

File System Metadata

Apple has done a very creditable job in merging file system metadata. Both Mac OS and BSD programs have access to the sorts of information they need; users and even programmers can largely ignore the underlying issues. To take full advantage of the combined environment, however, we need to know what kinds of metadata it supports.

The tables below list a variety of file system metadata and ways for users to get access to their values. Programmers will have to look up the appropriate libraries and/or frameworks, but the discussion below may provide some useful hints.

Perl programmers should look at Dan Kogai's MacOSX::File module, which provides access to a wealth of file metadata. Perl versions of several Mac OS X file access commands are included in the distribution (available on the CPAN: cpan.perl.org).

My table entries are a bit cryptic, so some explanation may be in order. Get Info (GI) is part of the standard Mac OS X distribution. File Buddy (FB) and Super Get Info (SGI) are available from SkyTag Software (www.skytag.com) and Bare Bones Software (www.barebones.com), respectively.

As their terse names hint, du and ls have been in the Unix lexicon for a number of years. GetFileInfo (and a companion command, SetInfo) are part of the Developer Tools and may be found (along with several other file-related utilities) in /Developer/Tools.

I realize that I may have left out some interesting data items (not to mention useful tools!); please drop me a line if you spot an egregious error or omission. As Apple continues to evolve the file system, I'm sure that a follow-on article will be needed...

Access
   Ownership     FB, GI, SGI   ls -l
   Permissions   FB, GI, SGI   ls -l
   set[gu]id                   ls -l
   sticky                      ls -l

Prior to OSX, Mac OS didn't have much in the way of access control. In fact, the only limitations were on file sharing. As convenient as this may have been, it did not protect users from each others' mistakes, nor protect the system files from user errors. BSD, as a multi-user system, has long had the notions of file ownership and permissions. OSX now has these, as well, but you can expect to see other (e.g., capability-based) forms of access control being added over time.

The setgid and setuid bits allow programs to run with the gid (group id) or uid (user id) of the file, rather than that of the user running the command. This is a convenient way to let programs do privileged things, in a controlled manner.

The "sticky" bit has different meaning, depending on the item involved. Adding the sticky bit to a directory makes the directory "append only"; see sticky(8) for details. On some systems (but not OSX), adding the sticky bit to an executable file causes its binary image to stay in memory after the program has finished.

The first character of the permission string printed by "ls -l" indicates the type of the file. This is not the same as the Type (four-character code) shown by some Mac utilities. In fact, it is more similar to the "Kind" field: it specifies whether the item is a plain file, a directory, a symlink, etc.

Date/Time
   Access            FB            ls -lu
   Backup            FB
   Creation          FB, GI, SGI   ls -cl, GetFileInfo
   Modification      FB, GI, SGI   ls -l,  GetFileInfo

Size
   Data fork
      file length    SGI
      size on disk   SGI   ls -l
   Resource fork
      file length    SGI
      size on disk   SGI
   Total
      file length    FB, GI, SGI   ls -l
      size on disk   FB, GI, SGI   du -k

The access and modification times, as you might expect, are the last times that the file's contents were accessed or modified, respectively. The creation time, on the other hand, is quite inconsistent and potentially confusing.

In both BSD and Mac OS, the creation time is initially defined as the time that the file was created. In BSD, however, the creation time gets updated when the file's directory entry (e.g.., its permissions) has been changed. To get the latter version, use ls -cl.

Because BSD does not have the notion of file "forks", standard command-line tools do not deal with them in a particularly consistent manner. In particular, ls may list the size of a resource fork as zero.

Attributes
   Alias file          FB            GetFileInfo
   Bundle              FB            GetFileInfo
   Custom icon         FB            GetFileInfo
   Desktop                           GetFileInfo
   Extension hidden    FB, GI, SGI   GetFileInfo
   Inited              FB            GetFileInfo
   Invisible           FB, GI, SGI   GetFileInfo
   Locked              FB, GI, SGI   GetFileInfo
   No INIT resource    FB            GetFileInfo
   Shared              FB            GetFileInfo
   Stationery Pad      FB, GI, SGI   GetFileInfo
   System file                       GetFileInfo

Comments
   Attached            FB
   Finder              FB, GI, SGI

Creator, etc.
   Creator             FB, SGI       GetFileInfo
   Kind                FB, GI, SGI
   Open with           SGI
   Path                FB, SGI
   Type                FB, SGI       GetFileInfo

Miscellany
   Name locked         FB
   Package             FB
   Preview             FB, GI, SGI
   Version             FB, SGI

To the best of my knowledge, all of these items are irrelevant to most BSD commands. That is, BSD commands neither affect nor respond to these items. Consequently, I won't discuss them in any detail.

The "Comments" information is, however, an exception. OSX stores comments in a binary file called ".DS_Store", located in the same directory as the file being commented upon. It also performs some hidden magic, copying the comments to the appropriate copy of .DS_Store when a file is moved, etc.

Because the BSD commands know nothing about this mechanism, they do NOT perform the same magic. So, if you use "cp" or "mv" on a file, don't expect the comments to follow along!

In Other News

If you're reading this column, you should probably take a look at O'Reilly 's "Mac OS X for Unix Geeks" (Jepson & Rothman) and "Learning Unix for Mac OS X (Taylor & Peek). Both volumes are slim and unassuming, but contain valuable nuggets of OSX lore .

O'Reilly has just released the second (and substantially larger) edition of "Mac OS X: The Missing Manual", by David Pogue. This looks like a handy book for all sorts of Mac OS X users, whatever their background. If you want a quick reference to the parts of OSX that aren't in your area(s) of specialization, give this book a once-over.


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Ember 1.8.3 - Versatile digital scrapboo...
Ember (formerly LittleSnapper) is your digital scrapbook of things that inspire you: websites, photos, apps or other things. Just drag in images that you want to keep, organize them into relevant... Read more
Apple iTunes 12.1 - Manage your music, m...
Apple iTunes lets you organize and play digital music and video on your computer. It can automatically download new music, app, and book purchases across all your devices and computers. And it's a... Read more
LibreOffice 4.4.3 - Free, open-source of...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
FoldersSynchronizer 4.2.1 - Synchronize...
FoldersSynchronizer is a popular and useful utility that synchronizes and backs-up files, folders, disks and boot disks. On each session you can apply special options like Timers, Multiple Folders,... Read more
Simon 4.0.2 - Monitor changes and crashe...
Simon monitors websites and alerts you of crashes and changes. Select pages to monitor, choose your alert options, and customize your settings. Simon does the rest. Keep a watchful eye on your... Read more
Cocktail 8.1.2 - General maintenance and...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Cyberduck 4.6.4 - FTP and SFTP browser....
Cyberduck is a robust FTP/FTP-TLS/SFTP browser for the Mac whose lack of visual clutter and cleverly intuitive features make it easy to use. Support for external editors and system technologies such... Read more
Herald 5.0.2 - Notification plugin for M...
Note: Versions 2.1.3 (for OS X 10.7), 3.0.6 (for OS X 10.8), and 4.0.8 (for OS X 10.9) are no longer supported by the developer. Herald is a notification plugin for Mail.app, Apple's Mac OS X email... Read more
DEVONthink Pro 2.8.3 - Knowledge base, i...
Save 10% with our exclusive coupon code: MACUPDATE10 DEVONthink Pro is your essential assistant for today's world, where almost everything is digital. From shopping receipts to important research... Read more
Boom 2 1.0.1 - System-wide pro audio app...
Boom 2 is a system-wide volume booster and equalizer app that is designed especially for OS X 10.10 Yosemite. It comes with a smart interface, self-calibrates itself according to your Mac, offers... Read more

Playworld Superheroes Review
Playworld Superheroes Review By Tre Lawrence on January 30th, 2015 Our Rating: :: HERO CRAFTINGUniversal App - Designed for iPhone and iPad It’s all about the imagination, fighting bad creatures — and looking good while doing so.   | Read more »
Join the SpongeBob Bubble Party in this...
Join the SpongeBob Bubble Party in this New Match 3 Bubble Poppin’ Frenzy Posted by Jessica Fisher on January 30th, 2015 [ permalink ] | Read more »
Handpick Review
Handpick Review By Jennifer Allen on January 30th, 2015 Our Rating: :: TANTALIZING SUGGESTIONSiPhone App - Designed for the iPhone, compatible with the iPad Handpick will make you hungry, as well as inspire you to cook something... | Read more »
Storm the Halls of Echo Base in First St...
Storm the Halls of Echo Base in First Star Wars: Galactic Defense Event Posted by Jessica Fisher on January 30th, 2015 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Contradiction Review
Contradiction Review By Tre Lawrence on January 30th, 2015 Our Rating: :: SPOT THE LIEiPad Only App - Designed for the iPad Contradiction is a live action point and click adventure that’s pretty engaging.   Developer: Tim Follin... | Read more »
Unlock Sunshine Girl in Ironkill with th...
Unlock Sunshine Girl in Ironkill with this special 148Apps code Posted by Rob Rich on January 29th, 2015 [ permalink ] Robo-fighter Ironkill has been out on iOS a | Read more »
Crossroad Zombies Review
Crossroad Zombies Review By Jordan Minor on January 29th, 2015 Our Rating: :: CROSSWALKING DEADiPad Only App - Designed for the iPad Crossroad Zombies is a rough draft of a cool genre mash-up.   | Read more »
Blood Brothers 2 – Tips, Cheats, and Str...
War is hell: Is it the kind of hell you want to check out? Read our Blood Brothers 2 review to find out! Blood Brothers 2, DeNA’s follow-up to the original Blood Brothers, is an intriguing card collecting / role-playing / strategy hybrid. There’s... | Read more »
Blood Brothers 2 Review
Blood Brothers 2 Review By Nadia Oxford on January 29th, 2015 Our Rating: :: AN AGGRAVATING RELATIVEUniversal App - Designed for iPhone and iPad Blood Brothers 2 is built on a simple, solid foundation, but its free-to-play system... | Read more »
I AM BREAD, the Toast of the Town, is Ro...
Have you ever dreamt of being deliciously gluten-y? Do you feel passionate about Rye and Wheat? The guys at Bossa Studios do and that is why they are bringing I AM BREAD to iOS soon. The loafy app will feature all the new content that is being... | Read more »

Price Scanner via MacPrices.net

Intel Aims to Transform Workplace With 5th-Ge...
Intel Corporation today announced the availability of its 5th generation Intel Core vPro processor family that provides cutting-edge features to enable a new and rapidly shifting workplace. To meet... Read more
iOS App Sharalike Introduces New Instant Smar...
Sharalike slideshow and photo management software for iOS, is making it easier than ever to create shareable meaningful moments with its new instant SmartShow technology. Staying organized is a goal... Read more
Apple Becomes World’s Largest Smartphone Vend...
According to the latest research data from Strategy Analytics, as global smartphone shipments grew 31 percent annually to reach a record 380 million units in the fourth quarter of 2014. Apple became... Read more
Cut the Cord: OtterBox Resurgence Power Case...
Dead batteries and broken phones are two of the biggest issues for smartphone users today. Otterbox addresses both with the new Resurgence Power Case for Apple iPhone 6, promising to make those panic... Read more
13-inch Retina MacBook Pros on sale for up to...
B&H Photo has 13″ Retina MacBook Pros on sale for $200 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.6GHz/128GB Retina MacBook Pro: $1199.99 save $100 - 13″ 2.6GHz/... Read more
15-inch 2.5GHz Retina MacBook Pro on sale for...
 B&H Photo has the 15″ 2.5GHz Retina MacBook Pro on sale for $2319.99 including free shipping plus NY sales tax only. Their price is $180 off MSRP, and it’s the lowest price available for this... Read more
Back in stock: Refurbished iPod nanos for $99...
The Apple Store has Apple Certified Refurbished 16GB iPod nanos available for $99 including free shipping and Apple’s standard one-year warranty. That’s $50 off the cost of new nanos. Most colors are... Read more
Apple lowers price on refurbished 256GB MacBo...
The Apple Store has lowered prices on Apple Certified Refurbished 2014 MacBook Airs with 256GB SSDs, now available for up to $200 off the cost of new models. An Apple one-year warranty is included... Read more
New Good Management Suite Simplifies Enterpri...
Good Technology has announced the availability of the Good Management Suite, a comprehensive cross-platform solution for organizations getting started with mobile business initiatives. Built on the... Read more
15-inch 2.0GHz Retina MacBook Pro (refurbishe...
The Apple Store has Apple Certified Refurbished previous-generation 15″ 2.0GHz Retina MacBook Pros available for $1489 including free shipping plus Apple’s standard one-year warranty. Their price is... Read more

Jobs Board

*Apple* Solutions Consultant- Retail Sales (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
Event Director, *Apple* Retail Marketing -...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global engagement strategy and team. Delivering an overarching brand Read more
At-Home Chat Specialist- *Apple* Online Stor...
**Job Summary** At Apple , we believe in hard work, a fun environment, and the kind of creativity and innovation that only comes about when talented people from diverse Read more
SW QA Engineer - *Apple* TV - Apple (United...
**Job Summary** The Apple TV team is looking for experienced Quality Assurance Engineers with a passion for delivering first in class home entertainment solutions. **Key Read more
*Apple* Retail - Multiple Positions(US) - Ap...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you re also the Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.