TweetFollow Us on Twitter

File System Metadata

Volume Number: 18 (2002)
Issue Number: 12
Column Tag: Section 7

What's in there, beside the data...

by Rich Morin

One of Apple's larger challenges, in merging Mac OS and BSD, has been the need to reconcile differences between their respective ways of handling files. Previous columns have touched on some of these issues; this month, we'll go into the topic in a bit more depth.

In order to keep both BSD and Mac OS program(mer)s happy, Apple has had to implement nearly every feature found in either HFS+ (Mac OS's "Extended Hierarchical File System") or UFS (BSD's "Unix File System"). This is a challenging task; the systems evolved independently and a variety of decisions were made in different manners.

A look at links

To take just one example, consider file links (BSD) and aliases (Mac OS). Conceptually, these are very similar: they provide ways to have multiple names for a single file. The devil, of course, lies in the details.

Hard links have been around since the earliest days of Unix. A hard link is a directory entry, within the same file system, to the same item. So, for example, the "." and ".." entries that you see in an "ls -al" directory listing are actually implemented as hard links.

Because all of BSD's file metadata is stored in the file's inode (information node), anything that affects any hard link to a file affects every hard link to the file. So, for example, you can't change the ownership or permissions on just one of a file's hard links.

Symbolic links (i.e., symlinks) were invented by the BSD developers, but other versions of Unix quickly picked up on the idea. A symlink is a specially-interpreted a file, containing the name of some other file, directory, etc. When the kernel is asked to access a symlink, it grabs the contained file name, then wanders off to resolve it. The referenced file may be anywhere on the system; in fact, it may not even exist!

Aliases are quite a bit like symlinks, but they differ in some subtle ways. For instance, Mac OS will try to find the file that an alias references, even if it has been moved or renamed. As long as the referenced file stays in the same file system, Mac OS will typically succeed in finding it. Symlinks, in contrast, either match completely or not at all.

These issues aside, symlinks act (mostly) like aliases, when viewed from the Finder. An alias, however, is unusable from the BSD command line; it simply appears to be a zero-length file. In short, if you need a link that is visible everywhere, use either a hard link or a symlink...

File System Metadata

Apple has done a very creditable job in merging file system metadata. Both Mac OS and BSD programs have access to the sorts of information they need; users and even programmers can largely ignore the underlying issues. To take full advantage of the combined environment, however, we need to know what kinds of metadata it supports.

The tables below list a variety of file system metadata and ways for users to get access to their values. Programmers will have to look up the appropriate libraries and/or frameworks, but the discussion below may provide some useful hints.

Perl programmers should look at Dan Kogai's MacOSX::File module, which provides access to a wealth of file metadata. Perl versions of several Mac OS X file access commands are included in the distribution (available on the CPAN: cpan.perl.org).

My table entries are a bit cryptic, so some explanation may be in order. Get Info (GI) is part of the standard Mac OS X distribution. File Buddy (FB) and Super Get Info (SGI) are available from SkyTag Software (www.skytag.com) and Bare Bones Software (www.barebones.com), respectively.

As their terse names hint, du and ls have been in the Unix lexicon for a number of years. GetFileInfo (and a companion command, SetInfo) are part of the Developer Tools and may be found (along with several other file-related utilities) in /Developer/Tools.

I realize that I may have left out some interesting data items (not to mention useful tools!); please drop me a line if you spot an egregious error or omission. As Apple continues to evolve the file system, I'm sure that a follow-on article will be needed...

Access
   Ownership     FB, GI, SGI   ls -l
   Permissions   FB, GI, SGI   ls -l
   set[gu]id                   ls -l
   sticky                      ls -l

Prior to OSX, Mac OS didn't have much in the way of access control. In fact, the only limitations were on file sharing. As convenient as this may have been, it did not protect users from each others' mistakes, nor protect the system files from user errors. BSD, as a multi-user system, has long had the notions of file ownership and permissions. OSX now has these, as well, but you can expect to see other (e.g., capability-based) forms of access control being added over time.

The setgid and setuid bits allow programs to run with the gid (group id) or uid (user id) of the file, rather than that of the user running the command. This is a convenient way to let programs do privileged things, in a controlled manner.

The "sticky" bit has different meaning, depending on the item involved. Adding the sticky bit to a directory makes the directory "append only"; see sticky(8) for details. On some systems (but not OSX), adding the sticky bit to an executable file causes its binary image to stay in memory after the program has finished.

The first character of the permission string printed by "ls -l" indicates the type of the file. This is not the same as the Type (four-character code) shown by some Mac utilities. In fact, it is more similar to the "Kind" field: it specifies whether the item is a plain file, a directory, a symlink, etc.

Date/Time
   Access            FB            ls -lu
   Backup            FB
   Creation          FB, GI, SGI   ls -cl, GetFileInfo
   Modification      FB, GI, SGI   ls -l,  GetFileInfo

Size
   Data fork
      file length    SGI
      size on disk   SGI   ls -l
   Resource fork
      file length    SGI
      size on disk   SGI
   Total
      file length    FB, GI, SGI   ls -l
      size on disk   FB, GI, SGI   du -k

The access and modification times, as you might expect, are the last times that the file's contents were accessed or modified, respectively. The creation time, on the other hand, is quite inconsistent and potentially confusing.

In both BSD and Mac OS, the creation time is initially defined as the time that the file was created. In BSD, however, the creation time gets updated when the file's directory entry (e.g.., its permissions) has been changed. To get the latter version, use ls -cl.

Because BSD does not have the notion of file "forks", standard command-line tools do not deal with them in a particularly consistent manner. In particular, ls may list the size of a resource fork as zero.

Attributes
   Alias file          FB            GetFileInfo
   Bundle              FB            GetFileInfo
   Custom icon         FB            GetFileInfo
   Desktop                           GetFileInfo
   Extension hidden    FB, GI, SGI   GetFileInfo
   Inited              FB            GetFileInfo
   Invisible           FB, GI, SGI   GetFileInfo
   Locked              FB, GI, SGI   GetFileInfo
   No INIT resource    FB            GetFileInfo
   Shared              FB            GetFileInfo
   Stationery Pad      FB, GI, SGI   GetFileInfo
   System file                       GetFileInfo

Comments
   Attached            FB
   Finder              FB, GI, SGI

Creator, etc.
   Creator             FB, SGI       GetFileInfo
   Kind                FB, GI, SGI
   Open with           SGI
   Path                FB, SGI
   Type                FB, SGI       GetFileInfo

Miscellany
   Name locked         FB
   Package             FB
   Preview             FB, GI, SGI
   Version             FB, SGI

To the best of my knowledge, all of these items are irrelevant to most BSD commands. That is, BSD commands neither affect nor respond to these items. Consequently, I won't discuss them in any detail.

The "Comments" information is, however, an exception. OSX stores comments in a binary file called ".DS_Store", located in the same directory as the file being commented upon. It also performs some hidden magic, copying the comments to the appropriate copy of .DS_Store when a file is moved, etc.

Because the BSD commands know nothing about this mechanism, they do NOT perform the same magic. So, if you use "cp" or "mv" on a file, don't expect the comments to follow along!

In Other News

If you're reading this column, you should probably take a look at O'Reilly 's "Mac OS X for Unix Geeks" (Jepson & Rothman) and "Learning Unix for Mac OS X (Taylor & Peek). Both volumes are slim and unassuming, but contain valuable nuggets of OSX lore .

O'Reilly has just released the second (and substantially larger) edition of "Mac OS X: The Missing Manual", by David Pogue. This looks like a handy book for all sorts of Mac OS X users, whatever their background. If you want a quick reference to the parts of OSX that aren't in your area(s) of specialization, give this book a once-over.


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Capto 1.2.9 - $29.99
Capto (was Voila) is an easy-to-use app that takes capturing, recording, video and image editing to the next level. With an intelligent file manager and quick sharing options, Capto is perfect for... Read more
Opera 51.0.2830.40 - High-performance We...
Opera is a fast and secure browser trusted by millions of users. With the intuitive interface, Speed Dial and visual bookmarks for organizing favorite sites, news feature with fresh, relevant content... Read more
GarageSale 7.0.13 - Create outstanding e...
GarageSale is a slick, full-featured client application for the eBay online auction system. Create and manage your auctions with ease. With GarageSale, you can create, edit, track, and manage... Read more
1Password 6.8.7 - Powerful password mana...
1Password is a password manager that uniquely brings you both security and convenience. It is the only program that provides anti-phishing protection and goes beyond password management by adding Web... Read more
Evernote 7.0.1 - Create searchable notes...
Evernote allows you to easily capture information in any environment using whatever device or platform you find most convenient, and makes this information accessible and searchable at anytime, from... Read more
MacUpdate Desktop 6.2.0 - $20.00
MacUpdate Desktop brings seamless 1-click app installs and version updates to your Mac. With a free MacUpdate account and MacUpdate Desktop 6, Mac users can now install almost any Mac app on... Read more
HoudahSpot 4.3.5 - Advanced file-search...
HoudahSpot is a versatile desktop search tool. Use HoudahSpot to locate hard-to-find files and keep frequently used files within reach. HoudahSpot will immediately feel familiar. It works just the... Read more
EtreCheck 4.0.4 - For troubleshooting yo...
EtreCheck is an app that displays the important details of your system configuration and allow you to copy that information to the Clipboard. It is meant to be used with Apple Support Communities to... Read more
WhatsApp 0.2.8361 - Desktop client for W...
WhatsApp is the desktop client for WhatsApp Messenger, a cross-platform mobile messaging app which allows you to exchange messages without having to pay for SMS. WhatsApp Messenger is available for... Read more
iClock 4.2 - Customize your menubar cloc...
iClock is a menu-bar replacement for Apple's default clock but with 100x features. Have your Apple or Google calendar in the menubar. Have the day, date, and time in different fonts and colors in the... Read more

Latest Forum Discussions

See All

The best games like Florence
Florence is a great little game about relationships that we absolutely adored. The only problem with it is it's over a little too soon. If you want some other games with some emotional range like Florence, check out these suggestions: [Read more] | Read more »
Angry Birds Champions adds cash prizes t...
Collaborating with developer Rovio Entertainment, GSN Games has released a twist on the Angry Birds formula. Angry Birds Champions features the same bird-flinging gameplay, but now you can catapult Red and co for cash. | Read more »
Around the Empire: What have you missed...
148Apps is part of a family. A big family of sites that make sure you're always up to date with all the portable gaming news. Just like a real family, I guess. I don't know, my mum never told me anything about Candy Crush to be fair. [Read more] | Read more »
The Battle of Polytopia Guide - Tips for...
The addition of multiplayer to The Battle of Polytopia has catapulted the game from a fun enough time waster to a fully-fledged 4X experience on your phone. We've been playing quite a few matches over the past week or so, and we've put together a... | Read more »
All the best games on sale for iPhone an...
Hi there, and welcome to our round up of all the best games that are on sale for iOS at the moment. It's not a vintage week in terms of numbers, but I'm pretty sure that every single one of these is worth picking up if you haven't already played... | Read more »
Disc Drivin' 2 Guide - Tips for the...
We're all still playing quite a bit of Disc Drivin' 2 over here at 148Apps, and we've gotten pretty good at it. Now that we've spent some more time with the game and unlocked more powerups, check out some of these more advanced tips: | Read more »
Alto's Odyssey Guide - How to Tackl...
Alto’s Odyssey is a completely stunning and serene runner, but it can also be a bit tricky. Check out these to try and keep your cool while playing this endless runner: Don’t focus too much on tasks [Read more] | Read more »
Here's everything you need to know...
Alto's Odyssey is a really, really good game. If you don't believe me, you should definitely check out our review by clicking this link right here. It takes the ideas from the original Alto's Adventure, then subtly builds on them, creating... | Read more »
Alto's Odyssey (Games)
Alto's Odyssey 1.0.1 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.1 (iTunes) Description: Just beyond the horizon sits a majestic desert, vast and unexplored. Join Alto and his friends and set off on an endless... | Read more »
Vainglory 5v5: Everything you need to kn...
Vainglory just got bigger. [Read more] | Read more »

Price Scanner via MacPrices.net

Sale! Amazon offers 13″ 2.3GHz MacBook Pros f...
Amazon has 2017 13″ 2.3GHz Apple MacBook Pros on sale today for $151-$150 off MSRP, each including free shipping: – 13″ 2.3GHz/128GB Space Gray MacBook Pro (MPXQ2LL/A): $1148 $151 off MSRP – 13″ 2.... Read more
Apple AirPods in stock today for $159, free s...
Adorama reports stock of Apple AirPods today for $159 including free shipping, plus pay no sales tax outside of NY & NJ. See our Apple AirPod Price Tracker for the latest prices and stock status... Read more
Saturday Sale: Amazon offers 12″ 1.3GHz MacBo...
Amazon has Silver and Gold 2017 12″ 1.3GHz Retina MacBooks on sale for $250 off MSRP. Shipping is free: – 12″ 1.3GHz Silver MacBook: $1349.99 $250 off MSRP – 12″ 1.3GHz Gold MacBook: $1349.99 $250... Read more
Use your Apple Education discount and save up...
Purchase a new Mac using Apple’s Education discount, and take up to $400 off MSRP. All teachers, students, and staff of any educational institution with a .edu email address qualify for the discount... Read more
Apple Canada offers 2017 21″ and 27″ iMacs fo...
 Canadian shoppers can save up to $470 on the purchase of a 2017 current-generation 21″ or 27″ iMac with Certified Refurbished models at Apple Canada. Apple’s refurbished prices are the lowest... Read more
9″ iPads available online at Walmart for $50...
Walmart has 9.7″ Apple iPads on sale for $50 off MSRP for a limited time. Sale prices are for online orders only, in-store prices may vary: – 9″ 32GB iPad: $279.99 $50 off – 9″ 128GB iPad: $379.99 $... Read more
15″ Apple MacBook Pros, Certified Refurbished...
Save $360-$420 on the purchase of a 2017 15″ MacBook Pro with Certified Refurbished models at Apple. Apple’s refurbished prices are the lowest available for each model from any reseller. An standard... Read more
Amazon restocks MacBook Pros with models avai...
Amazon has restocked 15″ and 13″ Apple MacBook Pros with models on sale for up to $251 off MSRP. Shipping is free. Note that stock of some Macs may come and go (and some sell out quickly), so check... Read more
Lowest price of the year: 15″ 2.8GHz Apple Ma...
Amazon has the 2017 Space Gray 15″ 2.8GHz MacBook Pro on sale today for $251 off MSRP. Shipping is free: – 15″ 2.8GHz Touch Bar MacBook Pro Space Gray (MPTR2LL/A): $2148, $251 off MSRP Their price is... Read more
Apple restocks full line of Certified Refurbi...
Apple has restocked a full line of Apple Certified Refurbished 2017 13″ MacBook Pros for $200-$300 off MSRP. A standard Apple one-year warranty is included with each MacBook, and shipping is free.... Read more

Jobs Board

*Apple* Media Products Commerce Engineering...
# Apple Media Products Commerce Engineering Manager Job Number: 56207285 Santa Clara Valley, California, United States Posted: 26-Jan-2018 Weekly Hours: 40.00 **Job Read more
Digital Platforms Lead, Today at *Apple* -...
# Digital Platforms Lead, Today at Apple Job Number: 56178747 Santa Clara Valley, California, United States Posted: 23-Feb-2018 Weekly Hours: 40.00 **Job Summary** Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.