Volume Number: 24 (2008)
Issue Number: 01
Column Tag: Backups

Networked Backups Using Time Machine

Making easy backups even easier

by Rich Warren

You've heard of Time Machine, right? One of Leopard's most-advertised new features, a complete backup solution so simple even my Aunt Agnes could use it. Just plug in an external drive, and Leopard asks if you want to use it for your backups. Click Yes, and then forget about it.

You won't find a lot of customization. Take a look at the System Preferences. There are only three buttons. You can turn Time Machine on and off, select or change your backup drive, and choose files or folders to ignore. Most people will never need to touch these preferences, since Time Machine provides reasonable defaults. It will back up your entire hard drive and any internal drives, but ignore any attached volumes. Once the initial backup is complete, it will make an incremental backup every hour, recording any and all changes.

Unfortunately, Time Machine cannot backup a File Vault protected home directory while that account is logged in. File Vault users also cannot browse their history or restore individual files. Instead, File Vault users must restore their entire directory using the Leopard install disk.

Time Machine will keep hourly backups over the last 24 hours, daily backups over the last month and weekly backups forever, or at least until you run out of disk space. If Time Machine needs to free up some space, it will begin deleting the oldest backups first.

Ok, let's see who was paying attention. Show of hands, who sees a problem here? Yep, the oldest backup was the only full backup of our entire hard drive. The newer backups simply record information that has changed. If we delete the oldest backups, won't we lose most of our information?

Not really. There's a little Unix magic going on under the covers here. But, before I explain that, let's look at how the backups are saved.

Time Machine stores local backups in a folder named Backups.backupdb. Within Backups.backupdb you will find more folders--one for each computer using this drive. Yes, multiple computers can back up to the same drive. It just uses up disk space faster.

Inside your computer's folder, you will find even more folders--one for each backup. The folders are labeled with the time and date of that backup. Finally, inside each of these you will find your entire hard drive.

There are two important points here. First, the backups are not encrypted, compressed or stored in an impenetrable binary format. You still need permission to access the files, so other users can't browse through your backups, but Time Machine gives you full access to all your files and folders. [Ed Note: Of course, these permissions only apply to the computer that the drive is initially connected to. All bets are off for anyone taking this drive to another computer, or, for anyone with admin level access on your Time Machine computer. So, be cautious with this drive if you have any sensitive data.]

That seems like a little thing, but this little thing warms my geeky heart. Apple has given us complete access to our backups. That means third-party developers can create new utilities to work with Time Machine. I can even browse and restore old files by hand--not that I'd want to. But I can, and that makes me happy.

Time Machine's backups only have one real limitation, we cannot boot from them. However, this seems like a minor quibble. In case of emergency, just boot to the Leopard disk. From there, we can restore our entire system from any backup.

OK, now for the second point: each backup folder contains an entire copy of your hard drive.

Yes, you read that correctly. So, lets see. I have 70+ GB of semi-random data stored on my laptop (don't ask why--I'm not sure myself). My backup drive can hold 500 GB. That means I can make seven backups before my hard drive fills up, right?

Wrong. This is where the Unix magic comes in. Remember, Time Machine only copies files that have changed since the previous backup. If a file has not changed, Time Machine simply makes a hard link to the old file. If nothing inside an entire folder has changed, it will make a hard link to the folder instead.

Links are the Unix equivalent of aliases. It's a shortcut that points to a file or folder. You can make them from the command line using the ln command. Type man ln in the terminal for more details.

Links come in two flavors: hard and soft. Soft links are almost identical to aliases. You can use them to access the original file, but if the original file is deleted, the link will now point off into empty space.

Hard links are something different. Special, in a witchy sort of way. From Leopard's man page: "By default, ln makes hard links. A hard link to a file is indistinguishable from the original directory entry; any changes to a file are effectively independent of the name used to reference the file. Hard links may not normally refer to directories and may not span file systems."

Basically a hard link creates another name for the information on disk. The original file name and each hard link point to the same information. You can delete the original file, and as long as at least one other hard link remains, the information stays intact. All existing hard links can still access it--as far as they're concerned, nothing has changed. You have only erased the original file name.

If you want to erase the file, you must erase all hard links to that file. You automatically erase the file with the last hard link.

Apple has taken Unix hard links and mixed in a bit of their own black magic. As the man page suggested, hard links usually cannot point to directories. However, in Leopard they can. [Ed. Note: See Greg Miller's article on DTrace in the November issue of MacTech for more details on this "magic."] This makes Time Machine considerably more efficient. If an entire directory tree is unchanged, Time Machine can make one hard link to the root of that tree. Without directory hard links, you would need one link for each and every file stored inside.

So, when we delete our oldest backup, we're not necessarily erasing the information. If any other backups have a hard link to a given file or directory, that file or directory will remain untouched. Deleting the oldest backup only erase information unique to that backup.

This bit of magic makes it look like each and every backup contains a complete copy of our hard drive.

You cannot see it using the Finder, but each backup folder also contains a hidden file named .Backup.log. These logs record verbose runtime messages from the entire backup procedure. You can mine these logs for interesting information. This includes: the number changed items, the size of the changes, the amount of time spent performing each step of the procedure, and a list of all old backups that Time Machine deleted during cleanup.

Browsing through the log files can give you a better feel for what Time Machine does under the hood. Unfortunately, Leopard's finder no longer has an option to show hidden files, so you will need to use a third party tool like Path Finder, or browse the files using the Terminal.

Obviously Time Machine cannot scan every file on your hard drive once each hour. Most backup software saves craziness like this for 4:00 a.m. (or 4:00 p.m. if you're a night owl). Instead, Apple uses the new FSEvents framework to efficiently determine which files have changed.

Apple designed the FSEvents API to passively monitor large sections of your file system. When a change occurs, FSEvents notifies all listening applications. These notifications are course-grained, both in scope and time. FSEvents tells you that the contents of a directory have changed, but it does not tell you which files were actually modified. Additionally, FSEvents combines multiple recent changes within a single directory into one notification. These limitations help keep FSEvents lightweight and efficient.

While notifications are nice, FSEvents goes one step further. It does not just broadcast these events; it also saves them into an event database. This allows FSEvents notifications to persist, even across reboots.

Practically speaking, this means your application does not need to actively listen for events. You can catch events even when the application is not running. When you want to check for events, simply launch the application, and have it query the event database for any changes over the desired time period.

Not surprisingly, FSEvents form the backbone of Time Machine. Fortunately, Apple has opened this API, letting third-party developers use FSEvents in their own projects.

So far everything sounds good. Remember, this is supposed to be a simple system--so simple that everyone will use it. But there's a problem in paradise. Attaching an external drive is fine for a desktop computer, but I do most of my work on a laptop. Constantly plugging and unplugging my external drive seems like a pain.

I hate to admit it, but if I have to plug and unplug, I would probably forget to back up my machine. Don't get me wrong. Like most people, I know I should do regular backups, and I would probably be really good about it--for a week, maybe two. But, eventually things would start to slip, and all too soon, months will go by between backups.

If Time Machine hopes to become a real backup solution for the masses, then it has to be invisible. Practically speaking, it must to work with a server or network drive--preferably wirelessly.

Originally, Apple promised backups to a hard disk attached to an AirPort Extreme Base Station. Unfortunately, Apple pulled the Air Disk backups just before Leopard's release. Air Disk support may show up in a future update (possibly before you read this). But, as I'm writing, the Air Disk is simply not an option.

Still, all is not lost. Time Machine can back up to another Mac running Leopard (as well as Leopard Server or Xsan storage devices). The host Mac must enable Personal File Sharing using Apple Filing Protocol (AFP).

In my case, I have a Mac Mini with a 500 GB external drive. I use that drive to back up my laptop, my wife's laptop and the Mini itself. The laptops automatically back up whenever they're attached to my wireless network. It all works, almost like magic.

Let's take a second to really think about what we're doing. We want to copy everything on our computer to a network accessible drive. By definition, network drives are designed to allow easy access. Sure, we can try to protect the information with passwords and access control, but a networked drive will never be as secure as a tightly firewalled computer. Additionally, we now store all our vital information in two different locations; that was the whole point of the backup after all. Unfortunately, this also means that hackers can access the information if either location is compromised.

Security experts often talk about the struggle between security and convenience--well, networking Time Machine is incredibly convenient, especially when we use a wireless network. We'll take steps to harden our network in a moment, but lets face facts. If someone is determined enough (or possibly just bored enough), they can probably break in.

So stop for a second. Take a deep breath, and really think about this. What do you have on your computer? What would happen if someone swiped that information? On the other hand, what would happen if your computer crashed and you lost everything? Talk about rocks and hard places.

For myself, as long as I can make hacking my network inconvenient enough to keep out most script kiddies, I feel reasonably safe. After all, I often leave my laptop attached to my home LAN, even when I'm not using it. If someone really wanted the data, they could always go for the originals.

One last word of warning, I'm not a security expert. These are the steps I use on my own machines, but don't just take my word for it. If you're worried, go out and read up on the subject. Of course, you may not sleep nights afterwards, but at least you'll be forewarned.

In many ways, it's much easier to protect a traditional Ethernet LAN. Machines must be physically attached to the network to even see it. Obviously, if the network has no connections to the outside world, then no one can see your data.

Unfortunately, most home LANs are connected to the internet, which then opens them to the entire world. Still, all communication between your home LAN and the internet must pass through a single door--your router. So, just like protecting your real property, shut that door and lock it. Keep a careful eye on who comes in, and what goes out.

Start by looking at your router's security options. It's hard to talk in specifics, since each router is different, but basically, if you don't need it, close it. You might look for things like port forwarding or UPnP. Try to make sure you understand what's turned on and why.

The same principle applies for each individual computer on your network. Look at your Sharing preferences. If you don't need them, turn them off. I would also recommend turning on your computer's firewall, and heeding the information in Apple's KB article about the Leopard firewall at http://docs.info.apple.com/article.html?artnum=306938.

Of course, even the best firewalls cannot completely protect you. So far, Mac users are lucky. We don't face the same plague of spyware, trojans, viruses and other malicious applications. But that is no reason for lenience. It's just a matter of time.

Malicious software can put your data at risk, so let's be careful out there. Know what you're downloading, and who you're downloading it from. Also, think about the other users on your computer, or the other machines on your network. If any of them become infected, they might leak information that puts you at greater risk.

Adding wireless networking automatically makes things harder. Radio waves go through walls, and once we start broadcasting, it's hard to limit how far the information travels. We can't simply barricade the front door anymore.

Right now, a team of over-caffeinated, sweaty-palmed L33T HAXORZ might be cruising through your neighborhood in a beat-up '72 Pinto with a duct-taped Pringles-can antenna and a laptop full of cracking software. Stranger things do happen.

More seriously, many people use flawed security practices on their wireless networks. Some exclusively rely on MAC filtering (that's Media Access Control, not Mac the computer) to prevent outsiders from logging into their network. Unfortunately, if your network traffic is not encrypted, there's nothing to stop outsiders from monitoring the unencrypted information that you're broadcasting. Worse yet, you can find software that lets you scrape out and mimic a valid MAC address from the network chatter. On it's own, MAC filtering provides no real security.

The WEP encryption protocol also falls depressingly short. While WEP may keep your neighbor from stealing your wireless to download porn, several weaknesses have been found. Modern software can often break WEP encryption in just a few minutes. Despite this, WEP remains the default on many wireless routers.

WPA encryption addresses the weaknesses in WEP, but it has problems of its own. The "Personal" mode used on most small networks requires a password. Obviously, your network is only as secure as your password. All the advice on picking secure passwords applies here--with one additional twist. For WPA encryption, longer is better. A typical 6 to 8 character password just won't cut it.

Bottom line, here's my advice: turn on both MAC filtering and WPA encryption. Give it a good, long password (you only need to type it once per computer, so go nuts). And you might want to change your password periodically. Of course, everything I said about wired networks still applies. Shut off any services that you don't need, and enable the firewall on all your machines.

There are three basic steps for setting up the host machine: setup the hardware, create the shared account and then share the drive.

Technically, you don't need to add an additional hard drive to the host machine. You could just use the host's hard drive, as long as it's big enough. However, I doubt this is a practical solution. In most cases, you will want to attach an external hard drive.

How large should the hard drive be? That's a very good question. We don't have a lot of real-world experience using Time Machine yet, so I'm afraid I don't have a rock-solid answer. Still, the basic rule of thumb seems simple; buy the biggest drive you can afford. After all, I've never been upset about having too much free space; I always find some way to fill it.

At a minimum, the drive needs to be bigger than all the drives you intend to back up added together. Of course, Time Machine is designed to browse through your hard drive's history. Using a bare minimum hard drive, that history will remain amazingly short. This might let you recover your files when your hard drive crashes, but you won't be able to find that email from your boss. You know, the one you accidentally deleted last week.

Instead, I recommend getting a drive at least twice as big as the total size of all drives you're backing up. That should give Time Machine enough room to back up a reasonable history for each computer.

If you're still having disk issues, you might want to look for the files that are causing the problem. Remember, time machine backs up entire files. If a single bit in a 10 GB file changes, time machine will back up the entire 10 GB.

Usually, this isn't a problem. Frequently changing data tends to live in small files. And large files, like videos, don't typically change. Still, there are a few applications that could cause pain. Virtualization software, like Parallels, often saves your virtual hard drive as a single file. This file can easily be 10s of gigabytes in size. If you use Parallels a lot, the virtual drives will change frequently. This could easily bog down even the largest backup drives.

If you find problematic files like these, simply exclude them from Time Machine's backup. Of course, you probably need another way to back up that information, but leave that up to you.

First things first, create a new user on the host machine. This will serve two purposes. First, it adds a layer of protection for your backups. If someone accidentally downloads a malicious program to the host machine, hopefully it will only affect their account, and your backups will remain untouched.

Also, opening a shared drive does present a potential security weakness. If someone manages to break in using the drive, you want the hacker stuck in a restricted account with limited access to the rest of the machine.

To set up the account, open the System Preferences. From the System row, select Accounts.

You will probably need to unlock the Accounts preferences. Click on the lock icon in the bottom left corner, and type in an administrator user name and password.

Click the "+" button under the accounts list. Enter the user name and a secure password for this account. Again, you will only manually enter this password a few times, so go ahead and make it extra passwordy.

Make sure to set the new account type to "Sharing Only". This will create an account specifically designed for remote access. Sharing Only accounts do have a home directory; however, you cannot log into them locally. As the name suggests, they are perfect for our purposes.

Click the Create Account button, and you're done.

In the System Preferences, select Sharing from the Internet & Network row.

Enable File Sharing. By default, your Mac will use AFP. You can enable FTP or SMB sharing by clicking the Options... button, but Time Machine only requires AFP.

Add the external drive to the list of shared folders. Click the "+" button under the Shared Folders list, and select the drive. Note: other computers will not have access to this drive when the host computer is sleeping. You probably want to go into the Energy Saver settings and make sure the computer never sleeps.

Back in the Sharing preferences, make sure the external hard drive is selected in the Shared Folders list. The Users list now shows the access controls for this drive. Unfortunately, we can only make limited changes here. Instead, let's open the drive's Info window.

Find the hard drive icon on your desktop. Control-click on the disk icon and select Get Information. In the Info window, expand the Sharing and Permissions section. Here, you can modify the access permissions to this drive for any number of users or groups. To unlock the control, simply click the lock icon in the bottom right corner, then enter an administrator name and password.

For Time Machine to work, the shared user must have both read and write permission. I also gave the administration group similar permissions; this lets me access the drive locally when necessary. Unfortunately, my current account shows up on this list, and I cannot delete it. Since it is already an administrator account, I just mirrored those permissions. Finally, everyone else should default to No Access. If you're not the shared account or an administrator, you don't get to touch this drive.

Once the permissions are set, go back to the Sharing window. Check the access restrictions listed there. They should match the settings from the Info window. If they don't, make any necessary changes. You can also remove the shared folders, if you're not going to use them.

Finally, Leopard only mounts external drives when a user is logged into the host machine. If no one is logged in, you cannot access the shared drive remotely. To fix this, open the command line and type the command listed below (all on one line).

sudo defaults write /Library/Preferences/SystemConfiguration/autodiskmount AutomountDisksWithoutUserLogin -bool true

After running the command, restart the host machine. Once it reboots, as long as the machine is turned on, the drive will remain active.

First the good news, clients are much easier. There's only one catch, you must log into the drive before setting it up. In Leopard, this is easy. The host machine should show up in the Finder's SHARED sidebar. Simply click on the icon, then log in using the shared user and password we created earlier.

You should see the external hard drive in the finder. Now, open Time Machine's preferences (from the System row of the System Preferences). Click the Choose Backup Disk... button. Select the remote disk, and click Use for Backup. Time Machine will ask you for the user name and password again. These lets Time Machine automatically log into the external drive when it performs its backups.

Selecting the drive should automatically turn on Time Machine. Your first backup will run in about two minutes.

As I described earlier, Time Machine saves local backups to the Backups.backupdb folder. Remote backups are a bit different. Time Machine stores these in sparsebundles. Sparsebundles are a new type of disk image. Like other disk images, they can be mounted and browsed, provided you have the necessary permissions. You can even create new sparsebundles using the Disk Utility application.

Unfortunately solid details about sparsebundles are hard to come by. According to Apple, sparsebundles are more reliable, efficient and scalable than the older sparse images, and they recommend using sparsebundles for any persistent sparse images, as long as the backing bundle is acceptable. As the name suggests, sparsebundles are bundles � in other words, a folder that the Finder treats as a single entity. Since it is a folder, you can dig into the contents. Do so and you'll see a 'bands' directory with slices of that disk image in chunks that may be up to 128MB each. The man page for hdiutil has a little more information.

While sparsebundles can be mounted remotely, you cannot mount them when directly connected to the drive, even when using an administrator account. If you try, you will get a socket error. Finder won't even let you open the bundle's contents locally. Ironically, this makes remote backups slightly more secure than local backups.

Time Machine's use of sparsebundles creates at least one small problem. My home wireless network is not the most reliable thing. I tried to backup my entire 70+ GB hard drive over wireless, just letting it run overnight. That never worked. It typically crashed around 12 GB. On the few occasions when shoving massive amounts of data over the air didn't kill the network outright, the transfer took forever. Only a fraction of the work was done by morning, and I don't have that kind of patience.

Ideally, I want to plug the backup drive directly into my laptop, do a complete local backup, then convert it to a remote backup. I made a few attempts at copying the contents of my local backup into a mounted sparsebundle, but I kept running into permission errors or corrupted bundle errors. I'm not saying it's impossible, but I couldn't get it to work.

Instead, I wired all the computers together into an Ethernet network and ran the initial backups over the wires. Time machine tagged my sparsebundle image with my Ethernet card's MAC address. Still, when I switched to a wireless connection, Time Machine recognized and used the proper sparsebundle. I've tested this on both my MacBook Pro and an older G4 PowerBook, and it worked fine on both machines.

Still, some people are reporting problems when switching between wired and wireless networks. The issue seems to be incorrectly named sparsebundles. If you're having trouble, try creating an alias or even renaming the sparsebundle. Look for the 12-character hexadecimal string after the underscore. Replace that with the MAC address for your wireless card.

Incremental backups tend to be rather small--usually less than 1 GB, so the Wireless network typically handles them with little trouble. But, it's not 100%, at least not on my network. I may not get every hourly backup, but this doesn't bother me too much. My computer gets several backups a day, and that seems sufficient. Anyway, once the backups are older than 24 hours, Time Machine will only keep one per day.

Time Machine itself runs a little more sluggishly over the wireless network. It can take a minute or more to mount the sparsebundle. But, once it's running, it usually to work reasonably well, network crashes notwithstanding.

So far, I am quite pleased with my wireless Time Machine setup, but it may not be the right solution for everyone. The two main deal breakers are security and performance.

Any time you open a service on your computer, you are opening yourself to possible attacks. No one can prove their software is bug free--so there's always a chance someone will discover a new exploit and use it against your computer.

Performance wise, your experience will very greatly depending on the quality of your wireless network. I think it works well enough as a general backup solution, even on my slow, temperamental network. However, if you frequently dig through older copies of files, then you may want something a bit snappier.

Even if you want to use Time Machine, consider this: Is Time Machine your complete backup solution, or is it just one piece in a larger system.

Time Machine backups work great if you accidentally delete a file, or if your computer's hard drive burns out. But what happens when there's a fire or hurricane or flood? Say someone breaks into your home and steals all your computer equipment? Typically, your backup hard drive will be in the same general location as your backed-up computer. A disaster that strikes one will likely take out the other as well.

I highly recommend using some sort of off-site backup in addition to Time Machine. You have any number of options here; depending on how much you're willing to pay and how much work you want to do.

If you're a .Mac member, then Backup is probably your simplest option. Or, for a more hands-on approach, you could burn all your important files to DVD and store the backup disks at work, or at someone else's house. I also like using the extra space on my iPod. After all, if there's any one item that I'm likely to have on my person as I run out of a burning building, it's my iPod.

Also look at third party solutions. Many people recommend SuperDuper!, especially for bootable backups. There are also a number of online storage services, like Amazon's S3. Online storage is nice, since the servers are probably in an entirely different state.

The bottom line is simple. Time Machine's quick setup and fire-and-forget usage makes it the ideal first line of defense, but it should not be your only backup solution. That's just asking for trouble.

For this networked Time Machine solution to work, you need a host computer connected to your network 24/7. So, that raises the question, what other services can this machine provide? After all, most of the time it's just sitting there. Here are a few suggestions, but these only scratch the surface.

By definition, you already have a shared drive connected to your network. If you have extra space, you can use that drive as a file server. You don't even need to do anything, the setup described above will let you upload and download files from the networked drive.

The host computer is also the ideal choice for a print server. Simply attach a printer to the host computer and enable Printer Sharing.

You might want to host your primary iTunes library on the host computer. If you share the entire library, bandwidth willing, you can access all of your media from any computer on your local network.

Additionally, you could set up your own web server by enabling Web Sharing. Leopard also includes Mongrel and Ruby on Rails, if you want more-complex web applications. I want to make a small grocery list application. Everyone in my family could add items to the list, and I could print out a copy just before heading to the store. Of course, in the age of blogs and social networks, rolling your own web site is not as exciting as it used to be, but it's still a good learning experience.

To make your web site even more useful, you could use a dynamic DNS server. With a few changes to your router's firewall, anyone on the internet could access your web pages. I wouldn't recommend this for a high-volume site, or for a business critical pages. But it might be fun for something small.

Finally, Back to My Mac opens a whole world of interesting possibilities. If you're a .Mac member, Back to My Mac lets you access your host computer from anywhere in the world. You can even remotely control the host using screen sharing. I'm sure we will find any number of interesting tricks for this technology.

Here's a quickie. Say you're hanging out at an internet café. There's a large file that you'd like to download, but you don't want to wait for it. Fire up Back to My Mac and use the screen sharing. Launch Safari on the host computer, and start the download there. Now you can shut down your laptop and forget about it. The download will be waiting when you get home.

You even have limited access to Time Machine while on the road. You can launch the Time Machine application--though, in my case it ran painfully slow. I had more success mounting the backup image and browsing it with the finder. Also, my laptop would not automatically schedule its next backup. I was able to force a remote backup by manually setting time machine to the back-to-my-mac mounted disk. However, after nearly 45 minutes of "preparing", my connection to the host computer crashed. Bottom line, I wouldn't recommend backing up from a coffee shop's shared wireless network. On the other hand, if you can plug into an office LAN while on a business trip, it might be worth a try.

Time Machine is an excellent backup solution, provided you understand and respect its limitations. Time Machine's main strengths come from its ability to do frequent, incremental backups. Unfortunately, this only works if your computer is regularly connected to the backup drive. For a desktop computer, no problem; however, I don't like having my laptop constantly tethered to an external disk. Laptop users need a remote solution.

In this article we explored remote backups to another Leopard machine. Unfortunately, this solution can be somewhat expensive . Hopefully, Apple will enable Air Disk support soon. Still, using a full-blown computer opens a range of additional possibilities. I highly recommend using Time Machine to back up to a host computer. You need to understand the security risks involved, and it does not replace a good off-site solution. Still, the next time your hard drive burns out, you will thank me.

Rich Warren lives in Honolulu, Hawaii with his wife, Mika, daughter, Haruko, and his new son, Kai. He is a software engineer, freelance writer and part time graduate student. When not playing on the beach, he is probably writing, coding or doing research on his MacBook Pro. You can reach Rich at rikiwarren@mac.com.