TweetFollow Us on Twitter

Data Overflow: What's Your Back-up Plan?

Volume Number: 23 (2007)
Issue Number: 12
Column Tag: Hardware

Data Overflow: What's Your Back-up Plan?

Why you must protect an ever-increasing volume of valuable data

by Mike Cobb

Introduction

Have you ever been stuck in a traffic jam on the freeway and thought, "Gee, they really ought to widen this road and add a couple of lanes to handle all this traffic?" Except, you know what would happen. Those lanes would fill up with cars in no time and you'd be stuck again. Or, as someone said, "Nature abhors a vacuum."

That's sort of how it is with today's hard drives. As manufacturers have been able to create more and more digital capacity in less and less physical space, users-Mac users in particular-have eagerly poured in more and more data to fill that extra space, often right to the limit.

The problem is that it takes longer to back up these expanding data sets. Many individuals don't bother, and many companies don't budget adequately for the right tools to keep up with the volume.

Meanwhile, the drives themselves have become more intricate, with smaller components and tighter tolerances. So the risk of failure is increased. Drive failure has been a fact of life since the first computers, but with the capacity of today's machines and the propensity of users to use it all, the sheer volume of data at risk today is staggering.

Whether you edit home movies on your laptop for fun or manage a room full of servers for business, it's time for a back-up plan.

A data salvager's perspective

As a Director of Engineering at DriveSavers, I'm responsible for overseeing the data recovery process for Mac and Unix systems, which includes everything from iPods to Xserve RAID. We deal with the challenge of data overflow every day from both the end-user and the enterprise level.

When you've been doing this for a few years-13 in my case-you get an interesting perspective on how much data volume has grown. A typical data recovery job in 1994 involved a hard drive with storage capacity in the 20-40 megabyte range. For the recovery process, we used 240 MB hard drives to hold the data we recovered and the average file count, including all the OS files as well as the user's data, was around 25,000 files per recovery. And in those days, floppy discs were the primary back-up medium and the media DriveSavers used to send customers their recovered data.

Today the average recovery for a Mac is 60 gigabytes and the average number of files is 160,000. Only 25% of the recoveries can go on CDs or DVDs. Most are sent back in new internal or external hard drives because of the large data sets and file sizes. In fact, a lot of our customers have files that are bigger than the 8-gig capacity of a dual-layer DVD. External drives-not CDs or DVDs-have become today's floppies.

Of course, the Mac world is different from the PC world, as you might expect. The average PC recovery is 10-15 GB, so most of those recoveries go out on DVDs. On the Mac side we work for a lot of creative professionals like photographers, filmmakers and audio engineers who are creating huge files with applications like ProTools and FinalCutPro. It sounds like a cliche straight out of the "Mac vs. PC" commercials, but it's true. Mac and PC users have one thing in common, however-they keep all their email and e-mail attachments forever.

Drive failure happens. Here's why.

So how do all these cluttered drives end up in our shop? Drive failure is inevitable, and its causes are many. A few are extreme. We have recovered data from computers that have been dropped, run over, burned, drowned and shot. But those "disk-asters", as we call them, are the exception. The everyday causes of drive failure are more mundane, breakdowns in the inner workings of the drives themselves, brought on by the very complexity that makes them so powerful.

Many drives come out of the factory with some kind of defect that will eventually surface. The average service life of a drive these days is 3-5 years. Drive manufacturers claim the failure rate is about 1% of all drives in use per year, but some independent estimates put it as high as 4% and even up to 13%.

Just as it can't be avoided, neither can it be predicted. There have been various efforts at "smart" failure prediction, but a majority of drive failures happen immediately, like bad accidents, without warning. When the drive heads suddenly decide they're going to crash into a spinning platter, no one can see it coming.

Simple electrical failure caused by a few inherently bad sectors used to be the primary cause of failure. Nowadays, we see less electrical failure and more physical media damage resulting from the tight packing of ever-shrinking, fast-moving mechanical parts, especially the head-to-platter surface interface.

Power surges are a common cause, too. They're especially bad for the users who are conscientious about backing up, because their back-up drives are usually plugged into the same power source as their main ones.

Here's one that surprises people: hard drives are sensitive to altitude. They have a higher rate of failure over 10,000 feet, even in pressurized airplane cabins where every other person seems to have a laptop or an iPod. In a depressurized environment, like a mountaintop in the Andes, they simply won't function over 10,000 feet.

User error is a less common cause of failure, but it certainly happens. It might be as simple as unplugging a Firewire or USB drive without first "ejecting" it. You might even get away with it nine times and get lulled into thinking nothing's going to happen-until the tenth.

At the enterprise level, we repeatedly hear from IT managers who thought that, in a RAID server, back-up was "built in"-if one drive failed, another would take over-never anticipating that a second drive failure would crash the entire system. In fact, when one drive fails the remaining drives start working overtime, running faster and hotter than normal, increasing the risk of complete failure.

IT support people often call DriveSavers, too, after they've done a reinstall on a desktop system to try to eradicate some kind of corruption, only to find there were some crucial documents that the user didn't back up to the server.

People ask us how they can prevent their disks from failing. The short answer is you can't. It's not a question of if, but when. The more pertinent question, though, is, how can you prevent your data from being lost, or avoid going through the downtime and costs that a recovery entails? Based on my experiences, there's only one answer.

Back up. Back up. And back up some more.

Being told you need to back up regularly is kind of like having the dentist tell you that you need to floss. You know it's true. You vow to be better about it. You have the means and every intention. But you forget, or you put it off. And next thing you know, you're getting a root canal.

Cost used to be an impediment. After paying for a computer, who really wanted to shell out the money for an extra hard drive? But external drives are comparatively cheap now. We see more and more of them coming in for data recovery. The reason? Users are buying external drives for back-up, and then they wind up using them for data overflow. So, the data on their external drives is just as much at risk as that on their computers. The only data that is not at risk is data that has been backed up.

So, what's the best back-up system for the heavy hobbyist or small creative business? The answer is the one you're most likely to use-if it encourages you to back up instead of discouraging you, it's right for you. If you have large files of photos, movies, music and the like, CDs or DVDs are simply not a practical option. The handling and storage of them is also a bit cumbersome. Tape back-up was once the standard in business, but nowadays it's costly and slow compared to other options and does not give you the flexibility to restore on other systems. Firewire or USB external drives are the way to go-provided you're not tempted to use that extra capacity for your data overflow. If that's inevitable, you need to buy another drive. If you can't be bothered to remember to back up on a regular basis, there are programs you can buy to schedule automatic backups.

Of course, in a creative business, you probably have very large data sets and a large number of files, possibly more than you can back up in one night. And if your back-up time cuts into your work time, that translates to downtime. One solution is to upgrade your network to gigabit. It's ten times faster than 100 Base-T Ethernet-currently the standard in many businesses-and you can transfer as much as two gigs per minute (vs. 200 Mb.)

Strategies for the RAID Environment

The issue is trickier for enterprise IT departments that measure their volume in terabytes rather than gigabytes, because it comes down to the fundamental business tradeoffs of time and money. If you have a RAID 1 mirror or RAID 5 striped with distributed parity, you're off to a good start. But if there's corruption on any of the drives, it will be mirrored as well. You still need a backup. Tape can take days to back up a high volume of data. Now, you can get external drives capable of holding up to 2 terabytes, allowing you to back up to multiple drives and restore the data to any Mac-an advantage that tape doesn't give you.

Do you need to back up your entire system every time you back up? One strategy to consider is incremental back-ups. Start with a full system back-up, and then back up only the data that has been changed in succeeding intervals. You'll still need to do full back-ups regularly, and you'll need to determine the schedule based on the amount of your data and the nature of your business-for instance, incremental back-ups nightly and full back-ups on the weekends.

The real killer in the movement of data is not always the amount of data in gigabytes or terabytes, but the file count. Is there a way to consolidate multiple files (directories, for example) into a single file? Fewer, larger files will be easier and faster to back up and restore than more, smaller files, even if the total volume of data is the same.

Of course, all the backup planning in the world is for naught if you don't also have a plan and the means for restoring the data. That means having yet another server dedicated to restoration. It's advisable if not imperative to do an occasional practice restore on another computer-obviously you can't restore to one that's failed. It doesn't have to be the identical computer, just one that's sufficiently robust to handle the data.

Managing the movement and storage of data is a secondary if not even a more distant priority at most companies relative to their primary business, which is why IT departments have to keep cajoling management on the importance of having adequate backup resources-and the cost of downtime. At our company, handling data is the main thing we do, so our experience may be instructive for other companies.

We didn't even have servers when I started. Now the Mac group has 40 terabytes of online fibre storage and 60 terabytes of offline disk storage using 500 GB and 750 GB SATA and PATA drives. The combined storage capacity is the equivalent of 100 million floppy discs. All of our systems are on a gigabit network, through which we push between 6 and 8 terabytes per day-almost more than the total of data permanently stored at the Library of Congress. Where it used to take us a week to back up 3 terabytes to tape and another week to restore it, we can now back up this amount of data to a server in just one day. This is over a 14-fold savings in time and resources.

Our business is all about keeping data safe, but when you think about it, it really ought to be a priority for any business.

When all else fails: data recovery

So, you vowed to be better about backing up, but you got too busy and just plain forgot. Or everything got backed up except one crucial file. Or your well-planned back-up system didn't do its job properly. Well, good news. Your chances are good, actually better than good, that the data is not really lost and that it can be restored.

At our company, the data gets imaged from theĘoriginal drive in the cleanroom then sent to our Mac engineers. Once we ascertain whether the damage is logical or physical, we'll determine the best strategy for an optimal recovery. We typically call our customers to find out what is most important to them and what they are hoping to have recovered. The recovered data is returned on external hard drives or DVDs, and also backed up onto our own servers. The data is held for a period of time in case the customer has any issues or questions. The length of turnaround and cost is based on the service selected (priority, standard or economy), the drive capacity, the operating system and the complexity of the recovery.

More data than ever, that matters more than ever.

As we've witnessed the increase in data volume over the years, we've also noted that it is proportional to the rise in the value of data-because so much of our work and lives now exists primarily in digital form. (Ironically, our company started out doing drive repair, until we figured out that people were more concerned with their data than the drives themselves.) Data loss can cripple businesses and send everyday users to the edge of despair.

With today's data recovery capabilities, despair is unwarranted in the vast majority of cases. However, the best recovery plan is the user's own. Think about what that growing volume of data is really worth to you and figure out how to safeguard it before it overflows. Hard drive failure may be unavoidable, but with the right tools, strategies mindset, you can avoid losing what really matters.


Mike Cobb is the Director of Macintosh and Unix Engineering at DriveSavers, a data recovery services company. He joined the company in 1994, and has performed recoveries on all types of hard drives. Before joining DriveSavers, he worked as a tech support supervisor and beta-test coordinator for a manufacturer of Mac-based RAID mirroring hardware and software, among other products.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Monolingual 1.6.4 - Remove unwanted OS X...
Monolingual is a program for removing unnecesary language resources from OS X, in order to reclaim several hundred megabytes of disk space. If you use your computer in only one (human) language, you... Read more
CleanApp 5.0 - Application deinstaller a...
CleanApp is an application deinstaller and archiver.... Your hard drive gets fuller day by day, but do you know why? CleanApp 5 provides you with insights how to reclaim disk space. There are... Read more
Fantastical 2.0 - Create calendar events...
Fantastical is the Mac calendar you'll actually enjoy using. Creating an event with Fantastical is quick, easy, and fun: Open Fantastical with a single click or keystroke Type in your event details... Read more
Cocktail 8.2 - General maintenance and o...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Direct Mail 4.0.4 - Create and send grea...
Direct Mail is an easy-to-use, fully-featured email marketing app purpose-built for OS X. It lets you create and send great looking email campaigns. Start your newsletter by selecting from a gallery... Read more
jAlbum Pro 12.6 - Organize your digital...
jAlbum Pro has all the features you love in jAlbum, but comes with a commercial license. With jAlbum, you can create gorgeous custom photo galleries for the Web without writing a line of code!... Read more
jAlbum 12.6 - Create custom photo galler...
With jAlbum, you can create gorgeous custom photo galleries for the Web without writing a line of code! Beginner-friendly, with pro results Simply drag and drop photos into groups, choose a design... Read more
Lyn 1.5.9 - Lightweight image browser an...
Lyn is a lightweight and fast image browser and viewer designed for photographers, graphic artists and Web designers. Featuring an extremely versatile and aesthetically pleasing interface, it... Read more
Sublime Text 3080 - Sophisticated text e...
Sublime Text is a sophisticated text editor for code, markup, and prose. You'll love the slick user interface, extraordinary features, and amazing performance. Goto Anything. Use Goto Anything to... Read more
WALTR 1.0.11 - Drag-and-drop any media f...
WALTR is designed to make it easy to upload and convert any music or video file to an iPad or iPhone format for native playback. It supports a huge variety of media file types, including MP3, MP4,... Read more

Bio Inc's New Expansion is Infectin...
Bio Inc., by DryGin Studios, is the real time strategy game where you infect a human body with the worst virus your evil brain can design. Recently, the game was updated to add a whole lot of new features. Now you can play the new “Lethal”... | Read more »
The Monocular Minion is Here! Despicable...
Despicable Me: Minion Rush, by Gameloft, is introducing a new runner to the mix in their latest update. Now you can play as Carl, the prankster minion. Carl has a few new abilities to play with, including running at a higher speed from the start.... | Read more »
Dungeon of Madness (Games)
Dungeon of Madness 1.0.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0.0 (iTunes) Description: Dungeon of Madness is an action game where you rotate tiles to create our own route. Help the hero by connecting the... | Read more »
Filters for iPhone (Photography)
Filters for iPhone 1.0 Device: iOS iPhone Category: Photography Price: $.99, Version: 1.0 (iTunes) Description: | Read more »
Jump'N'Shoot Attack (Games)
Jump'N'Shoot Attack 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: A mobile game for gamers! Join Louise Lightfoot, the legendary "Master of Jumping and Shooting", on her mission to save... | Read more »
Space Bounties Inc. (Games)
Space Bounties Inc. 1.4 Device: iOS Universal Category: Games Price: $1.99, Version: 1.4 (iTunes) Description: SuperGameDroid: 4/5 "Satisfying futuristic RPG combat, high replay value, and a heavy dose of nostalgia make Space... | Read more »
Gamebook: Pocket RPG (Games)
Gamebook: Pocket RPG 1.0.11 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0.11 (iTunes) Description: Walk into the Land of Lanthir Lamath ruled by wicked skeletons and fight for your life in a thrilling adventure.... | Read more »
Kids Can Mix, Match, and Catch with Tata...
Tatadada MixMatch, by Tatadada Ltd, is a mobile version of the classic game of mix & match. The game uses brightly colored creatures to train your children's pattern matching skills and hand-eye coordination. It's aimed at children around age 5... | Read more »
The Trace: Murder Mystery Game (Games)
The Trace: Murder Mystery Game 1.2.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.2.0 (iTunes) Description: | Read more »
This Week at 148Apps: March 16-20, 2015
Spring Roars In At 148Apps How do you know what apps are worth your time and money? Just look to the review team at 148Apps. We sort through the chaos and find the apps you're looking for. The ones we love become Editor’s Choice, standing out above... | Read more »

Price Scanner via MacPrices.net

Logitech Says MX Master Is Its Most Advanced...
Logitech’s new MX Master Wireless Mouse incorporates the best of Logitech’s many computer mouse innovations into a striking hand-sculpted design. The company claims that the MX Master creates a new... Read more
Save up to $300 on a new Mac, $30 on an iPad,...
Purchase a new Mac or iPad at The Apple Store for Education and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free,... Read more
Apple refurbished 2014 MacBook Airs available...
The Apple Store lowered prices on Apple Certified Refurbished 2014 MacBook Airs recently, with models now available starting at $679. An Apple one-year warranty is included with each MacBook, and... Read more
Mac Notebook Evolution; A Desktop Replacement...
More often than not right from the beginning, Apple’s Macs have tended to skew toward small. The original Macs were called “compacts,”, and notwithstanding a few exceptions like the honking Big Mac... Read more
13-inch 1.4GHz/128GB MacBook Air (Apple refur...
The Apple Store has Apple Certified Refurbished 2014 13″ 1.4GHz/128GB MacBook Airs available for $759 including free shipping plus Apple’s standard one-year warranty. Their price is $240 off original... Read more
YEP! Alternative Browser for iOS Now Supports...
Pfaeffikon, Switzerland based Power App AG has announced the release of an update to their Yep! Web Browser (v1.3.0) for iOS8 iPhone and iPad. Yep! hit the App Store shortly after the release of iOS... Read more
15-inch Retina MacBook Pros on sale for up to...
B&H Photo has the new 2014 15″ Retina MacBook Pros on sale for up to $250 off MSRP for a limited time. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $... Read more
Clearance 13-inch Retina MacBook Pros availab...
B&H Photo has leftover 2014 13″ Retina MacBook Pros on sale for up to $250 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.6GHz/128GB Retina MacBook Pro: $1098... Read more
Clearance 2014 MacBook Airs on sale for up to...
B&H Photo has MacBook Airs on sale for up to $180 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 11″ 128GB MacBook Air: $789.99 110 off original MSRP - 11″ 256GB... Read more
Apple refurbished Time Capsules available for...
The Apple Store has certified refurbished Time Capsules available for $100 off MSRP. Apple’s one-year warranty is included with each Time Capsule, and shipping is free: - 2TB Time Capsule: $199, $100... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - D...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Systems Engineer - Pre Sales, Educat...
…is responsible for proactively providing technical expertise to drive sales of Apple solutions into assigned accounts. The SE architects, validates, and assists in Read more
Sr. Technical Services Consultant, *Apple*...
**Job Summary** Apple Professional Services (APS) has an opening for a senior technical position that contributes to Apple 's efforts for strategic and transactional Read more
*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.