TweetFollow Us on Twitter

LBackup: Flexible backup for system administrators

Volume Number: 24 (2008)
Issue Number: 08
Column Tag: Backup

LBackup: Flexible backup for system administrators

by Henry Shustak, Infinity Architect

Project History

Since the project started over eight years ago various people have contributed to LBackup in one way or another. In 2007 the project was released under the GNU GPL and a website for the project was also established.

If you had asked me what I thought about backup ten years ago, I would probably have replied, "It is an important but extremely boring topic". However, my attitude towards backup since then has dramatically changed.

I was put in charge of backups at work. The company already had established a backup system. They were using commercial backup software to manage the data which were copied to tapes by an expensive and extremely slow (by today's standards) tape drive unit. I had been tasked with keeping the system running.

The backups were often running behind schedule. Initially, I assumed this was due to the tape errors which would often greet me on Monday mornings. It slowly dawned on me that the tape errors were not the only reason that I was perpetually playing backup catch-up. The primary issue was actually the limited amount of time that was available for the backup to run. Looking through the logs, I calculated that the backup would often take longer than the time available for the backup to run. I wanted to find a solution to this problem, but there were no inexpensive solutions readily available. This is what initially sparked my interest in backup technology.

A couple of years later, I enrolled at university. Initially, I used the school computers and the backups of my computer files were managed for me by the system administrators at the university.

There were not enough computers in the labs at university and the times which undergraduates were allowed access to the computer labs was strictly regulated.

Luckily, just prior to starting university, I had been gifted a second-hand laptop. This was a great gift and I convinced myself that working remotely from home would not only be more convenient, but also more productive. However, in reality, working remotely from home was intolerably slow. Therefore, it was not long until I had many important documents stored locally on my laptop rather than on the university computer system.

Backing up these locally stored files manually was not an attractive option. In addition, I did not want to return to the daily competition with other students for computer access.

If something was to happen to my laptop and was to I lose all my schoolwork, I could fail multiple courses. The danger of my situation was reinforced by the recent memory of installing a new hard drive for the previous owner of my laptop due to a hard drive failure.

It was time to take action. I begin researching various backup solutions that would protect my precious files. The initial aim of this project was not to build a backup system; it was to find an existing backup tool that was reliable, simple, unobtrusive and most importantly inexpensive.

After testing various backup systems I eventually, settled upon rsync. I was delighted that my important schoolwork was now backed up. I had no idea at the time that this script which had been whipped up for testing was going to evolve into LBackup.

Not long after installing and configuring the script on a few machines, I realized there were some new features which I desperately required on some of these new setups. I manually added the new feature to each installation as required. After repeating this process a few times it became clear that there had to be a better way. A more sensible approach would be to separate the code which performed the backup, from the configuration information which contained the details of how to perform the backup, on each individual machine.

I spoke to a friend whom I had met at university about the problem and together we worked out a solution. It was not long before an example configuration directory was included and the system became commonly referred to as LBackup.

Adding the configuration system meant that additional features could easily be added as required.

About LBackup

LBackup is an open source backup tool. It relies on many other open source tools in order to function.

A basic LBackup configuration file often takes less than 10 minutes to setup (excluding the time required for testing). Although the current version of LBackup is not yet as easy to setup as Time Machine, it is completely customizable.

Below is a list of the key LBackup features as of version 0.9.8q6.

Designed for user data backup

Full backup tree

Encrypted backups (Transport and Storage)

Email reports

Hard links (Saving Space)

RSync back-end (Reliable)

Pre and post actions (such as mounting and un-mounting volumes)

Network backup

Automation

All these features are documented in further detail on the LBackup website.

Basic Backup Tutorial

After completing this tutorial you should have an idea of whether LBackup is right for your needs. This tutorial aims to provide you with a feel for how LBackup works. This tutorial assumes you are running Mac OS 10.4.x or later and that you are familiar with using Mac OS X.

Step (1): Create a new directory on in the /Users/Shared directory called source.

Step (2): Create another new directory in the /Users/Shared directory called destination.

Step (3): Visit the LBackup web page and download the latest version of LBackup

Step (4): Run the LBackup installer and follow the on screen instructions.

Step (5): Open the root directory on the system which you just installed LBackup.

Step(6): Open the lbackup directory within the root directory.

Step (7): Copy the example_backup_config directory into the /Users/Shared directory.

Step (8): Rename the directory you copied in the previous step to testinglbackup.

Step (9): Locate the file called example_backup.conf within the testinglbackup directory.

Step (10): Rename the example_backup.conf file to testinglbackup.conf.

Step (11): Edit the configuration file so it is the same as the listing below.

 backupConfigurationVersion=1006
 useSSH="NO"
 sendGrowlNotification="NO"
 backupSource=/Users/Shared/source
 backupDest=/Users/Shared/destination
 log_fileName="TestingLBackup.log"
 excludes_filename=excludes.txt
 numRotations=8
 ssh_permit_standard_rsync_version="YES"

Step (12): Open a Terminal.

Step (13): Type the following into the terminal and then press return.

/usr/local/sbin/lbackup /Users/Shared/testinglbackup/testinglbackup.conf

Next you should see something similar to the following as output on the terminal:

Loading Backup Script Configuration Data...
Checking for Pre Action Scripts...
First Run Full Copy...
Synchronizing...
Number of files: 1
Number of files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 62
Total bytes sent: 74
Total bytes received: 20
sent 74 bytes  received 20 bytes  188.00 bytes/sec
total size is 0  speedup is 0.00
real   0m0.153s
user   0m0.004s
sys   0m0.018s
Rotating Backups...
Performing Atomic Swap...
Checking for Post Action Scripts...
Backup Completed Successfully

You have just configured and run LBackup. Check inside the destination directory you created in step (2) and you will see a new directory called Section.0. This directory contains the latest successful backup of the source directory which you created in step (1).

Add some files or directories into the source directory. Then run LBackup again by following the instructions in Step (12) and Step (13). There will now be two directories in the destination directory. One called Section.0 and one called Section.1. If you open the latest backup directory (Section.0), then you should see a directory called source. If you open this source directory then any of the files and directories you placed in the backup source directory should also be visible.

If you would like to exclude files from being copied from the source directory to the destination directory then edit the excludes.txt file located within the testlbackup directory which is located within the /Users/Shared directory.

MetaData Preservation

On Mac OS X systems, many file system objects have associated metadata. The extended file system (HFS+) supports the storage of this metadata. It is important that you check your backups and ensure that any required metadata is being preserved before you accept any backup system as a suitable backup solution.

The compilation options of rsync and the run time flags specified by LBackup determine which metadata is preserved when you run LBackup.

On a Mac OS X systems, the user who executes the backup is also an important factor in preserving metadata. For example, if your backup policy states that preserving the POSIX (portable operating system interface) owner and groups of files and directories is important. Then you should run the backup and check that this data has in fact been preserved.

The ownerships are a special case when performing a backup on Mac OS X systems. This is because the only user who is capable of the modifying the owner of existing file system objects is root. Therefore, if you are backing up files that belong to various users, it is important to run LBackup as root so this metadata is able to be preserved. For example, if you were to run the backup as user with UID (user identifier) 508 then all the files which are backed up will have the POSIX owner set to 508 regardless of whether the file system object being backed up had this UID.

Restoring a File or Directory

To restore a file or directory to the desktop simply locate the file or directory you would like to restore within the backup destination directory and then copy this file or directory to the desktop.

When restoring a file or directory, metadata preservation may still be important. For example, if you were going to restore a network home directory, then it would be advisable to copy (restore) the directory as root. This will preserve the permissions of the NHD and its contents.

User Access to Backups

Because LBackup stores the backup(s) in a file tree rather than in an archive, it is possible to configure the backup directory so it is available via the network, as a read only share point. Such a setup will allow users to perform file level restores, without needing to contact the backup operator.

Why is Lbackup Open Source?

In 2007, it was decided that LBackup should be available to anyone and everyone for free. One motivation behind the decision was that I and many of the companies I was working with preferred to use an open source backup solution when available and appropriate. Another reason behind the open-source move was that rsync had also been released under the GNU GPL and as such this was an obvious license to release LBackup under. However, the most important reason for open sourcing the project was to ensure that other people would have a tool which could be quickly and easily modified to meet their own specific needs.

Essentially this means that if you add a feature, then everyone else using LBackup also benefits from your work and vice versa. Open source development makes sense on many different levels for this kind of project.

Why the Crazy Version Numbers?

I have been using LBackup at home for more than eight years and at work for more than four years. A goal of a "1.0" release is a GUI which will allow users to create a basic LBackup configuration.

The future of LBackup

I would like to see a clean up of the code. It is nothing a little re-factoring will not fix. I am sure this will happen at some stage in the near future.

In addition to a general code clean up, there are some other new features listed below which I believe people would find handy:

A GUI offering even basic backup setup options.

Some bundled scripts for shutting down and starting up various databases.

An installer or some sort of automated setup of network backup client(s).

A system for handling network backup clients that come onto and off the network.

(I have heard that TimeCapsule and Time Machine does a pretty good job under 10.5)

Some bundled scripts for performing various database dumps.

A bundled script to handle backup locking

A system to make the setup of encrypted backups a little easier.

A pre- or post- script which performs a malware scan of the client or client data.

Possible integration with Link-Backup (which looks like an interesting backup system!).

There are of course many more features that other people are sure to need and may even develop. The LBackup team is always open to ideas. If you feel like adding a feature or just tidying-up some code then please consider joining the LBackup team, thus making the world a better place.

LBackup Support

If you require assistance with the setup or integration of LBackup, then Lucid Information Systems is available to help. Lucid offers on-site support for LBackup in selected regions. Visit their web page for contact details.

LBackup Pre and Post Scripts

LBackup supports pre and post scripts. This per-configuration feature is useful, if you would like to perform certain actions before or after a backup.

The LBackup scripting sub-system opens up a great deal of flexibility to those who are comfortable with programming. In addition LBackup ships with a variety of example scripts. The example pre and post scripts offer a way for you to get started quickly with the LBackup scripting subsystem.

The pre and post action examples may be found in the LBackup example_backup_config directory. If you open the resources directory and then open the example-scripts directory you will be able to browse though the example scripts.

One of the example scripts for Mac OS X users, allows LBackup to perform a push backup via SSH to an encrypted disk image. This script requires that you have installed MacFUSE and the MacFUSE SSH File System module. This pre script will mount the directory of a remote machine via SSH. Then it will proceed to mount an encrypted disk image that may be configured as the backup destination for this LBackup configuration.

There are also associated post scripts to unmount the encrypted disk image and the SSH mount point.

LBackup also ships with a variety of scripts to handle local drives and image files. With regards to the LBackup scripting interface, your imagination is the limit.

The LBackup scripting subsystem and the bundled scripts allow the configuration of redundant encrypted backups for a single data source. This way should something happen to one of your encrypted backups, your data should still be safe in one of the other encrypted redundant backups.

More Scripting

LBackup is command line and configuration file driven. Therefore, it is possible to build control systems which wrap around or extend LBackup.

For example you may want to create backup chains or integrate LBackup with third party (or your own) analysis and monitoring tools.

Documentation and Development

The LBackup development and documentation is open for editing. Provided this is not abused, the intention is to keep it this way.

This means you are able to look into the future of LBackup and even express your opinion with regards development decisions and directions..

In addition, you have the ability to fix any spelling or grammatical errors you may find within the documentation. You may even end up adding some pages on how to perform some sort of special backup routine using LBackup.

Conclusions

Let your backup policy dictate your requirements. Then find a system that is a good fit. Also, try different backup systems and see which system is the best for you and your requirements.

You may find that you end up using more than one backup system. For example, one system may not cover all of your requirements or you may find that one system will work better in certain situations, while another will work better in other situations.

Have a look at the LBackup web page and screen casts and then give it a try.

Other Notable Free Backup Tools For Mac OS X

If you are looking for a free backup tool for use with Mac OS X then the following free tools are defiantly worth investigating.

Carbon Copy Cloner

rdiff-backup

rsync

Time Machine (requires 10.5 or later.)

Rsnapshot

Link-Backup

Related Links

LBackup: http://www.lucidsystems.org/lbackup

GNU GPL: http://www.gnu.org/copyleft/gpl.html

Carbon Copy Cloner: http://www.bombich.com/software/ccc.html

rdiff-backup: http://www.nongnu.org/rdiff-backup/

rsync: http://samba.anu.edu.au/rsync/

Rsnapshot: http://www.rsnapshot.org/

Time Machine: http://www.apple.com/macosx/features/Time Machine.html

TimeCapsule: http://www.apple.com/timecapsule/

Link-Backup: http://www.scottlu.com/Content/Link-Backup.html

MacFUSE: http://code.google.com/p/macfuse/

MacFUSE SSHFS: http://code.google.com/p/macfuse/wiki/MACFUSE_FS_SSHFS

Lucid Information Systems: http://www.lucidsystems.org


In 1982, Henri used an Apple Computer for the first time. It was an Apple ][e and ever since he has enjoyed programming. Henri is not the only one who has loaded data onto a computer and has then later been unable to retrieve this data. However, in spite these experiences, he continues to use computer systems designed by Apple to store various digital information. More recently he has made an effort to prevent the loss of this data by trying, building, deploying and testing various digital data protection systems.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

ExpanDrive 6.1.8 - Access cloud storage...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
DiskCatalogMaker 7.2.7 - Catalog your di...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast Finder-like intuitive look and feel Super-fast search algorithm Can compress catalog data for... Read more
DiskCatalogMaker 7.2.7 - Catalog your di...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast Finder-like intuitive look and feel Super-fast search algorithm Can compress catalog data for... Read more
Iridient Developer 3.2.1 - Powerful imag...
Iridient Developer (was RAW Developer) is a powerful image-conversion application designed specifically for OS X. Iridient Developer gives advanced photographers total control over every aspect of... Read more
BusyContacts 1.2.7 - Fast, efficient con...
BusyContacts is a contact manager for OS X that makes creating, finding, and managing contacts faster and more efficient. It brings to contact management the same power, flexibility, and sharing... Read more
MegaSeg 6.0.5 - Professional DJ and radi...
MegaSeg is a complete solution for pro audio/video DJ mixing, radio automation, and music scheduling with rock-solid performance and an easy-to-use design. Mix with visual waveforms and Magic... Read more
BusyContacts 1.2.7 - Fast, efficient con...
BusyContacts is a contact manager for OS X that makes creating, finding, and managing contacts faster and more efficient. It brings to contact management the same power, flexibility, and sharing... Read more
MegaSeg 6.0.5 - Professional DJ and radi...
MegaSeg is a complete solution for pro audio/video DJ mixing, radio automation, and music scheduling with rock-solid performance and an easy-to-use design. Mix with visual waveforms and Magic... Read more
Iridient Developer 3.2.1 - Powerful imag...
Iridient Developer (was RAW Developer) is a powerful image-conversion application designed specifically for OS X. Iridient Developer gives advanced photographers total control over every aspect of... Read more
iFFmpeg 6.6.1 - Convert multimedia files...
iFFmpeg is a comprehensive media tool to convert movie, audio and media files between formats. The FFmpeg command line instructions can be very hard to master/understand, so iFFmpeg does all the hard... Read more

Latest Forum Discussions

See All

The best deals on the App Store this wee...
A new week means new discounts on the App Store. This week's deals run the gamut of action-adventure titles, puzzle games, and one of the best narrative adventure series out there. If you're looking to fill out your mobile gaming library on a... | Read more »
What you need to know about Animal Cross...
We hope you've been hard at work on collecting all of those holiday items in Animal Crossing: Pocket Camp, because you're about to get a whole new list of fun things to do as the game receives its first big update sometime soon. There are a lot of... | Read more »
Reigns: Her Majesty guide - how to use e...
Ruling a kingdom isn't easy--doubly so for a queen whose every decision is questioned by the other factions seeking a slice of power. Reigns: Her Majesty builds on the original game's swipey tactics, adding items that you can use to move the story... | Read more »
The best new games we played this week -...
Friday has crept up on us once again, so it's time to honor the best new games we've played over the past few days. This past week was a pretty exciting one, with the debut of lots of beautiful new indies and some familiar faces returning to the... | Read more »
Portal Knights guide- beginner tips and...
Portal Knights is finally making the jump to iOS and Android, and it's already climbing the ranks to become the next big MMO experience on mobile. This sprawling sandbox game will let you pursue any adventure you wish, whether you want to sling... | Read more »
Reigns: Her Majesty guide - how to swipe...
Reigns: Her Majesty is storming the App Store this week, bringing more tinder-esque kingdom building to eager players everywhere. If you've played the original Reigns, you'll know that leading a kingdom is never easy. It's a careful balancing act... | Read more »
Getting Over It (Games)
Getting Over It 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: A game I madeFor a certain kind of person To hurt them. • Climb up an enormous mountain with nothing but a hammer and a pot.•... | Read more »
Reigns: Her Majesty (Games)
Reigns: Her Majesty 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: | Read more »
Pocket Legends Adventures guide - how to...
Pocket Legends Adventures is a fun action adventure RPG that takes control when you want it to, but also opens itself for player input, too, if you're looking to tkae a more active role in combat. Regardless of play style, the game can be quite... | Read more »
Portal Knights (Games)
Portal Knights 1.2.4 Device: iOS Universal Category: Games Price: $4.99, Version: 1.2.4 (iTunes) Description: Craft your adventure. Forge your hero. Become the ultimate Portal Knight! | Read more »

Price Scanner via MacPrices.net

Green Monday deal: 15″ 2.8GHz MacBook Pro on...
B&H Photo has the 15″ 2.8GHz Space Gray MacBook Pro on sale for $250 off MSRP for today only as part of their Green Monday/Holiday sale. Shipping is free, and B&H charges sales tax for NY... Read more
Green Monday sale: B&H offers 12″ Apple i...
B&H Photo has 12″ iPad Pros on sale for up to $150 off MSRP as part of their Green Monday/Holiday sale. Shipping is free, and B&H charges sales tax in NY & NJ only: – 12″ 64GB WiFi iPad... Read more
Holiday deal: 21″ and 27″ Apple iMacs on sale...
MacMall has 2017 21″ and 27″ Apple iMacs on sale for up to $200 off MSRP. Shipping is free: – 21″ 2.3GHz iMac: $999 $100 off MSRP – 21″ 3.0GHz iMac: $1199 $100 off MSRP – 21″ 3.4GHz iMac: $1379 $120... Read more
Holiday deal: Apple Mac minis for up to $150...
MacMall has Mac minis on sale for up to $100 off MSRP, each including free shipping: – 1.4GHz Mac mini: $399 $100 off MSRP – 2.6GHz Mac mini: $599 $100 off MSRP – 2.8GHz Mac mini: $949 $50 off MSRP... Read more
Beats by Dr. Dre – BeatsX Earphones on sale f...
Best Buy has BeatsX Earphones on sale for $109, $40 off, on their online store. Sale price for online orders only. Choose free store pickup, if available, or choose free shipping. Read more
10″ 64GB WiFi Apple iPad Pros on sale for $59...
MacMall has 10.5″ 64GB Apple iPad Pros on sale for $599 including free shipping. That’s $50 off MSRP and among the lowest prices available for these iPads from any Apple reseller. Read more
15″ 2.2GHz MacBook Pros on sale for $200-$300...
B&H Photo has the 15″ 2.2GHz MacBook Pro available for $200 off MSRP including free shipping plus NY & NJ sales tax only: – 15″ 2.2GHz MacBook Pro (MJLQ2LL/A): $1799 $200 off MSRP Apple has... Read more
Holiday sale: 15″ MacBook Pros for $200-$420...
MacMall has 15″ MacBook Pros on sale for $220-$300 off MSRP, each including free shipping: – 15″ 2.8GHz MacBook Pro Space Gray (MPTR2LL/A): $2179, $220 off MSRP – 15″ 2.8GHz MacBook Pro Silver (... Read more
Holiday sale: 13″ MacBook Airs for up to $150...
B&H Photo has 13″ MacBook Airs on sale for $100 off MSRP as part of their Holiday sale. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13″ 1.8GHz/128GB MacBook... Read more
The best Holiday sale prices on 13″ MacBook P...
B&H Photo has 13″ MacBook Pros on sale this weekend, with models available for $100-$150 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13-inch 2.3GHz... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Information Security - Security Data...
# Apple Information Security - Security Data Analyst Job Number: 113119545 Austin, Texas, United States Posted: 10-Nov-2017 Weekly Hours: 40.00 **Job Summary** This Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.