Backup strategies

---

Have you backed up your computer lately? Have you backed up all your important and precious photographs to another hard drive; a CD or DVD? If not, stop reading this right now and go and do it. I'm serious. What are you still doing here? Stop reading and go make copies of all the data that is important to you? Use whatever mechanism you have available to you but go and do it now.

Aah, so you're back. How did it go? It's a bit painful and laborious isn't it which is probably why we tend to put it off and put it off until....well it might be too late.

The fact of the matter is that hard drives fail. How often do they fail? Well a respected Seagate hard drive comes with a 5 year warranty but on average a given hard drive will have an MTBF (mean time between failure) of 3-5 years if you're lucky, sometimes a lot less than that. So the simple fact of the matter is that the hard drive on which all of your photographs are now residing will fail at some point in the not so distant future.

Fortunately there are a plethora of solutions in the market-place to satisfy both the hardware and software requirements of automated backups to alleviate some of this pain. It is important however to formulate a strategy to make the process as seamless, accurate and painless as possible. Because there is such a variety of options available to you, you should make up your own mind as to how best to go about it. All I can do here in this blog is to relay my own experiences and what works best for me currently. It's important to note in reading my personal strategy however that I have a large investment in hardware and software of all kinds that does help the process. But this is not a requirement. The most fundamental part of any backup strategy is to ensure that you have at least one other working copy of your data at all times. You should preferably have two (or more) copies but that is not always feasible or practical. If you're completely paranoid one of those copies should be off site.

In my working machine at my office I have four hard drives which are utilised in the following fashion. Two drives are 160GB 10,000 RPM Western Digital Raptors. These drives are configured for RAID-0 with the sole purpose in inmproving performance as these super-fast RAID-0 devices allow the file system to be distributed across all disks and reduce latency for both read/write operations. What is important to realise is that RAID-0 is NOT a backup or redundancy strategy and this is an important consideration when talking about full system backups (later on in this article). In the event of failure of even one of the hard drives in a RAID-0 configuration it will be almost impossible to recover as the data on any one drive will be corrupt and incomplete.

The two other drives in my system are not setup for RAID but are configured with one drive being storage for my raw digital negatives that are in a work-in-progress status. The other drive stores the photograph derivatives that have been processed either in Lightroom, DxO Optics Pro or Adobe Photoshop and subsequently prepared for web and/or print output. These two drives are constantly backed up using synchronisation software and completed photographs are then deleted to make space for new incoming work. I use an application called Allways Synch and this software allows you to migrate only the changed files or folders from source to a backup destination. The software also allows for dual or one way synchronisation as you please however it works best for me by simply ensuring that my backup drives have an exact copy of whatever has been placed onto the RAW or DERIVATIVE drives at any stage. I use Allways synch in this manner to ensure that my Lightroom Catalogs, Digital Negatives, layered Photoshop files, web JPG output, print prepared TIFF files, documents, saved games etc are always backed up. Since the installation of Allways Synch I have saved about 32 hours of file copy operations as the application ensures that it only copies the new or changed files. That's also 32 hours of hard-drive thrashing that has been avoided increasing the longevity of those drives. There is no manual scouring of files and folders to determine what exactly needs copying over. The application is scheduled to run daily at midnight and this way I can walk away from my machine knowing that it will be securely backed up.

So where does Allways Synch backup to? It currently writes to a Drobo Unit that has four 1 terabyte (TB) drives in it. The Drobo unit utilises a data striping redundancy mechanism that is very similar in nature to RAID-5 however it works much better. It's better and simpler because with the Drobo unit you can mix and match drives as your need and budget allows. The unit can hold up to four drives of a maximum of 4TB in size each for a maximum storage capacity of 16TB per unit. However let's assume starting off with only two drives; once they start to full up the Drobo Unit alerts you to the fact. You can then simply slot in another drive of whatever capacity and the unit then makes that space available. It also then automatically and redundantly stripes data across all three drives for you in the background. Remove a drive and Drobo knows how to recover from just the remaining drives. Nothing could be simpler. With RAID-5 all drives have to be of equal capacity and recovering from a RAID-5 crash is a long process as the damaged drive is rebuilt from the working ones. It can be done, I recently had a hard drive crash on my Intel NAS RAID-5 server and it was simply a matter of removing the damaged drive, replacing it with another similar drive (500GB in this instance) and wait the requisite 24 odd hours while that drive was rebuilt. It worked well, it just took a lot longer than the Drobo and the Intel doesn't support drives of varying capacities.

So the Drobo is where Allways Synch backs up to. Because there is redundancy already built into the 4 1TB drives that's where most of my photographs now reside. If I was completely and utterly paranoid then these drives in itself should also be backed up to some other device (DVD for instance) and stored off-site. However because I am intrinsically lazy and confident that I can recover these photo's if I need them I have left it as is. Besides I can't honestly imagine backing up 4TB to DVD!! Some really very important photographs however (such as the wedding photo's for Renett and Johann are backed up to a seperate drive - my Intel NAS RAID - to be absolutely sure that data is uncorruptible). Previously I used a Digital Asset Methodology espoused by Peter Krogh in terms of creating RAW 'buckets' of 4.2GB each that could be backed up directly to DVD but since I have moved my DAM catalogs over to Lightoom 2 I have dispensed with that idea as I'm not writing to DVD anymore. For more on a DAM philosophy go to Peter's web site - The DAM Forum.

It has been proven also that backing up to DVD is not a fool-proof exercise. Writing DVD's can be prone to failure and the longevity of DVD's is also in doubt. Will we be able to read these DVD's in years to come? Look at those with tape drive backups of their data - they currently have no way of retrieving that. More simplistically can you still play those Betamax video tapes anywhere? With BluRay now replacing normal DVD's who knows what the future of this technology holds? So for now, I personally have dispensed with DVD backups altogether?

It acutally might be worth taking a step backwards for a second to talk about the process I follow when importing images from my cameras CF cards or the Epson P-5000 which I use to store photographs on when in the field. The process is effectively the same; it's just that in the field if a CF card is filled up then I will offload that to the Epson P-5000 so that I can reuse the card if necessary. If a card is filled up and not backed up to the Epson then it is placed in a Gepe card safe 'upside-down' so that I know it cannot be used until offloaded to either the PC or the Epson. Once a card is placed right side up in the card safe then I know that it must always be formatted in-camera before re-use. When back at the PC I begin the import process into Lightroom. During the import I convert my RAW images to Adobe's Digital Negative format (.DNG). I have adopted DNG across all my photographs for longevity and am comfortable that Adobe will continue to support and enhance the DNG format. During the import however for safety's sake I make an automatic backup copy of the images to a IOMega 250GB Firewire drive (supported by the Lightroom import process). This is just a temporary measure as due to the small nature of this drive the files are only kept there until that drive begins to fill up and then I delete the oldest imported files as we go. It's merely a safety net in case something goes wrong during the import process.

So what about the Intel NAS RAID-5 Server. This server has 4 500GB hard-drives configured as RAID-5 for data redundancy and recovery. This server comes into play with full-system backups and backups of all other purchased software, music and any other important data that I have. Most importantly for the purposes of this discussion however, the Intel NAS server holds the full-system backups of each PC in my office. These full system backups are generated using Acronis True Image Home which runs on each workstation. This application is not scheduled although in truth it could be. Once a system has been built to spec with all the requisite software (Windows, hardware drivers, Photoshop, Lightroom, Office etc) then I will use Acronis True Image to image the entire hard drive (remember those RAID-0 drives?) to the Intel NAS server. In this way if one of the system volumes should ever fail then that machine can be rebuilt from scratch quite quickly and will be up and running as before.

I have also recently just installed a Microsoft Windows Home Server on my network to determine it's efficacy. This is only a 180-day trial so we'll see how it goes but in essence all my machines on the network (all 6 of them) are automatically backed up each night by the server. These backups are stored on the Windows Home server and again should any machine fail it can be restored quite quickly. I have to ask myself the question though, shouldn't I be backing up those full system backups to the NAS server? Perhaps I will set that up tonight?

Does this all sound paranoid? Possibly? Overkill? Probably? But then again, I take comfort in the fact knowing that my data is secure and redundantly stored in the event of failure. I say again - hard drives fail! It's just a matter of time. Have you backed up your data lately?

---