Best way of preserving your digital images?

My belief is that the best backup policy is to diversify, diversify, diversify. The underlying principle is that if one backup set breaks, the others likely won't be ... if they're diversified.

What does this mean?
Don't keep all your sets on the same type of media. (What if it turns out the media you chose has some serious long-term issues that was only discovered yesterday?)

Don't keep all of your sets in the same geographic location. (What if you keep them all in your home, but the unexpected happens? Theft, fire, flood ...) I think a hybrid hard disk + DVD backup strategy works well.

CD and DVD Longevity: How Long Will They Last? Good reading

With today's price level of 300GB hard drives and soon DVD which can store 2x there is really no excuse any more not to save your master files as TIFF.

From Steves Digicam I Quote October 2004: "
What's the Bottom Line?:
The JPEG image format offers a way to save images using less space, but with some loss in image quality. Typically, a first generation save will be almost as good as a lossless TIFF as long as you use quality levels close to the highest available. Some "die hards" claim that you should never use a camera in JPEG mode when you have TIFF or RAW available as an option, and one cannot argue that you get the best quality and best editing capability with TIFF or RAW when compared to JPEG. That said, JPEG is a perfectly valid format to use even when capturing images the first time in your camera, especially when memory space, shooting speed, or the ability to print images without post processing is important. Remember that JPEG's are processed and ready to view/print, whereas RAW images require post processing to "develop" the images from the raw data. This takes additional time and can complicate your shoot-to-print workflow. A first generation JPEG will offer quality comparable to any other final or ready-to-print format, however, cannot offer latitude for correcting exposure and other shooting issues like a RAW image or a 48 bit TIFF. Bottom line: choose what works for you, but be sure to take the pros/cons of each format into consideration.
— Mike Chaney "

I quote following comments from different discussions groups: You can read the whole message at the link "archiving photo images" below!

• .....But JPEG compression is so dependent on the type of image being compressed that blanket statements about its quality should be approached with suspicion.

---

• .....I would use TIFF rather than JPEG because JPEG is a lossy format. It throws away data whenever you open it, do something, and save it again.

---

• .... Retouched" images are not archive. Once you've got the picture the way you want it save it as a different version. Never, ever, delete your original scan.

---

• ....The more information there is, the easier and more successful subsequent modifications and corrections will be. It is almost tragicomic when proponents of the use of JPEG as a general storage format fail to understand the ramifications of lossy JPEG compression.

---

• ...I bet that is what the original poster is getting at... You should ALWAYS archive THE EXACT FILES FROM THE CAMERA MEMORY CARD (they are your digital "negatives") without changing them in any way (I don't even trust "lossless rotation"), J PEG or whatever they are...and ALWAYS work on copies of these files. There is nothing to be gained by converting JPEG's to TIFF or any other lossless format and then archiving them...the JPEG artifacts will still be there, but the file size will be huge.

---

• There is no loss in saving as tiff. Ever.
  
  When you look at a picture, it's just a picture. An arrangement of pixels.
  
  When you save a picture you have to save it on disk in a picture format. The nearest to what you are seeing is for Windows users, BMP format. Each dot on the screen is a value in the file. However this gives a big file and is not readable by other than Windows.
  
  So the nearest universal format is uncompressed TIFF. There's also compressed TIFF whereby the data is compressed but not lost. For instance (I'm being simplistic here) if there is a bunch of pink pixels in the middle of the screen the compressed format says "100 pixels, all pink, occurring here") and this takes less than 100 individual identical values in the file. But you can see the data has not changed or quality lost by saving in this type of "non-lossy" compression.
  
  This is called file compression and there are lots of different ways of compressing files without any loss.
  
  However photographs are rarely compressed by very much by this non-lossy process because patterns or blocks are quite rare in pictures, so compression does not help much. Here is an actual list example:
  
  Uncomp TIFF Comp TIFF PNG format
  A sunset 5.49Mb 1.75Mb 1.73Mb
  Cornfield 5.49Mb 5.52Mb 4.06Mb
  
  (PNG is an alternative compressed format)
  
  I chose these 2 pictures because a sunset has very little detail and so compression works well. The cornfield has lots of detail and compression works badly - in fact it fails in one case as the compressed file is larger than the original!
  
  Now enter jpg format. When the data is saved **and only when it is saved** some data is thrown away from what you see on the screen. The file on disk is less accurate that what you are seeing in front of you. (Until you re-load the file on the disk, in which case you see the imperfections).
  
  This is because colour changes have been thrown away and averages made in places where it probably doesn't matter much. But this loss occurs only in the actual save to jpg on disk. And with jpg you can define roughly how much detail you throw away, or how much smaller you want the file to be. However I can say that best quality jpgs of the files above are about this size:
  
  Sunset 0.6Mb
  Cornfield 1.9Mb
  
  So that's at most a third the size, and at least a tenth. Again, note the detail on the original dictates the file size.
  
  But the important thing to realize is that image deterioration only occurs **when you save as a jpg**. There is no quality penalty for changing formats from jpg to tiff: and the only quality loss is when you save as a new jpg file, e.g. converting tiff to jpg.
  
  So the advice normally given is:
  
  1 Shoot so that your camera either saves jpg or tiff, depending on your card capacity and quality requirements.
  2 When editing, save intermediate files in TIFF as there is no loss of quality
  
  However my own opinion is that "best jpg" is indistinguishable on paper from TIFF that I always save intermediate files in that format because of the lower file size. But no doubt others disagree... :-)
  
  Bernard Hill (bernard@braeburn.co.uk)
  Braeburn Software
  Selkirk, Scotland




---

• .... Repeated saves in the .jpg format will degrade the picture quality. Repeated saves in .TIFF format will not. Yes, it eats disk space, but with 100 gigabyte drives going for under $100, that doesn't really matter much any more.

---

• Try to avoid visible artifacts ('squares' or 'blocks' introduced by JPEG compression) in images beginning from the early stages of image processing! The JPEG noise is approximately uniform when high quality compression (low compression rate) is used. However, low quality compression makes noise non-uniform and more difficult to get rid of. Therefore, one recommend using the highest quality of compression whenever possible if you still insist in archiving in the JPEG format.

---

• Following quotation is from one out of 85 replies in a newsgroup December 2003. The question was: JPG vs. TIFF Resolution Test
  
  Actually, you won't see any difference on-screen or in-print at home when comparing a 16-bit TIFF (48-bit color) to an 8-bit TIFF (which, as you note, will still look a lot better than JPEG). Monitors and home printers are 24-bit color limited (even 32-bit monitor displays are only 24-bit color).
  
  When you have your RAW (36-bit color) created 16-bit Tiffs (48-bit color space) printed on a true 36-bit LightJet or Durst Lambda (both have true 36-bit output), only then will you see the effect of not throwing away 4095/4096ths of the color space when degrading RAW images to 8-bit Tiffs. And you toss half of that again, when using a prosumer format like JPEG.
  Eric Witte (ewitte@hotmail.com)
  


Jerry McG (gmcgeorge.REMOVE@frontier.net) wrote 2003-12-04 : I've always used the Nikon NEF (RAW) format with my D1x, however the Nikon 5400 only permits TIFF or JPG. What I was intrigued by was that there's no real visible difference between the moose-like TIFF and the most compressed, basic JPG. Naturally, I convert the Jpgs and NEFs to TIF for processing in PS7. One great aspect of the RAW files that you point out is the dynamic range, which is apparent when I compare files shot on the D1x with the little 5400.



---

• In an digital imaging forum I read the following in June 2004:
  I strongly suggest refraining from using the words "archival" and "CD" in the same sentence. All the CD longevity ratings are based on accelerated testing that may or may not account for all the variables that affect the medium in the real world. Remember the debacle with the ozone-fading issue on the Epson 1270 inks? Henry Wilhelm wound up revising his testing protocols to include gas-fading after that, but before that, who knew?
  
  I just assume that CDs are likely to fail after 10 years, and handle my data accordingly.
  
  File formats are also an issue. Well-documented open-standard formats like TIFF and JPEG don't worry me. 200 years from now we'll still have the specs for these formats. I'm a great deal more worried about .NEFs and .CRWs....
  Bruce Fracer

---

Backing up:
Don't forget that TIFF format is a lossless compression compared to JPEG. Use always a lossless 24-bit format as TIFF (or PNG, PPM, etc) while working on the image, then JPEG it when you are ready to file it away or send it out on the net. If you expect to edit your image again in the future, keep a lossless master copy to work from. The JPEG you put up on your Web site should be a derived copy, not your editing master.

Therefore I always save my original files, as master copies, in TIFF-formats, in spite if that most digital cameras store the images as JPEG'S, and save and back them up on a CD-R disk. And for all of you , who are shooting RAW files and converting to 16-bit tiffs, it is a must, if you are not going to lose all that information.) Not a CD-RW as you accidentally can overwrite your originals. There's considerable information on CD-R stability. Life expectancy is measured in decades not year, as in a century or more. I am told! It is very inexpensive but not perfect. Writing to a CD-R is a two-step process, and scratching a CD-R is easy. However the main reason I like CD-R is that it is a universal format. The makers of hard drives, Zip Drives, and CD's all say these media will last a very long tome if you handle them properly. but that will not matter if you do not have a device that reads them. Ten years from now, we may not have USB ports, Zip Drives, or any of today's media cards, but I bet that you will have a drive that can read a 5.25-inch optical disc. There is too much content out there for them to disappear quickly. I will not recommend any DVD formats yet, because the standards war are still raging.

Will your grandchildren be able to figure out what those shiny round discs in the shoebox in the attic really are?

("The Mitsui Photo Worried About Your CD-R Disc Archive? This week's Sunday Morning Photographer column, (May 09, 2004) might just save you some major grief in the near future. Lately there has been a lot of talk about CD-R corruption and failures. Want to know what the best brand of CD-Recordable discs is and where to get the "gold?" -- give Mike's column a read. Maybe MAM-A the leading manufacturer of quality recordable CDs and DVD, is maybe the only archival available nowadays?
Note: CD-R is a good way to archive images. you can find more information on following links.
-Andy McFadden's CD-R FAQ
-Media Sciences' CD-R FAQ
-Recording LP records to CD-R FAQ
-Other CD-R FAQ (Goggle search)
-Things you should Know about CDs

---

Can I archive my raw-files?

The public, archival format for digital camera raw data Raw file formats are becoming extremely popular in digital photography workflows because they offer creative professionals greater creative control. However, cameras can use many different raw formats — the specifications for which are not publicly available — which means that not every raw file can be read by a variety of software applications. As a result, the use of these proprietary raw files as a long-term archival solution carries risk, and sharing these files across complex workflows is even more challenging.

The solution to this growing problem? The Digital Negative (DNG), a new, publicly available archival format for the raw files generated by digital cameras. By addressing the lack of an open standard for the raw files created by individual camera models, DNG helps ensure that photographers will be able to access their files in the future

---

File Formats for Digital Masters , a very informative article by The Research Libraries Group, a nonprofit coalition of universities, libraries, and archives, maintains some excellent pages

---

What do they say about archiving photo images in the newsgroups? I have collected some questions and answers hear

---

Archiving Images: Approaches to Storage & Retrieval by Michael H. Reichmann at The Luminous landscape.
Everyone has their own approach to image storage and retrieval. Today's 6MP digital cameras produce 18MB RAW files, and at about 30 Megabytes for each scanned 35mm frame even the largest built-in hard disk fills up quickly. How to store, index and safely archive all of these files? There are a number of conflicting issues, including cost, convenience and speed of retrieval, and finally, security. What to do?
Photographers have several choices, each with benefits as well as downsides. Here is my current approach. It has recently changed — and I'll explain both what I used to do, and what I now do to store, retrieve and archive my work.

---

At last my personal advice.
Save in TIFF format. You'll probably take the pictures in JPEG format, but plan to save them in TIFF format on your hard disk. TIFF is preferable to JPEG because the latter is a "lossy" format. You'll lose image information each time you change and resave a JPEG file. TIFF is a "lossless" format, so image quality will not be degraded. So don't ever save your master file as JPEG if you plan on doing any more editing to the image in the future as each recompression destroys a little more detail, rather like photocopying a copy then photocopying that copy etc .

Tip: Do you find TIFF files too large? If so, check to see whether your image-editing software offers a lossless compression option, such as ZIP. That will make the image files smaller without losing valuable data. I quote from the Photoshop guru Bruce Fraser: " I've been using TIFF with ZIP compression for both layers and background as my standard format for about 4 years now.... TIFF stores everything that PSD can hold, and it always contains a flattened composite so even if the TIFF reader can't understand layers it should be able to read the flattened version. ZIP-compressed TIFFs are typically about the same size as PSD saved without backwards compatibility (no composite), and much smaller than PSD daved with the composite.&

Save the master file. Even a large computer hard disk can quickly fill up with numerous image files. Yet it's important to save the master file: the original, unedited image. Saving the master file without changes lets you return to the original image later to make a third version, with special effects, for example.

(Note: Comparison of typical file sizes produced by the 8-megapixel Nikon Coolpix 8700 with 3,264x2,448-pixel image size.
RAW (12-bit NEF) 11MB
HI (8-bit RGB TIFF) 23MB
FINE (High-quality 8-bit RGB JPEG) 3.6MB
NORMAL (Standard 8-bit RGB JPEG) 1.6MB
BASIC (Highly compressed 8-bit RGB JPEG) 0.9MB)

Burning your images onto a CD-R! When testing several high speed 20x CD burners, I found that the system often produced errors. Not all of the images were recorded, and I had to start over, wasting time and discs. You can minimize such problems with two simple guidelines:

• Use a CD-R 1x to 20x (or 1x to 24x) disc of superior quality, often designated as premium, plus, gold, and so on. (This is usually the most expensive type of disc for any of the major brands such as Fuji, Kodak, Mitsui or TDK- which all use protective coatings.) Sore them in hard plastic cases, too. Be aware, also, that the shelf life of a blank CD-R is only five years, so use your dis ASAP. Use a felt-tip pen to label your disc and avoid adhesive labels.
• For the most reliable recording, select a medium speed setting (8x or 12x, for example), and not 20x.

Note : I have, as per December 2003, an awful lot of CD's (not all filled) and I'm thinking of transferring them to DVD. Just thinking while I wait for that the standards war between the DVD+R/RW and DVD/R/RW camps to be fully resolved. Even if DVD "only" lasts 25 years, I suspect there will be something I'll be transferring them something newer/cheaper faster in half that time.

I've heard figures of 100 years for DVD but who knows. I will not be around to argue.

" Anyway longevity is normally not determined by aging effects, although manufacturers quote studies that predict lifetimes of 20, 50, or even 100 years. Instead, longevity is usually limited by the cumulative effects of small scratches and contaminants that are introduced through normal handling and use. Do not tempt fate by assuming that your discs are bulletproof and can withstand abuse. CD-ROM CD-R longevity can only be achieved by starting with a high quality disc and by exercising care in handling and storage. "

---

Checking your CDs!
One way, of course, would be to copy the contents of the CD back onto your hard disk, which can be a slow process. However, there's a free utility, CD Speed 2000, available from CD Speed. You can i.e. perform a surface test before you start backing up. Or a quality check. (This is part of the same mob that produces Nero, the CD/DVD writing software package; CD Speed comes as part of the Nero Suite).

CD Tips- Things You Should Know About CDs and the proper care and handling of CDs - you won't believe this

---

DVD And CD Longevity Examined! Recently, scientists at the National Institute of Standards and Technology (NIST) looked at CDs and DVDs to see how long digital information recorded on to them would survive. They concluded that most CDs and DVDs will last 30 years or more if handled with care, but many factors can slash their longevity. However ,a big part of the problem is that nobody really knows exactly how they're going to age, and there are many other factors at play. Researchers can simulate what they think will happen in 20 years, but unless you've got a large sample of DVD-Rs that are 20 years old (which you don't) you're not going to know for sure.

---

Checking your CDs! One way, of course, would be to copy the contents of the CD back onto your hard disk, which can be a slow process. However, there's a free utility, CD Speed 2000, available from CD Speed. (This is part of the same mob that produces Nero, the CD/DVD writing software package; CD Speed comes as part of the Nero Suite).

---

From The Nordic Museum in Sweden i quote and translate January 2006:

How to preserve your old images?
CDs are NOT a solution for long term storage of digital images. For occasional and short term storage you can use CDs, provided they are stored correctly ( not exposed to moisture, sunlight and so on.).

For long term storage you need to renew the media, on which the images are stored. Instead buy an external hard disk which today is affordable. For US$ 100 you get one with 200GB to connect to your PC and which you can use for back-up.

Note. This museum has digitized a lot if images with the purpose to preserve them for generations to come

---

IBM expert warns of short life span for burned CDs (Burned CDs have a life span of between two to five years, storage expert says january 2006)