Jump to content
Dallas

How Do You Store Your Digital Archives?

Recommended Posts

As I recently purchased a new Mac and am in the process of de-commissioning my old Mac Mini to my son, I find myself pondering a better archiving solution for my old work and personal photos. I don't keep a lot of photos and my main drive which is a 2TB external is sitting at about 75% capacity. The 2TB Time Machine keeps a full backup of that drive.

 

I have bought another 2TB drive for all new files, this time a portable Seagate, but realise that I won't be able to add this to the Time Machine because there won't be enough space. I'd like to keep all my archives easily available on the current system, so what would the ideal solution for storage and backup be? Should I be looking at a dedicated file server or maybe just get a 4TB external to use as a new Time Machine and then use the old one as a new storage unit for the new files? Repeat when the next drive fills up (which could take a few years if the current one is anything to go by)? 

Share this post


Link to post
Share on other sites

Obviously I don't need the capacity that you do! My current archive is about fifteen GB for which I have a 650 GB USB drive by Samsung. I back up every when I eventually remember to do so. :D

Share this post


Link to post
Share on other sites

External 2TB WD My Passport.  I arrange my photos in folders, by subject, year, and month, and sometimes by day of month for hikes and outings, and archive them at the end of each month.

Share this post


Link to post
Share on other sites

External Thunderbolt hard drive, plus backup.  I keep my iMac's SSD for applications, and all data on a separate hard drive.  Both internal and external drives are backed up to Time Machine, and I have a separate hard drive as backup for the internal SSD, and another as backup for the data drive, managed by Super Duper.

 

Raid is not a backup.

Share this post


Link to post
Share on other sites

I have ~20TB here and externel mirror of the same far away at a friends place.

 

If I knew the ten best shots out of all my shots, the ones that could pay my bills for the rest of my life, I would need less space.

 

I cannot even tell the 10.000 best of my ~1 Million shots since 1983.

 

Still, tasking photos is a challenge and fun, so I continue, see where it will take me.

 

Storage is cheap and learing is my elixier.

Share this post


Link to post
Share on other sites

I have two external RAID drives and also leave one copy at my 3TB fusion drive of the Mac. Plus have three more external drives that I make copies once every two months.

I also have a time capsule, but I DON´T TRUST ON THESE TIME CAPSULES!!!!! This is my second unit and is not working again. I called Apple twice and they don´t even know how the problem is happening. My total HD size is 1,5 TB and the time capsule has 3TB but it keeps warning it does not have enough space. I´ve formatted that time capsule twice and the problem persists. I leave all the external drives disconnected during the back up process of the time capsule and disable the time machine when connect the hard drives.

I won´t work with these time capsules again.

Share this post


Link to post
Share on other sites

As Ann does. A bunch of labelled hard drives, unmounted and stored. Each year gets a new hard drive, plus the previous year's data. Ergo two backups of everything, by year, in the long run.

 

As my files are saved by date and stored by year, it makes a search relatively easy.

  • Like 1

Share this post


Link to post
Share on other sites

I like the approach that Ann and Alan are using. I wonder if creating small previews of the old stuff for use in Lightroom will create too much bloat on the catalog if I wanted to keep a visual record of everything from the archives available to search? 

Share this post


Link to post
Share on other sites

All my data is on an external Thunderbolt Raid drive. In addition to that I have two Thunderbolt backup drives with big HDD's, one of them is ALWAYS offsite (office at work). Each night a backup is made from the Raid storage to the backup drive then present by means of Carbon Copy Cloner, and the two backup drives will be exchanged at least on a weekly basis. But they will be NEVER at the same time at home.

Edited by Noct

Share this post


Link to post
Share on other sites
8 hours ago, Ann said:

As you know, I find Lr much too slow for my needs anyway and adding more items to its catalogue will almost certainly slow it down even further.

 

Having more than a single catalogue in Lr complicates matters even further because you have to load the right Catalogue each time before you can see its contents.

 

A true DAM program (which Lr is NOT!) is probably what you really need but since Cumulus abandoned Single Users and pursued only the mega-corporations instead, I don't know what to recommend.

 

Bridge, combined with my own reasonably efficient mental indexing methods, works very effectively for me.

 

But run a search on "DAM" and also see if there is an answer in this one:

http://www.zdnet.com/article/my-finally-successful-quest-for-a-good-media-asset-management-tool/

 

I'm busy checking this out, thanks for the link, Ann. 

 

I'm not so concerned about catalogs as I am about keeping disks organised and properly backed up, but if a catalog can direct me to where an image is stored and I can easily retrieve it from the HDD in question, then it serves a good purpose. 

Share this post


Link to post
Share on other sites
2 hours ago, DDFZ said:

 

I'm busy checking this out, thanks for the link, Ann. 

 

I'm not so concerned about catalogs as I am about keeping disks organised and properly backed up, but if a catalog can direct me to where an image is stored and I can easily retrieve it from the HDD in question, then it serves a good purpose. 

I have Neofinder.  It integrates moderately well with my workflow, so I use it only to find images, or to provide an analysis of shooting data.  It is very fast, inexpensive, and accurate.    There is a trial version which is perpetual.

Share this post


Link to post
Share on other sites

I installed the trial last night and pointed a catalog to my images drive. It took a while but I do like the results. Does it automatically re-index when you add new images or do you have to manually get it to include new additions to the folders it has included? 

Share this post


Link to post
Share on other sites

Just to clarify things a little better regarding my original question, I'm looking more at the way you set up your hardware than software for your digital library of images. As a Mac user I use external powered drives (3.5") and Time Machine. I don't have any off-site storage, but I think that I should probably use the new 2TB external drive I bought to make a copy of the Time Machine back-up and store that somewhere else. Can one simply copy the TM folder to a new drive? Will it behave the same as the original if the original is lost or would it be better to make native copies of the folders TM has backed up? 

Share this post


Link to post
Share on other sites
51 minutes ago, DDFZ said:

Just to clarify things a little better regarding my original question, I'm looking more at the way you set up your hardware than software for your digital library of images. As a Mac user I use external powered drives (3.5") and Time Machine. I don't have any off-site storage, but I think that I should probably use the new 2TB external drive I bought to make a copy of the Time Machine back-up and store that somewhere else. Can one simply copy the TM folder to a new drive? Will it behave the same as the original if the original is lost or would it be better to make native copies of the folders TM has backed up? 

 

You can make TM backups to more than one drive in parallel; e.g. you can also make a TM backup to your new 2TB drive. TM will then backup to your original backup disk at the odd hour, and to the new 2TB at the even hour. After dismounting the new 2TB TM will default again to a backup every hour to the original backup drive, and remounting the new 2TB will restart the other hour scheme. In case you leave the new 2TB disconnected for a long time, TM will notify you.

  • Like 1

Share this post


Link to post
Share on other sites
4 hours ago, DDFZ said:

I installed the trial last night and pointed a catalog to my images drive. It took a while but I do like the results. Does it automatically re-index when you add new images or do you have to manually get it to include new additions to the folders it has included? 

You have to update manually, but this is much quicker than the original indexing.  Autoupdater allows you to specify an updating schedule, but you need a business licence for this.    Items 3.4 and 3.5 in the User Guide deal with updating.  http://www.cdfinder.de/guide/

  • Like 1

Share this post


Link to post
Share on other sites
4 hours ago, Noct said:

 

You can make TM backups to more than one drive in parallel; e.g. you can also make a TM backup to your new 2TB drive. TM will then backup to your original backup disk at the odd hour, and to the new 2TB at the even hour. After dismounting the new 2TB TM will default again to a backup every hour to the original backup drive, and remounting the new 2TB will restart the other hour scheme. In case you leave the new 2TB disconnected for a long time, TM will notify you.

 

Very handy information, Eddie. Thank you for sharing, I didn't know that TM could do this. 

 

So, I think the solution for me is to convert my old 2TB TM into a new storage space for new work, then purchase a new high capacity drive like a 4TB unit and make that the new TM. My personal work I think I will make a second manual back up onto the portable 2TB drive and store that somewhere else, maybe at my son's place. Ideally I would like to have 2x4TB drives and use the TM as Eddie suggests, but there's an additional cost to that which I am not sure I want to incur right now. 

 

@Anthony thanks for the input on the Neofinder program. I think that it is actually very nice and once I have had a better chance to fiddle some more I will investigate getting the license for it. I see it has put an icon on the task bar which allows me to enter a keyword and quickly locate images without having to open Lightroom. That's probably worth the license cost on its own. 

 

Thanks everyone for your input. :) 

Share this post


Link to post
Share on other sites

I buy the bare drive like Ann does, but put them in these enclosures for getting to them easily.  They are great with fans and all.

 

https://www.amazon.com/Rosewill-RX-358-U3C-BLK-Enclosure/dp/B005KGNXTE/ref=sr_1_1?ie=UTF8&qid=1475510184&sr=8-1&keywords=rosewill+hard+drive+case

Share this post


Link to post
Share on other sites

I keep 3 files of all of my keepers.

 

1. one on my computer's hard drive

2. one in the "Google Drive" cloud storage

3. one in the "Dropbox" cloud storage

 

My consulting computer guru tells me that cloud storage drives are more safe and secure than are external hard drives.

 

Also, I read some where that Ansel Adams' goal was to get 10 or more keeper images in a single calendar year.  My keeper images number in the hundreds.  Those of you who have thousands/millions of keepers are either not deleting enough of your poorer images, or you are a better photographer than Ansel.  :no:

 

 

Share this post


Link to post
Share on other sites
1 hour ago, webco said:

 

Also, I read some where that Ansel Adams' goal was to get 10 or more keeper images in a single calendar year.  My keeper images number in the hundreds.  Those of you who have thousands/millions of keepers are either not deleting enough of your poorer images, or you are a better photographer than Ansel.  :no:

 

 

And then there are those of us who still haven't had to look up what "CH" and "CS" stand for on the selector dial, which is simply left on the "S" setting for the life of the camera. The net side effect of that seems to be having just 7000-8,000 photos to deal with per year in total (never mind just the few dozens of keepers), and not 30-40 similar versions of every shot we ever pressed the shutter release button on. :D

 

Even so, data upload speeds and limits prevalent here in Australia make cloud storage a precarious proposition, particularly given its base in a completely different country, and being reliant on the company offering the service actually remaining solvent.

Share this post


Link to post
Share on other sites

Ann was saying that by Saturday she had already shot over 6,000 frames on our week-long stay in Sabi Sabi. ::) :D

Share this post


Link to post
Share on other sites

Since I stack focus a lot, I have huge storage requirements for the originals, a real PITA. I have no trust in the catalogues that Adobe Lightroom creates, especially the metadata recording. So, here is how I handle this.

 

I ONLY use Lightroom to prepare images to be transformed into .TIFF files, ready for Zerene Stacker, my focus-stacking software of choice. That’s it. I don’t edit in Lightroom, because it is inferior to editing in Photoshop and is way too slow for me. Incidentally, along with every stacked photo’s filename, I add which lens was used. So, a file name might look like this:

 

8AA_7818 Otus55

 

The lens in the above filename would be the Zeiss Otus 55mm APO, and so on. I store files in folders by date, in this format:

 

2016-12-03

 

Like some others here, I use a hard-drive DOCK to handle many external 1TB-4TB drives, and, since I also do video, I have dozens of such drives, each labelled and stored in tight plastic cases. I have many DOCKS, but my main one contains four external drives in one DOCK.

 

Lately, I have begun to use the very small SSD (solid state) drives as they are coming down in price and going up in capacity. They take up less room, are not so exposed, and don’t have moving parts. Obviously, they are the future, at least for my work.

 

As for the editing process, as mentioned, I store my daily shoots in folders by Year-Month-Day and I use Adobe Bridge as a way of looking at and manipulating them. I do all of my editing in Adobe Photoshop CC, and save the results back to the same folder with various acronyms added to the filename so that I can find a group of them quickly.

 

I use a Wacom tablet in the editing process, because a mouse cannot dodge and burn properly. I do this in Photoshop CC. Lightroom’s attempt to dodge and burn is ludicrous. Photoshop and a tablet do it properly.

 

As a database engineer, you might think I would want some exotic file-handling system, but I don’t. My file storage, as mentioned earlier, is all I use, along with Adobe Bridge and its ability to label files. I don’t know why I don’t use something larger, perhaps because I no longer trust ANY program that catalogues my files. That’s why. And I like the sense of the rubber meeting the road that Adobe Bridge offers.

  • Like 1

Share this post


Link to post
Share on other sites
On 10/3/2016 at 14:35, webco said:

I keep 3 files of all of my keepers.

 

1. one on my computer's hard drive

2. one in the "Google Drive" cloud storage

3. one in the "Dropbox" cloud storage

 

My consulting computer guru tells me that cloud storage drives are more safe and secure than are external hard drives.

 

Also, I read some where that Ansel Adams' goal was to get 10 or more keeper images in a single calendar year.  My keeper images number in the hundreds.  Those of you who have thousands/millions of keepers are either not deleting enough of your poorer images, or you are a better photographer than Ansel.  :no:

 

 

My internet connection finally got upgraded to optical fiber, and a free upgrade for my home wifi router, this gave me a 10x increase in speed, so ... I'm going to evaluate doing cloud storage vs buying another external hard disk

  • Like 1

Share this post


Link to post
Share on other sites
11 hours ago, armando_m said:

My internet connection finally got upgraded to optical fiber, and a free upgrade for my home wifi router, this gave me a 10x increase in speed, so ... I'm going to evaluate doing cloud storage vs buying another external hard disk

 

Armando, good luck!  Even with ultra fast upload speeds it will take hours and days to upload your photo archives.   I usually start the upload when going to bed and when I wake up it is done.  Because upload speed varies during the hours of the day due to traffic/bandwidth, your fastest upload speeds are usually in the late evening and early morning hours.

 

I have no doubt that Google, Dropbox, and Amazon cloud servers are much more reliable than the external hard drive that I used to use.  I expect they use redundant backups and have highly qualified computer technicians maintaining their servers.

 

Also, should thieves brake into my home and steal my computer and external hard drive I'd loose all my photos that I stored on them.   Or, what if your home burned down.   I don't know exactly where these cloud drive servers are, but I expect they are thousands of miles from where I live.  It would take an earth ending natural disaster to wipe out both my home computer's hard drive and the cloud drive servers that I use (and if that happened I would not care)  ::).

 

Dave

Edited by webco

Share this post


Link to post
Share on other sites
On 29/11/2016 at 18:47, Ann said:

There was so much happening at Sabi Sabi that the camera could barely keep up with it all!

 

I only machine-gunned when the action got really fast (and sometimes furious) — otherwise it was mainly two or three shots on a single situation but I still arrived home after three weeks in various parts of S.A., and then in the mountain-kingdom of Lesotho, with some 13,000 "keepers".

 

I am still thoroughly enjoying editing and rating them (and re-living those most precious moments!) between doing the other tasks around here which need my daily attention.

 

 

Ann, how many of the 13,000 do you think you will process?

 

I ask because my processing backlog gets larger and larger;  I just do not have the time to deal with them all.  There are many shots from a trip that I want to keep, but are not necessarily worth taking time over.   I am beginning to think that I need to shoot raw + jpeg fine, and only process those that will benefit greatly from personal attention.  

Share this post


Link to post
Share on other sites

×

Important Information

By visiting this website you are agreeing to our Terms of Use, Privacy Policy & Guidelines.