Title™: Long Term Data Storage Options.
 

I come before you with an issue I've been thinking about lately.

With a large amount of data, I am begining to run into the issue of storage, thinking long term while maintaining accessability. I have considered Writable CDs and DVDs, but have read articles about the dye degrading over time, and also have concerns about contamination of said medium.

As I near another bout of Hard Disk purchases, I'm starting to wonder if I've missed other possible mediums. Are there any that you may know of that I have missed?

And barring that, should I remain with my Internal Hard Disk Method, I have another question. I have used Maxtor Hard Drives nearly exclusively for a while now. I have yet to have a failure (Fingers Crossed!), with any of my disks, starting from the PC I built in '99, culminating with the first rackmount PC I assembled back in early '05. This all occured in spite of many reviews and first hand experiance from family members and friends.

With creating a Raid array being a given once I do acquire these newest additions, and SATA II also being required, as I want faster transfer rates then my current ATA provides, does anyone have a brand preference, preferabely with first hand experiance? Although I may be a posterboy for Maxtor, I can only assume I am just freakin' lucky as all hell. I am mostly looking for reliability, noise not being a major factor.

All advice is greatly appreciated.

If you want a lot of storage for a reasonable price, you might want to check into the various cards and other removable drives. Or, failing that, you could always, y'know, delete something. I have an 80GB drive which is largely free because music goes on CDs and I only essentially have the stuff on here that I could ever possibly be using in the forseeable future. As in, I see no reason anyone should ever have a 200TB drive FULL OF FREAKING MUSIC AND MOVIES. You're wasting space if you're keeping the stuff you don't want, don't see yourself wanting anytime soon, but are keeping in case you're wrong about yourself.

Just an FYI, Loki is wayyyy into the TB range of anime, etc.

I think that DVD burning is probably the best way to go. Keep it in data format and you can store a bunch of stuff, and it's totally safe

Except from like, fire.

I hate to be the bearer of bad news but there really is no way to future proof your data storage. As new storage mediums evolve old ones become obsolete and its harder and harder to find ways to retrieve the information. So even if your DVD's were rated to faithfully store uncorrupted data for 20,30, or even 50 years there is a good chance they'll be obsolete in about 10-15. At the very least it will become harder to find ways of retrieving your data.

Even the lowly hard disk is starting to be replaced. In the next few decades we will probably see the birth of a solid state three dimensional data storage system. The long and the sort of this is that you're just going to have to continually migrate your data to new mediums every time a new medium becomes the standard. I wouldn't migrate to Blu-Ray or HD-DVD just yet but sooner or latter you probably will have to migrate to one or the other.

I share the same compunctions about optical media as you do, Loki; I'm working on moving all of my stuff onto magnetic media as well.

Personally, I'm thinking about the "Not a computer, not an eHDD" method. Basically I'd build a bare-bones box, preferably with a ton of SATA ports and 3.5" slots, set up a RAID 5 (For the scale you're thinking of, this is the best option in terms of needed redundancy and space), and buy a multitude of the biggest drive your manufacturer makes. Then construct the array and set it as a network drive, and connect the array box to the network. There's probably an enclosure designed to make this easier, but I have not looked, and this would probably be cheaper and allow for you to customize your parts/have a spare computer around. You might want to have a separate disc for the OS, however; a spare 20GB drive would suffice. Takes up a SATA spot, but hey, with 500GB drives, RAID 5 and 6 ports, you'd still end up with ~2TB of storage.

Now, the manufacturers. Personally, I'm extremely fond of Seagate, as they have a 5-year warranty. Normally, that's not worth very much; your data is just as gone. But with the redundancy of RAID 5, you just pop in the new disc and wait for the reconstruction. If you get to a very large number of discs, however, performance falls (I have no idea what this number is, however; you should be safe).

Now, I know you probably know most of this, and in summary I'll say that I second your RAID idea and suggest RAID 5 with Seagate 500GB drives. But this helped me organize my thoughts, and might just inspire others to try it out themselves.

Finally, unless you're looking to back up data for use 10+ years from now, you should have absolutely no problems with interfacing. I mean, pATA has been around for 21 years, and they're still making new motherboards with it. Even with accelerating rates of standards change, you should be good. Good luck, my fellow pack rat!

You know Eltargrim if you are going to soak that much money into a networked storage solution you might as well by a blade server. If I am remembering right each blade can be set up to act in several RAID configurations. Something like this although its a bit pricey. Which would give you 16TB of storage and could be set up in a RAID 10 configuration. If you bought something like this and filled it with the previously mentioned blades you'd get about 256TB of data storage and a power bill through the roof. (Considering the thing would eat about 12.8 KW having 16 800W power supplies. Plus cooling would be an issue.)

Something like this mounted in something like this would probably be a better idea. Not only can you swap out the drives in the blade you can also add more blades later for more storage. In fact you could probably find a similar one without harddrives and just buy 4 1TB drives to go in it. Then later repeat the process to upgrade your storage to a total of 8 blades at 4TB a piece or about 32TB of storage. You really shouldn't have to pay more then say $3000 for a 4TB blade.

The advantage this has over building a barebones box and networking it with your computer is that you don't pay for stuff you don't need. If you build a barebones box its going to need ram, some kind of display adapter so you can set it up, it'll probably have onboard sound, and the CPU is going to be vastly more powerful then it needs to be. After all that it both takes up more room and is less upgradeable then just running a rudimentary blade server.

Of course you could always buy something like this and be done with it.

Edit: Can you tell I'm having a few storage issues of my own?

Oh yes, yes I can.

I suppose it really depends on the scale of your need. 2TB would be plenty for me, and the solution I set up would also give me a spare computer, which is something I'll never say no to. If Loki is already well into the TB range, then it probably wouldn't be enough.

As to the issue of RAID, I prefer arrays with parity rather than mirroring, if only because you lose a smaller percentage of your total space to redundancy. RAID 5 is also easier to expand than RAID 10, which is another stroke in it's favour.

Well any of those system can run in basically any RAID format you want. I prefer blade server solutions myself for a few reasons.

1) Scalability; I can always add more almost without limit. If one blade fills up a buy a new one and slap it under the old one. That is until the rack fills up but then I can just buy a new rack.

2) Modular Elements; Almost like scalability except that it means I can purchase the system piece by piece and still have a functioning system in between. This applies not only to the blades but the Hard Drives in the blades as well.

3) Upgrade-ability; the twin sister of modular elements. I can buy better Hard Disks later and just stick them into the older blades. (To a point.)

4) Footprint; A blade even a stack of blades takes up far less space per TB of data than a whole other computer would. I mean you can get like maybe 6 Hard Disks into a case and that's really pushing it. I can easily buy a blade that holds 4 and stack 3 or 4 of these blades in the exact same area that the barebones box occupies. That's somewhere between 200 and 267% more space per unit of area.

5) Efficiency; you aren't paying for a parts to a whole other computer you may never actually use. Further you aren't paying to continuously run a whole other computer meaning blades generally use less energy per TB of data. (Although some of the bigger blades are mini-computers in themselves)

6) Ease; They're darn near plug in play if you by the fully assembled ones with Hard Drives included. Not to mention the drives can be set up to appear to your computer as if they were actually internal drives. A set up like that is much harder to achieve with two networked computers.

Edit: Actually go with this. Its got basically all the benefits of a blade plus it is fully automated. Not to mention cheap.

Two things:

External Hard Drive.

Fireproof Safe.


I have all of my writing in one, at my mothers house in the back of the closet. I know the fucker still works.