This past week my local section of the IEEE hosted a talk by Tom Coughlin on consumer data storage. One thing that struck me is the sheer amount of data consumers are storing.
The survey shows storage doubling every two years and starting at a larger amount than I would have guessed.
New technologies for creating content, such as video recording on phones, increase the need for storage. An example of increased storage is the 750GB hard drive made for the iPod. The user of larger hard drives contributes to the demand for more flash because users typically copy a portion of downloaded content to portable Flash-based players. Netbooks, which have hard drives but not built-in optical drives, also drive the demand for SD Flash as a convenient way of transferring data between the netbook and a PC.
One idea proposed to reduce unnecessary redundant storage is a “home virtualization system”. This would a protocol to spread storage intelligently all devices in the home. It would allow redundant copies of data (and keep them synchronized) only on devices that are used away from the home network. “Cloud” storage on the Internet will play a part in this, but it’s unclear how because it requires trusting the remote storage system and because privacy rules on data physically stored outside the home are less strict.
One problem not addressed by modern data storage is longevity of the data. Even the optical media break down after a few decades. The building in Springfield, IL where Abraham Lincoln did his banking, which now houses a branch of a large national bank, has a sample of his account ledger on display. It’s not clear that records from our time will survive 150 years. It’s unfortunate that although we can record data more densely than ever before, its longevity is much worse.