1/2 inch, consumer, open reel video format used in the 1960s-70s. This format suffers from Sticky Shed Syndrome, making playback difficult. Working machines for this long-obsolete format are scarce. Source: IUB Media Preservation Initiative, used with permission. Note: IUB has asked me to stress that the above photo is not representative of all media collections at the university.

The Media Preservation Site at Indiana University – Bloomington (IUB) makes its message loud and clear the moment you first set your eyes on it: “Our History is At Risk.”

Home to at least 3 million media objects, including sound and moving image recordings, photos, documents, and artifacts, IUB has come to grips with the issue at hand: a great deal of their heritage is locked within obsolete electronic and analog playback formats for sound and moving images.  As an old format becomes obsolete, it gets harder by the day to find working equipment to play back these objects.  And that assumes that the objects can be played back, not having succumbed to age, wear and physical decay. Lacquer from old aluminum audio discs can delaminate, making them unplayable. Video tapes from the 1960s, 70s and 80s can suffer from a condition where the binding agent that holds the recording material to the plastic base sheds, allowing audio and video recordings to literally flake into nothingness. Film, too, has its own serious decay problems.

With its vast collection, faculty and staff at IUB knew the situation could become serious if nothing was done.  Their first step was to take stock of the situation, and consult outside experts (myself included) to get input how how best to address the problem.

Their efforts began nearly 18 months ago when a group of IUB faculty and staff, concerned about the potential fate of important special collections on campus, approached their Office of the Vice Provost for Research about the critical issues of media, and to impress upon them that time was of the essence to address these issues.

“Even though [IU Bloomington's] needs are now documented, and it is far better equipped than most universities in the country to meet them, there is no guarantee that IU can adequately preserve its collections in the near future.”

— The State of Recorded Sound Preservation in the United States, Council on Library and Information Resources for The Library of Congress, Washington 

The culmination of their efforts to date have been documented on the IUB Media Preservation Website, where they document their comprehensive effort to preserve IUB’s vast audio, video, and film holdings.  Some important documents from their study and efforts including IUB’s Director of Media Preservation Services Mike Casey‘s  Media Preservation Survey (PDF), outlining the collection holders, preservation stakeholders, the risks involved, and potential preservation strategies.  A follow-on public report (12MB PDF) also lays out the situation and what steps are being taken to save their special collections and historic content.  Continued engagement, updates, and discussion on decisions made and procedures undertaken are regularly made available on their Media Preservation Blog.

The IUB Media Preservation Blog

IUB has many years of work ahead of it, not only to transfer older content into more modern digital formats, but also to continue to maintain those archives, preserve new content, and keep pace with new technologies and formats to ensure that their collections are accessible.  It’s encouraging to see them in action, and their efforts stand as a potential framework for other organizations in a similar bind to model their initiatives after.

A cloud storage datacenter, housing multiple storage bays. Each red box holds 135 Terabytes of storage. Source: backblaze.com

Previously in this series, I focused on local storage; primarily, hard drives and similar media were discussed.  But a lot has changed since that last article was posted!  In particular, flooding in Thailand threatened to severely restrict the supply of hard drives, and retail prices had doubled and nearly tripled for the most common capacities and models.  Pricing for hard drives has begun to stabilize, but supplies are still constrained, and the situation is not expected to go back to anything resembling normal until March 2012 at the earliest.  For you and I, this means that the price of a hard drive, or even a new computer, might bit higher for the next few months and harder to come by, until the region can recover and production of hard drive components can resume.

And so, it makes sense to look at other solutions for backup strategies, with cloud storage being a lead contender.  Cloud (or online) backup services have become quite popular over the last few years.  They offer an attractive option for keeping your stuff safe: for a fee, you get the ability to send your files to a remote datacenter, where maintaining the storage and hardware required for backing up all of your data becomes the responsibility of the backup service you subscribe to.  They upgrade the hardware when it needs to be upgraded.  They fix and replace hard drives that go bad. The idea is to further simplify the backup process so that even buying hard drives and hooking them up to your computer aren’t part of the equation.

To be sure, cloud computing providers will also be impacted by the hard drive crunch.  After all, cloud computing is nothing more than large scale computing and storage platforms housed in data centers and they, too, must buy hard drives and equipment from many of the same vendors that businesses and consumers would get them from. However, such large scale outfits still have an advantage in that they buy in bulk, and can sometimes ride out price spikes for hardware with the excess capacity they’ve already built up.  So, if there is a price hike in cloud backup services, it might not be as huge as the cost of retail hard drives.

In any case, cloud backup solutions have proliferated quite a bit, and users have many options.  But, how does a cloud backup work?  And what should you look for in choosing a cloud storage vendor?

What the cloud can do for you

The primary promise of cloud backup services is that they seek to offer a hassle-free experience for their users.  There are no hard drives to buy or hook up, and like modern local backup software, the backup operation can be a set-and-forget affair.  What you need is a computer of fairly recent vintage, a broadband internet connection,  and some software that runs on your computer to monitor and schedule the backup processes.

In most cases, you can tell the cloud backup service what files or folders  you want backed up and which you’d prefer they not touch, but by default, most will try without any specific instructions to backup your computer’s entire primary hard drive.  From there, you can expect an initially large backup session to start with, as the software sends a full snapshot of all of your files to the cloud.  Form that point, backups are incremental: only new or changed files are then uploaded to the cloud backup service going forward.  And if it turns out that you need to rely upon that backup later, you can either download the whole archive to rebuild everything, or just select individual files that you might need.

Online backup services generally come with a fixed monthly or annual fee.  Some people might find this more attractive than the upfront cost of buying hard drives for backup purposes, as the cost is typically pretty low.  For this, you might get a set mount of storage that you ca use to back up whatever you like, or you might get an “unlimited” amount of storage offered by certain services.. though they sometimes carry limitations on how much you can upload in a given month.

The Advantages

Set and Forget. For many, the allure provided by online backup services is that it usually provides a truly stress-free option for backing up your data.  As long as you pay into the service, you won’t have to worry about buying and maintaining hard drives. All of that is taken care of by the backup provider, who can enjoy  economies of scale by buying all of their storage and needed hardware in bulk.  As a result, multiple customers sharing the resources of a massive storage system somewhere means that the online backup service can usually buy their hard drives for a lower price than most individual consumers can.  Which brings us to our next advantage…

Price. Most online backup services aren’t expensive.  Some offer the service for as low as $5 a month for a single computer, or $50 for an entire year, regardless of how large that computer’s drive may be.  Such low price points make it easier for most users to want to try such a service.

Off-site. online backup services are, by definition, remote operations. This means that you backup is usually pretty safe, either ion a secure server room, or spread out over multiple locations.  This is ideal in a case where your home or office might suffer the effects of a disaster, and an on-site backup might be destroyed along with the backed-up computers.  With the cloud, your work site might be gone, but your data isn’t.

Expandability.

To allay most users’ fears about privacy, most online backup services offer encryption capabilities, so that only you (or someone with your password) can gain access to what’s in your backup.

The Drawbacks

Although the advantages make cloud backup service compelling, there are some caveats that must be considered as well.

High Bandwidth Usage. Online backup services, by virtue of storing your content in remote locations that are accessible on the internet, will use your internet connection to make those backups happen. It’s important to carefully consider the capabilities and limitations of your internet connection.  High-speed broadband is absolutely required to make cloud services work. Consider that most computer hard drives can store hundreds of megabytes – in some cases even terabytes – of information.  If you choose to backup everything, your very first complete backup could take days to complete, although subsequent backup sessions will only record changes to your files, and should take a much shorter time.

Another important consideration is whether your internet provider enforces a usage cap.  Some ISPs, like Comcast and AT&T in the US, and Rogers in Canada, limit how much data you can transfer in a monthly period, ranging from as little as 50GB to 250GB.  Other ISPs, like Verizon, do not impose such limits.

These problems can be mitigated through some careful timing of your initial backup session.  Your cloud provider might offer a feature that lets you schedule when and how much data gets backed up in a set amount of time, helping you “budget” the usage of any limited broadband services.  Some services even allow you, for a fee, to send in a hard drive with your initial backup image, to save time and bandwidth.

Longevity of the backup service. Another concern is whether the online backup service you’ve chosen has been in business for a one time, as stands of good chance of staying in business.  Startup services like Backify have entered the fray and then abruptly shut down, leaving users who thought their files were being safely backed up suddenly in the lurch. Even big names like Vodafone and HP are not immune from instances where they tried to offer backup services to the masses, but found they couldn’t make a profit and chose to shut down.

You can help mitigate this by having a good relationship with the provider you choose.  See if they have a blog, Twitter feed or Facebook page that’s regularly updated, and keep checking it to ensure they are active and still dedicated to serving their customers’ backup and storage needs.

Some other things to consider

Privacy and encryption.  It’s very likely that you’ll want to backup sensitive information, such as personal documents, financial data, saved passwords, and other information you might not want others to see and have unrestricted access to. This is why, if you’re considering the cloud, that you make sure the cloud backup service you’re considering offers a private key encryption feature.  This will allow you to encrypt your data locally, on your computer, using a password that you set. Effectively, this prevents even the backup service from gaining access to your files unless you provide the password to unlock your backup. And, remember to use a truly strong but easy to remember password to secure your backup.  With a private key system, your data is so secure that if you lose or forget your password, you’re out of luck!

Also be sure to carefully read the backup service’s privacy policy, and make sure you are comfortable with any limitations they spell out.

Getting your files back. A good backup service will not only clearly explain how their service works, but how you’re going to get your files back when you need them. Ideally, you should be able to browse the contents of your backup image online (after you provide you encryption password, of course!), and select either the individual files you need, or pull down your whole backup image to recover from something catastrophic.

The control panel for BlackBlaze's cloud backup service. Here, the status screen tells us the computer is mostly backed up, save for about 75MB in newly created files that will be uploaded at 2:00 am the following morning, presumably when no one is using the internet connection.

Some common cloud backup providers

No one backup service is perfect for everyone.  Each cloud provider offers different features that work best for different users, so it’s a good idea to research a few before making a decision.  Below, I’ve listed a few major backup services for desktops that have been around for a while, have been highly recommended by colleagues, and appear relatively stable:

Carbonite: Probably the most heavily advertised backup service, Carbonite offers “unlimited” backups for as low as $59 per year.  There is a catch though: “unlimited” on the basic plan refers to files in the “User” folders of your desktop, and user generated files.  Software and operating system files aren’t included, unless you play for premium services.  Carbonite will also slow down the speeds of backups that exceed 200GB in a month.

Backblaze: Backblaze provides native Mac and Windows backup support, and offer unlimited backups for $50 per year without any catches. They also allow you to budget your internet usage, allowing backups to occur at night, and allowing you to set limits on how much data is sent up within a set period of time.  Their blog also details the insights of the company, keeping everyone up to date on their efforts, and even describing the custom hardware they use to provide their service.

Incidentally, it is their ease of use, reasonable pricing and consistent openness that makes Backblaze the service that I use for my cloud backup strategy.

Mozy: This service is a bit more up front on their limits, with their home service being offered as either a 50GB or 120GB plan. With that limitation though, comes perks; namely, the ability to access your backed up files on iOS and Android mobile devices.

Crashplan: This service has a bit of a unique crowdsourcing-type model for backups.  In addition to the traditional cloud backup service, Crashplan also allows you to share your backup resources with friends and family, for free. It’s worth having a look if you have a network of friends you can rely on with fast internet connections that are willing to pool storage resources with you.

These and many other services are available for you to try out Cloud backup services.  They definitely have a place in many users’ personal data security plans. I suggest checking some of these services out and see if they fit your needs!

An External Hard Drive: the easiest method for making a quick copy of your important stuff. Photo taken by flickr user Miss Karen

All, right, so you’ve heard it over and over and again, and you know it’s true: you need to make regular backups of your stuff. But how?  What options do typical computer users have?

In the past few years, the options for preservation and backups have expanded quite a bit, giving users an enormous array of solutions to choose from.  Of course, the diverse options can be confusing: what’s the best choice for you?

The backup options out there can be grouped into two major categories: local storage and cloud storage.  Each has their strengths and weaknesses, and will appeal to different users based on where and how they use their computers.  Some of the best and most secure backup strategies make use of both solutions… a backup-of-the-backup, so to speak.  I’ll discuss that further in a later write-up.

In this article, I’ll talk about local storage.  In the next article, we’ll go into cloud-based solutions.

Read the rest of this entry »

A hard disk drive with damaged platters, caused by a head crash. The data on this drive is not recoverable.

Whether we like it or not, those of us who rely on electronics to get our work done are guaranteed one thing: a data loss event.  This means that at least once in our  lifetimes (and sometimes more than that), every one of us who uses a computer, laptop, tablet, smartphone or similar device is going to one day stare at our screens and realize that the piece of information we expected to be there, just isn’t.

It can happen any number of ways.  Sometimes, we users make a mistake and accidentally erase something we shouldn’t have… or someone else might’ve accidentally deleted something of ours that they shouldn’t have. Other times, it’s the computer’s fault: buggy software might’ve claimed to save something but didn’t, or a 10-year-old hard drive finally decided to give up the ghost.  And sometimes, acts of nature (power outage, natural disaster, or other events beyond our control) will intervene and cause vital work to be lost.

Of course, we’ve all heard it time and time again: to protect your documents, photos, drawings, artwork, and other important data, you need to have backups.  Unfortunately, while we all have heard this before and know it to be true, we don’t always follow through.  In the past it’s been tedious to do regular backups; a chore we all dread.  And so, it always falls but he wayside, and often, we get back into a backup regimen only after something bad has happened, and it’s already too late.

But take heart.  A lot has changed recently.  There ARE personal backup solutions out there that are surprisingly easy… and even automatic!  keeping your stuff safe doesn’t have to be a tedious chore anymore… as long as you’re willing to invest a little time and effort at the beginning, and in some cases a small amount of cash on an ongoing basis.

Not only are there different options, but different methods for backing up your online stuff.  There’s hard disks, removable media, cloud storage, duplication… even tape if your needs are hardcore.

Obviously, the uninitiated (and maybe even some veterans using older methods) will have questions.  Like, what IS cloud storage exactly? Is just dragging-and-dropping my files to another hard drive good enough?  Do I really have to buy extra software?

Over the next few days, I’ll be posting a series of articles that lays out all of the options, why they work best for, and describe whether and how they might fit how you work and play online.  Hopefully most of your questions will be answered, or least these articles will help arm you with enough information to get you started.  And with luck, the information provided will help make sure that you don’t have to face an expected data loss disaster without a well-rehearsed Plan B.

Michael Stern Hart and Gregory Newby, founders of Project Gutenberg, an effort to digitize and make available public domain books in electronic forms.

My colleagues and I sadly mourn the passing of a pioneer in digital curation.  Michael Stern Hart, the founder of Project Gutenberg, has passed away at age 64.

While not a household name, Hart has an important place in modern technology, given he is credited as the inventor of the eBook.  His contribution can be felt every time someone reads a book from a Kindle, Nook or iPad.

Some might say that fans of traditional paper books, and those bemoaning the demise of bookstores such as Borders, might take issue with his work.  But everyone should agree that his legacy will be the irreversible revolutionizing of how we read, going forward.

From Project Gutenberg’s obituary for Mr. Hart:

Hart was an ardent technologist and futurist. A lifetime tinkerer, he acquired hands-on expertise with the technologies of the day: radio, hi-fi stereo, video equipment, and of course computers. He constantly looked into the future, to anticipate technological advances. One of his favorite speculations was that someday, everyone would be able to have their own copy of the Project Gutenberg collection or whatever subset desired. This vision came true, thanks to the advent of large inexpensive computer disk drives, and to the ubiquity of portable mobile devices, such as cell phones.

Project Gutenberg is an effort to freely make available a collection of 36,000 out-of-copyright books that were painstakingly proofed and digitized into various open digital formats.