So much of our personal histories are now being recorded digitally, a point I make quite often in this blog.  In particular, smartphone sand social media have made it so that chances are good that the past several years of your life, and the foreseeable future, will be documented in some way by you, in the form of photos, videos, text messages, and even live streaming social media posts.  With that being the case, the ability to document them the right way, and make those memories last, is becoming a necessary life skill.  And so, Personal Digital Archiving is now the emerging buzzphrase among the digital preservation community.

Preservationists like myself are certainly recognizing this need, and we’re starting to lend some of our expertise to the public. As part of this effort, myself and Krista White, Digital Humanities Librarian at Rutgers Libraries, gave a seminar both in-person and on the web that describes some of the terminology and technical issues that people need to be aware of when recording something for posterity. The session, titled “Preserving Your Digital Life” was sponsored by the Association for Library Collections & Technical Services.

The video of the webinar is available on YouTube, and can be seen below.  I hope it helps people who have personal media and need to know what the first steps are to keeping it around for years to come.

A 35mm film projector in operation.

As part of preservation-level digital standards, myself and colleagues have worked since 2004 to develop a best practice specification for digitizing moving images.  Our initial standard document was developed for the NJVid Portal, and was very basic in its specification.

Since then, some minor tweaks have periodically been added to the document.  But recently, some major developments have occurred with our campus infrastructure that have resulted in our need to consider slightly more substantial changes to our spec:

• RUcore is in the process of implementing Wowza as it’s new streaming server platform, and it is already in operation for the libraries reservation streaming media service.  In conversations with the reserves and media teams handling video for those efforts, there has been some real-world testing of video quality improvements for MP4 streaming, and tweaks have been suggested for improving the quality of our streamed videos.
• RUwireless Wifi campus connections have been upgraded to support higher bitrates (up to 3Mbps per connection).  Previously they were capped at 1Mbps.  This means that we can now push video and other content at nearly triple the data rate we were accustomed to over campus Wifi, given proper conditions.

In light of this, coupled with user demand for improvements in video streaming quality, and in preparation for Wowza streaming support on RUcore, we’re proposing changes to the digitization specs for moving images. A draft for comments is available.  Changes are noted in the document red, but to summarize:

1. MPEG-4 streaming bitrates have been increased to a minimum of 860kbps, recommendation of 2.1Mbps for high quality.
2. HD resolution is now supported at a minimum of 720p resolution.

Language has also been added to address digitization of motion picture film, and calls for a minimum of DCI 4K resolution, with support for MXF wrappers and Motion JPEG2000 where appropriate. Motion Picture Film scanning is still a moving target however, and mention is made that film digitization projects should start with a Digital Curation consult.

A Peirce 55-B dictation wire recorder from 1945. Courtesy of Stanford University Libraries. Source: Wikipedia.

A good portion of our nation’s heritage has been immortalized in sound recordings.  From the late 19th century to the present, sound recordings have been used to capture music, speeches and historic events, the oral histories of people who have lived through important events in our nation’s history.

As with many electronic and mechanical recordings, however, this vast heritage is in danger.  In an effort to save what we can of these timeless recordings, the Library of Congress has put together a blueprint in the form of a National Preservation Plan.  This plan is the result of nearly a decade of work that was mandated by Congress as part of the National Recording Preservation Act of 2000.

As the Library of Congress puts in in their press release:

Experts estimate that more than half of the titles recorded on cylinder records—the dominant format used by the U.S. recording industry during its first 23 years—have not survived. The archive of one of radio’s leading networks is lost. A fire at the storage facility of a principal record company ruined an unknown number of master recordings of both owned and leased materials. The whereabouts of a wire recording made by the crew members of the Enola Gay from inside the plane as the atom bomb was dropped on Hiroshima are unknown. Many key recordings made by George Gershwin no longer survive. Recordings by Frank Sinatra, Judy Garland, and other top recording artists have been lost. Personal collections belonging to recording artists were destroyed in Hurricanes Katrina and Sandy.

The National Preservation Plan for sound recordings is available as a PDF file. In it, multiple recommendations are made, including:

• Create a publicly accessible national directory of institutional, corporate and private recorded-sound collections and an authoritative national discography that details the production of recordings and the location of preservation copies in public institutions;
• Develop a coordinated national collections policy for sound recordings, including a strategy to collect, catalog and preserve locally produced recordings, radio broadcast content and neglected and emerging audio formats and genres;
• Establish university-based degree programs in audio archiving and preservation and continuing education programs for practicing audio engineers, archivists, curators and librarians;
• Construct environmentally controlled storage facilities to provide optimal conditions for long-term preservation;
• Establish an Audio-Preservation Resource Directory website to house a basic audio-preservation handbook, collections appraisal guidelines, metadata standards and other resources and best practices;
• Establish best practices for creating and preserving born-digital audio files;
• Apply federal copyright law to sound recordings created before February 15, 1972;
• Develop a basic licensing agreement to enable on-demand secure streaming by libraries and archives of out-of-print recordings;
• Organize an advisory committee of industry executives and heads of archives to address recorded sound preservation and access issues that require public-private cooperation for resolution.

A Con Ed worker makes efforts to restore power in the wake of Hurricane Sandy. The East Coast’s Infrastructure was heavily damaged by this storm, testing not just our survival skills, but how well we preserve our data and memories. Photo by Robert Francis on Flickr.

No greater a test of our resolve and our preparedness exists than a true trial by fire, and the past two weeks have been living proof of this. There isn’t any way to overstate or exaggerate it.  Hurricane Sandy devastated New Jersey, New York and other sections of the East Coast, taking away lives, homes, power, and safety.  It could take years before a sense of normalcy is restored for the lives of those most impacted, and as has been stated by many officials and news outlets: it will never be the same.

A stark reality coming from this is the notion that landmarks and attractions washed away by the storm will now exist only in memories, and in people’s photo and video archives.  But what hasn’t quite been acknowledged fully just yet, is that Sandy has also taken a toll on the archival and digital front as well.

For some, Sandy was just a rather annoying inconvenience.  Power and heat were out for a while.  Cell phones didn’t work as well as they used to. Internet access was scarce, and websites were taken offline for a few days until power was restored. This very blog, for instance, had an emergency plan that kicked in when its usual home base at Rutgers sat safe but idle, without electricity. It was digitally “evacuated” to a backup cloud datacenter in Los Angeles for a few days, until all was clear and the power was back on.

These were the lucky ones. Others fared so much worse.

With homes and businesses being washed away, so too were all of the things inside.  We’re starting to hear about this in the media: trillions of dollars in on-paper riches, potentially wiped out. Computers with important family documents and personal data, gone.  Photos, and keepsakes, destroyed as in this article chronicling the situation in Breezy Point:

[Shamus] Barnes, 43, has spent every summer he can remember here at the sandy tip of the Rockaway Peninsula. Those years were lovingly documented in photos of what his family calls “the pyramid” — the intergenerational group photo op that seemed to grow larger each year.

Those photos were lost when, in the midst of Sandy’s assault, fire destroyed more than 100 houses on Monday, including Barnes’ and his parents’ homes.

“We’ll never be able to replace those things,” he says. He is standing in the mud, holding the lighthouse-shaped sign for No. 16 Fulton Walk, all that was left of his bungalow. “It’s just pictures, but they show the legacy of what’s gone on here. That’s the backbone of everything out here — memories.”

Sadly, the events of these past two weeks have meted out a cruel lesson: use the technology you have to save your important memories, before it’s too late to save them.  Storms can take away our physical possessions, but our photos, videos, recordings and documents can always be saved, if we work to keep them safe. We can’t always get back what is lost, but we can take steps to prepare for what may come.

Always Have a Plan B

How do we do this?  A year ago, I wrote a couple of articles on keeping your stuff safe.  The information in those articles is still relevant today, now more than ever:

1.  Keep a local backup of all your important stuff.
2. Supplement that local backup with a remote, or cloud, backup.  And I’ll add: do some research to make sure that this cloud provider is in a different geographical area from where you normally keep your digital stuff. This way if your local backup is just as waterlogged as your computer, you can still get your stuff back.
3. I haven’t written about this part yet, but you should even back up your mobile devices. It’s easy to do, whether you have an iPhone or an Android device.

“If you cannot access your data from your computer or storage device, no matter what its been through, the data may still be recoverable,” said Chris Bross, Strategic Technical Alliance Engineer at DriveSavers Data Recovery. “We have repeatedly been successful in recovering data from storage devices that have been exposed to sustained water and fire damage, corruption, corrosion and erosion. We have the most advanced technology and methods available to help Hurricane Sandy victims get their data back safely.”

Here’s to hoping that no one will ever have to experience such great losses again. But in case we must, let’s take the steps we need to better prepare for next time.

1/2 inch, consumer, open reel video format used in the 1960s-70s. This format suffers from Sticky Shed Syndrome, making playback difficult. Working machines for this long-obsolete format are scarce. Source: IUB Media Preservation Initiative, used with permission. Note: IUB has asked me to stress that the above photo is not representative of all media collections at the university.

The Media Preservation Site at Indiana University – Bloomington (IUB) makes its message loud and clear the moment you first set your eyes on it: “Our History is At Risk.”

Home to at least 3 million media objects, including sound and moving image recordings, photos, documents, and artifacts, IUB has come to grips with the issue at hand: a great deal of their heritage is locked within obsolete electronic and analog playback formats for sound and moving images.  As an old format becomes obsolete, it gets harder by the day to find working equipment to play back these objects.  And that assumes that the objects can be played back, not having succumbed to age, wear and physical decay. Lacquer from old aluminum audio discs can delaminate, making them unplayable. Video tapes from the 1960s, 70s and 80s can suffer from a condition where the binding agent that holds the recording material to the plastic base sheds, allowing audio and video recordings to literally flake into nothingness. Film, too, has its own serious decay problems.

With its vast collection, faculty and staff at IUB knew the situation could become serious if nothing was done.  Their first step was to take stock of the situation, and consult outside experts (myself included) to get input how how best to address the problem.

Their efforts began nearly 18 months ago when a group of IUB faculty and staff, concerned about the potential fate of important special collections on campus, approached their Office of the Vice Provost for Research about the critical issues of media, and to impress upon them that time was of the essence to address these issues.

“Even though [IU Bloomington’s] needs are now documented, and it is far better equipped than most universities in the country to meet them, there is no guarantee that IU can adequately preserve its collections in the near future.”

— The State of Recorded Sound Preservation in the United States, Council on Library and Information Resources for The Library of Congress, Washington 

The culmination of their efforts to date have been documented on the IUB Media Preservation Website, where they document their comprehensive effort to preserve IUB’s vast audio, video, and film holdings.  Some important documents from their study and efforts including IUB’s Director of Media Preservation Services Mike Casey‘s  Media Preservation Survey (PDF), outlining the collection holders, preservation stakeholders, the risks involved, and potential preservation strategies.  A follow-on public report (12MB PDF) also lays out the situation and what steps are being taken to save their special collections and historic content.  Continued engagement, updates, and discussion on decisions made and procedures undertaken are regularly made available on their Media Preservation Blog.

The IUB Media Preservation Blog

IUB has many years of work ahead of it, not only to transfer older content into more modern digital formats, but also to continue to maintain those archives, preserve new content, and keep pace with new technologies and formats to ensure that their collections are accessible.  It’s encouraging to see them in action, and their efforts stand as a potential framework for other organizations in a similar bind to model their initiatives after.