Salman Rushdie. Source: Wikipedia. Click on image for link to source.

The New York Times today published an article that reflects some of the challenges of preserving born digital content – that is, documents, data and other content that has been created digitally, on a computer or electronic device, and for which there is no physical original (such as on paper).

In particular, they highlight the efforts of Emory University, in preserving Salman Rushdie’s archival materials.

Among the archival material from Salman Rushdie currently on display at Emory University in Atlanta are inked book covers, handwritten journals and four Apple computers (one ruined by a spilled Coke). The 18 gigabytes of data they contain seemed to promise future biographers and literary scholars a digital wonderland: comprehensive, organized and searchable files, quickly accessible with a few clicks.

But like most Rushdian paradises, this digital idyll has its own set of problems. As research libraries and archives are discovering, “born-digital” materials — those initially created in electronic form — are much more complicated and costly to preserve than anticipated.

Electronically produced drafts, correspondence and editorial comments, sweated over by contemporary poets, novelists and nonfiction authors, are ultimately just a series of digits — 0’s and 1’s — written on floppy disks, CDs and hard drives, all of which degrade much faster than old-fashioned acid-free paper. Even if those storage media do survive, the relentless march of technology can mean that the older equipment and software that can make sense of all those 0’s and 1’s simply don’t exist anymore.

Imagine having a record but no record player.

An interesting aspect of this collection and its exhibition is that it emulates the experience Rushdie had in creating the content.  Rather than just viewing the finished documents, you get to see the computer desktop as he saw it, open up the same applications he used, all in the 1980s and 1990s technological contexts… and not using the modern, Web 2.0, Windows 7 or Mac OS X trappings we’re accustomed to in today’s computers.

I think this article is an excellent read, irrespective of what one’s views may be on the subject matter.  Material of all kinds, in increasing amounts, faces the same perils as this collection every day, and archivists everywhere, including this one, wrestle with how best to retain it all.  So far, the only tried and true method for such types of preservation is to obsessively manage and migrate the content, and that requires making tough decisions as to how to proceed, what formats to migrate to, and hoping the decisions made are the right ones to keep the content viable, at least until the next generation of technology requires that the hard decisions be made again.

Of all the work I do, I think dealing with older formats, and just figuring out how they work, is the most interesting aspect.

A few weeks ago, a stack of old open real tapes arrived, along with a similar-vintage tape player.  The recordings were done in the early 1950s, as part of a project to record the oral histories of various labor officials who were active in the early 20th century.  The recordings made it unequivocally clear that the intent was to allow students and researchers from decades into the future to get insight on the history of the labor movement in the state.

Well, for quite a few years, these tapes remained shelved and seldom accessed, until a faculty member from the School of Management and Labor Relations learned of their existence and wanted to use them in his courses.  Owing to the age of the recording format, the scarcity of playback equipment, and the condition of the tapes, there is no way that multiple students would practically access the tapes and have them survive.  But, that doesn’t mean the content should stay inaccessible.

And so, after getting a demonstration from out Special Collections staff on the best way to handle the tapes, and after mustering the courage to risk handling them, the player was hooked up to more modern digital recording equipment, and the digitization had begun:

I’ve always heard people talk about what wonderful sound fidelity the old open reel tape formats had, and they’re right; the sound quality is great, particularly for 55+ year old recordings. The physical condition of the tapes left much to be desired though: one reel had a paper backing, and was extremely fragile. Just playing it back was a white-knuckle experience. It’s a shame too, because one thing you do miss in the migration of old content to digital formats is the experience of handling these old things, and getting them working again. The operation of the tape deck; threading the tape, feeling the very mechanical-ness of the format and how it worked… these are things that modern digital formats have yet been unable to duplicate or preserve.

Additional photos of the setup and the reels themselves appear below the cut.
Read the rest of this entry »

The facility I work in at Rutgers, known as the Scholarly Communication Center (SCC), has a fairly short history in the grand scheme of academia, and yet a fairly long one when it comes to the rapid changes in technology it has seen in its lifetime.  It was originally started in the 1996, and meant to be a location for university students and faculty to access a growing body of the then-nascent collection of digital content.

Back then, the internet still wasn’t very fast and wasn’t nearly as media-rich as it it seems today.  And so, most of the data-heavy reference materials arriving in digital form came to the SCC as CD-ROMs (and later, DVD format).  To accommodate this, the SCC had a lab of ten desktop computers (known as the Datacenter), dedicated solely to accessing this type of material.

But the times changed, and so did the way people accessed digital material.  As the ‘net grew in size and capacity, it no longer made sense to ship reference material on disc, and so the access moved online.  Students migrated from visiting computer labs to bringing their own laptops (and later, netbooks and handheld mobile devices).  Traffic at the datacenter dropped to virtually nothing.  The space had to be re-tooled to continue to be relevant and useful.

And so, with my taking on the newly-minted role of Digital Data Curator, and in collaboration with my colleagues, a new plan for the former datacenter was developed.  Instead of being a place to merely access content, we would be a place to create it.  Analog items that needed to be digitized would be assessed and handled here.  New born-digital content would be edited, packaged, and prepared for permanent digital archiving in our repository.  We would be a laboratory where students getting into the field – and even faculty and staff who have been here a good while – would learn, hands-on, how to triage and care for items of historical significance, both digital and analog, and prepare them for online access.

The concept for a new facility was born.  And we call it the Digital Curation Research Center.

The center is still in “beta,” as we plug along with some internal projects for testing purposes along with a couple of willing test subjects within the university and surrounding community.  This is so we can test out the workflow of the space and make tweaks and optimizations as needed.  Our plan is to officially launch the space in the Spring of 2010, with a series of workshops and how-to sessions for the various things that make digital curation vital (e.g. digital photography, video editing, audio and podcasting, and scanning).

The plan is that this will be a continual, evolving learning experience for all involved.  People who have never really used cameras and recording equipment in a historical context will learn just how increasingly valuable the content they create, and the stories it will tell, can become over time.  And those of us in the DCRC day in and day out will encounter things that we’ve never run into before, and will have to wrap our heads around the issue of preserving it effectively.

Below are related documents that provide additional information about the DCRC.  More information will be coming up as we get closer to the official launch:

• Initial Proposal for converting the Datacenter to the Digital Curation Research Center – Includes Draft Floorplan (PDF File)
• Draft Access Policy (PDF File)
• Poster Session Presentation (PDF File)

As the person responsible for handling the various file formats in RUcore, the digital library repository for Rutgers University Libraries, I’ve been looking with trepidation at the increasing sizes of the digital assets people are starting to create.  In 2004 when the architecture for this was first envisioned, very few digital items grew past the hundred-megabyte point.

How things have changed!  Video and even audio files are routinely pushing into the gigabytes, now that technology has progressed to the point where high-definiteion video and audio can be originated for ubiquitous mobile devices.  And as RUcore and other large repositories seek to preserve this content, we are finding ourselves running into a hurdle we did not anticipate: the ability for our architectures to handle these very large digital files.  In particular, files larger than 2 Gigabytes has posed some exceptions forFEDORA, our infrastructure of choice, and this is a very big deal for video content in particular.  Consider that 2 Gigabytes can comprise less than 5 minutes of HD content, and you can see our dilemna.

Added mechanisms to support these large items has been slow in coming, and have presented some difficulties of their own in implementing.  For this reason, I’ve drafted a document which explains our position on why we need uniform large file support in digital repositories.  Feel free to have a look and provide feedback.

With any luck, developers will heed the call presented here and in other institutions,a nd work to make better support for big files a reality.

• White paper: “The Case for Large File Support in FEDORA/Duraspace Object Management Architectures” (101KB, PDF File)

Even as technology marches on, there is a slowly growing enthusiasm for looking back on the history of computing, and preserving those still-working specimens of obsolete systems and vintage hardware and software that exist. The movement is known as retrocomputing, and although it’s mostly done as a hobby, I would argue that the people who collect and preserve these older specimens are doing a bit of digital curation in their own right.

It’s one thing to read about older computers in the mid to late 20th century that shaped technology as it stands today. But looking up articles on PDP minicomputers, Apple IIs, TRS-80s, Commodore 64s and the like from your modern LCD desktop or iPhone just isn’t the same as actually getting to sit in front of one of these Old Greats and using them. And so, every so often, some of these retrocomputing enthusiasts get together and allow the interested public to do just that.

Recently the East Coast Vintage Computer Festival rolled into Wall Tonwship, NJ, and packed the InfoAge Science Center’s exhibit hall wall to wall with computer systems from decades past… most working well, some not so much. Even some of the much older minicomputers on display worked well for a while, until the systems consumed way too much power and blew the science center’s fuses, rendering them inoperable. But of the many old historic systems that did work, visitors could sit down and give them a spin. It was fascinating to be able make an IMSAI 8080 talk just like it did when fictional teenage hacker David Lightman used it to access military computers in the movie Wargames. Playing old cartridge games on a genuine Atari 800 was pretty neat as well. And although my BASIC programming was a little rusty, the old fascination I felt the first time I wrote a program in second grade came back, as I did it all over again.

Operating old computers can be quite a loud affair, as you’ll see in this video. Here, David Gesswein demonstrates a DEC PDP-8 minicomputer, connected – as a PDP-8 likely would have been in the mid-1960s – to an electromechanical ASR-33 Teletype machine. If you think CRT monitors are old-school, consider that this type of setup had no graphics at all. You had just the equivalent of an electric typewriter, a bank of switches and indicator lights, and some reel-to-reel tape machines to interact with. My, how we’re spoiled today…

Unfortunately, the clatter of Teletype and and the din of many excited geeks talking loudly drowns out the audio, but if you want to know more about the PDP-8, David Gesswein has his own website explaining its intricacies, and even lets you interact with a working PDP-8 online.

A wide assortment of photos taken at the festival appear after the jump link below. Enjoy!

Image Gallery: Vintage Computing Festival East 6.0