Today, all eyes on the tech world were focused on Apple, who as expected, wowed onlookers with its yearly sneak-peek of what’s to come from Cupertino. And among those announcements came a bombshell for those of us in digital imaging. A major product announcement came in the form of high-resolution, near-print quality image displays that are available beginning right now, on a high-end line of Apple MacBook Pro computers. These Retina displays are boasting 2880 x 1800 pixel densities, at a resolution of 220 pixels per inch, equivalent to some 5.2 megapixels.

By comparison, if you’re reading this on a desktop or laptop, you’re likely seeing it at a resolution of between 72 and 150 pixels per inch, weighing in at a mere 1.2 to 1.8 megapixels.

At the moment, this is only available (at least for desktops and laptops) on a very top level, premium line of systems whose price tags start at $2,200. However, we can expect over the next 12-18 months for these types of displays to become more common on less expensive computers, on all operating systems. And of course, displays of this resolution (and higher) are already available on tablets and mobile devices, such as iPads, iPhones and some high-end Android equivalents.

The immediate impact is that most web-formatted images that are meant to be displayed at the customary 72 ppi, will appear much smaller, and less defined, on these newer screens. This will mean that in order to deliver quality image content to users in the long term, images will have to be larger and more detailed. It’ll be possible for a greater amount of detail from high-resolution digital images to be seen online, instead of having to print these images to get the full effect. Most digital text will be clearer to read. Our computer screens will better approximate print. And hopefully, using a computer to read a document should be a lot easier on users’ eyes.

Fortunately for us at RUcore (and many other digital preservation projects), we’re still well ahead of the curve, with imaging preservation standards that set a baseline of 400 to 600 dpi scanned images. This was originally meant to help us re-create the full print quality of a lot of the documents we scan and preserve, should someone, someday decide to re-print them. But now, it’s clear that computer displays of all types are evolving to catch up with print, and give users an image experience that was never before possible on an electronic screen.

With over 12 billion 140-character messages and growing, Twitter has exploded onto the social networking scene since the first Tweet ever posted roughly fours ago.  Those tiny text-based messages add up: That’s an estimated 1.5 terabaytes of data, and growing!

It looks like the Library of Congress sees the social impact and significance of the medium, and even believes there is a potential academic treasure trove waiting to be unearthed within this mass of single-sentence missives.  And so, the LoC has announced – via Twitter, of course – that it has acquired the entire Twitter archive.

That’s right. Every public tweet, ever, since Twitter’s inception in March 2006, will be archived digitally at the Library of Congress.

…

So if you think the Library of Congress is “just books,” think of this: The Library has been collecting materials from the web since it began harvesting congressional and presidential campaign websites in 2000. Today we hold more than 167 terabytes of web-based information, including legal blogs, websites of candidates for national office, and websites of Members of Congress.

Twitter also made its own announcement via its blog:

It is our pleasure to donate access to the entire archive of public Tweets to the Library of Congress for preservation and research. It’s very exciting that tweets are becoming part of history. It should be noted that there are some specifics regarding this arrangement. Only after a six-month delay can the Tweets will be used for internal library use, for non-commercial research, public display by the library itself, and preservation.

The specific details of the arrangement are still a bit sketchy, and I do have some questions about how this will play out.  For instance, there’s not much direct mention of whether this archive will include the numerous photos and videos that are frequently linked to users’ tweets, but are often hosted via third party add-on sites such as TwitPic and Posterous.  A lot of Twitter users tend to use the platform as a springboard towards linking to websites and other external content, the permancnce of which can be pretty dubious.

This is still a very promising start though, and hopefully the archived twittersphere will in fact prove useful to researchers in the future.

Some may question the importance or singificane of this decision.  But Twitter isn’t just mindless banter. The LoC lists a few socially significant tweets in the archive.  Among them, the first “Victory tweet” by a president-elect.  There’s also quite a bit of historical influence that was set in motion by Twitter: political prisoners in the Middle East have used it to get their message across to followers; sometimes it was the very medium that got them into trouble, and other times it spread the word that helped set them free. Politicians in the West from all ends of the political spectrum have and continue to use Twitter to marshall their troops, as it were.  And the media have documented cases where Twitter became the source of social change in countries ruled with an iron hand, so much so that the potential outage of the service due to maintenance was once considered a serious threat to activism.  There’s PLENTY of social significance there.

The facility I work in at Rutgers, known as the Scholarly Communication Center (SCC), has a fairly short history in the grand scheme of academia, and yet a fairly long one when it comes to the rapid changes in technology it has seen in its lifetime.  It was originally started in the 1996, and meant to be a location for university students and faculty to access a growing body of the then-nascent collection of digital content.

Back then, the internet still wasn’t very fast and wasn’t nearly as media-rich as it it seems today.  And so, most of the data-heavy reference materials arriving in digital form came to the SCC as CD-ROMs (and later, DVD format).  To accommodate this, the SCC had a lab of ten desktop computers (known as the Datacenter), dedicated solely to accessing this type of material.

But the times changed, and so did the way people accessed digital material.  As the ‘net grew in size and capacity, it no longer made sense to ship reference material on disc, and so the access moved online.  Students migrated from visiting computer labs to bringing their own laptops (and later, netbooks and handheld mobile devices).  Traffic at the datacenter dropped to virtually nothing.  The space had to be re-tooled to continue to be relevant and useful.

And so, with my taking on the newly-minted role of Digital Data Curator, and in collaboration with my colleagues, a new plan for the former datacenter was developed.  Instead of being a place to merely access content, we would be a place to create it.  Analog items that needed to be digitized would be assessed and handled here.  New born-digital content would be edited, packaged, and prepared for permanent digital archiving in our repository.  We would be a laboratory where students getting into the field – and even faculty and staff who have been here a good while – would learn, hands-on, how to triage and care for items of historical significance, both digital and analog, and prepare them for online access.

The concept for a new facility was born.  And we call it the Digital Curation Research Center.

The center is still in “beta,” as we plug along with some internal projects for testing purposes along with a couple of willing test subjects within the university and surrounding community.  This is so we can test out the workflow of the space and make tweaks and optimizations as needed.  Our plan is to officially launch the space in the Spring of 2010, with a series of workshops and how-to sessions for the various things that make digital curation vital (e.g. digital photography, video editing, audio and podcasting, and scanning).

The plan is that this will be a continual, evolving learning experience for all involved.  People who have never really used cameras and recording equipment in a historical context will learn just how increasingly valuable the content they create, and the stories it will tell, can become over time.  And those of us in the DCRC day in and day out will encounter things that we’ve never run into before, and will have to wrap our heads around the issue of preserving it effectively.

Below are related documents that provide additional information about the DCRC.  More information will be coming up as we get closer to the official launch:

• Initial Proposal for converting the Datacenter to the Digital Curation Research Center – Includes Draft Floorplan (PDF File)
• Draft Access Policy (PDF File)
• Poster Session Presentation (PDF File)

As I’ve mentioned in some of my previous articles, I’m incredibly skeptical of the long-term success of any digital storage attempt that relies on a single medium as its container. In particular, I find CDs and DVDs to be extremely suspect in terms of their longevity. Those of us who have been around the block with these technologies know that Gold CDs and DVDs were supposed to be the last word on digital preservation, lasting decades if not centuries. But mere years later, we all learned a new term to add to our archival vocabulary: Disc Rot.  It became very clear to all of us that not all discs – not even the gold ones – are made equal, and some even from “trusted” brands can delaminate, fade, have their media layers flake off, or otherwise deteriorate and become unreadable in rather short periods of time… sometimes with painfully devastating results for smaller archives who banked it all on CDs and DVDs.

As a result, a type of storage medium that we were told would last decades has been widely recognized as only being good for about two to five years.

So clearly, it’s not without some deep suspicion and trepidation that I view a recent announcement by an unknown start-up known as Cranberry, in which they claim to solve the Disc Rot problem, and has introduced a type of DVD that they say will last 1,000 years.

A Cranberry DiamonDisc is a DVD made of high tech stone.
Memories carved on a DiamonDisc will last as long as the pyramids. No reflective surface. No ink layer. No fading. Problem solved. The Library of Congress is studying our technology for storage of the national archives. It’s the only solution for permanent, digital storage.

Oh, really?

This is a pretty bold claim from a company that no one has heard from until recently.  But what hard evidence does Cranberry have to – forgive the pun – back it up with?

Unfortunately, you have to dig pretty deeply into their site, to a FAQ section way down at the bottom before you get to a far more realistic statement:

How can you prove that the Cranberry Disc will last for centuries?
No one can prove that anything will last for centuries, but there are international standards for estimating the archival lifetime of optical media. The ECMA‐379 (2nd edition, December 2008) which tests the effects of temperature and relative humidity is widely recognized. Researchers at Millenniata have tested the Cranberry Disc using the ECMA‐379 temperature and humidity (85°C / 85% RH) testing as a standard to develop the most rigorous testing possible. They have combined temperature and humidity (85°C / 85% RH) tests with exposure to the full‐spectrum of natural light. The Cranberry Disc is the only survivor after this rigorous testing. Considering the combination of the Cranberry Disc’s test results and its rock‐like data layer, it is reasonable to conclude that the Cranberry Disc has a greater longevity and durability than competitors who claim a 300‐year shelf life.

Here’s what worries me about the above statement: Millenniata does not appear to be a dedicated research firm.  In fact, if you go to their website, they seem to be marketing their own archival-quality disc storage under the brand name of M*ARC.  Per Millenniata:

Millenniata is the sole provider of a permanent, backwards-compatible archiving solution for the digital age. Located in Springville, Utah, Millenniata is poised to become the world’s leader in digital data preservation. Millenniata is the result of pioneering inventions from Brigham Young University.

So… what’s going on here exactly?  It’s difficult for me to accept that Cranberry claims to have cornered the market in creating DVDs that last a millenium, only to point to research done by a company that claims it holds the sole solution to the disc rot problem.

Both companies claim that their respective formats are not something that the mundane DVD burners on standard computer equipment can burn.  Cranberry claims the needed equipment to make a DiamonDisc is “out of the reach of most consumers,” and it wants you to send your data to them so that they can “etch” a DVD for you, at a cost between $29.99 to $34.95.  Millenniata, however, will gladly sell you the special burner required to author these discs, which according to some news sources will cost up to $5,000 for the drive and a pack of 10 discs.  Once created though, both products will purportedly read just fine on any DVD drive. The capacity of these discs are 4.7GB, the same as a single layer DVD-R.

My as-yet-unconfirmed speculation is that, based on the descriptions both entities provide for their products, the storage products both companies offer might actually be one and the same.  If that’s true, then I’d have to conclude that the research done on these discs’ longevity is hardly objective and unbiased.

So, should archivists invest money in these discs?

My philosophy has always been that you should never blindly trust a vendor’s claim about how long their storage media should last, nor should you trust a single storage solution for archiving your digital content.  If a curator wants to spend multiple thousands of dollars for an M*ARC drive, or spend $30-$35 for Cranberry to author each disc, that’s their decision, but it should be supplemented by a secondary solution, be that hard drives, tape, solid-state media, or some other well-known container format.   And that data needs to be checked periodically to verify its integrity.

Cranberry claims that the Library of Congress is studying the technology for its own archival use. Personally speaking, I would like to see the definitive results of such a study, before I would feel comfortable making a decision on whether M*ARC or DiamondDiscs are worth the premium.  And even then, I still would not back away from my two-format philosophy to long-term digital storage.

It’s ironic that Cranberry is hyping up NARA and LoC studies, citing them as reasons why you should be very,very afraid about storing your precious data on standard writeable disc media.  Let’s hope for their sake that the LoC verifies their claims are true, or else their marketing literature would prove pretty embarrassing.

For two years, owners of Apple’s iPhone have complained bitterly about the lack of video recording capability, something that has become more and more common on mobile devices, if not yet matured.  Three weeks ago, they finally got their wish, and the results have been quite dramatic.

The weekend after Apple’s latest reveal – the iPhone 3GS – was released to the public, YouTube reported a massive 400% surge in the number of mobile video uploads, attesting to the mass appeal of Apple’s product and opening the floodgates for new video content.

Admittedly, seeing the sudden unleashing of newly-minted cellphone videographers made me cringe a bit, at first.  For a while now, I’ve been kind of annoyed with the popularity of cell phone video.  Let’s face it, although lots of people want something small and compact to make quick videos with, the image and sound quality coming from these devices has been utterly deplorable.  Blocky video, muddy sound… in general, not something you really want to store cherished memories with.  But what really stuck in my craw was when news sources started relying on cellphone videography as poor substitutes for actual newsgathering.  I began to ask myself: When did yanking video off a YouTube account start to pass as acceptable broadcast-quality content?

Within the past couple of months, however, things have changed quite a bit.  It actually started when camera makers like Canon began including video capabilities into their Digital SLR cameras, such as the Canon T1i.  Although it looks nothing like a camcorder, this model actually records some incredibly good HD-quality video, and can store it on a postage-stamp sized SD card.  It’s still rather bulky and expensive though; not the kind of thing an average person on the street would carry with them at all times.

Apple, however, changed that.  They finally decided to include video capabilities on their latest device.  And, they did the job so well that some aspiring independent film makers are now shooting films solely with the iPhone.  While the critical acclaim of the subject matter is best left open to judgement, one can’t deny the video quality is astoundingly good, compared to what everyone is used to from a cellphone.

Additionally, a professionally produced music video has been recorded and edited using the same gadget. But you wouldn’t know this came from an iPhone unless someone told you in advance:

Of course, the raw video underwent a lot of post-processing before yielding the finished product we see above.  But the unprocessed, full-color raw footage direct from the iPhone camera shows that it’s certainly no slouch on its own:

The ramifications are clearly huge.  There’s been lots of talk about how accessible good video recording technology has become, but now the technology to have a high quality video recorder literally in your pocket is available to the masses.

The technical specs of the videos created aren’t lightweight by any means, either.  Some users have reported that an hour of video from an iPhone 3GS can take up as much as 8GB of storage.  Still not on par with the roughly 20GB per hour that broadcast-level DV video consumes, but not anything to sneeze at, either.  The wide availability of this level of video production is going to require lots of supporting memory and storage to back it up.  And the metadata and details surrounding each and every video produced could be enough to turn the average cell phone user into amateur video catalogers, as well.

For professional curators, this could be both good and bad news.  From my perspective, it’s good that there’s a better chance history-making content will actually be recorded on a camera worthy of capturing it, as opposed to poor specimens of history like this. On the other hand, this means that good quality video will be coming from multitudes of sources, more than we’ve ever been accustomed to, as more and more individuals have the technology within their grasp.