As the person responsible for handling the various file formats in RUcore, the digital library repository for Rutgers University Libraries, I’ve been looking with trepidation at the increasing sizes of the digital assets people are starting to create. In 2004 when the architecture for this was first envisioned, very few digital items grew past the hundred-megabyte point.
How things have changed! Video and even audio files are routinely pushing into the gigabytes, now that technology has progressed to the point where high-definiteion video and audio can be originated for ubiquitous mobile devices. And as RUcore and other large repositories seek to preserve this content, we are finding ourselves running into a hurdle we did not anticipate: the ability for our architectures to handle these very large digital files. In particular, files larger than 2 Gigabytes has posed some exceptions forFEDORA, our infrastructure of choice, and this is a very big deal for video content in particular. Consider that 2 Gigabytes can comprise less than 5 minutes of HD content, and you can see our dilemna.
Added mechanisms to support these large items has been slow in coming, and have presented some difficulties of their own in implementing. For this reason, I’ve drafted a document which explains our position on why we need uniform large file support in digital repositories. Feel free to have a look and provide feedback.
With any luck, developers will heed the call presented here and in other institutions,a nd work to make better support for big files a reality.
