NARAtions » National Archives Digitization Tools Now on GitHub

Just want to alert you to a great blog from the National Archives.  This post in particular discusses some digitisation tool that they are developing and making available. Check it out: NARAtions » National Archives Digitization Tools Now on GitHub

More Managing that Metadata

Many have talked about embedding data in the digital file for some time. TIFF actually stands for Tagged Image File Format. Many pieces of hardware already use that capability to embed data in the file for other hardware to use to interpret the file's visual data. Now people can also do so, with almost as much ease.  Two groups continue to work on this the IPTC (International Press Telecommunications Council) and the VRA (Visual Reource Association).

The VRA's embedded Metadata Working Group has created a VRA metadata panel for Adobe's products, which uses the VRA core metadata structure, a similar though more complex than Dublin Core structure.

The IPTC Photo Metadata Working Group has issued a Manifesto encouraging this effort in May 2011.

Photographers, film makers, videographers, illustrators, publishers, advertisers, designers, art directors, picture editors, librarians and curators all share the same problem: struggling to track rapidly expanding collections of digital media assets such as photos and video/film clips.

With that in mind we propose five guiding principles as our "Embedded Metadata Manifesto":
    1.    Metadata is essential to describe, identify and track digital media and should be applied to all media items which are exchanged as files or by other means such as data streams.
    2.    Media file formats should provide the means to embed metadata in ways that can be read and handled by different software systems.
    3.    Metadata fields, their semantics (including labels on the user interface) and values, should not be changed across metadata formats.
    4.    Copyright management information metadata must never be removed from the files.
    5.    Other metadata should only be removed from files by agreement with their copyright holders.

Check out these two sites to learn more.

New Digital Preservation Tools

A new free and open source package of digital preservation tools is being developed. It is called Archivematica and is a product of many collaborators.  It brings together several digital preservation tools that would normally be used separately such as JHOVE and DROID and puts them into a workflow system.  

Their description of their process is in itself interesting and while it is still in beta form, it is worth the wander to go and peruse what they say and plan for:

http://archivematica.org/wiki/index.php?title=Main_Page

University of Chicago to use Finding Aid for large scale Digitization Initiative

Kathleen E. Arthur, Head of Digitization at the Preservation Department of the University of Chicago Library recently announced their launch of a digitization initiative on the Digipres listserv. What I found particularly interesting is their use of finding aids, but these are not your mother's finding aids. As you can see by reading her post below, their use allows them to catalog at various levels of detail, through the use of EAD tags links placed at various places in the finding aid.

But Kathleen describes the whole project so much better, Read it for yourself.

11/11/10 to Digipres from Katheen Arthur:

The University of Chicago Library’s Special Collections Research Center has a launched an initiative for the digitization of archives and manuscript collections. The digital images are being made available via the online finding aid for each collection. This will recreate for the online user the experience of a researcher encountering the original materials in the SCRC Reading Room, with documents displayed as they are housed in each folder, and with description of the contents in the form of folder headings.

Individual, high-resolution images of each page will be permanently preserved in the Library's digital repository, and can be made available for publication or other research needs. Due to provisions of copyright laws, digitization efforts are currently focused on materials in the public domain, or those for which the University of Chicago holds copyright.

Collections with digitized content now available online include:

• The Ida B. Wells Papers contain diaries, correspondence, manuscripts and photographs documenting the life of the teacher, journalist, and anti-lynching activist.   http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.IBWELLS
• The Dr. Harry Bakwin and Dr. Ruth Morris Bakwin Soviet Posters Collection contains nineteen Soviet political posters produced in the early 1930s, collected by the American physicians Dr. Harry Bakwin and Dr. Ruth Morris Bakwin during two trips to the Soviet Union.  http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.BAKWINPOSTERS
• The Fielding Lewis Papers contain business, personal and legal records documenting life on a plantation on the James River in Virginia, both before and after the Civil War.  http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.LEWISF
• The University of Chicago Laboratory School Work Reports are made up of reports about the Elementary and Secondary division of the Laboratory School, and document classroom activities in the School's first decade. http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.LABSCHOOLREPORT
• The Jefferson Davis Trial Papers document the legal entanglements, ambiguous delays, political floundering, and shifting of responsibilities surrounding Jefferson Davis' first indictment for treason.   http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.MS979JDavis
• The Thomas Winston Papers relate primarily to Winston's activities as a surgeon with Illinois troops during the Civil War.   http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.WINSTONT
• The Middle Eastern Poster Collection produced by government offices and private organizations, primarily in Iran and Afghanistan.  http://pi.lib.uchicago.edu/1001/scrc/ead/ICU.SPCL.MEPOSTERS

For those of you who are interested in the production details, the Large Scale Digitization Initiative is a collaborative effort involving the Special Collections Research, the Preservation Department, and the Digital Library Development Center. The initiative has been guided by definitions of, and requirements for, mass digitization provided by funding agencies such as the National Historical Publications and Records Commission. These guidelines stress expedited scanning workflows, without sacrifice of image quality, and with close attention to preservation concerns, and the use of existing descriptive metadata, such as that provided by a finding aid.

The collections are scanned by Preservation staff. The documents are scanned in color, in the order in which they are filed in each folder, and a .TIFF file is created for each page image. A naming scheme is used for the files which can be extended to other collections scanned as part of the initiative. The TIFF images from each folder in the physical collection are combined into PDFs for delivery. PDF was chosen as a delivery format because of its simplicity, stability and ubiquity. It is expected that the vast majority of users will have PDF viewers on their computers, and will be able to use them to enlarge, decrease, rotate, print, and otherwise easily view the images. Although the images are delivered as pdfs, the tifs of each page will be stored in the digital repository, and will be available if needed for other purposes.

Links to the digital files are added to the online finding aid by SCRC staff. The Encoded Archival Description (EAD) tags chosen allow links to be created at any level of description in the finding aid, from series, to folder, to item, and for multiple links to be attached to a particular description. DLDC staff updated the style sheets to allow display of the links in the finding aids database. DLDC is also hosting the digital files, which will be retained in and delivered from the digital repository.

The procedures developed for large scale digitization of archives and manuscript collections are simple and extensible. Future plans call for the delivery of digital audio files and, eventually, of born-digital content, and for full-text searching of digitized typescript documents, which can be made keyword searchable through optical character recognition (OCR).

Large Scale Digitization Team include: Eileen A. Ielmini, Kathleen Feeney, Daniel Meyer, Kathy Arthur, Karen Dirr, Charles Blair, and the student scanners in Preservation.

CONVERTING TIFF TO PDF

For those looking to adopt the Acrobat file type pdf PDF as their Archival or Display file type Andrew Stawowczyk Long of the National Library of Australia has recently posted the following to Imaglib listserve.

"I wrote a free image to PDF batch converter. It's a simple application but does what's intended. Have a look at http://home.netspeed.com.au

Just a word of caution - I didn't have time to test it extensively but it seems to be working quite well."

Regards
Andrew Stawowczyk Long
Strategist
Digital Preservation Standards
National Library of Australia
anlong@nla.gov.aua

He graciously gave me permission to repost it here as he would welcome feedback, including any further development one might wish for.

The use of PDF as a preservation file type (Archival) is still abit controversial, but many including Archive.org and some of the LOC's efforts are using it. It is great for Text work.

JPEG2000 here to stay?

For all those studying and debating the pros and cons of the file format JPEG2000, here is an announcement of a blog dedicated to just those issues.
"The Wellcome Library has launched a new blog dedicated to JPEG 2000. The blog charts our progress in determining what type of JPEG 2000 we will use, how we use it, and how it impacts on the rest of the Digital Library infrastructure. The blog is also fed to our new Twitter account, Wellcome Digital, where you can also keep abreast of our news and views on UK PubMed Central and the development of our Digital Library and digitisation programme."

http://jpeg2000wellcomelibrary.blogspot.com/


Get informed!

Digital Curation

A new term has been buzzing around library literature for the past few years: digital curation. Traditionally, curation was the act of organizing and maintaining a collection of artworks or artifacts. Digital curation, broadly interpreted, is about maintaining and adding value to a trusted body of digital information for current and future use. It is the active management and appraisal of digital information over its entire life cycle. Digital preservation focuses on the actions taken to ensure the accessibility of digital information across time and new technologies.

Digital curation most often refers to the management of open access publishing, or literature that is digital, online, free of charge, and free of most copyright and licensing restrictions. It can include born-digital research, images and projects produced by scholars, students and institutions. Those of us in the visual resources management field have been curating digitally for the past decade.

Most academic libraries have built collections by doing one of two things:

-They have purchased collections to support their local organizations.

-They have curated special collections of unique or valuable items.

The focus for most libraries or collections is changing to concentrate on the second role. In the future, the bulk of what will be curated will be digital. These two highly complementary concepts should take into account the needs of current and future users.

There are several challenges to in this new approach to image collection building: First, the visual resources collection will need to develop the skills and infrastructures to manage collections of open access content, while developing a solid strategy for long-term preservation of digital information. The visual resources librarian will be required to assist the faculty in the creation, evaluation and collection of this digital content. Second, such a strategy will be difficult or dangerous, because in effect it will require canceling or not purchasing vendor materials, or producing visual resources itself, on the assumption that the content will be available in an open access format. This may or may not be true in the long term. It requires a clear collection development policy and budgeting in our visual resources collections and libraries.

Given a definition that ranges from managing to archiving to preserving data along the data life cycle, there are various points were digital curation services can be pursued by visual resources managers: at a point of visual project initiation (articulating the project and pursuing funding); at a point of recent or ongoing visual production; at a point where a larger community needs to be engaged (broadening access); and at a point where time considerations are important (archiving and preserving in a repository).

It behooves us to keep abreast with resources about digital curation; there are many web sites and blogs, conferences and online classes, papers and panels that can be located.

An article of interest: David W. Lewis, “A Strategy for Academic Libraries in the First Quarter of the 21^st Century,” http://hdl.handle.net/1805/953

“Born-Digital” and Born Again

I read a timely newspaper article on the plane to Atlanta recently to attend the Visual Resources Association annual conference: it mentioned the current exhibition at Emory University (in Atlanta) of an unusual “born-digital” archive of Salman Rushdie. It included four Apple computers (one ruined by a spilled Coke), which housed electronically produced book drafts, correspondence and editorial comments. This archive had presented the university archivists with the choice of simply saving the contents of the digital files, or of also preserving the organization and experience of using these early files. Emory opted for recreating the writing experience of Rushdie that gallery viewers can share in and play with. They can see the progress of written drafts and even make their own editorial comments (which takes a certain amount of chutzpah, considering that it is Rushdie!) “I know of no other place in the world that is providing access through emulation to a born-digital archives,” said Erika Farr, the director of born-digital initiatives at the Robert W. Woodruff Library at Emory.

The unique way the Emory archivists approached the thorny issues of digital preservation nicely illuminates the remarks that Trudy made in her initial blog on “Digital Preservation and Digital Copies.” Trudy pointed out the differences in the value of digital access and digital preservation, and the costs and difficulties that attend to each. Hammering home the significance of digital preservation for me was a session late in the VRA conference titled “Embedded Metadata: share, deliver, preserve.” Digital preservation should not be overlooked in this spectrum when evaluating an archival collection, despite our apparent primary focus on digital access that the visual resources field seems to favor. Context can be as important as content in the visual world, although it is often harder to capture and replicate. The difficulties and expense of archiving generations of equipment, software and the web environment can be overwhelming, but it must be done in order to allow our increasingly rich “born-digital” culture to be captured for future research and enjoyment.

“Then Into the Dumpster They Go…” Really?

This is a good opportunity to build on Leigh's and Barbara's recent posts weaving into this mix the practical advice offered in a recent thread on the Visual Resources Association Listserv.

The initiator of the thread sought collective professional advice on what to do with her academic institution's instructional collection of 160,000 35mm slides. This collection which had supported an academic program and contains some original photography as well as many slides of works that are included in ARTstor and other image databases is no longer being used. The former users and their successors now find their materials online, and the space that this analog collection occupies is needed for other purposes. Our visual resources colleague wanted advice about: 1) the criteria for keeping or discarding a slide, 2) whether or not some slides should be kept on site rather than to send the entire collection into storage, and 4) on the most environmentally friendly way of disposing of discarded slides.

The advice that our colleague received to guide her weeding project was typically pragmatic:

• Consider retaining slides that are camera originals contributed by your faculty and students. They can be scanned and added to your institutional collection.

• Discard items that match images in existing ARTstor and other local and licensed image collections available in your institution. This may require checking to find exact replications of the images you have in 35mm format although exact matches may not be important if the overall digital coverage of an artist or area is sufficient. Some groups of images will require image by image or site by site checking while others can be eliminated more expeditiously.

• Target sections of your slide collection for evaluation that either are or are not well represented in the digital image databases your users have access to. Check item by item—artist by artist—work by work. When you find a digital equivalent of decent quality that is identical or close enough, discard the slide.

• Discard slides that you have already added to your institutional digital image collection.

• Use your local digital asset management database to generate reports of existing digital holdings to identify those artists, sites, or subject areas for which you have many works and for those works for which you have many digital images. Obviously the reverse strategy might also be useful.

• Replace slides purchased from vendors with licensed subscriptions to digital equivalents.

• Identify groups of slides that can be quickly cataloged, scanned, and added to your institutional collection. Develop a prioritized list of works or buildings which should be digitized from slides remembering that it is more efficient to scan an image directly from a source publication than to digitize the same image from a slide.

• Discard slides that were copied from text book sets; discard slides in teaching sets.

• Prioritize the building of your local digital image collection to support current teaching needs.

• Consult with the faculty who teach with material from a specific area before undergoing a mass de-accessioning. Some may want to keep these slides for their own collections; some will have important advice about what is important to save and what isn’t. They all need to understand the process.

• Use student employees—graduate students, SLIS students, and art history students—by assigning them sections of the collection to evaluate. Lay out a group of slides on a light table to facilitate the process of discarding out of focus, pink, moldy, and duplicate slides before beginning the checking process. This process is often iterative.

• It cost less to spend the time checking to determine whether a digital image already exists than to scan, catalog, and upload a new image file.

• Consolidate your slide drawers to compact your remaining holdings; expansion space is no longer needed.

• While slides are not recyclable, they are often useful to faculty and students for art projects. You may be able to give them away to faculty members or other interested individuals. They could be transformed into collectible items such your own version of the Getty’s “Art to Go Bags” (http://www.gettymuseumstore.org/arttogobag.html). You want to avoid the dumpster which leads to a trip to the local landfill if possible.

• Determine whether your weeded collection should be placed in accessible storage so interested patrons of staff will have limited access to it or whether it should go into archival storage.

This brings us to Leigh’s food historian gourmand. The visual fruits of this scholar’s research into all things tasty—his classified collection of 35mm slides as well as the lectures that accompanied these images--is at risk of being lost to scholarship because of a lack of financial and administrative support to transform this corpus into an online resource. Many academic institutions currently face similar situations because those scholars of the visual who are now retiring primarily created and maintained analog slide collections. The digital collections being created by current scholars while equally valuable for preservation present a different set of preservation and descriptive issues.

It is these visual pockets of original research and contribution to visual learning and knowledge that are important to identify, save, and preserve. They play a role in defining the unique at an institution by preserving the essence of the work and personalities of eminent scholars and inspiring teachers over time. These collections contribute to the reputation of an institution by helping to define and high light those intellectual endeavors that distinguish this institution from all others. It is these collections that in digital form enrich learning within the institution and can be shared with others. This is the material that must be rescued from the dumpster to be saved and preserved.

Now comes Barbara’s part—finding the funding to make these projects possible. Her advice is sound and approaches the problem from the bottom starting locally and building upon prior experience. Committed faculty support and endorsement of these projects is a critical part of this process. Yes, funding is often available, but you must persevere to get it.

Bon appetit!

Recently it was reported that one of the most important photographic archives of the twentieth century was being shipped by trailer truck from New York City to Austin, Texas. The entire collection, more than 180,000 images known as press prints, had been amassed by the Magnum photo cooperative, whose members had been among the most important photojournalists of their time. The archive has been sold to the private investment firm for the family of Michael Dell, the Texas computer tycoon. Fortunately, the new owners have reached an agreement with the Harry Ransom Center at the University of Texas at Austin to locate it there, for study and exhibition, for at least the next five years.

Thomas F. Staley, the director of the Ransom Center, said that it planned to scan every image in the collection, of which Magnum itself had scanned fewer than half, in order to order to make them accessible for historical research and exhibitions.

This is just one of several significant archives of vintage visual material, including those of the New York Times and National Geographic Society, which has recently been targeted for digitization. These are invaluable and irreplaceable repositories of our culture and times. This is heartening, especially in light of the expense and labor involved. But it has me worried about the more modest, locally significant collections with which we are all familiar, which are being consigned to the landfills of history, for want of expertise, direction and funding.

For example, an art historian emeritus at the School of the Art Institute had left, as part of his estate, his considerable personal collection of slides, amassed over forty years of teaching. Much of the collection could be found duplicated in any other academic visual resources collection; however, there were pockets of extraordinary images documenting his particular interest in Food, and Food in Art. He was an original foodie, a gourmand, and enthusiastic lover of the sensual rewards of both art and food. He was a popular lecturer locally at culinary institutes, dining establishments and the art school. His slides included photographic documentation of noteworthy meals, the history of food, and the local cuisine found on his many travels, especially in Mexico. There were whole drawers labeled “French Revolution and food” or “Early American feasts” or even “Spam”!

This fascinating collection may never be accessible to the many and diverse audiences that might appreciate it, mostly because of lack of administrative and financial support. Food historians, art historians and other food enthusiasts will perhaps never taste the joys of this particular research and assiduous documentation. Even more frustrating, in a way, is the lack of a way to capture the pedagogy this historian employed, the apposite and humorous observations he made in his lectures. But at least we have the slides as a visual record--for now.

We need to try harder to identify, evaluate and preserve these visual treasure troves close to home. We are in a strong position to be able to advise and direct digital projects to make these images accessible to those who would appreciate them. There is some urgency to this mission, but also the promise of great adventure and astonishment.

Digital Preservation and Digital Copies

From now on, I am making it a rule to refer to digital files made of physical objects as digital copies. This is to differentiate them from those files now referred to as born-digital, where there is no physical original. I not only believe this is more accurate, but also when discussing the preservation of these files, it is important to note if you are preserving a copy or an original.
When you are preserving a copy, you are mostly concerned with protecting your financial investment represented in real costs as well as in effort. In terms of digital copies, there are several ways that your investment is at risk. Besides the normal liabilities of nature and god, there is digital corruption and mechanical failure. There is also the concern that the format or method of storing the file may become obsolete. In addition, there is the real concern that the standards, under which the file was created, may become obsolete or the quality of your files no longer meets the needs of the rapidly advancing technology. In other words, you may need to recreate them for a more sophisticated technology. This last risk is one many early adopters have already experienced. Actually, several visual resource professionals have observed this within their own careers, where photographs were replaced by lanternslide, which were replaced by film slides, which are now being replaced by digital copies.
Given that we do want to protect these copies as long as they are viable, what is the best method of protecting these digital copies? Two things are important, back up and documentation. Fortunately, digital copies can be duplicated without loss of quality. A copy can be stored on several types of media in several locations, hopefully one in a remote location. In fact, it is best to store a copy of your digital collection in another entirely different geographic location. That is where documentation comes in. You need to know not only where the copies are stored, but also the method and the technical metadata documenting the standards under which the files were created.

When most people are discussing preserving a born-digital file, it is in terms of an archive. This is an original artifact, which may someday have inherent value in its own right. Is that not so? As an archival object, it should be stored in a safe and protected environment. In the digital realm, this means inspection, migration and redundancy and is very costly. In addition, if the digital artifact is valued in its own right, such as a Web posting of the 2004 election, then you must also preserve the software and hardware required to read the file. Think of Thomas Edison’s cylinders. If your collection contained some, would you not want to hear what they heard then?

However, the question to me is a born-digital file of value because of it format or its content. Clearly, it would be a shame not to be able to hear an Edison cylinder, but are we poorer for not preserving all of them? If you accept that it is the content not the format or media that is to be preserved, you might even progress to questioning what media is the best way to preserve that content. In other words is digital media the best way to preserve content.
When we first began to make digital copies of collections, it was adamantly stated that it was for the purpose of ACCESS not preservation. If we stick with that argument now that we have digital originals, for other then protection of investment, should we preserve the originals in their digital format? Do we preserve something, just because it exists or find the best a format in which to preserve it? We could of course always keep a digital copy for access.