Experiences in Mass Digitization:

Examining OCR Quality at Scale

Paul FogelCalifornia Digital LibraryUniversity of California

HathiTrust

The Partnership

ConsortiaCommittee on Institutional Cooperation Triangle Research Libraries NetworkUniversity of California

Individual InstitutionsArizona State UniversityBaylor UniversityBoston UniversityCalifornia Digital LibraryColumbia UniversityCornell UniversityDartmouth CollegeDuke UniversityEmory UniversityHarvard University LibraryIndiana UniversityJohns Hopkins UniversityLafayette CollegeLibrary of CongressMass. Inst. of TechnologyMichigan State UniversityNew York UniversityNew York Public LibraryNorth Carolina Central UniversityNorth Carolina State University

Northwestern University The Ohio State University The Pennsylvania State University Princeton University Purdue University Stanford University Texas A&M University Universidad Complutense de Madrid University of California Berkeley University of California Davis University of California Irvine University of California Los Angeles University of California Merced University of California Riverside University of California San Diego University of California San Francisco University of California Santa Barbara University of California Santa Cruz The University of Chicago

University of Connecticut University of Florida University of Illinois University of Illinois at Chicago The University of Iowa University of Maryland University of Michigan University of Minnesota University of Nebraska-Lincoln The University of North Carolina University of Notre Dame University of Pennsylvania University of Pittsburgh University of Utah University of Virginia University of Washington University of Wisconsin-Madison Utah State University Yale University Library

The Corpus

Visualizations

Issues of Scale:The Index

Size

Performance

or

Issues of Scale:Languages

Lots of them

A sad example:the page image

the extracted text

Complexity

Issues of Scale:OCR Failures and

Correction

OCR Failures

Corrections

Corrections

Corrections

Thank you.

paul.fogel@ucop.eduhttp://www.cdlib.org/services/collections/massdig/