Digitizing Entomology: The Biodiversity Heritage Library @ the Smithsonian
-
Upload
martin-kalfatovic -
Category
Technology
-
view
2.194 -
download
0
description
Transcript of Digitizing Entomology: The Biodiversity Heritage Library @ the Smithsonian
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Digitizing Entomology
The Biodiversity Heritage Library @ the Smithsonian
Martin R. KalfatovicSmithsonian Institution Libraries26 November 2007
A Presentation for the Department of Entomology, National Museum of Natural History, Smithsonian Institution
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Yet another physical difficulty is the task of assembling the library and indexes which will enable the student to work under proper conditions…. the beginner must now be prepared to spend liberally, or else must establish himself in an institution where a large library exists; if he work by himself with only a few books, he will have to confine himself to a very narrow specialty indeed.
'The Limitations of Taxonomy' by J.M. Aldrich, Science, April 22, 1927, vol. LXV, no. 1686, p.381
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
2003, Telluride. Encyclopedia of Life meeting
February 2005. London. Library and Laboratory: the Marriage of Research, Data and Taxonomic Literature
May 2005. Washington. Ground work for the Biodiversity Heritage Library
June 2006. Washington. Organizational and Technical meeting
August 2006. New York Botanical Garden. BHL Director’s Meeting.
October 2006. St. Louis/San Francisco. Technical meetings
February 2007. Museum of Comparative Zoology. Organizational meeting
May 2007. Encyclopedia of Life and BHL Portal Launch. Washington DC..
BHL Timeline
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
American Museum of Natural History (New York)Field Museum (Chicago)Natural History Museum (London)Smithsonian Institution (Washington) Missouri Botanical Garden (St. Louis)New York Botanical Garden (New York)
BHL Members
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Royal Botanic Garden, KewBotany Libraries, Harvard UniversityErnst Mayr Library of the Museum of Comparative Zoology, Harvard UniversityMarine Biological Laboratory / Woods Hole Oceanographic Institution
BHL Members
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Members
University of Illinois, Urbana-Champaign (contributing member)
Scheme for addition of European and Asian partners under consideration
Additional categories of membership under consideration
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Structure of the Encyclopedia of Life
Serine Molecule
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Serine Molecule
Synthesis CenterField Museum
BiodiversityHeritageLibrary
SecretariatSmithsonian Education &
OutreachSmithsonian/Harvard
InformaticsMarine Biological
Laboratory & MOBOT
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
EOL Species Pages
Built from a variety of new and existing sources
Views available for varying levels of expertise from novice to expert
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
EOL Species Pages
Legacy literature a key component of the EOL species pages
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Focus: Literature
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Focus: Literature
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
• Core literature pre-1923: 400,000 (80 million pages)
• All pre-1923: 600-750,000 (120-150 million pages)
• All literature: 1.4-1.6 million (280-320 million pages)
Mass. Zoological and Botanical SurveyReports on the fishes, reptiles and birds of Massachusetts, 1839
BHL Focus: Literature
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Selection Tools
Combined Serial list for selection of title to scan to avoid duplication of effort
Mongraphic “de-duping” algorithm
OCLC Collection Analysis
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
1.3 million catalogue records 73% are monographs
(remainder are serials at title-level)
63% is English language material
The next most popular language (9%) is German
About 30% of material was published before 1923
BHL Collections
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Selection
Marine Biological Laboratory/WHOI Marine monographs General Science
Museum of Comparative Zoology MCZ publications Herpetology monographs
and serials Ichthyology monographs and
serials
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Selection University of Illinois
Fieldiana Natural history of Illinois
American Museum of Natural History AMNH publications Ornithology
Natural History Museum NHM publications Major natural history general
serials
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Selection Botany Collections
Missouri Botanical Garden, New York Botanical Garden, Harvard Botany Libraries, and Royal Botanic Garden, Kew will cooperatively develop a methodology for botanical publications
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Selection Smithsonian Institution
Libraries Smithsonian publications Marine mammals Fishes Selected special collections
materials Entomology collection
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
The Internet Archive
• 501(c)(3) organization• Dedicated to “Universal Access to
Human Knowledge”• Founder of the Open Content
Alliance• Provides:
– Mass scanning– Archival storage of files– Image processing– Technology development
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Scribe Scanner
• Single Scribe Machine– Custom built by the
Internet Archive– Human operated– 3,500 page per shift per
week
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Scanning Centers
Northeast Regional Scanning Center 10 Scribe machines MBL/WHO (scanning) Harvard
New York Public Library 10 Scribe machines AMNH NYBG
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Scanning Centers
University of Illinois 2 Scribe machines
Natural History Museum, London 1 Scribe machine
Missouri Botanical Garden Non-Scribe operation
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Scanning Centers
Washington, DC 1 Scribe machine at
Smithsonian Libraries Planned 10 Scribe
facility in the Washington area (Spring 2008)
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Scanning Stats
2 million plus total pages scanned
250,000 plus from the Natural History Museum, London
150,000 from the MBL/WHOI library
Fieldiana, 15,000 plus pages
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Scanning Stats
Smithsonian Libraries 250,000 pages (non-Scribe
scanned, 1996-2007) 50,000 Scribe scanned
pages (since August 2007) Other libraries (non-Scribe)
MOBOT: 780,000 AMNH: 150,000
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
But what about ...
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Difficult (impossible?) to re-purpose much of the material
Quality of images often questionable
“Frankenbooks” Sketchy / inaccurate
bibliographic data
But what about
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Persistent Identifiers Stable URL Handle DOI BICI/SICI ISSN ISBN
http://www.biodiversitylibrary.org
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Structural Markup<article> <title>A BRIEF CONSIDERATION OF
CERTAIN POINTS IN THE MORPHOLOGY OFTHE FAMILY CHALCIDID^E.*.</title>
<author>L. O. HOWARD.</author> <volume>1</volume> <issue>2</issue> <start_page>65</start_page> <end_page>86</end_page> <start_count_page>85</start_count_page> <end_count_page>106</end_count_page>
<start_page_image_file>3908800908001101smthrich_0085.djvu</start_page_image_file>
<end_page_image_file>3908800908001101smthrich_0106.djvu</end_page_image_file>
</article>
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Semantic Markup
GoldenGATEThe intention of the GoldenGATE editor is to build a bridge between NLP components and XML markup of natural language text according to arbitrary XML schemas. It allows the deployment of NLP components to marking up the bodies of literature they were designed for. In this way, it enables transforming the texts into XML content according to an XML schema that was designed to gain maximum benefit from the knowledge provided in them.
Integrated Open Taxonomic Access (INOTAXA)
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
10.7 million name strings in NameBank
Uses sophisticated algorithm (TaxonGrab) to locate likely name strings in OCR text
Iterative processing of BHL texts will both increase the number of name strings in NameBank and increase the accuracy of name string recognition
Taxonomic Intelligence
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL & Publishers
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Permissions
• Seek permissions from copyright holders
• Opt in Copyright Model: The BHL will actively work with professional societies and associations to integrate their publications into the BHL in a way that serves the societies’ missions and goals
• BHL will digitize learned society backfiles and mount them through the BHL Portal at no cost.
• Will provide a set of files to the publishers for reuse as they see fit.
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Advantages• Use of the articles will increase
as evidenced by citation upsurge• Long-term management of the
digital assets is provided by the BHL at no cost
• Publishers’ content is embedded in the emerging knowledge ecology that is sweeping biology in this century
• Structural markup of backfiles into conformance with NLM DTD (just starting)
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Successes
• Entomological News• Journal of Hymenoptera
Research
• Herpetological Review
• Publications of the San Diego Natural History Museum
• University of Kansas natural history publications
• And more ...
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
BHL Portal• Library catalog-like interface
to BHL literature• Enhanced structural
analysis to provide volume/issue/article page access to the literature
• Iterative development based on feedback from user community
• Provide access to two key audiences:–Humans–Machines
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Page Delivery
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Taxonomic Intelligence
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Search Browse
Web 2.0 Features
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Discovered Bibliographies
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
• Initial grant from the MacArthur and Sloan Foundations (as part of the Encylopedia of Life grant)
• Additional support from parent institutions
• Additional grants being actively pursued by BHL and individual members
Funding & the Future
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
• Co-evolving bioinformatics resources produce a rich information ecology:– Consortium for the
Barcoding of Life (CBOL) with gene sequences deposited in GenBank.
– GBIF’s Electronic Catalog of Taxonomic Names
– Hebaria and museum specimen databases
Funding & the Future
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Financial Sustainability Strategy
• Quick ramp-up high early costs – development, mass scanning, etc. Drive long-term costs down the asymptote toward zero.
• Derive some long-term costs from the operating budgets of the member institutions. (examples under consideration: acquisitions budget, staff positions, etc.)
• Integrate functions/tasks with wider efforts where appropriate, e.g. mass storage.
• Clear roles for staff who wear multiple hats. Two full-time grant funded positions currently but >15 staff who make substantive contributions.
Funding & the Future
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
• The Long Now Strategy– Institutions that are creating the
BHL exist to persist through time. That’s an important part of their business. Use them!
– The future is uncertain, the technology landscape changes, people pass on. So create consortial structures that are low-overhead, flexible, and can respond quickly
Funding & the Future
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
In any well-appointed Natural History Library there should be found every book and every edition of every book dealing in the remotest way with the subjects concerned. One never knows wherein one edition differs from or supplements the other and unless these are on the same table at the same time it is not possible to collate them properly.
Charles Davies Sherborn, Epilogue to Index Animalium, March 1922
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Thank You
MARTIN R. KALFATOVIC :: SMITHSONIAN LIBRARIES :: DEPT. of ENTOMOLOGY :: WASHINGTON, D.C. :: 26 NOVEMBER 2007
Thanks to: Chris Freeland, Missouri Botanical
Garden Tom Garnett, The Biodiversity Heritage
Library Project Suzanne C. Pilsk, Smithsonian
Institution Libraries The staff at the Internet Archive
Images from The Galaxy of Images, Smithsonian
Libraries (www.sil.si.edu/imagegalaxy) Martin R. Kalfatovic Keri Thompson Bernard Scaife
CREDITS