Improving the Linking WorldBy Enhancing

Knowledge Bases-- A Joint NISO/UKSG Project

Todd CarpenterManaging Director, NISO

November 13, 2007 SSP TechBlitz

Where are we headed this morning?

• What it is a Knowledge Base?• Some background about the NISO/UKSG

Knowledge Base project• What role they play in information

distribution• What problems exist in the supply chain• Some recommendations and next steps• Questions

Thanks to UKSG and James Culling, SIS Lead author for much of this content

November 13, 2007 SSP TechBlitz

What is NISO?

• NISO - National Information Standards Organization• NISO is the only ANSI-accredited organization

tasked with the development of standards in the field of Information and Documentation

• Work with publishers, libraries, agents and other systems vendors to develop community consensus

• Develop wide range of standards– Paper permanence and steal shelving– Accessibility issues– Bibliographic formats and exchange– OpenURL, Metasearch, SUSHI

November 13, 2007 SSP TechBlitz

Knowledge Base Background

• OpenURL system developed in 1999– Developed by Herbert Van de Sompel, Patrick

Hochstenbach, and Oren Beit-Arie – First Commercialized by ExLibris SFX in 2001

• Standardized by NISO as Z39.88 in 2004• Providers structure for selecting “appropriate

copy” of among digital content options• Allows links between resources to be library-

defined• Does not depend solely on the linking services

defined by the information providers or on a specific set of identifiers and/or communications protocols

November 13, 2007 SSP TechBlitz

Framework for Knowledge Base Exchange

November 13, 2007 SSP TechBlitz

UKSG Survey

United Kingdom Serials Group (UKSG) commissioned report in 2006

• Explore data flow of metadata into knowledge bases that underlie resolver systems

• Conducted by Scholarly Information Strategies • Published in May 2007GOALS

– Understand the supply chain of information in knowledge bases– Define what is not working well– Consider methods for alleviating problems identified– Consider alternative approaches– Identify areas for further research

Interviewed 30 suppliers, librarians, agents and publishersOnline questionnaire responses from 118 librarians

November 13, 2007 SSP TechBlitz

Populating the Knowledge Base

• The exchange of information that serves as the keystone of the entire linking process is not well understood

• Certainly not standardized

• How does information get from there to here?

November 13, 2007 SSP TechBlitz

•Metadata about titles, content packages and linking systems descriptions are made available to link resolver systems.

•Typically includes titles, coverage dates, etc, generally in delimited text format.

•Some data is pushed, some pulled

Link Resolver Suppliers Gather Data

November 13, 2007 SSP TechBlitz

•Link resolver suppliers normalize data and compile it in proprietary database.

•Quality assurance layer

•Linking syntax programmed into software

•Additional value-add information added to knowledge base

Suppliers Create Knowledge Base

November 13, 2007 SSP TechBlitz

Suppliers Create Library Specific KB

•Link resolver suppliers distribute master knowledge base to library specific implementations.

•Frequency of updates to implementations vary

•Possible delays in library-hosted resolvers due to library IT staff

November 13, 2007 SSP TechBlitz

Subscription Holding Data Gathered

•Maintaining the resolver, librarians collect holding data from internal systems•(ILS, catalogues, A-Z lists, ERMS)

•Request downloads holding details from agents and publishers

•Might not be required where package details are common among all subscribers

November 13, 2007 SSP TechBlitz

Library Adds Holdings Data to KB

•“Standard” packages are activated based on “trusted data” from resolver suppliers

•Compiled holdings data from other packages or resources is processed and normalized

•Holdings data is uploaded into localized knowledge base to form customized functioning KB

November 13, 2007 SSP TechBlitz

Process of Exchanging Link Data

November 13, 2007 SSP TechBlitz

Problems in the Process

• …“just because full-text finding tool vendors update their products regularly does not mean that the lists are actually up-to-date, because full-text finding tool vendors get updates from content providers who have various updating schedules and practices, and thus are of varying quality”

Chen, X. “Assessment of Full-Text Sources Used by Serials Management Systems, OpenURL Link Resolvers, and Imported E-Journal MARC Records.” Online Information Review 28.6 (2004): 428-434. <http://dx.doi.org/10.1108/14684520410570553>

November 13, 2007 SSP TechBlitz

Issues and Barriers

Lack of awareness Lack of cooperationInaccurate & incomplete data Content package issuesTitle changes and transfers Responsibility for data

qualityLack of data standards Timing issuesOpenURL issues and compliance Inbound linking problems

Role of subscription agents in the processBroadening scope of the use of Knowledge Bases

OPENURL COMPLIANCE ISN’T KNOWLEDGE BASE COMPLIANCE

November 13, 2007 SSP TechBlitz

Recommendations

Create an organization to foster development of code of practice definitions

Similar in structure to Project COUNTERWould manage best practices and

issue compliance measures

November 13, 2007 SSP TechBlitz

Code of Practice: Compliance

• File format structure• Mandatory fields and sequencing• Content formats• Frequency• Collections mechanisms• Package information hierarchy• Documentation• Syntax adherence

November 13, 2007 SSP TechBlitz

Other Potential Solutions

• Web services - similar to ONIX SOH or SUSHI equivalent– Exchange is only one problem

• Centralized Model– Considered as a central repository for release

and availability information– Knowledge bases – Suppliers are differentiated by the quality of

their KB.

November 13, 2007 SSP TechBlitz

Where does it stand?

• UKSG accepted the final report at their May board meeting

• Decided to pursue partnership with NISO in September

• Discovery to Delivery Topic Committee reviewing proposal

• Expect to charge Working Group• Recruiting participants• 18-24 month development cycle

November 13, 2007 SSP TechBlitz

Working Group Charge

• Develop and publish guidelines for best practice to effect smoother interaction between members of the knowledge base supply chain.

• Provide educational events to address the roles of each party in the supply chain

• Create a centralized information portal

– Providers and their customers will benefit from higher quality data

– Publishers will benefit from accurate linking to their content and subsequently increased usage

November 13, 2007 SSP TechBlitz

More Information

• UKSG Reporthttp://www.uksg.org/projects/linkfinal

• Summary of UKSG presentationhttp://liveserials.blogspot.com/2007/04/framework-for-improving-link-resolver.html

• NISO - OpenURLhttp://www.niso.org/standards/resources/Z39_88_2004.pdfhttp://www.niso.org/standards/resources/OpenURL_FAQ.html

OCLC - OpenURL Maintenance Agencyhttp://alcme.oclc.org/openurl/servlet/OAIHandler?verb=ListSets

November 13, 2007 SSP TechBlitz

Thank you!

Todd Carpenter, Managing Directortcarpenter@niso.org

One North Charles StreetSuite 1905Baltimore, MD 21201 USA(301) 654-2512(410) 685-5278www.niso.org