The New World of Metadata
Growing, Shifting, MergingDiane
Hillmann
Metadata Day, Worcester 2
Intro to (New) Metadata
0 What’s the problem, huh?0 Are the old questions the right questions?0 Are our assumptions about metadata still valid?0 What exactly are the new realities we need to
consider?0 What are some survival strategies for the times to
come?
5/9/12
After we empty the card catalog …
We hear that there’s a crisis in libraries, but we still haven’t realized how pervasive it is
Reality: we’ve gotten rid of the cards, now we need to get rid of the catalog.
If we don’t, we may lose our institutional support, our mission, and our way …
5/9/12Metadata Day, Worcester 3
Metadata Day, Worcester 45/9/12
“As librarians, we pride ourselves on operating outside of the commercial marketplace. However, whether we like it or not, we are working in an information environment the dynamics of which are very much like those of a free market, except the the currency spent by our “customers” is not money, but time and attention. … We may believe, for example, that our carefully-crafted catalog records provide excellent value in return for the time and energy required to use them—and we may be right. But if our patrons doubt that the catalog will return good value in exchange for the time and energy required to use it, then whatever value the catalog may actually contain becomes irrelevant.”
Rick Anderson, The Crisis in Research Librarianship, Journal of Academic Librarianship, July 2011
Metadata Day, Worcester 55/9/12
“Wikipedia is founded on the belief (largely correct, as it turns out) that crowds both can and will provide high-quality content and metadata to the world at no charge. For our part, in research libraries we still tend to treat books as if they are primarily tools for linear reading, and metadata records as artisanal products. We still build collections that are fenced off from the larger information world and encourage our patrons, against all reason, to begin their information searches within the confines of our artificially limited collections.”
Rick Anderson, The Crisis in Research Librarianship, Journal of Academic Librarianship, July 2011
Metadata Day, Worcester 65/9/12
“We must look with cold and hard-headed rationality at our current practices and ask ourselves not what value they offer, but rather what value our patrons believe they offer. If what we offer our patrons is not perceived as valuable by them, then we have two choices: change their minds, or redirect our resources. The former is virtually impossible; the latter is enormously painful. But the latter is possible, and if we do not undertake such a redirection ourselves, it will almost certainly be undertaken for us.”
Rick Anderson, The Crisis in Research Librarianship, Journal of Academic Librarianship, July 2011
Metadata Day, Worcester 7
Poking Holes in the Old SilosUnderstanding what we can gain by moving into the linked data world is really critical for ‘future-proofing’ our efforts
We have a great deal to offer to the wider world, but we can’t do it if we cling to the old familiar ways of doing things
Learning about other data available to us (that we don’t have to pay to create) is essential, despite our inclination not to trust any data but our own
5/9/12
Metadata Day, Worcester 85/9/12
“In the big picture, very little will change: libraries will need to be in the data business to help people find things. In the close-up view, everything is changing-- the materials and players are different, the machines are different, and the technologies can do things that were hard to imagine even 20 years ago.”
Eric Hellman http://go-to-hellman.blogspot.com/2011/07/library-data-why-bother.html
Metadata Day, Worcester 9
Questionable Assumptions
0 We’re going to continue to build records for library catalogs
0 We’ve always shared ‘records’ in cataloging, and that’s still the right way to do things
0 The choice of the ‘right metadata format’ (e.g., DC, MODS, RDA, etc.) is critically important
0 The proliferation of metadata formats is a bad thing0 The ‘old’ way of cataloging materials one-at-a-time
always produces better quality data than any other method
5/9/12
Metadata Day, Worcester 10
This Is NOT About Blaming the Messengers
5/9/12
Metadata Day, Worcester 11
“Today, we face another significant time of change that is being prompted by today’s library user. This user no longer visits the physical library as his primary source of information, but seeks and creates information while connected to the global computer network. The change that libraries will need to make in response must include the transformation of the library’s public catalog from a stand-alone database of bibliographic records to a highly hyperlinked data set that can interact with information resources on the World Wide Web. The library data can then be integrated into the virtual working spaces of the users served by the library.”
--Karen Coyle, Understanding the Semantic Web: Bibliographic Data and Metadata, Jan. 2010
5/9/12
Metadata Day, Worcester 125/9/12
If all of this sounds otherworldly and vague, it is because there is no specific vision of where these changes will lead us. The crystal ball is unfortunately shortsighted, in no small part because this is a time of rapid change in many aspects of the information ecology. The few things that are certain, however, point to the Web, and its eventual successors, as the place to be. For libraries, this means yet another evolutionary step in the library of our catalog: from metadata to metaDATA.”
--Karen Coyle, Understanding the Semantic Web: Bibliographic Data and Metadata, Jan. 2010
Metadata Day, Worcester 13
Model of ‘the World’ /XML
0XML assumes a 'closed' world (domain), usually defined by a schema:0"We know all of the data describing this
resource. The single description must be a valid document according to our schema. The data must be valid.”
0XML's document model provides a neat equivalence to a metadata 'record’
5/9/12
Metadata Day, Worcester 14
Model of ‘the World’ /RDF
0 RDF assumes an 'open' world:0 "There's an infinite amount of unknown data
describing this resource yet to be discovered. It will come from an infinite number of providers. There will be an infinite number of descriptions. Those descriptions must be consistent."
0RDF's statement-oriented data model has no notion of 'record’ (rather, statements can be aggregated for a fuller description of a resource)
5/9/12
Metadata Day, Worcester 15
What Are the Components of the Semantic Web?
0 RDF: Resource Description Framework0 Statements about Web resources in the form of subject-predicate-
object expressions, called triples0 E.g. “This presentation” –“has creator” –“Diane Hillmann”
0 RDF Schema0 Vocabulary description language of RDF
0 SKOS: Simple Knowledge Organisation System0 Expresses the basic structure and content of concept schemes such
as thesauri and other types of controlled vocabularies0 An RDF application
0 OWL (Web Ontology Language)0 Explicitly represents the meaning of terms in vocabularies and the
relationships between them
5/9/12
Metadata Day, Worcester 16
Semantic Web Building Blocks0 Each component of an RDF statement (triple) is a
“resource”0 RDF is about making machine-processable
statements, requiring0 A machine-processable language for representing RDF
statements0Extensible Markup Language (XML) can be used
0 A system of machine-processable identifiers for resources (subjects, predicates, objects)0Uniform Resource Identifier (URI) 0For full machine-processing potential, an RDF statement is
a set of three URIs
5/9/12
Metadata Day, Worcester 17
Things Requiring Identification0 Object “This presentation”
0 e.g. its electronic location (URL): http://hdl.handle.net/1813/11524
0 Predicate “has creator”0 e.g. http://purl.org/dc/terms/creator
0 Object “Diane Hillmann”0 e.g. URI of entry in Library of Congress Name Authority File
(real soon now?)0 NAF: nr2001015786
0 Declaring vocabularies/values in SKOS and OWL provides URIs0 Without such identifiers, the Web will never become Semantic
5/9/12
What’s the Point of Interoperability?
0 For users, it’s about resource discovery (user tasks)0 What’s out there?0 Is it what I need for my task?0 Can I use it?0 Reality: They don’t care about what metadata format
you’re using!0 For resource creators, it’s about distribution and marketing
0 How can I increase the number of people who find my resources easily?
0 How can I justify the funding required to make these resources available?
0 How can I maximize my human resources in a world of smaller budgets and larger expectations?
5/9/12 Metadata Day, Worcester 18
Metadata Day, Worcester 19
Questioning Our ModelsToday’s metadata is not about choices of formats, it’s about ensuring interoperability and harmonization for our data in the world
Our old model is based on catalog cards, regardless of the method of delivery through our online catalogs
The new metadata environment provides better ways to express relationships—both content to content and concept to concept
5/9/12
Why Use Models for Metadata?
0To understand what entities you are dealing with
0To understand what metadata are relevant to which entities
0To understand relationships between different entities
0To organize your metadata to make it more predictable (and be able to create and use it with automated tools)
5/9/12 Metadata Day, Worcester 20
Descriptive Metadata Models0 Conceptual models for bibliographic and authority
data0 Functional Requirements for Bibliographic Records
(FRBR)0 Functional Requirement for Authority Data (FRAD)0 Functional Requirements for Subject Authority Data
(FRSAD)
0 Dublin Core Abstract Model (DCAM)0 Some other models:
0 CIDOC Conceptual Reference Model (emerged from museum community)
0 INDECS (for intellectual property rights)
5/9/12 Metadata Day, Worcester 21
Bibliographic relationships (pre-FRBR)
0Tillett’s Taxonomy (1987)0 Equivalence0 Derivative0 Descriptive0 Whole-part0 Accompanying0 Sequential0 Shared-characteristic
5/9/12 Metadata Day, Worcester 22
FRBR Entities
0Bibliographic entities: works, expressions, manifestations, items
0Responsible parties: persons, corporate bodies
0Subject entities: concepts, objects, events, places
5/9/12 Metadata Day, Worcester 23
Group 1 Entities & Their Relationships
5/9/12 Metadata Day, Worcester 24
Work
Expression
Manifestation
Item
A Work “Is realized through”
An Expression
An Expression “Is embodied in”A Manifestation
A Manifestation“Is exemplified by”
An Item
An Expression“realizes”
A Work
A Manifestation“embodies”
An Expression
An Item “exemplifies”
A Manifestation
[Thanks to Sherry Vellucci for this slide.]
DC Abstract Model0Reaffirms the ‘One-to-One Principle’0Defines‘statement’as the atomic level0Distinguishes between “description”
and “description set”:0Description: “One or more statements
about one, and only one, resource.”0Description Set: “A set of one or more
descriptions, each of which describes a single resource.”
0RDA vocabularies developed to reflect the DC Abstract Model
5/9/12 Metadata Day, Worcester 25
Metadata Day, Worcester5/9/12 26
A Dublin Core View of the World
DCMI Abstract Model: http://dublincore.org/documents/abstract-model/
Metadata Day, Worcester5/9/12 27
A Dublin Core View of the World
DCMI Abstract Model: http://dublincore.org/documents/abstract-model/
Metadata Day, Worcester5/9/12 28
Anatomy of a Statement (Attribute/Value)
Place of Production: New York
Property Value
ValueString
Metadata Day, Worcester5/9/12 29
Anatomy of a Statement (Attribute/Value)
Place of Production: http://experimental.worldcat.org/fast/1204333
Property Value
RelatedDescription
Metadata Day, Worcester5/9/12 30
A Related Description
Metadata Day, Worcester5/9/12 31
Description Sets a Key Concept!
Metadata Day, Worcester
Description Set=“A set of one or more descriptions, each of which describes a single resource.”*
5/9/12 32
*DCAM Definition
Metadata Day, Worcester5/9/12 33
A FRBR Description Set “Package”
Work
Manifestation
Expression
representation
representation
statementstatement
descriptiondescription
record
descriptionset
description
statementproperty
value
representation
value string
relateddescription
is groupedinto
is instantiatedas
has oneor more
has one
has one
is representedby one or more
is a
is a
OR
A record consists of descriptions, using properties and values.A value can be a string or a pointerto another description.
5/9/12 Metadata Day, Worcester 34
Basic model: Resource with properties
A Play has the title “Antony and Cleopatra,” was written in 1606 by William Shakespeare, and is about “Roman history”
5/9/12 Metadata Day, Worcester 35
… related to other Resources
5/9/12 Metadata Day, Worcester 36
Metadata Day, Worcester 37
Pulling it together; writing it all downIf you want others to be able to use your data, you’ll need to be clear about your intentions and document them
An Application Profile is one good way to do this so that your data can be properly interpreted, and easily used in a variety of ways
This is an area where DCMI has been working for some time …
5/9/12
What Are Application Profiles?
0APs describe the set of metadata elements, policies, guidelines and vocabularies defined for a particular domain, implementation, or object type0 Declares the metadata terms an organization,
information resource, application, or community uses in its metadata
0 Documents metadata standards used in instance data, including schemas and vocabularies, policies, required elements, etc.
0 Called “application profile” or just “profile”
5/9/12 Metadata Day, Worcester 38
Metadata Day, Worcester 395/9/12
Less Flexibility, More Predictability0Many metadata standards are sufficiently
flexible that they need a mechanism to impose some constraints0 Profiles allow expression of the decisions made for a
project in machine-readable form
0Refining0 Allows a narrower interpretation of a standard to
suit the needs of a domain or project
0Combining0 Enable mixing elements from various different
standards (there are limits to this!)
5/9/12 Metadata Day, Worcester 40
Components of an AP 0Human readable documentation
0 Property descriptions and relationships0 Domain or project specific instruction0 Obligation and constraints
0Machine-readable versions:0 Specific encoding decisions and XML schemas
or RDF 0 Models of data relationships specific to the AP
represented in the schemas0 Functional requirements and use cases
supporting decisions and clarifying intent5/9/12 Metadata Day, Worcester 41
Using Properties from a Variety of Schemas
0 DC APs set stringent requirements for determining reusability of terms:0 Is the term a real “property” and defined as such
within the source schema?0 Is the term declared properly, with a URI and
adequate documentation and support?0 In general, properties whose meaning is partly or
wholly determined by its place in a hierarchy are not appropriate for reuse in DC APs without reference to the hierarchy.
0 Other styles of profiles have different requirements and strategies for developing machine-readability and validation
5/9/12 Metadata Day, Worcester 42
How Does Quality Happen?
0 Lessons from the library community0 Quality is quantifiable and measurable0 To be effective, enforcement of standards of quality must
take place at the community level
0 Looking more broadly:0 Data problems are not unique to particular communities0 General strategies can improve interoperability
0 Quality is not tied to any particular creation strategy0 Human created metadata can be extremely variable0 Machine-created metadata is far more consistent, but that
consistency may not be correct
5/9/12 Metadata Day, Worcester 43
Quality Measurement: Criteria
0Completeness
0Accuracy
0Provenance
0Conformance to expectations
0Logical consistency and coherence
0Timeliness (Currency and Lag)
0Accessibility
5/9/12 Metadata Day, Worcester 44
Metadata Day, Worcester 45
Is Trust Based on Quality?
0 In traditional libraries, trust of other library metadata is nearly automatic0 Based on many years of sharing data, common
agreements, and reality
0 On the Web, data comes from many sources, not all have the immediate trust of libraries0 Evaluation strategies and techniques of determining quality
must be at a collective level0 Standards for provenance being developed by W3C and
DCMI will improve the ability to assess quality and assign trust levels
0 Libraries might want to collaborate on and share evaluations
5/9/12
Metadata Day, Worcester 46
BREAK!!
5/9/12
Metadata Day, Worcester 47
Top Down vs. Bottom Up?
Where standards are developed and who supports them are critical to their success
A case study is the RDA Vocabularies
Is this an anomaly or the precursor of things to come?
5/9/12Flickr photo by freebird4
Metadata Day, Worcester 48
RDA Vocabularies: General Strategy
We used the Semantic Web as our “mental model”Wanted to create a “bridge” between XML and
RDF to support innovation in the library community as a whole, not just those at the cutting edge or the trailing edge
We registered the FRBR entities as classes in a ‘FRBR in RDA’ vocabulary, to enable specific relationships between RDA properties and FRBR
IFLA has followed suit using the Open Metadata Registry to add the ‘official’ FRBR entities, FRAD, FRSAR and ISBDThis provides exciting opportunities to relate all
the vocabularies together
5/9/12
Metadata Day, Worcester 49
Structure: Rationale & Decisions
0 Property and value vocabularies are registered in the Open Metadata Registry (formerly the NSDL Registry): http://metadataregistry.org/rdabrowse.htm
0 Used RDF Schema (RDFS), Simple Knowledge Organisation System (SKOS) and Web Ontology Language (OWL)
0 Decisions oriented to favor approaches that can be generalized to make other vocabulary based standards web-friendly, available for use in applications, and easily updated by communities
0 RDA still a work-in-progress, without agreements on maintenance, boundaries, etc.
5/9/12
Metadata Day, Worcester 50
The Structure, Simplified
0 FRBR in RDA Vocabulary declared as classes0 RDA Properties declared as a ‘generalized’ vocabulary,
with no explicit relationship to FRBR entities0 Subproperties for the generalized elements may be
explicitly related to FRBR entities (using ‘domain’)0 Label/Name includes (Work) or other class to
provide unique name (unless the entity name already appears in the name of the property)
0 Other generalized subproperties usable by others not tied to FRBR
5/9/12
Metadata Day, Worcester 51
Why Generalized Properties?
0 The generalized properties essential for relating RDA to the larger world0 Mapping will be cleaner using the generalized properties (since
most properties mapped to or mapped from will not be based on FRBR)
0 Extensions can be built more usefully from the generalized properties
0 Generalized properties much more acceptable to non-library
implementers (not often using FRBR)0 The ‘bounded’ properties are the first pass at an
Application Profile for RDA as a whole0 Discussions ongoing about completion of the AP, hopefully this year
5/9/12
Metadata Day, Worcester 52
Property (Generalized, no FRBR relationship)
Subproperty (with relationship to one FRBR entity)
FRBR Entity
SemanticWeb
Library ApplicationsThe Simple Case: One Property--
One FRBR Entity
5/9/12
Metadata Day, Worcester 53
Book format
Book format (Manifestation)
Manifestation
SemanticWeb
Library ApplicationsThe Simple Case: One Property--
One FRBR Entity
5/9/12
Metadata Day, Worcester 545/9/12
http://RDVocab.info/Elements/bookFormatManifestation
Metadata Day, Worcester 55
More Complex RelationshipsThere are multiple techniques used in RDA to make the
connection between FRBR entities and RDA propertiesWe tried to reconcile those in the RDA Vocabularies
Some properties related to more than one FRBR entityRelationships in Appendix J actually include the name of the
FRBR entity in the name and have separate definitions (we re-used this strategy for the FRBR-bounded properties)
Other properties and sub-properties appear multiple times in the text and ERDs, with the same definitions and no indication that they might be repeated elsewhere (we consolidated these)
5/9/12
Metadata Day, Worcester 56
Property (Generalized, no FRBR relationship)
Subproperty (with relationship to one FRBR entity)
Subproperty (with relationship to one FRBR entity)
FRBR Entity
FRBR Entity
SemanticWeb
Library Applications The Not-So-Simple Case: One Property—more than
One FRBR Entity5/9/12
Metadata Day, Worcester 57
Extent
Extent (Item)
Extent (Manifestation)
FRBR Item
FRBR Manifestation
SemanticWeb
Library Applications The Not-So-Simple Case: One Property—more than
One FRBR Entity5/9/12
Metadata Day, Worcester 585/9/12
Metadata Day, Worcester 59
Roles: Attributes or Properties?
0 In 2005, the DC Usage Board worked with LC to build a formal representation of the MARC Relators so that these terms could be used with DC
This work provided a template for the registration of the role terms in RDA (in Appendix I) and, by extension, the other RDA relationshipsRole and relationship properties are registered at
the same level as elements, rather than as attributes (as MARC does with relators, and RDA does in its XML schemas)
Applications can choose either approach
5/9/12
Metadata Day, Worcester 60
“Super” Property
Subproperty (with relationship to one FRBR entity)
Subproperty (Generalized, no FRBR relationship)
FRBR Entity
SemanticWeb
Library Applications The Roles Case: Properties, Subproperties
and FRBR Entities
Mapping,Etc.
5/9/12
Metadata Day, Worcester 61
RDA:Creator
RDArole:Composer (Work)
RDArole:Composer
Work
SemanticWeb
Library Applications The Roles Case: Properties, Subproperties
and FRBR Entities
Mapping,Etc.
5/9/12
Metadata Day, Worcester 62
Aggregated Statements
RDA sets up Publication, Distribution, Manufacture and Production statements very much the way they have been done since catalog card days:Assumed aggregation of Place, Name and Date are obvious
leftovers from catalog cards, and are not necessary to enable indexing or display of those elements together if libraries want to do that
We viewed those aggregations as ‘Syntax Encoding Schemes’ (as defined in the DCAM) and built in ways to accommodate them within the bounded propertiesThose using the generalized properties (outside libraries,
usually) need not be constrained by these traditional aggregations of properties
5/9/12
Metadata Day, Worcester 63
Aggregated Statement (no domain or range)
Aggregated Statement Subproperty
Range: RDA Syntax Encoding Scheme (Subclass of RDF Datatype)
Dom
ain: FRB
R Entity
Range: [Specific] Encoding Scheme (Subclass)
General Property (no domain or range)
Subproperty
Pre-coordinated Statements: Structure
Option 15/9/12
Metadata Day, Worcester 64
Publication Statement (no domain or range)
Publication Statement (Manifestation)
Range: RDA Syntax Encoding Scheme (Subclass of RDF Datatype)
Dom
ain: Manifestation
Range: Publication Statement Encoding Scheme (Subclass)
Place of publication (no domain or range)
Place of publication (Manifestation)
Pre-coordinated Statements: Example
Option 15/9/12
Metadata Day, Worcester 65
What Does This Structure Buy Us?
0 Release from the tyranny of records0 Potential for use with a variety of encodings0 Ability to maintain statements at a more granular level
0 Opportunity to re-think how we build, maintain and share data
0 Potential for sharing data beyond the library silo0 Both directions!
0 A challenge to our old notions of what library data can do and should be doing0 As our users migrate to the web and away from library catalogs,
we need to follow them (and lead them appropriately!)
5/9/12
Metadata Day, Worcester 66
Finding Our Way AroundThere’s not just one way to align bibliographic data: we don’t have to agree on one ‘authoritative’ mapping
‘Crosswalking’ strategies, aimed at use by particular applications, see that activity as primarily accomplished by networks
Crosswalks only recognize one relationship: sameAs—this is a very blunt instrument 5/9/12
Metadata Day, Worcester 67
Collapsing and Extending0 We’re starting to envision how mapping and
alignment of elements and concepts can provide a clearer pathway between schemas
0 More specificity for special materials can be accomplished by extending element vocabularies for particular purposes
0 As we build these maps, we can provide different views of data without necessarily having to change the data values themselves
5/9/12
Metadata Day, Worcester 68
Mapping
0 Building relationships between vocabularies is not necessarily a proprietary, high-level activity tied to particular applications or services
0 http://MARC21rdf.info provides a beginning point for mapping library data that is easily shared and modified to meet a variety of needs
0 New approaches go beyond ‘sameAs’ and suggest the potential for more granular, nuanced relationships
5/9/12
Metadata Day, Worcester 69
The sub-property ladder
m21:“Target audience of
books”
m21:“Target audience”
rdfs:subPropertyOf
m21:“Target audience oflanguage material”
m21:“Target audience of
music”
m21:“Target audience of
visual materials”
Slide: Gordon Dunsire5/9/12
Metadata Day, Worcester 70
m21:“Target audience”
frbrer:“has intended
audience”
rda:“Intended audience”
dct:“audience”
isbd:“has note on
use or audience”
unconstrained:“Intended audience”
unconstrained:“has note on
use or audience”
rdfs:subPropertyOf
rdfs:subPropertyOf
“Audience” map
Slide: Gordon Dunsire5/9/12
Metadata Day, Worcester 71
Vocabulary Extension
0 The inclusion of unconstrained properties provides a path for extension of RDA into specialized library communities and non-library communities0 They may have a different notion of how FRBR
‘aggregates’ (For example, a colorized version of a film may be viewed as a separate work)
0 They may not wish to use FRBR at all0 They may have additional properties to use, that
could have a relationship to the RDA properties
5/9/12
Metadata Day, Worcester 72
RDA:adaptedAs
RDA:adaptedAsARadioScript
hasSubproperty
5/9/12
Metadata Day, Worcester 73
RDA:adaptedAs
RDA:adaptedAsARadioScript
KidLit:adaptedAsAPictureBook
hasSubproperty
hasSubproperty
5/9/12
Extension using Generalized Properties
Metadata Day, Worcester 74
RDA:adaptedAs
RDA:adaptedAsARadioScript
KidLit:adaptedAsAPictureBook
hasSubproperty
hasSubpropertyKidLit:adaptedAsAChapterBook
hasSubproperty5/9/12
Extension using Generalized Properties
Metadata Day, Worcester 75
Where Are We Going?
0 Many of the older bibliographic standards are undergoing review in order to be ‘alignment ready’ (ISBD, FR Family)0 RDF versions will be available for all0 Application profiles being built
0 MARC21 now available in its entirety as RDF, with mapping in mind
0 New possibilities for interoperability and harmonization are being circulated widely
5/9/12
Metadata Day, Worcester 76
Links and Contact Info
RDA Vocabularies: http://rdvocab.infoMARC 21 in RDF: http://MARC21rdf.info
Diane: [email protected]
“RDA Vocabularies: Process, Outcome, Use”, DLib Magazine, Jan./Feb. 2010: http://dlib.org/dlib/january10/hillmann/01hillmann.html “A Reconsideration of Mapping in a Semantic World” http://dcpapers.dublincore.org/index.php/pubs/article/view/3622 5/9/12
Top Related