Digital preservation metadata Why is it needed and what ......Angela Dappert -Digital Preservation...

Digital preservation metadata

Why is it needed and what does

it look like?

Angela Dappert

The British Library

Angela Dappert -Digital Preservation Metadata and Improvements to PREMIS in Version 3.0

What is digital preservation metadata?

� Digital preservation metadata = Metadata to ensure long-term accessibility of digital resources

� Digital objects must be self-descriptive

� Must be able to describe, manage and discover independently from the systems that were used to create them

XML (machine and human readable)

17 October 20162

DP metadata supports preservation goals

Preservation Pyramid (from Priscilla Caplan)

17 October 20163

Domain

17 October 20164

Born digital

Digitized

Technology dependence

No direct access

• Not self-descriptive

• Complex formats

Complex environments

digital

17 October 20165

Technology dependence

Metadata:• Format information• Rendering information

• Software• Hardware• Other dependencies:

schemas, style sheets, encodings, etc.

17 October 20166

Complex structures

Metadata• Physical structural relationships

• Embedded files• File sequence

• Logical structural relationships

Abstract.html

Abstract.gif

Thumb.jpg

17 October 20167

Complex structures

Content file 1

Content file 2

Content file 3

17 October 20168

Content file 136

Content file 137

Content file 138

Complex structures

Title Page

TableOfcontents

17 October 20169

Supporting features

17 October 201610

Metadata:Semantic information for the designated community

Supporting features

17 October 201611

Metadata:Semantic information for the designated community

Example uses and queries

Context descriptions

Metadata: Context descriptions

• Original source

• Related items (e.g. migration source)

17 October 201612

Obsolescence

-> object transformations

� Pre-emptive preservation actions

� Bit migration

� Content migration

� Replacing part of the rendering stack

� Forensic transformation actions

17 October 201613

Obsolescence / object transformations

Goals Metadata

� Avoid rights violations

� Prove authenticity

� Rights information for preservation actions during copyright / license period

� Provenance metadata:

� History of all actions performed on the resource

� History of custodianship

17 October 201614

�Events�Dates�Changes and decisions�Agents (decision maker + tools used)

Obsolescence / object transformations

Goals Metadata

� Manage potential loss of object characteristics

� Demonstrate degree of authenticity

� Explain decisions

� Significant characteristics

� Lost characteristics

� Business rules (policy, strategy) guiding preservation actions

17 October 201615

� Documentation

Mutability

Goals Metadata

� Viability: the object is readable

� Fixity: the object is unchanged

� Data carrier metadata

� Type of medium

� Its preservation characteristics

� Age of medium

� Date of recording

� Usage patterns

� Checksums, message digests, hash function

� Event creating them� Algorithms creating them

� Date/time

� Originator

17 October 201616

� Intentional or accidental change

� Decay: rapid and potentially complete

Mutability

Goals Metadata

� Integrity: the object is whole and unimpaired

� Authenticity: the object is what it purports to be

� Event information for format identification and validation events (= provenance)

� Structural metadata

� Digital signatures

� Access rights

17 October 201617

� Intentional or accidental change

� Decay: rapid and potentially complete

What is PREMIS?

Angela Dappert

The British Library

The PREMIS standard

� International de-facto standard for metadata to support the preservation of digital objects and ensure their long-term usability.

� Information you need to know for preserving digital objectsPreservation Metadata: Implementation Strategies

� Developed by an international team of experts.

� Implemented in digital preservation projects around the world.

� Incorporated into commercial and open-source digital preservation tools and systems.

17 October 201619

The PREMIS standard

� Data Dictionary (PREMIS 3.0)� http://www.loc.gov/standards/premis/v3/premis-3-0-final.pdf

� Version 3 – major release

� XML schema v3.0� http://www.loc.gov/standards/premis/premis.xsd

� OWL ontology

� Supporting documentation

17 October 201620

Activities

� The PREMIS Editorial Committee

� Coordinates revisions and implementation of the standard

� PREMIS Implementors' Group forum (pig@loc.gov)

� Email message to listserv@loc.gov: Text: subscribe pig <your name>

� PREMIS Implementation Fair (PIF)

� User group meetings (@iPres)

17 October 201621

� What PREMIS DD is:

� Common data model for organizing/thinking about preservation metadata

� Standard for exchanging information packages between repositories

� Implementable

� Technically neutral

� Core metadata

17 October 201622

� What PREMIS DD is not:

� Out-of-the-box solution

� All needed metadata

� Lifecycle management of objects outside repository- increasing support for integration with outside

� Rights management standard- strong support for rights statements

17 October 201623

� What PREMIS DD is not:

� It is not limited to or customized for archives and libraries.

� It does not dictate that you need to use every feature.

� But you should examine for yourself which features you can knowingly ignore.

� It is not only useful if you implement metadata. You can use it to assess the metadata quality of systems you use.

� Everyone modeling the digital landscape can and should use the high-level modeling feature.

17 October 201624

Example: Object Entity semantic units

� 1.1 object Identifier

� 1.2 object Category

� 1.3 preservation Level

� 1.4 significant Properties

� 1.5 object Characteristics

� 1.6 original Name

� 1.7 storage

� 1.8 environment

� 1.9 signature Information

17 October 201625

Object

1.10 relationship

1.11 linkingEventIdentifier

1.13 linkingRightsStatementIdentifier

1.5 objectCharacteristics

1.5.1 compositionLevel

1.5.2 fixity

1.5.3 size

1.5.4 format

1.5.5 creatingApplication

1.5.6 inhibitors

1.5 objectCharacteristics

1.5.1 compositionLevel

1.5.2 fixity

1.5.3 size

1.5.4 format

1.5.5 creatingApplication

1.5.6 inhibitors

Sample Data Dictionary Entry

17 October 201626

17 October 2016

Tayloring PREMIS to needs

� Evolving metadata � Increasing experience ensuring the longevity of digital objects

� Changing future technical possibilities

� Changing future legal framework

� Tayloring solutions� Varying needs

� Content-types

� Institutional policies

� Intended use

� Off-the-shelf (OS / commercial )or custom-built

• Predefined metadata profiles• Out-of-the-box tools

Off-the-shelf systems

• Metadata profiles and tools

Configured, extended, adapted

• Metadata profiles and tools

Custom-built systems

From Version 2.0

to Version 3.0

Angela Dappert

The British Library

PREMIS: From V2 to V3

� Improving PREMIS based on user needs

� Add preservationLevelType semantic unit

� Add agentVersion semantic unit

� Add “unknown” values

� Add eventDetailInformation semantic unit

� Add authority for controlled vocabulary

� Make Intellectual Entity an Object category

� Make Environments independent Objects

� Add physical Objects

� Update conformance statement

17 October 201629

Add preservationLevelType semantic unit

� 1.3 preservationLevel

� 1.3.1 preservationLevelValue

� 1.3.2 preservationLevelRole

� 1.3.3 preservationLevelRationale

� 1.3.4 preservationLevelDateAssigned

17 October 201630

Add preservationLevelType semantic unit

� 1.3 preservationLevel

� 1.3.1 preservationLevelType

� 1.3.2 preservationLevelValue

� 1.3.3 preservationLevelRole

� 1.3.4 preservationLevelRationale

� 1.3.5 preservationLevelDateAssigned

� Associate type of preservation function with preservation level.

17 October 201631

� objectIdentifier

� objectIdentifierType: ARK

� objectIdentifierValue: ark:/9999/c1

� objectCategory: file

� preservationLevel

� preservationLevelType: Bit preservation

� preservationLevelValue: medium

� preservationLevel

� preservationLevelType: Functional preservation

� preservationLevelValue: migration

� objectCharacteristics

� compositionLevel: 0

� size: 726970368

� format

� formatDesignation

� format name: application/vnd.ms-excel17 October 201632

Add agentVersion semantic unit

� If agentType is software, � agentVersion can be used to refine agentName.

� 3.1 agentIdentifier � 3.2 agentName � 3.3 agentType

� 3.4 agentNote � 3.5 agentExtension � 3.6 linkingEventIdentifier � 3.7 linkingRightsStatementIdentifier� .

17 October 201633

� 3.1 agentIdentifier� 3.2 agentName� 3.3 agentType� 3.4 agentVersion� 3.5 agentNote� 3.6 agentExtension� 3.7 linkingEventIdentifier� 3.8 linkingRightsStatementIdentifier� 3.9

17 October 201634

� 3.1 agentIdentifier � 3.2 agentName � 3.3 agentType � 3.4 agentVersion � 3.5 agentNote � 3.6 agentExtension � 3.7 linkingEventIdentifier � 3.8 linkingRightsStatementIdentifier � 3.9 linkingEnvironmentIdentifier

17 October 201635

Unknown compositionLevel and format

compositionLevel and format:

� A value of unknown addedif the information is not available.

17 October 201636

Add eventDetailInformation semantic unit .

� 2.1 eventIdentifier

� 2.2 eventType

� 2.3 eventDateTime

� 2.4 eventDetail

� 2.5 eventOutcomeInformation

� 2.6 linkingAgentIdentifier

� 2.7 linkingObjectIdentifier

17 October 201637

Add eventDetailInformation semantic unit .

� 2.1 eventIdentifier

� 2.2 eventType

� 2.3 eventDateTime

� 2.4 eventDetailInformation

� 2.4.1 eventDetail

� 2.4.2 eventDetailExtension

� 2.5 eventOutcomeInformation

� 2.6 linkingAgentIdentifier

� 2.7 linkingObjectIdentifier

17 October 201638

17 October 201639

� eventIdentifier: eventIdentifierType: UUIDeventIdentifierValue: 908985d3-9600-4da4-a7e7-c6e9508bf24c

eventType: validation

� eventDateTime: 2014-07-03T23:18:19eventDetailInformation:

eventDetail: program="Jhove"; version="1.5"eventOutcomeInformation:

eventOutcome: faileventOutcomeDetail:

eventOutcomeDetailNote: format="JPEG"; version="1.02"; result="Not well-formed“

capturecompressioncreationdeaccessiondecompressiondecryptiondeletiondigital signature validationfixity checkingestionmessage digest calculationmigrationnormalizationreplicationvalidationvirus check

authority="premisEventType" authorityURI= "http://id.loc.gov/vocabulary/preservation/eventType.html" valueURI= "http://id.loc.gov/vocabulary/preservation/eventType/val.html

Implementation specific change:

Add authority for controlled vocabulary

17 October 201641

Make Intellectual Entity an Object category

Rights

Intellectual

Entity

V2:• Assumed to be

held in a container metadata schema

• No Intellectual Entity semantic units

• Exception: identifier to enable linking to a description

• PREMIS Objects link to it.

• A set of content that is considered a single intellectual unit for purposes of management and description

• For example, a particular book, map, photograph, or database.

ObjectRepresentation

Bitstream

17 October 201642

Rights

V3:• Possibility to

describe preservation aspects of intellectual entities

• Same semantic units as Representations

ObjectRepresentation

Bitstream

Intellectual Entity

Bitstream

Intellectual Entity

Representation

Bitstream

17 October 201643

� Relate to PREMIS Events and RightsStatements.

� Support structural and derivative relationships with Objects. E.g.record logical containment between an article and an issue

� Represent an aggregate, such as a collection, FRBR work, FRBR expression, fonds or series.

� Capture versioning information and metadata update events at the Intellectual Entity level

� Associate business requirements with them.

� Significant characteristics, risk definitions, guidelines for preservation actions, etc..

17 October 201644

Make Environments independent Objects

� What is needed to render or use an object

� Operating system

� Application software

� Hardware

� Computing resources

� A high-level data model

� No detailed characteristics specific to an environment type

17 October 201645

File File 2relationshipType: dependencyrelationshipSubType: requires

Software application

Operating system

Hardware architecture

Hardware peripheral

Software driver

Software library

Example: Environment stack and

dependency relationships

• Modularised environment aggregates as a network

• Re-usable and distributed environment descriptions

• across different Objects• across repositories and

registries

File 1

RepositoryRepository

Registry 1Registry 1

Registry 2Registry 2

17 October 201646

Data Model in PREMIS V2

17 October 201647

Object (including

Environment semantic

unit container)

Rights

Intellectual

Entity

Environment

Data Model in PREMIS V3

17 October 201648

Object Environment

Rights

identifiers

relationshipType: structuralrelationshipSubType: represents

Intellectual Entity

Representation

Bitstream

Intellectual Entity

Representation File

represents =relationshipType: structuralrelationshipSubType: represents

FileContent Object

Intellectual Entityhardware

Intellectual Entityoperating system

Intellectual Entitysoftware application

File ObjectISO image

File Objectexecutable file

represents represents

requires

Example:

An object and its rendering environment

requires =relationshipType: dependencyrelationshipSubType: requires

represents

Intellectual Entityfor content Object

17 October 201650

17 October 201651

1. Object to environment - specify computational context

2. environment to Object - documentation, specifications, surrogates

3. environment to environment - inclusion, dependency, derivation,other

4. environment is an Object – preserved software source code

5. Agent to Environment - role of an Agent

6. environment to Event - environment specific Events (provenance)

7. environment to RightsStatement - software license, policy

“Object”: here a traditional content Object

Rights

identifiers

4Object Environment

New relationships

Structural object relationships

Intellectual Entity

RepresentationFile

Bitstream

represents

is included in

is part of

Expanded relationship types

� deploys

� Requires

� is Compatible With

� has Source

� Generalizes

� documents

� supersedes

� emulates

� includes

� Represents

� has Part

� has Sibling

� has Root

� is Deployed On

� is Required By

� is Source Of

� specializes

� is Documented In

� is Superseded By

� is Emulated By

� is Included In

� is Represented By

� is Part Of

17 October 201653

� Dependency

� Derivation

� Logical

� Reference

� Replacement

� Structural

Semantic units only applicable to

environment Intellectual Entities� 1.9 environmentFunction

� environmentFunctionType

� environmentFunctionLevel

� 1.10 environmentDesignation� environmentName

� environmentVersion

� environmentOrigin

� environmentDesignationNote

� environmentDesignationExtension

� 1.11 environmentRegistry� environmentRegistryName

� environmentRegistryKey

� environmentRegistryRole

� 1.12 environmentExtension

� 1.13 relationship …� relatedEnvironmentPurpose

� relatedEnvironmentCharacteristic

objectIdentifierobjectIdentifierType: ARKobjectIdentifierValue: ark:/9999/b1

objectCategory: intellectual entityenvironmentFunction

environmentFunctionType: software environmentFunctionLevel: 1

environmentFunctionenvironmentFunctionType: operating systemenvironmentFunctionLevel: 2

XP Professional, Service Pack 3

17 October 201654

environmentFunctionType: softwareenvironmentFunctionLevel: 1

environmentFunctionenvironmentFunctionType: operating systemenvironmentFunctionLevel: 2

environmentDesignationenvironmentName: Windows XP ProfessionalenvironmentVersion: Service Pack 3environmentDesignationNote:

maintenance deadline: 2014-04

17 October 201655

environment Intellectual Entities

17 October 201656

� 1.9 environmentFunction � environmentFunctionType

environmentFunctionType: softwareenvironmentFunctionLevel: 1environmentFunction

environmentFunctionType: operating system environmentFunctionLevel: 2environmentDesignation

environmentName: Windows XP ProfessionalenvironmentVersion: Service Pack 3

environmentRegistryenvironmentRegistryName: PRONOMenvironmentRegistryKey: x-sfw/8environmenttRegistryRole: identity

� relatedEnvironmentCharacteristic Content Object

relationshipType: dependencyrelationshipSubType: requiresrelatedEnvironmentPurpose: renderrelatedEnvironmentCharacteristic: recommendedrelatedObjectIdentifier

relatedObjectIdentifierType: PUIDrelatedObjectIdentifierValue: x-sfw/8

x-sfw/8Description of Windows XP Professional in PRONOM

Alternative: Link to an external registry

17 October 201657

� 1.11 environmentRegistry

� environmentRegistryName

� 1.13 relationship …

� relatedEnvironmentPurpose

17 October 201658

17 October 201659

� 1.9 environmentFunction � environmentFunctionType

� 1.13 relationship � …

� relatedEnvironmentPurpose

environmentFunctionType: software application

Firefox 10.0objectCategory: intellectual entityenvironmentFunction

environmentFunctionType: software application

BlueGriffon 1.6

Content ObjectformatName: text/html

relationshipType: dependencyrelationshipSubType: requiresrelatedEnvironmentPurpose renderrelatedEnvironmentCharacteristic: known to work

relationshipType: dependencyrelationshipSubType : requiresrelatedEnvironmentPurpose: create

Add physical Objects

� A physical Object is � A content Object, such as a manuscript, or printed document� An environment Object, such as a physical hardware device.

� Representation: A digital or physical Object � Either one instantiates or embodies an Intellectual Entity

� Digital and non-digital Objects can be captured uniformly.� Physical Objects can relate to digital Objects and other physical Objects.

� In V3 storage is applicable to Representations. For physical Representations: the physical location, e.g. a shelf location.

17 October 201660

Add physical Objects

17 October 201661

[Physical representation]

objectIdentifierobjectIdentifierType: ARKobjectIdentifierValue:

ark:/9999/h1.version1objectCategory: fileformat

formatDesignationformatName: image/tiffformatVersion: 6.0

objectIdentifierobjectIdentifierType: ARKobjectIdentifierValue::ark:/12148/cb37367035f

objectCategory: intellectual entity

relationshipType: derivationrelationshipSubType: has sourcerelatedObjectIdentifier

relatedObjectIdentifierType: Internal call numberrelatedObjectIdentifierValue: Rés. Ye-3535

relationshipType: structuralrelationshipSubType: is represented as

17 October 201662

clarificationhttp://www.loc.gov/standards/premis/premis-conformance-20150429.pdf

Thank you!

� Resources: http://www.loc.gov/standards/premis/

� PREMIS Implementors Group Forum: PIG@listserv.loc.gov

17 October 201663

Digital preservation metadata Why is it needed and what ......Angela Dappert -Digital Preservation...

Documents

Transcript of Digital preservation metadata Why is it needed and what ......Angela Dappert -Digital Preservation...

Metadata for preservation

Preservation Metadata Initiatives and Standards

Metadata in Digital Preservation - ERPANET report.pdfERPANET Training Seminar, Metadata in Digital Preservation 4 Metadata in digital preservation Marburg, 3-5 September 2003 Introduction

PREMIS Data Dictionary for Preservation Metadata · 2011-01-06 · PREMIS Data Dictionary for Preservation Metadata, version 2.0. Since the publication of version 2.0implementation

Metadata for digital long-term preservation

PREMIS - Preservation Metadata: Implementation Strategies ...

Digital Preservation Metadata - Planets

Metadata For Preservation Delos

Don't WARC Away: Preservation Metadata & Web Archives

Preservation Metadata: The PREMIS Experience

Metadata for long term-preservation

Dataset Metadata, Tools & Approaches for Access & Preservation

PREMIS Data Dictionary for Preservation Metadata -- version 2 · PREMIS Data Dictionary for Preservation Metadata version 2.0. Development of the original PREMIS Data Dictionary The

DataONE Preservation and Metadata Working Group Report 2014

Preservation Metadata Extraction and Collection : Tools and Techniques

Preservation metadata

Practical maintenance of evolving metadata for digital preservation

PRESERVATION METADATA: IMPLEMENTATION STRATEGIES Preservation Metadata: The PREMIS Experience Priscilla Caplan Florida Center for Library Automation (FCLA)

Angela Dappert -Digital Preservation Metadata and ......eventIdentifierValue: E002.2 eventType: migration 27 May 2015 27 May 2015 agentIdentifier agenIdentifierType: gentID agentIdentifierValue:

Preservation Metadata: Implementation Strategies (PREMIS)