NE NDSA WorkshopWGBH, Boston MA

May 10, 2013Andrea Goethals, Harvard Library

NDSA Levels of Digital Preservation NDSR Boston

NDS*

Levels of Digital PreservationGuidelines created by the NDSA

Standards & Practices

InfrastructureInnovation

Outreach

Content

NDSADiverse working groups

Standards & Practices

InfrastructureInnovation

Outreach

Content

Levels of Digital

Preservation

Common Need

Common Need

• Simple, practical, documented levels of preservation services reflecting best practices, broadly useful– For those just starting out & those with mature

programs– Independent of formats, storage systems– Useful to educators & implementers

Niche

Personal Archiving

Advice

…

Levels of Digital

Preservation

…

Formal Certifications &

Audits

Levels of Digital Preservation, v1Level 1

Level 2

Level 3

Level 4

Category 1

Category 2

Category 3

Category 4

Category 5

Levels of Digital Preservation, v1Level 1

Level 2

Level 3

Level 4

Category 1

Category 2

Category 3

Category 4

Category 5 Bit-l

evel

Pro

tecti

on

Long

er-te

rm U

sabi

lity

Levels of Digital Preservation, v1Level 1 (Protect your data)

Level 2 (Know your data)

Level 3 (Monitor your data)

Level 4 (Repair your data)

Storage and Geographic Location

- Two complete copies that are not

collocated

- For data on heterogeneous media

(optical discs, hard drives, etc.) get the

content off the medium and into your

storage system

- At least three complete copies

- At least one copy in a different geographic

location

- Document your storage system(s) and storage

media and what you need to use them

- At least one copy in a geographic location with

a different disaster threat

- Obsolescence monitoring process for your

storage system(s) and media

- At least three copies in geographic locations

with different disaster threats

- Have a comprehensive plan in place that will

keep files and metadata on currently accessible

media or systems

File Fixity and Data Integrity

- Check file fixity on ingest if it has been

provided with the content

- Create fixity info if it wasn’t provided with

the content

- Check fixity on all ingests

- Use write-blockers when working with original

media

- Virus-check high risk content

- Check fixity of content at fixed intervals

- Maintain logs of fixity info; supply audit on

demand

- Ability to detect corrupt data

- Virus-check all content

- Check fixity of all content in response to

specific events or activities

- Ability to replace/repair corrupted data

- Ensure no one person has write access to all

copies

Information Security

- Identify who has read, write, move and

delete authorization to individual files

- Restrict who has those authorizations to

individual files

- Document access restrictions for content - Maintain logs of who performed what actions

on files, including deletions and preservation

actions

- Perform audit of logs

Metadata- Inventory of content and its storage

location

- Ensure backup and non-collocation of

inventory

- Store administrative metadata

- Store transformative metadata and log events

- Store standard technical and descriptive

metadata

- Store standard preservation metadata

File Formats- When you can give input into the creation

of digital files encourage use of a limited

set of known open formats and codecs

- Inventory of file formats in use - Monitor file format obsolescence issues - Perform format migrations, emulation and

similar activities as needed

http://www.digitalpreservation.gov/ndsa/activities/levels.html

Storage and Geographic LocationLevel 1Protect your data

Level 2Know your data

Level 3Monitor your data

Level 4Repair your data

Two complete copies that are not collocated

For data on heterogeneous media (optical discs, hard drives, etc.) get the content off the medium and into your storage system

At least three complete copies

At least one copy in a different geographic location

Document your storage systems(s) and storage media and what you need to use them

At least one copy in a geographic location with a different disaster threat

Obsolescence monitoring for your storage system(s) and media

At least three copies in geographic locations with different disaster threats

Have a comprehensive plan in place that will keep files and metadata on currently accessible media or systems

File Fixity and Data IntegrityLevel 1Protect your data

Level 2Know your data

Level 3Monitor your data

Level 4Repair your data

Check file fixity on ingest if it has been provided with the content

Create fixity info if it wasn’t provided with the content

Check fixity on all ingests

Use write-blockers when working with original media

Virus-check high risk content

Check fixity of content at fixed intervals

Maintain logs of fixity info; supply audit on demand

Ability to detect corrupt data

Virus-check all content

Check fixity of all content in response to specific events or activities

Ability to replace/repair corrupted data

Ensure no one person has write access to all copies

Information SecurityLevel 1Protect your data

Level 2Know your data

Level 3Monitor your data

Level 4Repair your data

Identify who has read, write, move and delete authorization to individual files

Restrict who has those authorizations to individual files

Document access restrictions for content

Maintain logs of who performed what actions on files, including deletions and preservation actions

Perform audit of logs

MetadataLevel 1Protect your data

Level 2Know your data

Level 3Monitor your data

Level 4Repair your data

Inventory of content and its storage location

Ensure backup and non-collocation of inventory

Store administrative metadata

Store transformative metadata and log events

Store standards technical and descriptive metadata

Store standard preservation metadata

File FormatsLevel 1Protect your data

Level 2Know your data

Level 3Monitor your data

Level 4Repair your data

When you can give input into the creation of digital files, encourage use of a limited set of known open formats and codecs

Inventory of file formats in use

Monitor file format obsolescence issues

Perform format migrations, emulation and similar activities as needed

Some Uses

• Identify community consensus on best practices

• Preservation service choices• Assessments – how do we compare with best

practices?– What should we improve next?– Where do we excel?– How will we improve after project X?– How have we improved over time?

Self-assessment example

Level One Level Two Level Three Level Four

Storage & Geographic Location

File Fixity and Data Integrity

Information Security

Metadata

File Formats

= satisfied with implementation

= will be satisfied with implementationafter current enhancement project

= implemented but could be improved

= not implemented

How you can help: provide feedback!

• Revisions will continue until the Levels stabilize on a broad professional consensus.

• Comments received by 8/31/2013 can affect the next revision

• Send comments by e-mailing the addresses listed at http://digitalpreservation.gov/ndsa/activities/levels.html

NDSR BostonIMLS-funded Residency Project

National Digital Stewardship Residency New residency program created by the

Library of Congress (LC) with the Institute of Museum and Library Services (IMLS)

To develop the next generation of stewards to collect, manage, preserve and make accessible our digital assets

NDSR DC

Cohort model (social learning) Focus on digital stewardship Graduates of any masters program Round out what’s needed for a successful career

◦ Hands-on experience with real projects in real world settings

◦ Building of portfolio, professional network, presentation skills

Directly beneficial to host institutions◦ Projects proposed by them◦ Collaborate with the other host institutions◦ Exposure to program training material, resident tools

Residency Model Highlights

10 Washington D.C.-area hosts◦ Project proposals

10 recent master’s graduates◦ Apply and choose top 3

NDSR DC

• Developing and promoting policies and services to make digital assets of research libraries accessible (Association of Research Libraries)• Management and preservation of digital assets (Dumbarton Oaks)• Born-digital preservation (Folger Shakespeare Library)• Taking action to mitigate format obsolescence (Library of Congress)• Developing a thematic web archive collection (National Library of Medicine)• The digital dissemination challenge (National Security Archive)• Broadcast media archive: appraisal and evaluation of at-risk media to support digitization initiative (PBS)• Time-based media art: specialized requirements for trustworthy digital repositories (Smithsonian Institution Archives)• Accessing born-digital literary materials (University of Maryland Libraries and MITH)• eArchives: memory of the world bank (World Bank Group Archives)

Begins with an intensive 2-week immersion workshop at LC on digital stewardship

Residents transfer to 1 of 10 Wash. D.C. institutions for 9 months◦ Hands-on experience working on digital

stewardship project(s)◦ With the cohort attend guest lectures, field trips,

make presentations◦ Start to build portfolio and professional network

NDSR DC: Residency

Hosts already identified Selected residents will be notified next week Sept. 2013: Immersion workshop Sept 2013 – May 2014: Residency

NDSR DC: Timeframe

IMLS-funded grants Two geographic areas

◦ NDSR Boston (Harvard / MIT)◦ NDSR New York (METRO)

Replicating the model

Timeframe (June 2013 – June 2016)

◦ Year 1: planning

◦ Year 2: 1st round of residents

April 2014: Hosts identified for 1st round

◦ Year 3: 2nd round of residents

April 2015: Hosts identified for 2nd round

NDSR Boston

Coordinated by an academic inst. (Harvard) Produce curriculum resources and model

documents Train-the-Trainers workshop Great environment for residents

◦ Public transportation system◦ Rich with potential host institutions◦ Many potential guest lecturers, site visits

NDSR Boston

Thanks!