Finnish Digital Preservation Service for Cultural Heritage

Mikko Tiainen

IT-Architect Mar. 2015

CSC at a Glance

Owned by Ministry of Education and Culture of Finland

Operates on a non-profit principle

Short history:

– Founded in 1971 as a technical support unit

for Univac 1108

– Connected Finland to the Internet in 1988

– Reorganized as a company, CSC – Scientific

Computing Ltd. in 1993

– Facilities in Espoo, close to Otaniemi campus

(of 15,000 students and 16,000 technology

professionals) and Kajaani

– Staff 269 (March 2015)

– Turnover 2014 ~33 million euros

Enterprise Architecture for NDL

Dis

sem

inati

on p

ackage

Users

Metadata Metadata

DIGITAL

PRESERVATION

Obje

ct

request

and other 3rd

party services

PUBLIC

INTERFACE

Meta

data

Subm

issi

on p

ackage

SUPPORT

SERVICES

STANDARD

PORTFOLIO

External Services

Ontology services

Authentication Service

Integration Platform

Reachability Information

Geographical Information

Online Payment System

LIBRARY, ARCHIVE AND MUSEUM SYSTEMS

Preservation aspects and focus of

NDL’s Digital Preservation Service D

P s

erv

ice

Part

ner

org

an

iza

tio

ns

Long-term

utilization

• Storage device

• Storage media

• Materials & replication management

• Preservation actions

• File formats

• Preservation planning

• Descriptive metadata

• Content knowledge and semantics

Bit-level preservation

Logical preservation

Semantic preservation

• Administrative & technical metadata

NDL DP Specifications

Specifications available at: – http://www.kdk.fi/en/enterprise-architecture

– (mostly in Finnish)

Recommended

file formats

Acceptable file

formats for

transfer

Administrative

and structural

metadata

Descriptive

metadata

BACK-END SYSTEM

Standard portfolio

NDL METS profiles

SUBMISSION INFORMATION PACKAGES (SIP)DIGITAL

PRESERVATION

Digital Preservation – core principles

Multi vendor approach

3 copies with different media Hard disk, HP Proliant SL4540Gen8

Tape1, IBM TS1140 4TB/tape

Tape2, Oracle T10000D 8TB/tape

Dark Archive 2 different tape technologies

Fixity of AIP is verified twice per 5 years AIP checksum SHA-256

IBM Tapes are LBP verified once in a year

Open source based platform Software components are designed to be replaceable

Component life cycle estimations

Hardware

– Hard disk storage 5 years

– Tape drives & medias 5 years

– Tape libraries, 10 years

Software

– Commercial support at least for 5 years

– Open source, maintained and developed until

replaced

Platform overview

Software stack

Integration and control layer ~13000 lines of Python code ”in house”

Archivematica + Gearman

Several OS components e.g. for file format identification and validitation

Middleware Storage software GlusterFS 3.5.2

MongoDB, MySQL

Keepalived

Operating system CentOS 6.6

Configuration mgmt Spacewalk 2.2

Monitoring Opsview community release

– Traditional Nagios plugins + snmp polling & traps

VMWare

High-quality digital

preservation

• Support services

• Maintenance of specifications

• Management

• Cooperation with ATT DP

Development of NDL DP System

Future

Research data preservation

Sertifications ISO 27001 sertification in H1/2015

Data Seal of Approval later on 2015

ISO 16363 sertification on 2017 or later

Software Python 2.6 lifespan, what’s after this

Storage layer battle: etc. GlusterFS, Ceph,…

Hardware Kinetic hard drive techology (Seagate) intergation

into storage layer software

Data Integrity Feature for Oracle LTFS