DevOps in the Digital Repository of Ireland

About DRI (http://www.dri.ie/)

● DRI is an interactive trusted digital repository for

contemporary and historical, social and cultural

data held by Irish institutions

● Exchequer funded; HEA PRTLI 5, €5.2M

● RIA (lead), NUIM, TCD, DIT, NUIG, NCAD

● Partners: academic, cultural, social, government

● Duration - Sep 2011 – Sep 2015

Objectives

● Trusted Digital Repository (TDR) ○ Provide reliable, long-term access to managed digital resources

Open Archival Information System (OAIS)

ISO16363 Trusted Digital Repository (TDR) Checklist

Certification (DSA)

● Central internet access point o Interactive multimedia tools

Organisation

User Interface

Data Management

Storage

configurable

bi-lingual

multi-modal

accessible

federated

secure

scalable

resilient

software architecture

tool-set

curation

preservation

Requirements

● Stakeholder interviews

○ Requirements

○ Policy

● Core DRI requirements

○ Multiple data types

○ Multiple metadata standards

○ Access controls

● Mapped to Cucumber features

Technology Choices

● Ruby on Rails (http://rubyonrails.org/)

● Git (http://git-scm.com/)

● Jenkins (http://jenkins-ci.org/)

● Cucumber (http://cukes.info/)

● Project Hydra (http://projecthydra.org/)

● Ansible (Dev/Test/Prod) (http://www.ansibleworks.com/)

● Vagrant (Dev) (http://www.vagrantup.com/)

● OpenNebula (Test/Prod) (http://opennebula.org/)

○ Originally FP7 Funded

Hydra

• Open-source repository

solution

• Open architecture

o Common core

o Solution bundles

• Community

o Developers and

adopters

OpenNebula

● Open-source IaaS

○ Private cloud

o Hybrid cloud support

• Lightweight

o Management interface

o VM hosts

o Storage

• Community support

Development Practices

● Communication

○ ‘Stand-ups’

○ Teleconference

○ Face-to-Face

● Collaboration

○ Issue tracking

○ Version control

○ Continuous integration

● Consistent environment

○ Virtualisation

Development Environment

● Development scaffold

○ Automated

○ Repeatable

● Production like environment

○ OS

○ Package versions

● System dependencies

○ Fedora

○ Solr

○ Redis/Resque

○ MySQL

○ Ceph S3

○ Passenger

○ ClamAV, ffmpeg….

Deployment

● Local test suite

○ Cucumber/RSpec

● Continuous integration

○ Automated build and test

● Deployment

○ Automated with Ansible

○ Development/production

Deployment

Operations

● Preservation and disaster recovery

○ File fixity

○ Replication

● Auditing

○ Security (authorisation and authentication)

○ Malware scanning

○ Characterization

● DOIs

○ Minting, Takedowns, Updates

● User support

Issues

● Translating requirements to specification

○ Maintaining Cucumber steps

● Initial stakeholder interaction

● Hydra rapid development

● Institutional networking restrictions

● RHEL (rvm/ruby, ceph, haproxy...)

● Performance issues

○ Many moving parts

Questions?

Team ● TCD - Jimmy Tang, Kathryn Cassidy, Peter Tiernan, Stuart Kenny

● NUIM - Damien Gallagher, Sinéad Redmond

● DIT - Eoin Kilfeather, Monica Harasimiuk, Anna Deegan