The PLANETS Testbed DPE, PLANETS and CASPAR 2nd Annual Conference Lisbon, 5–6 September, 2007
-
Upload
madaline-walker -
Category
Documents
-
view
20 -
download
0
description
Transcript of The PLANETS Testbed DPE, PLANETS and CASPAR 2nd Annual Conference Lisbon, 5–6 September, 2007
The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual ConferenceLisbon, 5–6 September, 2007
Max Kaiser, Austrian National [email protected], http://www.onb.ac.at
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
ToC
Why do we need Testbeds in Digital Preservation? What will the PLANETS Testbed deliver? Role of Testbed within PLANETS PLANETS Testbed services for Digital Preservation
community Testbed Methodology, Experiment 6-step Process Testbed Software Application Testbed Components User Interface Mock-Up Demonstration Next Steps
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Why do we need Testbedsin Digital Preservation? Concern about digital preservation rapidly increased
in recent years Several approaches to digital preservation
implemented by various programs and projects A number of digital preservation tools and services
(e.g. for migration and characterization) developed
Need for systematization, benchmarking and evaluation urgent in many fields:
Systematic analysis of current and potential preservation strategies
Evaluation and benchmarking of characterisation and migration tools
Assess the role of emulation tools …
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Why do we need Testbedsin Digital Preservation? If we want to
perform scientific research in digital preservation evaluate preservation approaches in diverse “real life” settings avoid duplication of work maximise use of invested resources
we need a dedicated research environment, that allows the systematic execution of experiments by different
institutions with experiments that
• follow a formalized methodology• are reproducible• are fully documented and accessible for further analysis and
comparison
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
A Testbed for Digital Preservation
Definition:
“A controlled environment for experimentation and evaluation, with metrics and benchmark content that allow comparison of preservation tools and strategies”
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
PLANETS Testbed
Platform for objective evaluation of preservation tools and strategies: Load content Perform digital preservation experiments:
• Design the experiment, select data and tools, specify resources and evaluation criteria, run the experiment, evaluate results against criteria, compare outcomes
Find support for validation of preservation strategies Benchmark tools and services
Will consist of Hardware, data storage, PLANETS software, Testbed application Benchmark and other content
Will provide services for The PLANETS project partners The preservation community Tool developers and vendors
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Role of Testbed in PLANETS | 1
Test and validate the technical solutions and approaches developed in PLANETS: provide a controlled hardware and software environment for
testing and evaluating preservation action (migration, emulation) and characterisation tools and services
record experiments data in registries for further analysis and comparison
assist the validation of the effectiveness of different digital preservation plans
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Role of Testbed in PLANETS | 2
Assess the suitability of the approaches across „real life“ scenarios in various organisations Analyse applicability of the outcomes of PLANETS in
existing workflows and organisational contexts Evaluate their efficiency in providing practicable solutions for
organisations engaged in digital preservation
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Role of PLANETS Testbed in theDigital Preservation Community In a second phase, the PLANETS Testbed will
Offer services to organisations outside PLANETS:• Support institutions to test preservation tools and
services against benchmark content• Assist institutions to validate their preservation plans
against their policies & content profiles Enable developers and third party vendors to submit tools for
benchmarking and certification:• Validate the suitability of their tools in preservation
workflows
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
PLANETS Testbed: Foundations
Build on work undertaken by the Dutch Digital Preservation Testbed and DELOS Testbed Framework
Integration in PLANETS Interoperability Framework Focus on formalisation of testing process design Strong focus on comparability and traceability of
results
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Main Participants
Austrian National Library Humanities Advanced Technology and Information
Institute at the University of Glasgow (HATII) Austrian Research Centers (ARC) National Archives of the Netherlands British Library Vienna University of Technology University of Cologne
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Testbed Methodology
Experiment Process: Refined to a focused 6-step process that each formal
experiment must follow
PLANETS Testbed experiment: consist of a consistent methodology for Defining its metadata (exploration area, scope, criteria,
references etc.) Selecting or creating a workflow of (orchestrated)
preservation services Evaluating the results against the experiment’s initial goals
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Basic Experiment Properties Definition
Basic Experiment Properties Definition
Tools and Data Specification
Tools and Data Specification
Declaration ofGo / No go
Declaration ofGo / No go
2. Design Experiment
2. Design Experiment
3. Specify Resources &
Outcomes
3. Specify Resources &
Outcomes
4. Go /
No go
4. Go /
No go
StartStart 1. Define Basic Experiment Properties
1. Define Basic Experiment Properties
Evaluation Checklist
Evaluation Checklist
Resource Specification
Resource Specification
• Name• Reference to other
experiments• Summary• Purpose• Participants• Scope• Type of Tools• Type of Objects• Approach• …
• Name• Reference to other
experiments• Summary• Purpose• Participants• Scope• Type of Tools• Type of Objects• Approach• …
• Specify Experiment Type• Migration• Characterisation• Workflow
• Specify Tools• Specify Data
• Specify Experiment Type• Migration• Characterisation• Workflow
• Specify Tools• Specify Data
• Estimate number of output files
• Estimate computer resource requirements
• Specify benchmark goals for experiment
• Estimate number of output files
• Estimate computer resource requirements
• Specify benchmark goals for experiment
• Automatic approval by system• Based on resource
specification and system monitoring
• Manual approval by administrator
• Automatic approval by system• Based on resource
specification and system monitoring
• Manual approval by administrator
Testbed Experiment Process | 1
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Workflow
Experiment Workflow
Experiment Results
Experiment Results
Experiment Evaluation
Experiment Evaluation
6. Evaluate Experiment
6. Evaluate Experiment
StopStop
4. Go /
No go
4. Go /
No go
5. Run Experiment
5. Run Experiment
• Experiments executed as BPEL workflows
• Processed by workflow execution engine of the Interoperability Framework
• Presentation of status report
• Experiments executed as BPEL workflows
• Processed by workflow execution engine of the Interoperability Framework
• Presentation of status report
• Presentation of input and output data
• Evaluation of benchmark goals
• Presentation of input and output data
• Evaluation of benchmark goals
Testbed Experiment Process | 2
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Testbed System Design
Initial understanding of the required characteristics of PLANETS Testbed derived from the Dutch Preservation Testbed and the DELOS Digital Preservation Cluster
Additional requirements gathered following a clearly structured software design process Interviews User Scenarios ~ 60 Use Cases Software Requirements
• IF Requirements (Authentication Interfaces, Logging, Versioning…)
• Perform Experiment, Define Basic Properties, Select A Tool, View Available Services, Handling Stalled Experiments…
Non-functional requirements• Usability, Reliability, Performance, Security, Supportability…
Design and Implementation Plan• High-level design of the Testbed application • Initial breakdown of the system into components
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Testbed Software
Platform-independent system based on Java Enterprise Edition (EE)
Three-tiered system:
Handles presentation of user interfaces; logic for dynamically creating content. Based on Java
Server Faces and Facelets
Provides business logic (core functionality). Based
on JavaBeans and JAX web service endpoints
Provides data persistency. Uses data registries provided by PLANETS Interoperability
Framework
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Interoperability Framework Service Bus
Security/Authentication/ Authorization
Monitoring/Logging/Auditing
Workflow Execution Engine
Error/Exception Handling
Transaction Manager
Database Layer
WorkSpace
Service Registry
RegistryServicesRegistry
Services
IF Applications
Wo
rkflow
De
sign
er
Ad
min
istra
tion
T
oo
l
Integrate
System Monitoring Client
Persistence
Scheduling Approval
TB Service RegistryClient
TB Data RegistryClient
Testbed Backend Components
Use
Configure
Testbed Application
Experiment
De
sign
er
Eva
lua
tor
Im/E
xpo
rter
Data
Data
Manager
Services
Service
Manager
GeneralAdm
inistration
Bro
wse
r
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Designer component | 1
Testbed Application
Experiment
De
sign
er
Eva
lua
tor
Im/E
xpo
rter
Data
Data
Manager
Services
Service
Manager
General
Adm
inistration
Bro
wse
r
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Designer component | 2
Guides the user trough design of an experiment User Interface for steps 1–3 of the experiment
process Define basic properties Design experiment Specify resources
Select whether experiment is formal or informal Select type of experiment
Migration Characterisation Emulation Preservation plan evaluation Execution of a predefined workflow
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Designer component | 3
Interaction with several other Testbed components, most notably the service manager and data manager
Interaction with Service Manager Browse PLANETS Service Registry Selection and configuration of (wrapped) tools for
Preservation Action (PA) and/or Preservation Characterisation (PC)
„Service“: PA/PC tool exhibiting well-defined functionality through a PLANETS IF compliant web service wrapper
Interaction with Data Manager Interface for handling data input and output via API for
accessing the IF Data Registry
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Designer component | 4
Specification of Resources E.g. number of output files
Specification of Outcomes of an experiment Will appear at the evaluation stage of experiment
Experiment stored as BPEL file and handed over to Approval component
Additional features: Loading existing experiments Viewing of experiments in “passive” mode (read-only) Commenting on experiments Re-use and Repeating experiments
• Re-use certain aspects of existing experiments, or• Repeat entire experiments with original data
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Designer component | 5
Rendering of BPEL workflows All experiments are executed as BPEL workflows “Simple” migration and characterisation experiments:
• BPEL files are created within Testbed based on templates
• Up to three orchestrated services (e.g. characterisation followed by migration followed by characterisation)
Complex workflows are created by using the IF Workflow Designer (or any other BPEL tool) and imported into the Testbed
Component does not process the actual execution of an experiment! Handled within the PLANETS Interoperability Framework by
the Workflow Execution Engine
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Approval & Scheduling | 1
System Monitoring Client
Persistence
Scheduling Approval
TB Service RegistryClient
TB Data RegistryClient
Testbed Backend Components
Testbed Application
Experiment
De
sign
er
Eva
lua
tor
Im/E
xpo
rter
Data
Data
Manager
Services
Service
Manager
General
Ad
min
istratio
n
Bro
wse
r
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Approval & Scheduling | 2
Approval component handles technical approval of experiments by the system and/or the administrator Makes use of information provided by the experimenter and
by the system monitoring component – e.g.:• Number of input files• Estimated number of output files• Current CPU / Memory usage• Threads / Processes• Available Disk Space
Scheduling component allows administrator to Select a date/time for an experiment to start Assign a maximum run time Check progress of experiments Stop experiments
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Evaluation | 1
Testbed Application
Experiment
De
sign
er
Eva
lua
tor
Im/E
xpo
rter
Data
Data
Manager
Services
Service
Manager
General
Adm
inistration
Bro
wse
r
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Evaluation | 2
View files and results that have been produced during the experiment
Capture experiment conclusions in a structured way Evaluate results based on the criteria defined during
experiment design process
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Importer / Exporter | 1
Testbed Application
Experiment
De
sign
er
Eva
lua
tor
Im/E
xpo
rter
Data
Data
Manager
Services
Service
Manager
General
Adm
inistration
Bro
wse
r
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Experiment Importer / Exporter | 2
Create BPEL files from existing experiments and make them available for download
Upload BPEL files to the Testbed Import whole experiments Import predefined workflows as part of the experiments
design process
Check validity of the BPEL file structure Check data contained within BPEL files Download experiment results
Information on experiments in PDF format created on-the-fly• Basic properties, tools used, data used, information
about approval process, information about execution of experiment, experiments results, validation of experiment
local demo WWW demo
Browse Experiments
Browse Experiments
Browse ServicesBrowse Services
My Experiments
My Experiments
New Experiments
New Experiments Browse DataBrowse Data Import
ExperimentsImport
Experiments
Login / LogoutLogin / Logout HomepageHomepage
View Experiment
Details
View Experiment
Details
Annotate ExperimentAnnotate
Experiment
Export Experiment
Setting
Export Experiment
Setting
Experiment Designer
Experiment Designer
Upload DataUpload DataRun
ExperimentRun
Experiment
Evaluate Experiment
Results
Evaluate Experiment
Results
Edit Conclusions
Edit Conclusions
View Metadata
View Metadata
ExperimenterPages
Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
Conclusions / Next Steps
PLANETS Testbed: controlled environment for experimentation that allows benchmarking of preservation tools, services and strategies
First release to be expected in February / March 2008 Will be used to experiment with the preservation tools and services
developed in PLANETS In a 2nd Phase (by end 2008) Testbed will offer services to
organisations outside PLANETS: Support institutions to test preservation tools and services against
benchmark content Assist institutions to validate their preservation plans against their
policies & content profiles Main Testbed Instance hosted by University of Glasgow (HATII)
• Dedicated hardware with sufficient processing power and storage space
The PLANETS Testbed
Max Kaiser, Austrian National [email protected], http://www.onb.ac.at
Thank you!
Questions?