1

Using Utility Analysis to Evaluate and Compare

Preservation Strategies

Carl Rauch, Andreas Rauber

Vienna University of Technology

http://www.ifs.tuwien.ac.at

2

Motivation

We have collections with different file formats and

preservation requirements myriads of potential preserveration approaches

(various converters, emulators, metadata schemes,…)

We need a structured approach to selecting and evaluating

preservation solutions, rather than un-transparent „out-of-the-guts“ decisions

3

Outline

Introduction Utility Analysis

Set objectivesEvaluate alternativesDefine preferences and decide

Summary

4

Selecting a preservation strategyProblem

Strategies that suit different requirements

Means to make strategies comparable

Measures to be equally applicable to new preservation strategies

Structured approach

Requirements

Several preservation strategies, none excels in all circumstances

Different requirements for different collections

Steady change and development of strategies and tools

Solution

Generic framework, which can be easily applied to specific environments

Decision support system, which clearly ranks possible preservation solutions

5

Utility Analysis

Developed in the 1970s Applied mainly for infrastructure projects,

such as dams, bridges, neighbourhoods Flexible and expandable Adapted to fit the preservation

requirements

6

Utility Analysis Procedure

Define project objectives

Assign effects to the objectives

Define alternatives

Measure alternatives performance

Transform measured values

Weight the objectives

Aggregate partial and total values

Rank the alternatives

7

Define Project Objectives

Collect set of project objectives Include all requirements and desiderata Rather complex, extensive Procedure:

Bottom-up approach: brainstorming session

Top-down approach: according to generic objective tree

Structure as an Objective Tree

8

Define Project Objectives

Bootom-up:

9

Define Project Objectives

Collection preservation

File characteristics

Process characteristics

CostsPersonel

Technical

Originality

Stability

Scalability

Usability

Appearance

Structure

Behavior

e.g. Maintenance

e.g. Hardware, software, per file

e.g. Complexity, functionality

e.g. Data increase

e.g. Supplier independency

e.g. Tracability of changes

e.g. Search, links, user inputs

e.g. Caption, tag description

e.g. Character, sound, video, ..

Top-down:

10

File Characteristics

Appearance

Structure

Behaviour

Size

Special Characters

Separation

Characters

Paragraph

Page

Picture Inclusion

Footnotes

Page Numbering

Page Borders

Page Break

Word Functionality

Define Project Objectives Integrate:

11

Assign effects to objectives

Objectivei

Measurable effects: for example in

- dpi resolution, mm difference, true/false

- EURO per year, person months

- seconds per file

- ……

Subjective evaluation:

subjective impression when no measureable evaluation possible, for example

- display quality- look-and-feel

12

Listing Alternatives

Migration and Standardisation Migrate documents to Adobe PDF using XXX Migrate documents to OpenOffice 1.0 Migrate documents to PostScript using XXX Migrate documents to MS Word 2003

Encapsulation Hardware Museum … Maintain current strategy No action

13

Alternatives' evaluation

Select files for evaluation Original files from collection Files from a testbed

Ensure that they cover collection characteristics Perform preservation steps according to

list of alternatives Measure results

14

Alternatives' evaluation

Word 2003 OpenOffice PDF 5.0 No changes

Page margins 0 mm + 3 mm 0 mm 0 mm

Ingest: sec. per file 10 sec 10 sec 15 sec 0 sec

Software costs per year 50 € 0 € 0 € 0 €

Numbering of chapters 3 N.A. 5 5

Paragraph formatting 3 2 5 5

Result: Table of performance measures

15

Transform Measured Values Need to make measured values comparable Define transformation table

5 4 3 2 1 N.A

Page margins +/- 0mm +/- 1mm +/- 2mm +/- 3mm +/- 4mm > 4mm

Ingest: sec. per file 0 -5 sec 5-10 sec 10-15sec 15-25sec 25-40sec >.40sec

SW costs/year 0 € 1-30 € 31-50 € 51-70 € 71-100 € > 100 €

Chapter numbering 5 4 3 2 1 N.A.

Paragraph formatting 5 4 3 2 1 N.A.

16

Transform measured Values

Word 2003 OpenOffice PDF 5.0 No changes

Page borders 5 2 5 5

Ingest: sec. per file 4 4 3 5

Software costs per year 3 5 5 5

Numbering of chapters 3 N.A. 5 5

Paragraph formatting 3 2 5 5

Transform measures:

17

Weighting

Objectives differ in importance / priority Assign weights to objectives Basically possible right after definition of

Objective Tree Recommended to perform after

measurement and transformation Weights per branch level sum up to 1

18

Weighting

File characteristics

Process characteristics

Costs

Appearance

Structure

Behaviour

0,6

0,3

0,1

0,4

0,3

0,3

Final weight of all leafs:

Appearance 0,6 * 0,4 = 0,24

Structure 0,6 * 0,3 = 0,18

Behaviour 0,6 * 0,3 = 0,18

Σ(leaf weights) = 1

Σ(w1,j) = 1

19

Aggregating part values

Calculate leaf values by multiplying transformed measurements with weights

Aggregate values per alternative If necessary, average or min/max over different

demo-files Provides performance per alternative

according to different branch levels, i.e. objective granularities

20

Final Ranking

Ranking of alternatives Not-acceptable alternatives are kept in ranking Final sensitivity analysis regarding non

measurable influences on the decision, such as: expertise in a specific alternative good relation to a supplier …

21

Summary

Composition of Objective Tree depends strongly on collection requirements

Different solutions vary mainly in Objective tree composition Objective‘s weights

A few „standard“ Objective Trees may evolve We now have:

A structured approach to make accountable preservation decisions

A transparent decision process

22

Next steps

Cooperating with institutions to elaborate "standard" Objective Trees

Cooperate on generating "exhaustive" listings of file format characteristics

Develop tool support for calculating different weighting scenarios

Evolve into decision support system