Document

51
Jean VARRA_ INA 2008 1 Managing a Preservation and Digitisation Plan of Audio-visual Archives Processes and technology Managing a Preservation and Digitisation Plan of Audio-visual Archives Processes and technology

description

http://iftadev.geckon.dk/wp-content/uploads/02.pdf

Transcript of Document

Page 1: Document

Jean VARRA_ INA 2008 1

Managing a Preservation and Digitisation

Plan of

Audio-visual Archives

Processes and technology

Managing a Preservation and Digitisation

Planof

Audio-visual Archives

Processes and technology

Page 2: Document

Jean VARRA_ INA 2008 2

INA at a glance...

• INA (Institut National de l’Audiovisuel - France) is in charge of the preservation and exploitation of one of the richest and oldest radio and television archive holdings.

• These holdings are mainly composed of programs from public national and regional TV and radio broadcasters since the beginning until now.

• They include cinematic news items and an important amount of photographs.

• Otherwise, INA has been in charge of the legal deposit of the French radio and TV programs since 1995.

• The other missions of INA are Production, Research and Training.

• In 1999, INA decided to launch a massive and systematic digitisation plan of its audio-visual archives.

• INA (Institut National de l’Audiovisuel - France) is in charge of the preservation and exploitation of one of the richest and oldest radio and television archive holdings.

• These holdings are mainly composed of programs from public national and regional TV and radio broadcasters since the beginning until now.

• They include cinematic news items and an important amount of photographs.

• Otherwise, INA has been in charge of the legal deposit of the French radio and TV programs since 1995.

• The other missions of INA are Production, Research and Training.

• In 1999, INA decided to launch a massive and systematic digitisation plan of its audio-visual archives.

Page 3: Document

Jean VARRA_ INA 2008 3

INA holdings

Radio holdings: 497 500 hours78RPM

disks27000hours

78RPM

disks27000hours

1/4 » audio tapes

476000 hours

1/4 » audio tapes

476000 hoursDATDAT

Photo holdings: 1 200 000 photos

TV holdings: 535 000 hoursVideo

1p and 2p70 000 hours

Video1p and 2p

70 000 hours

Film

35 and 16mm

143000 hours

Film

35 and 16mm

143000 hours

Video

3/4 inch

135000 hours

Video

3/4 inch

135000 hours

BETACAM TAPES

192000 hours

Page 4: Document

Jean VARRA_ INA 2008 4

Why a digitisation plan?

All archivists must facetwo important and unavoidable problems:

All archivists must facetwo important and unavoidable problems:

AND Obsolescenceof playback

means

Obsolescenceof playback

means

Old media inevitably degrade

Old media inevitably degrade

Page 5: Document

Jean VARRA_ INA 2008 5

3 main questions to answer...

Degradation of the carriers and obsolescence of techniques

Degradation of the carriers and obsolescence of techniques

Definition and setting of preservation and

safeguardingpolicies

Definition and setting of preservation and

safeguardingpolicies

Archives have commercialand cultural values

Archives have commercialand cultural values Definition of an access policyDefinition of an access policy

questionsquestions AnswersAnswers

Archive volumesincrease exponentially

Archive volumesincrease exponentially

Decision of exhaustiveness of preservation

or definition of selection criteria

Decision of exhaustiveness of preservation

or definition of selection criteria

Page 6: Document

Jean VARRA_ INA 2008 6

Typology of degradations

3 types of decay3 types of decay

Mechanical( changes in size

and shape)

Mechanical( changes in size

and shape)

Biological(mould, bacteria….)

Biological(mould, bacteria….)

Chemical(spontaneous

chemicalchanges:Acid…)

Chemical(spontaneous

chemicalchanges:Acid…)

Page 7: Document

Jean VARRA_ INA 2008 7

Some major degradations...

• Film– Flammability of decomposing nitrate films– Vinegar syndrome of acetate films causing shrinkage and decomposition.– Splice degradation( dry or greasy splices) and damaged perforations requiring heavy mechanical repair– image degradation like scratches, dirt, colour fading….

• Tapes (video and audio)– Stickiness – head clogging and dropout– mechanical deformation (shrinkage,wavering….

• Mechanical disks ( 78 RPM, LP) – (grooves disappear producing a fin white powder) – sensitivity to fungus, ultra violets and heat– crackling and shrinkage

• Optical disks– layer separation,lack of planarity,crackling and pinholes.– sensitive to pollutants , corrosion of the metallic layer and long light exposure

• Film– Flammability of decomposing nitrate films– Vinegar syndrome of acetate films causing shrinkage and decomposition.– Splice degradation( dry or greasy splices) and damaged perforations requiring heavy mechanical repair– image degradation like scratches, dirt, colour fading….

• Tapes (video and audio)– Stickiness – head clogging and dropout– mechanical deformation (shrinkage,wavering….

• Mechanical disks ( 78 RPM, LP)– (grooves disappear producing a fin white powder) – sensitivity to fungus, ultra violets and heat– crackling and shrinkage

• Optical disks– layer separation,lack of planarity,crackling and pinholes.– sensitive to pollutants , corrosion of the metallic layer and long light exposure

Page 8: Document

Jean VARRA_ INA 2008 8

The importance of good storage conditions

• “Old media are often archived in stores with bad climate conditions”.• the effects of climate conditions ( temperature , humidity,dust, air

quality…)on degradation and therefore on the life expectancy of carriers are now well known.

• Preserving original media in good climate conditions (controlled T°, Relative humidity and dust) and in suitable containers is always far less expensive than being obliged to restore degraded media.

• “Old media are often archived in stores with bad climate conditions”.• the effects of climate conditions ( temperature , humidity,dust, air

quality…)on degradation and therefore on the life expectancy of carriers are now well known.

• Preserving original media in good climate conditions (controlled T°, Relative humidity and dust) and in suitable containers is always far less expensive than being obliged to restore degraded media.

Vented cans Stil Design•Archives films

But relying only on good climate conditionsto preserve content in the long term,is not enough….

But relying only on good climate conditionsto preserve content in the long term,is not enough….

Page 9: Document

Jean VARRA_ INA 2008 9

Recommended climate conditions

Film• Main vault: Cold

– T= 5°C– RH:25%– Light over-pressure.– air filtering.

• transit storage: Cool– T= 15°C– RH:30% – dew point controlled to avoid

condensation • Temperature and RH stability

– T°= +/- 1°C over 1 hour– Relative humidity : +/- 5% RH over 1 day

• Frozen storage vault 0°C( for degrading acetate films with vinegar syndrome and or degrading nitrate

• Isolate degraded nitrate and acetate film to avoid contamination.

Film• Main vault: Cold

– T= 5°C– RH:25%– Light over-pressure.– air filtering.

• transit storage: Cool– T= 15°C– RH:30% – dew point controlled to avoid

condensation• Temperature and RH stability

– T°= +/- 1°C over 1 hour– Relative humidity : +/- 5% RH over 1 day

• Frozen storage vault 0°C( for degrading acetate films with vinegar syndrome and or degrading nitrate

• Isolate degraded nitrate and acetate film to avoid contamination.

Magnetic tapesAcetate tapes:

– Cool 12 °C max for 50% max RH– 17°C max for 30% max RH– 23°C max for 20% RH– frozen forbidden unless high vinegar

syndrome level (IPI 2 or higher).

Polyester tapes:– Cool 12 °C max for 50% max RH– 17°C max for 30% max RH– 23°C max for 20% RH– Cold to avoid– frozen forbidden

Disks– Store Vertically– Frozen is forbidden

Magnetic tapesAcetate tapes:

– Cool 12 °C max for 50% max RH– 17°C max for 30% max RH– 23°C max for 20% RH– frozen forbidden unless high vinegar

syndrome level (IPI 2 or higher).

Polyester tapes:– Cool 12 °C max for 50% max RH– 17°C max for 30% max RH– 23°C max for 20% RH– Cold to avoid– frozen forbidden

Disks– Store Vertically– Frozen is forbidden

Page 10: Document

Jean VARRA_ INA 2008 10

Why digitise our archives?

Only one way:

Digitisation.

Only one way:

Digitisation.

Today all archive owners have the same dilemma to solve:

Today all archive owners have the same dilemma to solve:

•How could this analogue heritage be preserved?•How could this heritage be everlasting?•How could this heritage be more valuable?

Digitisation

Page 11: Document

Jean VARRA_ INA 2008 11

Setting a preservation and digitisation plan

A methodological approach

Page 12: Document

Jean VARRA_ INA 2008 12

Some important preliminary considerations

Why a methodological approach?Why a methodological approach?

•Because the holdings are often huge.•Because the holdings are often poorly identified•Because budgets for preservation are poor compared to the mass of material to be treated. •Because priorities must then be defined.•Because the materials to be treated are often damaged and their life expectancy is limited.

Page 13: Document

Jean VARRA_ INA 2008 13

A 3 entries problem…

Volumes Time

MeansTechnical, Human, Money

Page 14: Document

Jean VARRA_ INA 2008 14

1 - Define the measurement unit of your holdings (nbr hours)1 - Define the measurement unit of your holdings (nbr hours)

2 - Map your holdings: (indexed) contents & physical storage media2 - Map your holdings: (indexed) contents & physical storage media

3 - Assess the condition of the physical storage media3 - Assess the condition of the physical storage media

4 – Define a preservation policy & processing priorities (+ preservation committee)4 – Define a preservation policy & processing priorities (+ preservation committee)

5 – Define the technical solutions (formats, storage media, workflows)5 – Define the technical solutions (formats, storage media, workflows)

6 - Estimate your human, technical, financial resources6 - Estimate your human, technical, financial resources

7 – Draw up annual budgets and processed volumes (over the pluriannual plan)7 – Draw up annual budgets and processed volumes (over the pluriannual plan)

A seven steps preliminary process...

Page 15: Document

Jean VARRA_ INA 2008 15

Some important preliminary considerations

Why is it important to define and set a massive, industrial and pluri-annual

digitisation plan?

Why is it important to define and set a massive, industrial and pluri-annual

digitisation plan?

• Bring an answer to the urgency to face to degradation and obsolescence of analogue technology.

• “Massive and industrial” in order to reduce costs and go fast.• Get a reference tool for negotiating funds and all other

negotiations : human resources, technical resources and budgets.

• Provide a tool to select and define priorities in terms of materials to be processed.

Page 16: Document

Jean VARRA_ INA 2008 16

Step 1

Measure your holdings (volumes)Measure your holdings (volumes)

•Number of materials•Number of catalogued documents •Number of programmes •Number of hours of programmes

•Evaluate “redundancy”

Page 17: Document

Jean VARRA_ INA 2008 17

Step 2

•Map your contents:•using cataloguing, indexing, searching tools

•Map your media:•make inventory of physical media •sort volumes according to the genre and type of media

Map your holdings (volumes)Map your holdings (volumes)

Page 18: Document

Jean VARRA_ INA 2008 18

Examples of mapping

CONTENTS 2004 2005 2006 2007 2008TOTAL

2008 2009-2015Amount to be treated

NEWSREELSNews 1000 1500 2800 2500 2000 9800 200 10000Collections of news magazines 500 1000 1000 500 1622 4622 2 499 7121

Total newsreels 1500 2500 3800 3000 3622 14422 2699 17121

DOCUMENTARIES AND MAGAZINESDocumentaries 300 350 300 200 100 1250 1316 2566Litterature programs 300 400 400 200 100 1400 1007 2407Art programs 200 400 400 200 100 1300 730 2030Cultural programs 200 100 100 100 100 600 2166 2766Music programs 300 300 300 300 200 1400 713 2113Cinema,TV, theater programs 400 300 300 200 200 1400 2660 4060Tourism programs 1560 1560

Total doc and mag 1700 1850 1800 1200 800 7350 10152 17502ENTERTAINMENTSPop, rock programs 200 200 175 100 100 775 6925 7700

Total entertainments 200 200 175 100 100 775 6925 7700SPORTSports magazines 300 300 300 200 100 1200 2556 3756

Total sports 300 300 300 200 100 1200 2556 3756YOUTH PROGRAMS 4296 4296ANIMALS PROGRAMS 3000 3000RELIGIONS 3207 3207GAMES 7162 7162

TOTAL FILM 3700 4850 6075 4500 4622 23747 39997 63744

NOT REAL

FIGURES

FILM 1 and 2 inches Umatic/BVU TOTALACTUALITE - NEWSREELSNews 10 828 Regional news 35 500 55 800

TOTAL NEWSREELS 46 328 55 800 102 128 FICTIONSFictions 5 000 650 770 6 420

TOTAL FICTIONS 5 000 650 770 6 420 DOCUMENTAIRES - DOCUMENTARIESliterature programs 2 407 959 749 4 115 art programs 2 030 850 650 3 530 cultural programs 2 766 951 1 771 5 488 music programs 2 113 2 085 1 727 5 925 science programs 2 790 1 024 718 4 532 cinema, theater programs 4 060 1 534 2 072 7 666 tourism programs 1 560 1 163 823 3 546 society programs 8 710 2 810 2 500 14 020

TOTAL DOCUMENTARIES 26 436 11 376 11 010 48 822 ENTERTAINMENTSrock, pop programs 7 700 2 900 3 968 14 568

TOTAL ENTERTAINMENTS 7 700 2 900 3 968 14 568 SPORTsport programs 3 756 1 900 2 429 8 085

TOTAL SPORTS 3 756 1 900 2 429 8 085 YOUTH PROGRAMS 4 296 3 540 1 897 9 733 ANIMALS PROGRAMS 3 200 1 695 802 5 697 RELIGIONS 3 207 1 490 720 5 417 GAMES 7 162 4 813 2 477 14 452

NOT REAL

FIGURES

Page 19: Document

Jean VARRA_ INA 2008 19

Step 3

•Evaluate the degradation level of each type of media and the corresponding volumes.

•Vinegar syndrome•Splice condition•Image degradation.•Sticky shed syndrome.•Mechanical deformations•…….

Define and estimate the level of difficulty to process them.

Know the condition of physical mediaKnow the condition of physical media

Page 20: Document

Jean VARRA_ INA 2008 20

Example: items for film condition mapping

NITRATE BASE FILM

16 mm Acetate base film

NEGATIVE

SEPMAG

COMMAG

SPLICED WITH ADHESIVETAPE SPLICED WITH GLUEVINEGAR SYNDROME

STATE OF THE ADHESIVETAPE

GOOD DAMAGED

DRY GREASY

LEVEL OF DEGRADATIONIPI levels

0 2 31

PHYSICAL STATE

STATE OF THE GLUE

GOOD BREAKS

SHRINKAGERETRAIN

Black and white

Colour

News

production

YEAR

Manufacturer andtype

Making statistics /decisions/processes...

PICTURE

SCRATCHES

Reconditionning? separation of film andsepmag ? storage conditions/isolation

decisions to transfer?

base gelatine colourgraded ?

GOOD FADEDCOLOR

specific telecine and orsepmag player

film processing? simple check up or repair?

Yes no

Color correction ?

Value

severity?

PERFORATIONS

DAMAGED?

Yes no

GRADINGNOTCHES

Yes no

KINESCOPE

type of copyACETATE

POLYESTER

smallRollls

Editedon one

reel

Page 21: Document

Jean VARRA_ INA 2008 21

Step 4

With the help of a preservation commissionWith the help of a preservation commission

Define prioritiesDefine priorities

combiningContent reasons

heritage value, commercial value, scientific value, shortage...

Technical reasons

Uniqueness of the carrier, decay level,exploitation difficulties...

Page 22: Document

Jean VARRA_ INA 2008 22

Step 5

•Prefer open media and standardised files.

•Specify precisely the encoding schemes and the various quality levels to get. (see next chapter)

Choose digital target media and filesChoose digital target media and files

Page 23: Document

Jean VARRA_ INA 2008 23

Step 6

• Define mean processing times for each type of media according to its physical condition.

• Take an inventory of the available work forces and skills, outsourcing (service providers) and indoor and evaluate them.

• Specify the processing costs for each media type.• Define the best cost effective technical solutions.

• Define the best cost effective technical solutions

Measure one’s forces:human and technical

Measure one’s forces:human and technical

Page 24: Document

Jean VARRA_ INA 2008 24

Step 5

Specifications of the technical jobSpecifications of the technical job

•The objectives of technical specificationsis to define precisely the technical expected results.

•Describe precisely all the steps of each process.

Page 25: Document

Jean VARRA_ INA 2008 25

•Set precisely the dispatch of:• annual investment and operating budgets•volumes according to: internal /external processing capabilities.

Set budgets and volumes(over the duration of the plan)

Set budgets and volumes(over the duration of the plan)

Step 7

Page 26: Document

Jean VARRA_ INA 2008 26

Volumes Pattern (INA) Digital preservation Plan ( volumes - hours)

Total remaining Achievement %

hours repartition over 2005 - 2015

to process achieved achieved Achieved 05 / 15

Years 99 / 02 03 04 05 06 07 08 09 10 11 12 13 14 15

vinegar syndrome 64500

FILM telecine 143 500

mechanical repair 132 400

VIDEO 3/4 inch 122 200

VIDEO 1 et 2inch 70 000

Total TV 335 700

DISQ 78 RPM 20 700

preservation restoration

1/4 ich tapes 465 000

DAT 11 800

Total RADIO 497 500

TOTAL 833 200

EXAMPLE

PATTERN

Page 27: Document

Jean VARRA_ INA 2008 27

Budget pattern (INA)Tableau 2 Preservation Plan Budgets ( M€

Total hours to process Total

Total PSNAnnées 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 05/15 99/15vinegar preservation vinegar mesasurementFILM TC 143 500mechanical repairVIDEO 3/4i 122 200VIDEO 1 et 2i 70 000Total TV 335 700

78 RPM 20 700preservation restoration14 Inch tapes 465 000DAT 11 800Total RADIO 497 500PhotoStoragestorageTOTAL INVESTInventory containersmagnetic media tapesCD/DVDothersmanagementlabour forces TOTAL FONCT

TOTAL

achieved

EXAMPLE

PATTERN

Page 28: Document

Jean VARRA_ INA 2008 28

Overall technical process

Composition

Identify and assemble

items

Digitisation and

new media creationnew digital

masterlow data rate

versionscreate

new archive item

quality control

Updatearchive

replaceold item with newupdate

metadata

Preparation of the sets

packing original media

job orders

planning

Reception of sets

performing expected job

work progress

Reception of sets

performing quality control

accept or reject work

organise corrective actions

Reception of sets

Statistics about job achievement : budget and volumes

Organisation of transports

Organisation of transports

Organisation of transports

Page 29: Document

Jean VARRA_ INA 2008 29

Digital solutions for preservation

Page 30: Document

Jean VARRA_ INA 2008 30

Digitisation is the key… But what digital?

Keys for everlastingness or “ permanence” of digital audio or video data are:

Keys for everlastingness or “ permanence” of digital audio or video data are:

The independence of dataregard to

the physical media

The independence of dataregard to

the physical media

The use of standardisedand “non proprietary” :

• Essence data • Metadata formats• Storage file formats• Exchange file formats

The use of standardisedand “non proprietary” :

• Essence data • Metadata formats• Storage file formats• Exchange file formats

Page 31: Document

Jean VARRA_ INA 2008 31

A few figures...

– Film digitisation: keeping the original definition of a 1 hour film.• 1 hour of 16mm film requires 595 GB ( Gigabyte)• 1 hour of film 35mm requires from 1008 GB to 3715GB• according on the 35 mm formats• a 4K (4096x3112) scan delivers a file of 4,13 TB (Terabyte)• a 2K (2048x1556) scan delivers 1 TB• a HD (1920x1080) telecine delivers a file of 560 GB

– Video SD digitisation:• One hour of uncompressed video (720 x 576) requires 72 GB

– Audio digitisation: one hour of stereophonic audio • (16 bits, 48KHz) requires 700 MB

Digitising film, video or, in a smaller scaleaudio, requires large storage capacities.

Digitising film, video or, in a smaller scaleaudio, requires large storage capacities.

Page 32: Document

Jean VARRA_ INA 2008 32

Current digital mediafor archiving

Page 33: Document

Jean VARRA_ INA 2008 33

Two ways to store digital audio and video...

There are 2 main approaches to store and archivedigital audio and video according to expected

uses and storage hierarchy.

There are 2 main approaches to store and archivedigital audio and video according to expected

uses and storage hierarchy.

Proprietary solutions:

Digital SD or HD video or

audio digital formats.

Proprietary solutions:

Digital SD or HD video or

audio digital formats.

Open solutions:File formats (data)

onHard disk servers

ordata tapes or optical disks

on the “shelves”or within

data tape librariesor disc jukeboxes.

Open solutions:File formats (data)

onHard disk servers

ordata tapes or optical disks

on the “shelves”or within

data tape librariesor disc jukeboxes.

Page 34: Document

Jean VARRA_ INA 2008 34

Possible current media for digital archiving

Proprietary solutionsProprietary solutions Open SolutionsOpen Solutions

SD digital videotapes

D1,D2,D3,D5,DCTDigital BETACAM

BETACAM SXIMX

DV, DVCAM ,DVC Pro 25- 50HD formats

VOODOO, D5 HD, DVC pro 100HDCAM-SR

Audio tapesDAT

SD digital videotapes

D1,D2,D3,D5,DCTDigital BETACAM

BETACAM SXIMX

DV, DVCAM ,DVC Pro 25- 50HD formats

VOODOO, D5 HD, DVC pro 100HDCAM-SR

Audio tapesDAT

Hard drives(RAID arrays)

Data tapesSDLTLTO

Optical mediaCD/DVDBlu ray

Holographic disk ?

Hard drives(RAID arrays)

Data tapesSDLTLTO

Optical mediaCD/DVDBlu ray

Holographic disk ?

Page 35: Document

Jean VARRA_ INA 2008 35

Open media

• Magnetic hard drive disks capacity is still growing: it doubles every 18 months (currently 1 TB/disk)

• Used in servers as RAID arrays ( Redundant array of inexpensive disks) Solves 2 main needs : it increases data bit rates for video and reliability and tolerance to disk failures

• Cost per GB is dramatically decreasing ( Currently 1 to 2€/GB)

• Magnetic hard drive disks capacity is still growing: it doubles every 18 months (currently 1 TB/disk)

• Used in servers as RAID arrays ( Redundant array of inexpensive disks) Solves 2 main needs : it increases data bit rates for video and reliability and tolerance to disk failures

• Cost per GB is dramatically decreasing ( Currently 1 to 2€/GB)

• Current data tape solutions are– SDLT: 1/2 inch linear

recording by quantum

– LTO (LTO4) 1/2 linear inch tape by IBM, Seagate, Hewlett Packard

• Still best cost per GB (0,3 €/GB)• LTO is the leader on the market

• Current data tape solutions are– SDLT: 1/2 inch linear

recording by quantum

– LTO (LTO4) 1/2 linear inch tape by IBM, Seagate, Hewlett Packard

• Still best cost per GB (0,3 €/GB)• LTO is the leader on the market

Page 36: Document

Jean VARRA_ INA 2008 36

Open media

• Blu-ray Disk– 12 cm and 8 cm– Capacity: 23 /25/ 27 GB– data bit rate: 36,/ 72 /144 Mb/s– Suitable for HD and compressed

video – Used in XDCAM ( SONY)– Currently 1,2€ /GB

• Blu-ray Disk– 12 cm and 8 cm– Capacity: 23 /25/ 27 GB– data bit rate: 36,/ 72 /144 Mb/s– Suitable for HD and compressed

video– Used in XDCAM ( SONY)– Currently 1,2€ /GB

• Holographic disk– Format HDV Alliance / InPhase

technology – Storage capacity: 200 GB to 1,6 TB– Data bit rate: 20 MB/s to 1 GB/s

• Holographic disk– Format HDV Alliance / InPhase

technology– Storage capacity: 200 GB to 1,6 TB– Data bit rate: 20 MB/s to 1 GB/s

Page 37: Document

Jean VARRA_ INA 2008 37

Evolution of digital media technology

• In the digital domain, media technology evolves very fast:– Road maps of the manufactured products show that:

• capacity ( GB) doubles every 2 years• recording and play back speed also.

– Backward compatibility of data tapes is 2 generations at the utmost.– Obsolescence of digital media comes very fast.

• In the digital domain, media technology evolves very fast:– Road maps of the manufactured products show that:

• capacity ( GB) doubles every 2 years• recording and play back speed also.

– Backward compatibility of data tapes is 2 generations at the utmost.– Obsolescence of digital media comes very fast.

To prevent from that, digital to digital migrationmust be considered as soon as possible.

To prevent from that, digital to digital migrationmust be considered as soon as possible.

Page 38: Document

Jean VARRA_ INA 2008 38

Essence dataFormats

Page 39: Document

Jean VARRA_ INA 2008 39

Preserving quality

The use of compression allows the user to match quality with different uses, while reducing storage needs and

data transfer rates.

Archive owners must learn to deal withseveral quality levels:

Archive owners must learn to deal withseveral quality levels:

Archivingquality

Archivingquality

Exploitationquality

Exploitationquality

local viewingquality

local viewingquality

distant viewingquality

distant viewingquality

Page 40: Document

Jean VARRA_ INA 2008 40

Preserving quality

UncompressedUncompressed Losslesscompression

Losslesscompression

Lossycompression

Lossycompression

?

Page 41: Document

Jean VARRA_ INA 2008 41

The central role of a preservation master file

PreservationMaster file

PreservationMaster file

Uncompressed

Loss less compression

“Visually loss less” compression

Exploitation fileExploitation file

“Light” compression for current use

Viewing fileViewing file

Compression rate adapted to networks and Internet

Original formatOriginal format

Film

SD Analogue video

SD Digital video

HD Digital video

Page 42: Document

Jean VARRA_ INA 2008 42

Current file formats for archiving quality

data files for audiodata files for audio

WavWav

AES/EBUAES/EBU

AIFFAIFF

UncompressedUncompressed

MPEG L2MPEG L2

MPEG 3MPEG 3

Lossy compressed

Lossy compressed

data files for film and videodata files for film and video

JPEG2000JPEG2000

Loss less compression Loss less

compression

MPEG2 422(I frame only

until 100 Mb/s)

MPEG2 422(I frame only

until 100 Mb/s)

DV25 /50/100

Mb/s

DV25 /50/100

Mb/s

H264?H264?

Lossy compression

Lossy compression UncompressedUncompressed

DPXDPX

AVIAVI

Page 43: Document

Jean VARRA_ INA 2008 43

Metadata and exchange format standardisation

Page 44: Document

Jean VARRA_ INA 2008 44

Metadata

• Metadata is data about data• Information that makes data

useful: – Technical metadata– Descriptive metadata– Rights

• Several initiatives related to different domains and environment ( libraries,broadcast, multimedia,consumer…)

• Trying to converge….

• Metadata is data about data• Information that makes data

useful:– Technical metadata– Descriptive metadata– Rights

• Several initiatives related to different domains and environment ( libraries,broadcast, multimedia,consumer…)

• Trying to converge….

Dublin core

Dublin core

SMPTESMPTE MPEG7MPEG7

TV anytimeTV anytime

AAF AAF SMEF andP/META

SMEF andP/META

Metadata initiativesMetadata initiatives

Page 45: Document

Jean VARRA_ INA 2008 45

Exchange file formats (wrappers)

A Wrapper gathers in one file format the content (essence) and its related metadata for data exchange between two different systems.

Main standardised wrappersMain standardised wrappers

AAFAdvance Authoring Format

AAFAdvance Authoring Format

MXFMedia exchange format

MXFMedia exchange format

BWFBroadcast Wave Format

BWFBroadcast Wave Format

Page 46: Document

Jean VARRA_ INA 2008 46

Current digital archive system at INA

Digitisation Storage DeliveryFilm

(telecine)

Film(telecine)

2”/1”tapes

2”/1” tapes

BETACAMtapes

BETACAM tapes

3/4 inchtapes

3/4 inch tapes

transfer to digital betacam

transfer to digital betacam

AutomatedMPEG2

and/MPEG1encoding

(flexicarts)

Automated MPEG2

and/MPEG1 encoding

(flexicarts)

ONLINE MPEG1storage

(NAS servers)

ONLINE MPEG1storage

(NAS servers)

distantBACK UP STORAGE

HARD DRIVE SERVERS

distantBACK UP STORAGE

HARD DRIVE SERVERS

Browsing and viewing

intranetInternet (Ina media)

delivery of viewing copies(DVD,VHS, DV….)

Browsing and viewing

intranetInternet (Ina media)

delivery of viewing copies (DVD,VHS, DV….)

Deliverydelivery of exploitation copies

(Digital BETA, IMX, DVCPro….)Delivery through dedicated links

(TF1,FTV)delivery over networks FTP

(all customers)

Deliverydelivery of exploitation copies

(Digital BETA, IMX, DVCPro….) Delivery through dedicated links

(TF1,FTV)delivery over networks FTP

(all customers)1/4 inchaudio tapes1/4 inch

audio tapes

1/4 inchaudio tapes

78 RPMdiscs

78 RPM discs

78 RPMdiscs

78 RPM discs

transfer to WAV files

BWF

transfer to WAV files

BWF

NEAR ON LINE MPEG2storage

(LTO 4 inADIC library)

NEAR ON LINE MPEG2storage

(LTO 4 inADIC library)

INA Information system ( Databases,workflows, networks,commercial and rights management,...

Page 47: Document

Jean VARRA_ INA 2008 47

Near future evolutions...

MPEG1 1Mb/sMPEG1 1Mb/s

Digital BETACAMDigital BETACAM

MPEG2 8 Mb/sMPEG2 8 Mb/s

H264 400Kb/sH264 400Kb/s

File lossless compression

JPEG2000 (SD and HD)other?

File lossless compression

JPEG2000 (SD and HD)other?

MPEG2 higher bit rateor H264 HD and SD?

MPEG2 higher bit rateor H264 HD and SD?

Page 48: Document

Jean VARRA_ INA 2008 48

A temporary conclusion about digital technology...

• Digital media and technologies are evolving and renewing extremely fast.

• In the digital domain, we must learn to think of“everlasting file” and forget the idea to find an “everlasting media” for a while.

• Once analogue media has been digitised, the job is not finished, we must foresee and prepare the regular “migration” of data from current digital media to new ones.

• Digital media and technologies are evolving and renewing extremely fast.

• In the digital domain, we must learn to think of“everlasting file” and forget the idea to find an “everlasting media” for a while.

• Once analogue media has been digitised, the job is not finished, we must foresee and prepare the regular “migration” of data from current digital media to new ones.

Page 49: Document

Jean VARRA_ INA 2008 49

A temporary conclusion about digital technology...

• The choice of a suitable preservation master file is fundamental.

• If standardised essence data formats are more and more used in televisions (MPEG2, DV, …),there is still a long way from theory to “real” use of standardised metadata;

• For standardised exchange formats, MXF and BWF seems to take the leadership.

• The choice of a suitable preservation master file is fundamental.

• If standardised essence data formats are more and more used in televisions (MPEG2, DV, …),there is still a long way from theory to “real” use of standardised metadata;

• For standardised exchange formats, MXF and BWF seems to take the leadership.

Page 50: Document

Jean VARRA_ INA 2008 50

Thank you for your attentionand don’t let your archives

become like that!

Page 51: Document

Jean VARRA_ INA 2008 51