USGS/EROSArchive Preservation

StatusUpdate

Stuart Doescher, USGS

(Cheryl Greenhagen)WGISS – 22

Sept 11- 15, 2006Annapolis, MD USA

TM and MSS Archive Conversion TM and MSS Archive Conversion System (TMACS)System (TMACS)

• Operations: 1992 to 2002.• Number of High Density Tapes (HDT)

transcribed: 55,503– Landsat MSS, TM

• Number of DCRSi Cassette Tapes (DCT)s output: 3,827

• Mostly in Telemetry stream format (Frame sync required to read)

WBVT summaryWBVT summary

• Operated: April 1999 - May 2002• Number of HDTs converted: 21,300• Number of Digital Linear Tape (DLT 7000)

generated: 143 and an additional 143 as backup • Format: BSQ ( MSS-X)• "orphan" data (no DIAT / SLAT) save on 172 DLTs• Bad input tapes: 476 (even with cleaning &

baking).

SPOT summarySPOT summary

• Operated: late April 2000 - February 2002

• Number of HDTs converted: 6,100

• Number of DLT 7000 tapes generated: 901

• Format: MDA's FRED

Landsat Archive Conversion System (LACS)

• Planned Operations: June 2004 – December 2006

• Input: About 5500 HDTs, 4000 DCTs, plus current Landsat 5 data

• Output: ~ 1100 tapes of "formatted" data and ~ 1200 tapes of "raw" data on 9940B. (~180G/tape)

• Format: BSQ

LACS DescriptionLACS Description

• Ingest current Landsat 5 data from disk files• Ingest data from DCRSi Cassette Tapes (DCTs)

– Multispectral Scanner Archive format (MSS-A) data– Multispectral Scanner Product format (MSS-P) data– Thematic Mapper Archive format (TM-A) data– Thematic Mapper raw (TM-R) data

• Ingest TM-R data from High Density Tapes (HDTs)• Archive the data to a high-density computer-compatible

digital tape: 9940B • Generate and archive browse imagery• Generate inventory information for the new archive.

Data to be converted by LACSData to be converted by LACS

Data SetNumber of

ScenesNumber of

Tapes Data Volume

MSS-P 65,128 118 DCTs 3.2 terabytes

MSS-A 262,088 277 DCTs 9.5 terabytes

TM-A 17,000 108 DCTs 3.6 terabytes

TM-R (1982 – 2005) 550,000 3,320 DCTs 140 terabytes

TM-R (1995 – 2002) 97,250 5,444 HDTs 27.0 terabytes

TM-R (new) ~ 14 terabytes

 1,031,762  Total ~200 terabytes

LACS Status• Started Operations: June 2004• Progress to Date: (July 31, 2006)

– TM-R DCTs complete (3,320 DCTs, 555,000 scenes)

– TM-R HDTs complete (5,444 HDTs, 112,000 scenes)

– TM-A DCTs complete (108 DCTs, 17,000 scenes)

– TM-R New Landsat 5 Data (86,000 scenes)

– MSS-A started (21 DCTs, 17,000 scenes)

• Transcription rate will decrease as we handle more data anomalies and problem tapes

• Estimated completion: December 2006

LACS Configuration

LACS1LTS 1: R C C Ingest

LACS3LTS 3: D C T Ingest

LACS2LTS 2: D C T Ingest

LAM

DCS

LIS 1D isk

SAN

LA C S 3D isk

LIS1LIS 1: D B ,

B rowse / A C C A

LIS2LIS 2: B rowse /

A C C A

LIS3LIS 3: B rowse /

A C C A

LCDSLA C S D BD C S D B

LACS4LTS 4: D C T Ingest

SILOLocal

D B linkSANnfs

LACS StatusLACS Status

• LTS (LACS Transcription Subsystem)

4 Strings:

– LTS1: Current Landsat 5 data(acquired at EDC or GLC, or received on DLT from Australia)

– LTS2: DCT Input (2 shifts per day, 5 days a week)

– LTS3: DCT Input (2 shifts per day, 5 days a week)

– LTS4: DCT Input (2 shifts per day, 7 days a week)

LACS StatusLACS Status

• LIS (LACS Integration Subsystem)

Performs Automated Cloud Cover Assessment,

Browse Generation, Quality Assignment

Provides Interactive Assessment Capability

Gives priority to current Landsat 5 data

LIS work is distributed across 3 strings

LACS Status

• Issues / Problems:– Data anomalies, tape degradation, cross-play issues– DCRSi tape drives require frequent cleaning and

maintenance

• Plans:– Complete tape transcription– Port LTS from SGI to Intel (Linux)