Conrad PDES Spring 2006

31
05/30/22 05/30/22 National Archives and Records Admin National Archives and Records Admin istration istration 1 The Electronic Records Archives (ERA) Program Program and Research Updates Program and Research Updates Spring 2006 PDES, Inc. Offsite Spring 2006 PDES, Inc. Offsite Mark Conrad Mark Conrad Archives Specialist Archives Specialist ERA Research ERA Research Division Division

description

Electronic Records Archives Progress Report to Long-Term Data Retention Team at PDES, Inc Spring 2006 Meeting

Transcript of Conrad PDES Spring 2006

Page 1: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 11

The Electronic Records Archives (ERA) Program

Program and Research UpdatesProgram and Research UpdatesSpring 2006 PDES, Inc. OffsiteSpring 2006 PDES, Inc. Offsite

Mark ConradMark ConradArchives SpecialistArchives Specialist

ERA Research DivisionERA Research Division

Page 2: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 22

Bob’s Not Here!Bob’s Not Here! NARA’s Representative to PDESNARA’s Representative to PDES Specific Questions – Specific Questions –

[email protected]@nara.gov

Who am I?Who am I? Archivist on Bob’s StaffArchivist on Bob’s Staff Continuing Contributor to PDESContinuing Contributor to PDES

Caveat Emptor!Caveat Emptor!

Page 3: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 33

Background: Background: NARA’s MissionNARA’s Mission

Enable Citizens to Inspect Records of What the Government has DoneEnable Citizens to Inspect Records of What the Government has Done

Enable Officials to Review Their Actions and Help Citizens Hold Them Enable Officials to Review Their Actions and Help Citizens Hold Them AccountableAccountable

Ensure Continuing Access to Essential Evidence DocumentingEnsure Continuing Access to Essential Evidence Documenting Rights of American CitizensRights of American Citizens Actions of Federal OfficialsActions of Federal Officials The National ExperienceThe National Experience

Page 4: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 44

Background: Background: NARA’s MissionNARA’s Mission

How Do We Carry It Out?How Do We Carry It Out?

Determine What Evidence is EssentialDetermine What Evidence is Essential

Ensure that Evidence is CreatedEnsure that Evidence is Created

Make it Easy for Users to Access that EvidenceMake it Easy for Users to Access that Evidence

Identify Technologies and Partners to:Identify Technologies and Partners to: Improve ServiceImprove Service Contain CostsContain Costs

Page 5: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 55

Background: Background: Federal RecordsFederal Records

A Federal RecordA Federal Record Any piece of information, regardless of form or characteristics, made or received by an Any piece of information, regardless of form or characteristics, made or received by an

agency in course of businessagency in course of business That is, or should be, keptThat is, or should be, kept

As evidence of that activityAs evidence of that activity Because it contains valuable dataBecause it contains valuable data

44 U.S.C. 330144 U.S.C. 3301

NARA NARA Provides direction, guidance, and assistance to all agencies in managing their records, Provides direction, guidance, and assistance to all agencies in managing their records,

and and Preserves records with “enduring” value in the National ArchivesPreserves records with “enduring” value in the National Archives Through lifecycle management of recordsThrough lifecycle management of records

Page 6: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 66

Electronic RecordsElectronic RecordsNARA’s ChallengesNARA’s Challenges

ScopeScope The Entire Federal GovernmentThe Entire Federal Government

VarietyVariety Different Types of RecordsDifferent Types of Records

ComplexityComplexity Records in Different FormatsRecords in Different Formats

VolumeVolume Enormous Amounts of RecordsEnormous Amounts of Records

ObsolescenceObsolescence Constantly Changing Constantly Changing TechnologyTechnology

Page 7: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 77

Clinton AdministrationClinton Administration 40 million email messages 40 million email messages

State DepartmentState Department

25 million electronic diplomatic messages25 million electronic diplomatic messages

Department of DefenseDepartment of Defense 54 million images from electronic official military 54 million images from electronic official military

personnel files annuallypersonnel files annually

Census Bureau Census Bureau 600 to 800 million image files (2000 census)600 to 800 million image files (2000 census)

Electronic Records Are Arriving!Electronic Records Are Arriving!

Page 8: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 88

Anticipated Incoming Permanent Anticipated Incoming Permanent Electronic Records CollectionsElectronic Records Collections

within the next 15 yearswithin the next 15 years

Year 20072007 20082008 20092009 20102010 20112011 20122012 20132013 20142014 20182018 20222022

Projected Accumulated

NARAHoldings

in petabytes

3.63.6 5.55.5 7.87.8 10.610.6 14.114.1 19.019.0 25.825.8 35.835.8 131131 347347

Page 9: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 99

How NARA RespondedHow NARA Responded The ERA VisionThe ERA Vision

ERA will be a comprehensive, systematic, and dynamic means of

preservingpreserving and providing continuing accesscontinuing accessto any typeany type of electronic record

free from dependencefree from dependence on any specific hardware or software,

created anywherecreated anywhere in the Federal Government.

Page 10: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1010

The ERA SystemThe ERA SystemRequirementsRequirements

PersistentPersistent• To be able to To be able to managemanage and and accessaccess records over time. records over time.

AuthenticAuthentic• To ensure that record To ensure that record is what it purports to beis what it purports to be, ,

and and has not been corruptedhas not been corrupted..

ScalableScalable• To To grow and adaptgrow and adapt to increasing volumes and to increasing volumes and

evolving typesevolving types of electronic records. of electronic records.• To serve a To serve a variety of user groupsvariety of user groups..

Page 11: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1111

The ERA The ERA SystemSystem GoalsGoals

Overcome technological obsolescence in a manner Overcome technological obsolescence in a manner that that preserves demonstrably authentic recordspreserves demonstrably authentic records

Build a Build a dynamicdynamic solution that: solution that: Accepts that Accepts that changes will happenchanges will happen

Incorporates Incorporates continuing changecontinuing change in IT in IT

Takes advantage of Takes advantage of new technologiesnew technologies

Maintains and Maintains and improves performanceimproves performance

Improves Improves customer servicecustomer service

Page 12: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1212

EnsuringEnsuringAuthenticityAuthenticity

The property of a record that The property of a record that it is it is

what it purports to bewhat it purports to be, , and and

has not been corruptedhas not been corrupted

Page 13: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1313

For traditional For traditional paperpaper records we must: records we must:

Preserve and maintain the medium (Preserve and maintain the medium (contentcontent))

Document and maintain a Document and maintain a chain of custodychain of custody

Maintain Maintain the the contextcontext of the record to other of the record to other

recordsrecords

EnsuringEnsuringAuthenticityAuthenticity

Page 14: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1414

For complex electronic recordsFor complex electronic records We must preserve the We must preserve the essential propertiesessential properties

of the record:of the record:

Examples of complex electronic records:

Databases, Web sites, GIS, Virtual Reality, Product Models, etc.

ContentContent

ContextContext

StructureStructure

BehaviorBehavior

EnsuringEnsuringAuthenticityAuthenticity

Page 15: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1515

Strategies Strategies that NARA will usethat NARA will use

Attack the critical Attack the critical preservation issuepreservation issue

Define Define Lifecycle requirementsLifecycle requirements

Find solutions in Find solutions in mainstream technologiesmainstream technologies

Keep paceKeep pace with the with the next generationnext generation national information national information

infrastructureinfrastructure

Align with overall direction of Align with overall direction of IT in the U.S. GovernmentIT in the U.S. Government

Page 16: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1616

ERA Program ERA Program Major MilestonesMajor Milestones

19981998

The Archivist of the United States tasks Ken Thibodeau with setting up a program to understand the challenges of preserving and accessing Federal Government electronic records

NARA receives seed money to engage Government and private sector research partners

199919992000200020012001

First proof-of-concept of research results are delivered and released NARA establishes the ERA Program Office Research results on preservation of Senate records are delivered

20022002

ERA System Requirements are developed ERA Research Program supports research at the National Center for Supercomputer

Applications (NCSA)

20032003 The ERA Research Program initiates operation of the Virtual Archives Lab

20042004

ERA Program is a “Line Item” Appropriation ($36M, P.L. 108-199) NARA participates in meetings conducted by the Executive Office of the President’s

National Coordination for Federal IT Research Office First test of NARA’s Research Prototype Persistent Archive (the “Data Grid”) NARA awards two development design contracts to Harris Corporation and Lockheed Martin

Corporation

20052005 Competing contractors deliver ERA prototype design demos NARA selects Lockheed Martin Corporation to begin the development of the ERA System.

Page 17: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1717

• 12/ 03 Release of the RFP• 8/3/04 Awarded Two Design Contracts

• 9/8/059/8/05 NARA Selects a Single DeveloperNARA Selects a Single Developer

2005 - 2011 Multiple Increments (Inc) w/ Multiple Releases:

• FY07 Initial Operating Capability

• 2011 Full Operating Capability

The ERA Program:The ERA Program:

Where we are nowWhere we are now

Initial Operating Capability

FY 200712/03

Release of the Request For

Proposal (RFP)ContractAwarded

NARA Selected NARA Selected Lockheed Martin Lockheed Martin CorporationCorporation

Full Operating Capability

2011

Inc 1

8/3/04Sep 2005

Inc 2 Inc 3 Inc 4 Inc 52 years 1 year 1 year 1 year 1 year

Page 18: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1818

Selected Corporate Team Selected Corporate Team

8 September 20058 September 2005 – NARA Selects Lockheed – NARA Selects Lockheed Martin Team to develop Operational ERA SystemMartin Team to develop Operational ERA System

Lockheed Martin Corporate teamLockheed Martin Corporate team - - Bearing Bearing Point, Fenestra Technologies, FileTek, EDS, History Point, Fenestra Technologies, FileTek, EDS, History Associates, Metier, Image Fortress, Science Associates, Metier, Image Fortress, Science Applications International Corporation, TessellaApplications International Corporation, Tessella

Value of Awarded ContractValue of Awarded Contract - - $308M$308M

Page 19: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 1919

Archivist’s DirectiveArchivist’s DirectiveResearchResearch

44 U.S.C Section 2904 (Chapter 244 U.S.C Section 2904 (Chapter 2))

The Archivist of the United States is

authorized by law to “conduct research “conduct research with respect to the improvement of records with respect to the improvement of records management practices and programs.”management practices and programs.”

Page 20: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2020

Why ERA Research?Why ERA Research?

Archival MoonshotArchival Moonshot

Red-tailed HawksRed-tailed Hawks

The WizardsThe Wizards

Page 21: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2121

Why ERA Research?Why ERA Research?The Archival MoonshotThe Archival Moonshot

Unprecedented ChallengesUnprecedented Challenges Technology Does Not Currently ExistTechnology Does Not Currently Exist Complexity &Volume Will Continue to GrowComplexity &Volume Will Continue to Grow

Unprecedented ResourcesUnprecedented Resources Recognized GOVERNMENTAL ChallengesRecognized GOVERNMENTAL Challenges White House Coordinated Research ActivityWhite House Coordinated Research Activity

Page 22: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2222

Why ERA Research?Why ERA Research?The Red-Tailed HawksThe Red-Tailed Hawks

Identify Potential in New TechnologiesIdentify Potential in New Technologies

Promote Knowledge & Technology Transfer Promote Knowledge & Technology Transfer from Research Projectsfrom Research Projects

Bring Together Partners with Diverse Domain Bring Together Partners with Diverse Domain ExpertiseExpertise

Leverage Other ResearchLeverage Other Research

Page 23: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2323

Why ERA Research?Why ERA Research?The WizardsThe Wizards

Unprecedented CollaborationsUnprecedented Collaborations

World-Class Expertise in Multiple DomainsWorld-Class Expertise in Multiple Domains

Recognized World-Class ChallengesRecognized World-Class Challenges

Page 24: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2424

NationalScienceFoundation

San Diego San Diego Supercomputer Supercomputer

CenterCenter

NARA’s Research PartnersNARA’s Research Partners

Global Grid Forum

National Center National Center for for Supercomputing Supercomputing ApplicationsApplications

Army Army ResearchResearchLaboratoryLaboratory

NISTNIST

…and many other Federal Agencies and their Records Officers

Page 25: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2525

National Coordination Office forNational Coordination Office forNetworking Information TechnologyNetworking Information Technology

Research and Development (NCO/NITRD) Research and Development (NCO/NITRD)

Page 26: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2626

FY 2007 FY 2007 NITRD Supplement to the President's Budget

See http://www.nitrd.gov/pubs/2007supplement/

Page 27: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2727

FY 2007 FY 2007 NITRD Supplement to the President's Budget:

Some Highlights

Human-Computer Interaction and Information Management Needs Include:

Next-generation methods, tools, and technologies to make it possible to access, integrate, analyze, and efficiently manage massive stores of widely distributed, heterogeneous information (e.g., science and engineering research data, Federal records)

Page 28: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2828

FY 2007 FY 2007 NITRD Supplement to the President's Budget

Some Highlights

Human-Computer Interaction and Information Management Needs Include:

Federal information management architecture testbeds: To evaluate issues in petascale collections of information

governed by differing requirements (e.g., national security, personal privacy)

Long-term preservation: Maintenance of and access to long-lived science and

engineering data collections and Federal records

Page 29: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 2929

FY 2007 FY 2007 NITRD Supplement to the President's Budget

Implications for PDES

Explicit Priority for R&D FundingExplicit Priority for R&D Funding

Monitor RFPs, RFIs, etcMonitor RFPs, RFIs, etc

ERA Research Will Post to the List any ERA Research Will Post to the List any Opportunities We are Made Aware ofOpportunities We are Made Aware of

Page 30: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 3030

ERA ResearchERA ResearchNewNew CollaborationCollaboration

Department of the NavyDepartment of the Navy

Engineering Engineering Drawings for Drawings for TWRTWR

Page 31: Conrad PDES Spring 2006

04/12/2304/12/23 National Archives and Records AdministrationNational Archives and Records Administration 3131

Your Contacts in theYour Contacts in the

ERA Program Management OfficeERA Program Management Office

Bob [email protected]

Mark Conrad [email protected]

ERA CommunicationsERA Communications(301) 837-0740

[email protected]

The ERA Website ishttp://www.archives.gov/era