Public Records in the Digital Age
description
Transcript of Public Records in the Digital Age
![Page 1: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/1.jpg)
Salvador BarraganCurator of Government Records
Nebraska State Historical Society
Public Records in the Digital Age
![Page 2: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/2.jpg)
What is a record?
• A ‘record’ is the complete set of documentation required to provide evidence of a business transaction.
![Page 3: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/3.jpg)
Shifting Media• Before Paper we stored valuable historical data on stone
and papyrus.• In our current period records of historical permanency
were stored on paper and kept in filing cabinets– When the cabinet was full, records were sent to file
room.
• Now records are stored electronically on computers– When the computer is ‘full’ – add more hard drives or
servers.
Basic skills to manage and maintain records has been lost, replaced by infinite storage
![Page 4: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/4.jpg)
Electronic Records Management Goals
1.Bring the record to the forefront of system design activities.
2.Identify electronic records functionality as part of system design.
3.Create electronic records that support legal, fiscal and evidentiary needs.
4.Create long term archival storage for both retention schedule and historical purposes.
![Page 5: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/5.jpg)
Goals con’t
5.Create electronic records that are accessible and usable over time (non-proprietary formats).
6.Integrate diverse document forms and formats into records.
7.Identify need for internal and external primary and secondary access to records.
![Page 6: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/6.jpg)
Three Functional Requirements for Electronic
Records Management & Preservation
![Page 7: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/7.jpg)
1. Records Capture – Records are created or captured and identified to support the business process and meet all records management requirements related to the process.
![Page 8: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/8.jpg)
2. Records Maintenance and Accessibility – Electronic records are maintained so that they are accessible and retain their integrity for as long as they are needed.
![Page 9: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/9.jpg)
3. System Reliability – A system is administrated in accordance with best practices in the information resource management field to ensure the reliability of the records it produces.
![Page 10: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/10.jpg)
What happens when you do not have a
RM system?
![Page 11: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/11.jpg)
![Page 12: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/12.jpg)
Higher Standards
• As electronic records become more integrated into society, producers of those records will be held to higher standards of conduct– HIPPA– Sarbanes Oxley– Federal and State Mandates– Case Law
![Page 13: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/13.jpg)
NE Public Records Laws002.01 Record. The Records Management Act (Revised Statutes of Nebraska, Chapter84, Article 12) defines a record as: "any book, document, paper, photograph, microfilm,sound recording, magnetic storage medium, optical storage medium, or other materialregardless of physical form or characteristics created or received pursuant to law, charter,or ordinance or in connection with any other activity relating to or having an effect uponthe transaction of public business." A record is information that is inscribed on a tangiblemedium or that is stored in an electronic or other medium and is retrievable inperceivable form.
http://statutes.unicam.state.ne.us/Corpus/statutes/chap84/R8412013.html
![Page 14: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/14.jpg)
Records Retention
The foundation of democracy in America is government accountability to the people and permanency of our culture and heritage.
![Page 15: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/15.jpg)
So the question becomes… who takes care of the
records, and do they have the knowledge
understanding of the new technology?
![Page 16: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/16.jpg)
Caretakers of Information• Historically records sent to file room, staff
maintained access to records and managed lifecycle based on need and legal requirements
• Now records are managed by users and IT staff, based on capacity and cost.
![Page 17: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/17.jpg)
Taking into account the goals of records management and the
function of Records, what are we to do?
![Page 18: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/18.jpg)
Or what is the solution?
•Best Practice Models
•Standards
•Systems
•Digital Archive?
![Page 19: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/19.jpg)
Best Practice Models
•OAIS Model
•Washington State Archives
http://www.digitalarchives.wa.gov/default.aspx
![Page 20: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/20.jpg)
OAIS Model
![Page 21: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/21.jpg)
www.digitalarchives.wa.gov
![Page 22: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/22.jpg)
Standards
When ever possible follow the prevailing best practices and standards.
![Page 23: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/23.jpg)
Standards for E-Records….
•Hardware
•Software
•Formats
•Management
•Authenticity
![Page 24: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/24.jpg)
Hardware• File Room of the 21st century• Capacity and Speed double every 18 months• Many choices
– Tape– Optical– Spinning Disc
First Immutable Law of Digital Archiving“What hardware you use today will be obsolete
within four years”
![Page 25: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/25.jpg)
Internet
DA-WEB1
DA-SE1
DA-BIZ-RS1BizTalk
Receive/SendLocation
Washington State Digital Archives Network Configuration, May 2, 2005
LegendFirewall
Database Server
REVISIONS
1.02.0
Final network architecture09/16/200405/02/2005
Actual configuration
Data Tier
Processing Tier
Services Tier(Search Services)
State/LocalOffice
Web/FTP Server
Web Services
BizTalk Server
Domain Controllers
DA-DC1
HP DL3802 *3GHz HT CPU2GB RAM36GB Mirrored HDWIN 2003 std
HP DL3802 * 3GHZ HT CPU
2GB RAM36GB Mirrored HDMS WIN 2003 std
SAN Storage
Internet Send/Receive
DA-WEB2
DA-DC2
HP DL3802 * 3GHz HT CPU2GB RAM36GB Mirrored HDMS WIN 2003 std2 Coyote HW Loadbalancers
DA-SE2 DA-SE3
DA-BIZ-RS2BizTalk
Receive/SendLocation
BizTalk 2004Database
Cluster
Digital ArchivesAsset Metadata
Cluster
HP DL3802 * 3GHZ HT CPU2GB RAM144GB RAID 5 HDMS WIN 2003 stdMS BizTalk 2004 ent
`
CitizenInternet
User
HP DL580 4 *3GHZ CPU4GB RAM36GB Mirrored HDMS WIN 2003 entMS SQL Server 2000MS Clustering Active/Active
EMC Clariion CX700 SAN 1TB 15K FC
4TB 7200 SATA
Tape Library
HP DL7408 *3GHZ HT CPU
8GB RAM36 GB Mirrored HD
MS WIN 2003 entMS SQL Server 2000
MS Clustering Active/Passive
Shared Disk Array
DA-BIZ-INBOX1 RAW Data “Temp” Storage
Image ConversionXML “Temp” Storage
Administration
ADIC iScalar 2000 10 LTO-2 drives 500 tape slots
Hardware Load Balanced
Hardware Load Balanced
IIS
IIS
IIS IISIIS
http/https
Secure FTP
DA-DMZ-DC2DA-DMZ-DC1
http/https
DA-Media1 & 2 (Images & Streaming
Media)
DA-SE4
DA-SE5
IIS
IIS
80/443
22
80
22
80
80
DMZ
DA-Tectia1 (Secure FTP)
HP DL3802 *3GHz HT CPU2GB RAM36GB Mirrored HDMS WIN 2003 std
HP DL580 4 *3GHZ CPU4GB RAM36GB Mirrored HDMS WIN 2003 ent
![Page 26: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/26.jpg)
Digital Archives Hardware• Network – Cisco Backbone end to end
– LAN and SAN• EMC – SAN storage
– 5 TB now, 20TB by end of Year• HP – Servers and desktops• ADIC – Tape Library for offsite, disaster
recovery (nightly or weekly back up, remember Katrina and 9-11)
• Microsoft – Software and Development
![Page 27: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/27.jpg)
Archival Software and File Format Standards
• Native• ASCII• TIF• PDF/A (Used by the Federal Courts)• XML http://www.thexmltoolkit.org/guides.asp (metadata
and interoperability)• DoD 5015.2-STD compliant system• Nebraska State Records Guidelines
Whenever possible seek the Open, documented solution!
Remember WordStar and DBase II ???
![Page 28: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/28.jpg)
Metadata & Interoperability
• Cross cultural and contextual boundaries• Interoperability • Interoperability & Metadata schema
![Page 29: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/29.jpg)
Interoperability and XML
![Page 30: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/30.jpg)
Content Management• Essential to maintain control of the information
explosion• Allows hard coded rules and information
exchange• BUT still requires a strong knowledge,
understanding and implementation of basic records management
Second Immutable Law of Digital Archiving:“Data is Data, a Record is a Record, It is the
content that drives retention, not the media”
![Page 31: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/31.jpg)
• DoD 5015.2-STD compliant systemhttp://www.dtic.mil/whs/directives/corres/pdf/50152std_061902/p50152s.pdf
Wrap original file in native format• Wrap XML copy• Apply metadata & XML for indexing,
searching & retrieval• Provide chain of custody & authenticity
‘Content Management’
![Page 32: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/32.jpg)
‘Content Management’• Microsoft Solution• SQL Server back end• BizTalk translation utility• SSH Tectia for secure transport
http://www.ssh.com/products/Washington State Archives Case Study
![Page 33: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/33.jpg)
Authenticity
• Maintain Chain of Custody• In the care of trusted 3rd party• Received from trusted, known source
![Page 34: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/34.jpg)
Data Security• Encrypted SSH FTP transmission• Issue Digital Certificate• Verify IP and computer information• MD5 Hash on all original files• Copy of FTP on tape prior to ingestion• DB backups on tape• Record Level Security for confidential Info
![Page 35: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/35.jpg)
Record Level Security• Restrict records at item, field or series
level• Restrict to individual, dept, office or global• Uses authenticated login to reveal fields• Anonymous users see ‘Restricted’
![Page 36: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/36.jpg)
Deep Storage XML
![Page 37: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/37.jpg)
Deep Storage XML SchemaRecord Common
•Who
•What
•When
•Where
•Original File
•‘web’ file
•Security
•Fixity
Vital Records• Type
Birth• Date of
• Father, Mother
• Hospital
![Page 38: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/38.jpg)
Ingestion Process• MUST be flexible• Microsoft BizTalk 2004• Transforms, adds metadata based on
business rules• Creates ‘deep storage’ copy wrapping
original file in XML, with Hash• Creates ‘web’ version of original file
![Page 39: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/39.jpg)
Archive Database• Designed around latest industry standards• Open source, non-proprietary file storage• Applies metadata ‘tags’ to save information about
record– creator, date, agency, subject, etc.
• Provides chain of custody & authenticity of record• Allow search and retrieval of archival records
through a web page
![Page 40: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/40.jpg)
Risks• Distributed, non-standardized environment• Limited technology expertise in some
agencies• Unpredictable data growth rate• Few business models• Emerging technologies• Limited internal expertise
![Page 41: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/41.jpg)
Management Issues• Authenticity of record• Metadata• File naming conventions• Corporate Culture• Start small with e-mail, web page• Use existing retention schedules• Educate• Shift AWAY from desktops…• …And move to central servers• Management Software is a must!• Privacy of sensitive data
![Page 42: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/42.jpg)
Third Immutable Law“Anything that you do today, will need major
overhaul in two years or sooner”
Technology and industry changing at unprecedented rates… But, more records are ‘lost’ every day!
– Key is to be flexible and address with systematic forethought
![Page 43: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/43.jpg)
How to handle Records over the Web.
![Page 44: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/44.jpg)
Open Record
![Page 45: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/45.jpg)
Restricted Record
Confidential
![Page 46: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/46.jpg)
E-Commerce
![Page 47: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/47.jpg)
Add to Shopping Cart
• Ecommerce Functionality– Add to Shopping cart
![Page 48: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/48.jpg)
Shopping Cart
![Page 49: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/49.jpg)
Billing Information
![Page 50: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/50.jpg)
View and Submit Order
![Page 51: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/51.jpg)
Why a Digital Archives?• Comply with statutory & regulatory mandates.
– The Law requires preservation of certain public records – it doesn’t specify whether those records are paper or electronic. All records must be given the same care.
• Avoid loss of legal & historical records– As technology changes, the older media (5 ¼” floppy disks, for instance)
become harder to read.• Centralize Records
– Centralization means uniformity in maintenance– ‘Trained professionals’ serve as caretakers
• Preserve rare and ‘at-risk’ paper records• Improved access for citizens
– By centralizing historical electronic records in one location, ‘one-stop shopping’ will provide the information quicker and easier
![Page 52: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/52.jpg)
The Digital Archives will:
• Preserve electronic records with long-term legal, historical and/or fiscal significance
• Assure platform-neutral retrieval 50, 100, or more years from now
• Provide security back-up of certain permanent electronic legal records (courts, vital records, land records, etc.)
![Page 53: Public Records in the Digital Age](https://reader035.fdocuments.us/reader035/viewer/2022070420/56815e61550346895dccdda4/html5/thumbnails/53.jpg)
Acknowledgements
• Adam Jansen, Digital Archivist for the Washington State Archives.
• Dr. Ed Papenfuse, State Archivist for the Maryland State Archives.
• Andrea Falling, State Archivist for the Nebraska State Historical Society.
• Cathy Danahy, Assistant Director of the Nebraska Records Management Division.