Guiding ideas of the FACIT Storage Architecture: UTK Presentation
-
Upload
flashdomain -
Category
Documents
-
view
171 -
download
0
Transcript of Guiding ideas of the FACIT Storage Architecture: UTK Presentation
Guiding ideas of the FACIT Guiding ideas of the FACIT Storage ArchitectureStorage Architecture
Library of Congress Storage Vendor Meeting
Sept 18, 2007Terry Moore, University of Tennessee, KnoxvilleLarry Carver, University of California at Santa
Barabara
What is FACITWhat is FACIT FACIT – Federated Archive
Cyberinfrastructure Testbed FACIT: project of National Geospatial Data
Archive (NDIIPP partner) Goal of FACIT: Create a testbed to test a
different approach to federated resource sharing, redundancy and access
FACIT partners: NGDA (UCSB and Stanford)Logistical Networking (UTK) – network storage techREDDnet (Vanderbilt) – NSF funded infrastructure
using LN for data intensive collaboration2
Typical design perspectiveTypical design perspective
3
recent
content
now
take
action
now
+
100 years
“The archive begins today”
FACIT design perspectiveFACIT design perspective
4
old
content
now - 50 now + 50now
take
action
contentancient
content
“mid-century perspective”
5
What archivist in the middle seesWhat archivist in the middle sees Repeated migrations across storage media
and storage systemspast and future – 20 to 30+ over a century
Repeated migrations across archive systemseach possibly necessitating transformation and
reorganization of archived content Repeated handoffs between institutions
each implementing different policies How can we create a “handoff process” that
can be sustained? Design for interoperability and deployment
scalability first How do you do that?
6
Generic storage stackGeneric storage stack
The common interface “virtualizes” the technology beneath it
What interface goes here?
Issue: Whatever you choose will become the basis for storage interoperability for adopters
LN hypothesis: Do it the way that the network people did it
““Bits are bits” infrastructure for storageBits are bits” infrastructure for storage
7
Standardize on what we have an adequate common model for Storage/buffer
management Coarse-grained data
transfer Leave everything else to
higher layers End-to-end services:
checksums, encryption, error encoding, etc.
Enable autonomy in wide area service creation: security, resource allocation, QoS guarantees…
Gain the benefits of interoperability today!
One infrastructure serves all
8
Basic elements of the LN stackBasic elements of the LN stack
Highly generic, “best effort” protocol for using storage
•Generic -> doesn’t restrict applications
•“best effort”-> low burden on providers
•Easy to port and deploy
Metadata container bit-level structure
•Modeled on Unix inode
•bit-level structure, control keys, …
•XML encoded
9
Sample exNodesSample exNodes
A B C
0
300
200
100
IBP Depots
Network
Tennessee
UCSB Stanford
REDDnet
Crossing administrative domains, sharing resources
Question: Where is data object C?
10
New federation members?New federation members?
• Add new depots
LoC
IBP Depots
Network
Tennessee
UCSB Stanford
REDDnet
• Rewrite the exNodes
• Copy the data
LN file downloadLN file download
11
IBP Depots
exNode – 4 copies
TCP streams
LN file downloadLN file download
12
REDDnet depot unit: COTSREDDnet depot unit: COTS
13
• Dual core 2.4GHz AMD 64 X2 processors with 4GB of memory,
• 4 x750GB SATA2 drives in hot-swap bays
• Dual GigE NICs.
• OS stored on a USB-header mounted transflash drive; all disk drives available for use
• >$700 per TB
• “But there’s so much it doesn’t do!” True
• Question: How much can we do in software on top? E.g. check-sums, error encoding, encryption, etc.