IBP and Condor Micah Beck Assoc. Prof. & Director
description
Transcript of IBP and Condor Micah Beck Assoc. Prof. & Director
![Page 1: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/1.jpg)
IBP and CondorMicah Beck
Assoc. Prof. & Director
Condor Week, Madison May 6, 2003
![Page 2: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/2.jpg)
Funding• Dept. of Energy
SciDAC• National Science
Foundation ANIR• UT Center for Info
Technology Research
Logistical Networking Research at UTK
University of Tennessee
• Micah Beck• James S. Plank• Jack Dongarra
University of California, Santa Barbara
• Rich Wolski
![Page 3: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/3.jpg)
IBP: The Internet Backplane Protocol
• A scalable mechanism for deploying shared storage resources throughout the network
• A general store-and-forward overlay networking infrastructure
• A state management infrastructure for distributed applications and active services
![Page 4: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/4.jpg)
The Network Storage Stack
Applications
Logistical File System
Logistical Tools
L-Bone
IBP
Local Access
Physical
exNode
• Our adaption of the network stack architecture for storage• Like the IP Stack• Each level encapsulates details from the lower levels, while still exposing details to higher levels
![Page 5: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/5.jpg)
IBP: How it Works
• Storage provisioned on community “depots”• Very primitive service (similar to block service, but
more sharable)• Goal is to be a common platform (exposed)• Also part of end-to-end design
• Best effort service – no heroic measures• Availability, reliability, security, performance
• Allocations are time-limited!• Leases are respected, can be renewed• Permanent storage is to strong to share!
![Page 6: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/6.jpg)
The Network Storage Stack
The L-bone:Resource Discovery& Proximity queries
IBP: Allocating and managing networkstorage (like a network malloc)
The exNode:A data structurefor aggregation
LoRS: The Logistical Runtime System:Aggregation tools and methodologies
![Page 7: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/7.jpg)
L-Bone: January 2003
Current Storage Capacity: 13 TB
![Page 8: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/8.jpg)
Multithreaded Transfers
![Page 9: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/9.jpg)
Caching/Staging
![Page 10: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/10.jpg)
Point-to-Multipoint
![Page 11: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/11.jpg)
Heterogeneous Multicast
![Page 12: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/12.jpg)
Relationship to Work of Condor Group
• Wide Area File Management/Access
• Management of Computation State
• As a Storage Allocation Layer for • Kangaroo, NeSt
• DiskRouter
• If routers can have disks, how about disks with processors?
![Page 13: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/13.jpg)
Routers, Depots and the Network Functional Unit
router
inin
out
depot
disk/RAM
inin
out
depot
RAM
inin
out
NFU
send
store
load
execute
![Page 14: IBP and Condor Micah Beck Assoc. Prof. & Director](https://reader035.fdocuments.us/reader035/viewer/2022062520/56815923550346895dc64c6f/html5/thumbnails/14.jpg)
Scalable Operations
• IBP Depots with NFU define a State Transformation Substrate
• Processes run at “endpoints” but can use NFUs to transform data in network
• Is it processor-in-storage or active networking?
• All state is exposed