University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage...
Transcript of University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage...
![Page 1: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/1.jpg)
University of Minnesota
Digital Technology Center
Intelligent Storage
Consortium
David H.C. Du
For more information on Intelligent Storage Consortium, please check
http://www.dtc.umn.edu/programs/DISC.htm
![Page 2: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/2.jpg)
Overall FocusEmphasize the application of Advanced Storage TechnologiesA Balanced approach to research that includes:
Applications that need/use storageAdvanced and Emerging Storage ArchitecturesAdvanced and Emerging Storage Technologies both software and hardwareBusiness Cases and aspects of the Storage industry
Market TrendsProduct DirectionsEffects of these disruptive technologiesAdoption rates
Provide consortium members with not just technology research but a more complete and significant outcome
![Page 3: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/3.jpg)
Initial Specific Focus
Applied Object-based Storage Device (OSD) Active Storage (Datanomic Concept)
The Application of OSD and Active Storage Devices to different real-world problems (MPEG-21, data mirroring, iSCSI based data sharing and file systems)Demonstrate the feasibility and advantages of OSD and Active Storage (modifying Lustre code)Develop an understanding of the limitations of OSD and Active Storage from a theoretical and practical standpoint (many research issues)
How OSD Active Storage addresses real-world problems facing the storage industry today and tomorrowMust make this a Win-Win value proposition for the University and Industry
![Page 4: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/4.jpg)
University Participation
The Digital Technology CenterJim Licari and Tom Ruwart
Cross-disciplinary University participantsComputer Science – Software Technology focus
David Du – Active disk, OSD, and NetworkingJon Weissman – Software systems, grid computingYongdae Kim- Computer and Network SecurityZhili Zhang – Networking and Internet Engineering
Electrical Engineering – Hardware Technology focusAhmed Tewfik – Signal processing, wireless networkDavid Lilja – Computer Architecture, Distributed systems
Carlson School of Business – Business focusBob Kauffman, Information and Decision SciencesAlok GuptaGediminas Adomavicius
![Page 5: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/5.jpg)
Potential Industrial ParticipationEMC, StorageTek– Storage SystemsSeagate, Qlogic, – Storage components
Development and demonstration of OSDDevelopment and demonstration of OSD Active Storage
Veritas – Storage SoftwareMicrosoft, Intel, IBM, Sun Micro – I/O Subsystem, Network Storage System and ApplicationsCisco- Network Storage SystemStorage Networking Industry Association (SNIA)
Involvement with Technical Working GroupsContact with leading-edge storage and networking companiesHelp and guidance for useful projects
![Page 6: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/6.jpg)
Changes - Technology
Intelligent devicesStorage capacity Ubiquitous connection
high speed IP networkwireless network
Reducing costAmount of data
![Page 7: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/7.jpg)
Changes – User&Application
User Demandanytime, anywhere accessheterogeneous/diversified user requirementgrowing needs for storage space
1997 1999 2001 2003
1,370MB 3,500MB 7,600MB 14,350MB
from trendfocus
storage needs from end users
Emerging data-intensive applications
![Page 8: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/8.jpg)
What is Datanomic?In an autonomic system, processes are performed automatically in response to internal causes and influences.Our proposed approach is to take a data-centric view of the system in which actions performed by, or on behalf of, a data object are taken at the direction of the data object itself. Therefore, such a system is called datanomic system.
![Page 9: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/9.jpg)
Datanomic Computing System behavior driven by characteristics of the data
Automatic optimization to ever changing data requirementsAllocate resources according to increase in demand of the dataTransform data formats to support different applications
Seamless data access from anywhere at anytimeLocation and context aware access to dataAdaptive performanceConsistent view of each user’s dataIndependent of platforms, operating systems, and data formats
Exploit active object and intelligent disk
![Page 10: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/10.jpg)
Basic Technologies and Assumptions of Datanomic
Intelligent Storage DevicesObject Storage Devices
Unique Object IdFlat Object Directory + Strong Search EngineData + Meta-data + Attributes
Network Attached Storage DevicesSeparation of Control and Data PathsMobility of Data, Clients and Storage DevicesWireless and Intermittent Connectivity
![Page 11: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/11.jpg)
System Architecture
IP NetworkWithin a Region
Desktop
Regional Manager
Laptop
App Server
IP NetworkWithin a Region
Desktop
Regional Manager
Laptop
App Server
IP NetworkWithin a Region
Desktop
Regional Manager
Laptop
App Server
IP NetworkWithin a Region
Desktop
Regional Manager
Laptop
App Server
IP Network
![Page 12: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/12.jpg)
Regional OrganizationPartition of regions:
based on physical or logical affinitySingle regional managerclientsIntelligent object-based storage devices
![Page 13: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/13.jpg)
Regional component (1)Regional Manager
Object metadata managementSecurity related issues within/outside regionNaming serviceObject replication, migration and consistencyClients and OSD devices management (including mobile clients and devices)
![Page 14: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/14.jpg)
Regional component (2)Client
End users or applications that access objects within a regionClient has a home region that stores important client information. The home region is allowed to moveClient can move freely among region
![Page 15: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/15.jpg)
Regional component (3)
Intelligent Object-based Storage Devices
OSD decides if a specific client is allowed to perform some operationsPerform data-directed operations specified by the object itself
![Page 16: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/16.jpg)
Coordination within a region
Desktop
Regional Manager
App Server
Laptop
IP Network
1
2
4
Intelligent OSD
5
3
![Page 17: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/17.jpg)
Coordination between regions
IP Network
Desktop
Regional Manager
Laptop
App ServerIP Network
2
Lookup(object ID/name)
4
8
IP NetworkDesktop
Regional Manager
Laptop
App Server
1
5
9
3
7
6
![Page 18: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/18.jpg)
Research Issues
Dynamic replica managementCaching and consistency managementSecurityEfficient and robust data pathsRepresentative applications
![Page 19: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/19.jpg)
ChallengesLarge scale of RMs, devices, objects and clientsWide area environmentUser, data, and device mobility
SolutionDynamic replica management Cooperative replication
Utilize the intelligence of object and device Goal
Reduce the burden of regional managerAvoid hot spotReduce the access delay and network trafficProvide continue access when RM is unavailable
Replication Overview
![Page 20: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/20.jpg)
How can active object help?
Object GUID
- system transparent - user-directed itinerary- pre-captured access pattern- lifetime- component objects
- copy(source, dest, format, bytes)
user/application customizationflexible functionsThree ways
Object MetadataAssociated MethodData Dependent Attributes
![Page 21: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/21.jpg)
How can intelligent device help?What does intelligent device know?
Object read/write trafficReal-time device load
What can intelligent device do?inside region object replication
when and where(device) to replicatediscover neighbor devicesload balance traffic to neighbors
guide cross region object replicationwhen and where(region) to replicate
provide continue access when RM is downalleviate RM burden (more scalable inside one region)
![Page 22: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/22.jpg)
Role of RM - Speculative replication based on
“open” request- Cross region replication- Backup or mirroring - Distributed location service- Quota Control
![Page 23: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/23.jpg)
Consistency Management OverviewRecognize replicas as the same objectAllow automatic duplication, deletion, or migration of objectsSupport concurrent access of the same logical object from nearby replicasAllow caching of objects at multiple levelsOvercome intermittent connections
![Page 24: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/24.jpg)
Levels of ConsistencyObjects indicate desired consistency levelStrong consistency
global consistency at any momentusing server locking
Weak consistencyallow temporary inconsistencyusing optimistic concurrency control
Balancing in-betweenCould allow one object to split into two versions
![Page 25: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/25.jpg)
Partition of Consistency Management Functions
Regional Managers + Intelligent Storage DevicesGoals:
reducing overheadsenhancing concurrency
Approaches:fine granularity smaller than data objectshierarchical management over RM/ISD hierarchy
![Page 26: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/26.jpg)
Differential Updates of ObjectsOptimistic concurrency control generates co-existing multiple versions of objectsUpdate history has to be maintained until final reconciliationFull copy of individual versions consume too much storageExploiting incremental/differential updates
Existing scheme for text file and database app.Seeking more general support for various objects
![Page 27: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/27.jpg)
Distributed CommittingUpdates of compound object may involve several referenced objectsSuch updates have to be performed in a transaction waySuch updates also have to support versions such that rollback is possibleDetermine global serializability of distributed updates
![Page 28: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/28.jpg)
Consistency Management under Mobility
Mobile Entities:Users w/ high mobilityActive objects w/ less mobilityIntelligent storage devices w/ less mobility
Different requirementsUsers require quick hand-off when moving across adjacent regionsActive objects & Intelligent storage devices requires least interruption of other services during the moving
![Page 29: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/29.jpg)
General Picture
RMRM
OSDOSD OSDOSD
Access OAccess Oii
Certificate/ CapabilityCertificate/ CapabilityReq, Certificate
Req, CertificateOOii
OSDOSD
ClientClient RMRM
OSDOSD
KK1,1,KK22
KK11 KK22
??
![Page 30: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/30.jpg)
Security IssuesTypically file managers are involved in all security-related functions
Issuing capability, access control, revocationExcessive load on the file managersCentral point of failureAttractive target of DoS attacks
Devices are directly attached to the networkExposed to various potential attacksDoS, masquerading, replaying, other active/passive attacks
![Page 31: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/31.jpg)
Security IssuesDirect interaction between client and devices
Client needs a capability or credential from the RMObject MobilityCompound objectsDevices need to perform additional functions
Authentication, access control
Traffic analysisTraffic trends or identity of the communicating party can reveal some sensitive information
Merger of two companies, business trends of ISPProviding anonymity of traffic is important
![Page 32: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/32.jpg)
AuthenticationGeneral authentication
Client – RM, client- device, device – RM, RM – RM
Message authenticationMessages sent from and between RMsResponses from devices
Non-repudiation Preventing RM, device, client from denying their operations or commands
![Page 33: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/33.jpg)
Access ControlCredential based
ACL on the RMRM issues a credential which contains the access rights for the clientThis credential by the client to probe to the device his access rightsCredential usually short lived, frequent requests to RMRM has to be online for the entire system to work
Identity basedACL associated with the objectClient receives an identity certificate from the RMPreferred if the object is mobileRevocation difficult, changes in client’s rights results in large number of updates
![Page 34: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/34.jpg)
Access ControlCompound objects
The component objects could be independent objects with different access rights than the main objectCould be located on different device or region
![Page 35: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/35.jpg)
Confidentiality and IntegrityInsecure Internet communication
Encryption on link necessaryPerson getting physical access to the device can read and modify the objects
System administrator, device theftsEnd to End object level encryption may be required (optional)
Ensuring integrity of the traffic on the link, cache etc. is importantCompound objects
How to encrypt?Accessing one object might require more than one keys
![Page 36: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/36.jpg)
Key Management and Keyword SearchKey management
Merging different groupsRevocation of the userEncrypted objects can reside on the device for a long time
Some parts of meta-data will be encryptedHow to support search operations using this meta-dataDecrypting every time will be expensive
If client wants to retrieve objects that contain certain keywords
RM will have to decrypt all of the relevant objectsHowever, RM cannot always be trusted (E2E)Decryption of the objects will be expensive and unnecessary
![Page 37: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/37.jpg)
Data Path RequirementsHigh performance
Reduce the protocol and system software overheadQuality of service
Object data delivery adapts to network conditionMobile client
Gracefully handle the intermittent connectionWireless connection
Optimize the frequency, power level allocationRobustness
Handle failure in the data path (e.g. network failure, end system crash, etc.)
![Page 38: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/38.jpg)
Software Overhead in Data PathSoftware overhead of TCP/IP in end system is two high:
Multiple memory copiesProtocol processing overhead Context switchingInterrupt servicing
The object device has limited processing powerCPU boundMemory bound
![Page 39: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/39.jpg)
Reducing Software OverheadOSD/Secure RDMA
Use RDMA to achieve zero memory copyLeverage RDMA over TCP/IP Incorporate security with RDMA mechanismIntegrate RDMA with TOEExploit the caching for different clientsDevelop a RDMA access interface
![Page 40: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/40.jpg)
OSD/Secure RDMA Architecture
OSD Client
OSD controller
OSDVIPL
ObjectManager
Buffers
DiskDriverVI NIC
driver
OSD Device
Application
OSDVIPL
Buffers
VI NICdriver
NICNIC
IP network
![Page 41: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/41.jpg)
Quality of ServiceActive object is a special application
may have its own QoS requirementLatency variancebandwidth
Underlying network condition unpredictableTCP is a best-effort protocol, no guarantee of BWTraffic can be bursty and varied
Congestion may occurHow to provide the QoS for object in datanomic?How to adapt the QoS smoothly when the network condition degrades?How to allot and schedule the BW in a OSD device?
![Page 42: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/42.jpg)
Adaptive QoS controlCollect status info of substrate networkExchange QoS info between application and substrate networkExploit application “hint”Tailor the application level data transmission based on the underlying network behaviorNetwork level transmission also adjusts to the above application requirements
![Page 43: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/43.jpg)
MobilityMobility
Mobile clients (PDA, wireless and mobile users)Mobile objectsMobile storage devices
ChallengesConnection can be intermittentAn entity can dynamically join and leave a regionA transition may happen in the process of data transmission or critical operation
![Page 44: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/44.jpg)
Wireless ConnectionLimited resource in wireless connection
Bandwidth, error rate, power consumptionEffective resource management
Optimize frequency allocation, power levels, and beamforming of access pointsAdapt to changing traffic patterns and QoSrequirementsExploit multiple channels
![Page 45: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/45.jpg)
RobustnessPossible component failures in datanomic system
Network failureStorage device failureClient failureRegional manager failure
Maintain robustness in the face of failuresData consistency Graceful degradationRoll-back mechanisms
![Page 46: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/46.jpg)
Representative Applications
MPEG-21-compliant applications:Meeting users’ heterogeneous capabilities, dynamic bandwidth changing, & display resolution by:
Replication of objects and compound objects in different formats and resolution.Layered Multimedia representations.On-the-fly transcoding.
Media personalization and Content-based search techniques.
![Page 47: University of Minnesota · EMC, StorageTek–S torage Systems Seagate, Qlogic, – Storage components zDevelopment and demonstration of OSD zDevelopment and demonstration of OSD Active](https://reader034.fdocuments.us/reader034/viewer/2022043011/5fa522f5d205bf17ba6c400b/html5/thumbnails/47.jpg)
Representative Applications (continue)
Backup strategies ( special case of replication) that operate at multiple time scales.
Potential application in the field of the medical research and data mining:
Using the “methods” stored together with active data objects to generate requested statistics or discovering undefined correlations based on a large number of patient records without effecting the confidentiality of patient records.