Extensible File Systems Yong Yao CS614 May 1 st, 2001.
-
date post
20-Dec-2015 -
Category
Documents
-
view
222 -
download
5
Transcript of Extensible File Systems Yong Yao CS614 May 1 st, 2001.
![Page 1: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/1.jpg)
Extensible File Systems
Yong YaoCS614
May 1st, 2001
![Page 2: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/2.jpg)
Problems of file systems Evolution of filing services is slow
Many innovations have been proposed, but few have become widely available
![Page 3: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/3.jpg)
Why File systems are large and difficult to
implement Key part of an operating system Interact with other core services
No well-defined interface to introduce new service easily Interfaces vary from system to system
![Page 4: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/4.jpg)
Benefit with an extensible interface
Compared to the way that additional services obtained at the user level
More available services at a far faster rate
![Page 5: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/5.jpg)
Motivation Modularity
![Page 6: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/6.jpg)
Sun Network File System
A system for accessing remote files across LANs
Goal: Allow some degree of sharing among a set of independent file systems.
![Page 7: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/7.jpg)
NFS Architecture Three major layers
Unix file-system interface
VFS layer NFS protocol
layer
![Page 8: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/8.jpg)
But If interface is evolving:
Compatibility problems arise – A change to the interface requires changes to each existing filing services
If keep static: Difficult to provide new services cleanly
![Page 9: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/9.jpg)
Desirable characteristics Extensibility: Filing must be robust to both internal
and external change – change management
Stacking: add new functionality to existing services– Modularity
Coherence: Data is consistent across multiple layers– Successful design
![Page 10: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/10.jpg)
Stacking and Extensibility
A conflict between two characteristics
Stacking: layer is bounded(above and below) by the same interface
Extensibility: layer is robust to change
![Page 11: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/11.jpg)
Stackable Design Decomposed a
complex filing service into several layers
Each layers can be developed independently
![Page 12: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/12.jpg)
Decomposition Make individual
components most reusable
Each encompasses a single abstraction
Example: Disk partition Files service Directory service
![Page 13: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/13.jpg)
Possible division
Physical storage management Directory services Compression and decompression Encryption and decryption Cache management Remote-access services Replication
![Page 14: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/14.jpg)
Layer substitution Support the evolution and replacement of
layers
![Page 15: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/15.jpg)
Symmetric interface Construct complex filing services from a
number of simple layers
Interfaces are identical above and below a layer Compared to Shell programming:
The pipe mechanism provides a simple byte-stream of data
ls -l | wc
![Page 16: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/16.jpg)
Bypass routine Interface is evolving, and any layer
can add new operations NFS: A routine for each operation
Default routine-pass unknown operations to a lower layer Handle variety of argument Metadata
![Page 17: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/17.jpg)
Nonlinear Stacking Not necessary to be a strict stack
Fan In Fan Out
![Page 18: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/18.jpg)
Address-space independence
Layers can execute in different address spaces or even on different machines
Distributed file system
![Page 19: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/19.jpg)
Transport Layer A stackable layer
that transfers operations from one address space to another
NFS: only a fixed set of operations, not extensible
Extend NFS to bypass new operations
![Page 20: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/20.jpg)
Example- replication layer of Ficus Ficus: a distributed
file system developed at UCLA
Provide a large scale replication service
Logical layer: single-copy, highly available file
Physical layer: implement the concept of a file replica
![Page 21: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/21.jpg)
Interposition Layers are
interposed between existing users of the stack and the old stack-top
Useful when operations must happen at run-time
![Page 22: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/22.jpg)
Implementation
Existing File-System Interfaces
Vnode:individual files Vfs: subtree Mounting Delayed binding
![Page 23: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/23.jpg)
Interface Extensibility Vnode interface: fix the formal definition of all
operations before kernel compilation
UCLA interface: maintain all interface definition until execution, then dynamically constructing the interface
Each one provide a list of all operations Take the union of these operations Customized to each file system
![Page 24: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/24.jpg)
Stack Creation Mounting for layer construction
Instantiate a new UFS from /layer/ufs/crypt.raw
Create an encryption layer(/usr/data) on top of the lower layer (/ayer/ufs/crypt.raw)
![Page 25: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/25.jpg)
Stack Data Caching
Individual stack layers cache data to improve performance
A cache manager coordinate page caching to keep consistent. Page-naming policy
[stack identifier, file offset]
![Page 26: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/26.jpg)
Performance
Layer overhead compared to monolithic file system
Facilitate file-system development
Compatibility problems
![Page 27: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/27.jpg)
Layer Performance Interface Performance
Compare a kernel supporting UCLA interface with a standard kernel
![Page 28: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/28.jpg)
Multiple-Layer Performance Null layers: forward all operations to the next
layer of the stack
![Page 29: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/29.jpg)
Object-Orientation and Stacking
Strong parallels exist between “object-oriented” design and Stacking Data encapsulation Late binding vs. run time stack
configuration Inheritance vs. bypass routine
![Page 30: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/30.jpg)
Conclusion
Focus: Improve the file-system development process
Stacking: build new services on old system
Extensible interface: add new services without invalidating existing work
![Page 31: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/31.jpg)
The Echo Distributed File System
Andrew D.Birrell, Andy Hisgen, Chuck Jerian, Timothy Mann, and Garret Swart
![Page 32: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/32.jpg)
Motivation
Virtues of centralized systems Easy sharing of data Centralized administration Simple failure model
Virtues of distributed systems Fault tolerance Scalability
![Page 33: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/33.jpg)
Properties
Global naming Global access Global security Fault Tolerance Global Scale
![Page 34: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/34.jpg)
Global naming Each file has a name which has
the same meaning to every users. NFS: unix2: /usr/shared/dir1
unix1: /usr/local/dir1
unix1
local
usr
shared
usr
unix2
dir1
unix1
local
usr
dir1
![Page 35: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/35.jpg)
Echo name space
Single global root Arc is independent of the node Volumes: a sub-tree of name space Junction: a leaf in one volume,
pointer to a further volume
![Page 36: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/36.jpg)
Volume classes
Global root volume: DNS provides world-wide service
Name service volume: Echo name server
Echo Filestore volume Store most of the files and directories
![Page 37: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/37.jpg)
Example: Resolve /-/com/dec/src/x/y/p/q
![Page 38: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/38.jpg)
Why two kinds of volumes Trade off between consistency and
availability High level: change quite rarely,
availability is more important Low level: change rapidly
Use different algorithms
![Page 39: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/39.jpg)
Global Access
Goal: clients see the same data Echo file servers use a replication
scheme with tight consistency Key point: reduce the load on
servers Write behind cache policy with
token management
![Page 40: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/40.jpg)
Fault Tolerance Replicate data storage to improve
reliability – two copies are enough
Replicate storage access paths(Name service volumes) to improve availability
Elect a primary by majority voting scheme
![Page 41: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/41.jpg)
Overview
Performance was better than most NFS
Global naming was satisfactory
Reasonable availability
![Page 42: Extensible File Systems Yong Yao CS614 May 1 st, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062407/56649d485503460f94a247ca/html5/thumbnails/42.jpg)