Ext2 On Singularity
description
Transcript of Ext2 On Singularity
![Page 1: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/1.jpg)
Scott FinleyUniversity of Wisconsin – Madison
CS 736 Project
![Page 2: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/2.jpg)
Basic, default Linux file system Almost exactly the same as FFS
◦ Disk broken into “block groups”◦ Super-block, inode/block bitmaps, etc.
![Page 3: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/3.jpg)
New from the ground up Reliability as #1 goal Reevaluate conventional OS structure Leverage advances of the last 20 years
◦ Languages and compilers◦ Static analysis of whole system
![Page 4: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/4.jpg)
Implement Ext2 on Singularity Focus on read support with caching Investigate how Singularity design impact
FS integration Investigate performance implications
![Page 5: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/5.jpg)
I have a Ext2 “working” on Singularity◦ Reading fully supported◦ Caching improves performance!◦ Limited write support
Singularity design◦ Garbage collection hurts performance◦ Reliability is good: I couldn’t crash it.
![Page 6: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/6.jpg)
1. Singularity Details2. Details of my Ext2 implementation3. Results
![Page 7: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/7.jpg)
![Page 8: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/8.jpg)
Everything is written in C#◦ Small pieces of kernel (< 5%) in assembly and C+
+ as needed Kernel and processes are garbage collected No virtual machine
◦ Compiled to native code Very aggressive optimizing compiler
![Page 9: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/9.jpg)
Singularity is a micro kernel Everything else is a SIP
◦ “Software Isolated Process” No hardware based memory isolation
◦ SIP “Object Space” isolation guaranteed by static analysis and safe language (C#)
◦ Context switches are much faster
![Page 10: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/10.jpg)
All SIP communication is via message channels
No shared memory Messages and data passed via Exchange
Heap Object ownership is tracked Zero copy data passing via pointers
![Page 11: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/11.jpg)
Application creates buffer in ExHeap
AppFile
SystemDisk
Driver
Exchange Heap
Empty Buffer
![Page 12: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/12.jpg)
Application send read request to file system◦ File system owns the buffer
AppFile
SystemDisk
Driver
Exchange Heap
Empty Buffer
![Page 13: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/13.jpg)
File system sends read request to driver◦ Driver owns the buffer
AppFile
SystemDisk
Driver
Exchange Heap
Empty Buffer
![Page 14: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/14.jpg)
Driver fills buffer and replies to file system
AppFile
SystemDisk
Driver
Exchange Heap
Full Buffer
![Page 15: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/15.jpg)
File system replies to application
AppFile
SystemDisk
Driver
Exchange Heap
Full Buffer
![Page 16: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/16.jpg)
Application consumes the buffer
AppFile
SystemDisk
Driver
Exchange Heap
![Page 17: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/17.jpg)
![Page 18: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/18.jpg)
Ext2Control: Command line application Ext2ClientManager: Manages mount points Ext2FS: Core file system functionality Ext2Contracts: Defines communication
![Page 19: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/19.jpg)
System service (SIP) launched at boot Accessible at known location in /dev
directory Does “Ext2 stuff” Operates on Ext2 volumes and mount
points Exports “Mount” and “Unmount”
◦ Would also provide “Format” if implemented 300 lines of code
![Page 20: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/20.jpg)
Command line application Allows Ext2 Client Manger interface access Not used by other applications 500 lines of code
![Page 21: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/21.jpg)
Core Ext2 file system. Separate instance (SIP) for each mount
point.◦ Exports “Directory Service Provider” interface
Clients open files and directories by attaching a communication channel◦ Internally paired with an Inode.
Reads implemented, Writes in progress 2400 Lines of code
![Page 22: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/22.jpg)
Client wants to read file “/mnt/a/b.txt”◦ Ext2 mounted at “/mnt”
1.App --CH0-->SNS: <Bind,“/mnt/a/b.txt”,CH1>2.App<--CH0-- SNS: <Reparse, “/mnt”>3.App --CH0-->SNS: <Bind,”/mnt”,CH1>4.App<--CH0-- SNS: <AckBind>
![Page 23: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/23.jpg)
5. App --CH1-->Ext2Fs: <Bind,“/a/b.txt”,CH2>6. App<--CH1-- Ext2Fs: <AckBind>7. App --CH2-->Ext2Fs: <Read, Buff, BOff, FOff>8. App<--CH2-- Ext2Fs: <ReadAck, Buff>
9. …
![Page 24: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/24.jpg)
1. Inodes: Used on every access2. Block Numbers: Very important for large
files3. Data Blocks: Naturally captures others All use LRU replacement Large files unusable without caching
◦ 8300X faster reading 350 MB file
![Page 25: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/25.jpg)
![Page 26: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/26.jpg)
Athlon 64 3200, 1 GB RAM Disk: 120GB, 7200 RPM, 2 MB buffer, PATA Measured sequential reads Varied read buffer size from 4 KB to 96 MB Timed each request File sizes ranged from 13 MB to 350 MB
![Page 27: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/27.jpg)
![Page 28: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/28.jpg)
![Page 29: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/29.jpg)
Linux is faster◦ Not clear that this is fundamental
Performance is not horrible◦ “Good enough” objective met◦ Garbage collection hurts, but not “too bad”
Not sensitive to file size
![Page 30: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/30.jpg)
System programming in a modern language System programming with no crashes Micro kernel is feasible
◦ Hurts feature integration: mmap, cache sharing◦ Clean, simple interfaces
![Page 31: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/31.jpg)
![Page 32: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/32.jpg)
![Page 33: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/33.jpg)
![Page 34: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/34.jpg)
![Page 35: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/35.jpg)
![Page 36: Ext2 On Singularity](https://reader033.fdocuments.us/reader033/viewer/2022051621/56814d4e550346895dba7ef9/html5/thumbnails/36.jpg)