QoS Support in Operating Systems

67
QoS Support in Operating Systems Banu Özden Bell Laboratories [email protected]

description

QoS Support in Operating Systems. Banu Özden Bell Laboratories [email protected]. Vision. Service providers will offer storage and computing services through their distributed data centers connected with high bandwidth networks to globally distributed clients. - PowerPoint PPT Presentation

Transcript of QoS Support in Operating Systems

Page 1: QoS Support in Operating Systems

QoS Support in Operating Systems

Banu Özden

Bell Laboratories

[email protected]

Page 2: QoS Support in Operating Systems

VisionVision

• Service providers will offer storage and computing servicesService providers will offer storage and computing services• through their distributed data centers through their distributed data centers • connected with high bandwidth networksconnected with high bandwidth networks• to globally distributed clients. to globally distributed clients.

• Clients will access these services via diverse devices and Clients will access these services via diverse devices and networks, e.g.:networks, e.g.:• mobile devices and wireless networks,mobile devices and wireless networks,• high-end computer systems and high bandwidth high-end computer systems and high bandwidth

networks. networks. • These services will become utilities (e.g., storage utility, These services will become utilities (e.g., storage utility,

computing utility).computing utility).• Eventually resources will be exchanged and traded Eventually resources will be exchanged and traded

between geographically dispersed data centers to address between geographically dispersed data centers to address fluctuating demand. fluctuating demand.

Page 3: QoS Support in Operating Systems

Eclipse/Eclipse/BSDBSD::an Operating System with Quality of Service Supportan Operating System with Quality of Service Support

Banu Ö[email protected]

Page 4: QoS Support in Operating Systems

MotivationMotivation• QoS support for (server) applications:QoS support for (server) applications:

• web serversweb servers• video serversvideo servers

• Isolation and differentiation of differentIsolation and differentiation of different• entities serviced on the same platformentities serviced on the same platform• applications running on the same platformapplications running on the same platform

• QoS requirements:QoS requirements:• client-basedclient-based• service-basedservice-based• content-basedcontent-based

Page 5: QoS Support in Operating Systems

Design GoalsDesign Goals• QoS support in a general purpose operating systemQoS support in a general purpose operating system• Remain compatible with the underlying operating systemRemain compatible with the underlying operating system• QoS parameters:QoS parameters:

• IsolationIsolation• DifferentiationDifferentiation• FairnessFairness• (Cumulative) throughput(Cumulative) throughput

• Flexible resource managementFlexible resource management• capable of implementing a large set of capable of implementing a large set of

provisioning needsprovisioning needs• supports a large set of server applications supports a large set of server applications

without imposing significant changes to their without imposing significant changes to their designdesign

Page 6: QoS Support in Operating Systems

Talk OutlineTalk Outline

• SchedulersSchedulers• Reservation File System (reservfs)Reservation File System (reservfs)• TaggingTagging• Web Server ExperimentsWeb Server Experiments• Access Control and ProfilesAccess Control and Profiles• Eclipse/BSD StatusEclipse/BSD Status• Related WorkRelated Work• Future WorkFuture Work

Page 7: QoS Support in Operating Systems

Proportional sharingProportional sharing• Generalized processor sharing (GPS) Generalized processor sharing (GPS)

weight of flow weight of flow iiservice received by flowservice received by flow i i in in set of flowsset of flows

• For any flow For any flow i i continuously backlogged in continuously backlogged in

• Thus, rate of flow Thus, rate of flow ii in is: in is:

FjtWtW

j

i

j

i ,),(),(

i

),( tWi

] , [t

] , [t

rjtFj

i

),(

] , [t

F

Page 8: QoS Support in Operating Systems

QoS GuaranteesQoS Guarantees

• Fairness Fairness

• ThroughputThroughput

• Packet delayPacket delay

Page 9: QoS Support in Operating Systems

Schedulers in EclipseSchedulers in Eclipse

• Resource characteristics differResource characteristics differ

• Different hierarchical proportional-share Different hierarchical proportional-share schedulers for resourcesschedulers for resources

• Link scheduler: WFLink scheduler: WF22QQ• Disk scheduler: YFQDisk scheduler: YFQ• CPU scheduler: MTR-LS CPU scheduler: MTR-LS • Network input: SRPNetwork input: SRP

Page 10: QoS Support in Operating Systems

server

company Bcompany A0.8 0.2

page 1 page 2

0.5 0.5

Hierarchical GPS ExampleHierarchical GPS Example

server

company Bcompany Apage 1

company Apage 2

0.4 0.4 0.2

hierarchical proportional sharing proportional sharing

Page 11: QoS Support in Operating Systems

SchedulersSchedulers• Hierarchical proportional-sharing (GPS)Hierarchical proportional-sharing (GPS)

descendant queue nodes of node n descendant queue nodes of node n serviced received by scheduler node serviced received by scheduler node

n n inin set of immediate descendant set of immediate descendant

nodes of the nodes of the parent of node n parent of node n

• For any node n continuously backlogged in For any node n continuously backlogged in n

m

m

n

n FmtWtW ,),(),(

queues(n)

queues(n)

),(),(i

in tWtW

] , [t

] , [t nF

Page 12: QoS Support in Operating Systems

Link Aggregation Link Aggregation • Need to incrementally scale bandwidthNeed to incrementally scale bandwidth• Resource aggregation is emerging as a solution:Resource aggregation is emerging as a solution:

• Grouping multiple resources into a single logical Grouping multiple resources into a single logical unitunit

• QoS over such aggregated links?QoS over such aggregated links?

link schedulerlink

scheduler

Page 13: QoS Support in Operating Systems

Multi-Server ModelMulti-Server Model• Multi Server Fair Queuing (MSFQ)Multi Server Fair Queuing (MSFQ)

• A packetized algorithm for a system with N A packetized algorithm for a system with N links, each with a bandwidth of r, that links, each with a bandwidth of r, that approximates a GPS system with a single link approximates a GPS system with a single link with Nr bandwidthwith Nr bandwidth

Nr

GPS

…r r r

MSFQ

Nr

Reference model

Packetizedscheduler

Page 14: QoS Support in Operating Systems

Sigcomm 2001

Multi-Server Model Multi-Server Model (Contd.)(Contd.)

• Goals:Goals:• Guarantee bandwidth and packet delay bounds Guarantee bandwidth and packet delay bounds

that are independent of the number of flowsthat are independent of the number of flows• Allow flows arrive and depart dynamicallyAllow flows arrive and depart dynamically• Be work-conservingBe work-conserving

• Algorithm:Algorithm:• When a server is idle, schedule the packet that When a server is idle, schedule the packet that

would complete transmission earliest under a would complete transmission earliest under a single server GPS system with a bandwidth of Nrsingle server GPS system with a bandwidth of Nr

Page 15: QoS Support in Operating Systems

MSFQ Preliminary PropertiesMSFQ Preliminary Properties

• Ordering: Ordering: a pair of packets scheduled in the order of their GPS a pair of packets scheduled in the order of their GPS finishing times may complete in reverse orderfinishing times may complete in reverse order

• GPS busy MSFQ busy, but converse is not true GPS busy MSFQ busy, but converse is not true • Non-coinciding busy periodsNon-coinciding busy periods

• Work backlog? Work backlog?

1 2

0 1 2 3 4

GPS

WFQ

time =

1a1 2

a2

serv1

2

0 1 2 3 4

GPS

MSF

Q

time =

1a1 2

a2

serv1serv2

),0(),0( WW

Multi-Server specific properties

MSF

Q

12

3

45

67

……

0 1 2 3 4 5 6 7 8 9 10

GPS

time =

1 2 3 4 5 6 7a1

a2

a3

a4

a5

a6

a7

serv1serv2serv3

Page 16: QoS Support in Operating Systems

MSFQ PropertiesMSFQ Properties• Maximum service discrepancy (buffer requirement)Maximum service discrepancy (buffer requirement)

• Maximum packet delayMaximum packet delay

• Maximum per-flow service discrepancyMaximum per-flow service discrepancy

rL

NrLN

dd ppp

max)1(

max),0(),0( NLWW ii

pd pd

Packet delay

timese

rvice

GPSMSFQ

Service discrepancytime

serv

ice

GPSi

MSFQi

),0( iW

),0( iW

max)1(),0(),0( LNWW

Page 17: QoS Support in Operating Systems

Schedulers (contd.)Schedulers (contd.)• Disk scheduling with QoSDisk scheduling with QoS

• tradeoffs between QoS and total disk performancetradeoffs between QoS and total disk performance• driver queue managementdriver queue management• queue depthqueue depth• queue orderingqueue ordering• fragmentationfragmentation

• Hierarchical YFQHierarchical YFQ• CPU scheduling with QoSCPU scheduling with QoS

• length of cpu phases are not known a priorilength of cpu phases are not known a priori• cumulative throughput cumulative throughput • Hierarchical MTR-LSHierarchical MTR-LS

Page 18: QoS Support in Operating Systems

Eclipse’s Key ElementsEclipse’s Key Elements

• Hierarchical, proportional share resource Hierarchical, proportional share resource schedulersschedulers

• Reservation, reservation file system (reservfs)Reservation, reservation file system (reservfs)

• Tagging mechanismTagging mechanism

• Access and admission control, reservation Access and admission control, reservation domaindomain

Page 19: QoS Support in Operating Systems

Reservations and SchedulersReservations and Schedulers

• (Resource)(Resource) reservations reservations • unit for QoS assignmentunit for QoS assignment• similar to the concept of a similar to the concept of a flowflow in packet in packet

scheduling scheduling • Hierarchical schedulersHierarchical schedulers

• a tree with two kinds of nodes:a tree with two kinds of nodes:• scheduler nodesscheduler nodes• queue nodesqueue nodes• each node corresponds to a reservationeach node corresponds to a reservation

• Schedulers are dynamically reconfigurableSchedulers are dynamically reconfigurable

Page 20: QoS Support in Operating Systems

• Hosting two companies’ web sites, each with two Hosting two companies’ web sites, each with two web pagesweb pages

cpu cycles

company Bcompany A0.8 0.2

network bandwidth

company Bcompany A0.8 0.2

page 1 page 2

0.5 0.5

page 1 page 2

0.5 0.5

disk bandwidth

company Bcompany A

page 1 page 2

0.5 0.5

0.8 0.2

Web Server ExampleWeb Server Example

page 1 page 2

0.5 0.5

Page 21: QoS Support in Operating Systems

Reservfs Reservfs

Web Server Video Server

ApplicationInterface

Scheduler Interface

CPU scheduler Link scheduler Disk scheduler 1

Reservation file system

Disk scheduler 2

CPU 1 CPU 1 Net 1 Net 2 Disk1 Disk2 Disk3

• We built theWe built the reservation file system reservation file system• to create and manipulate reservationsto create and manipulate reservations• to access and configure resource schedulersto access and configure resource schedulers

Page 22: QoS Support in Operating Systems

Reservfs Reservfs

• Hierarchical Hierarchical • Each Each reservation directoryreservation directory corresponds to a node at corresponds to a node at

a schedulera scheduler• Each resource is represented by a reservation Each resource is represented by a reservation

directory under /reservdirectory under /reserv

/reserv/reserv

cpucpu fxp0fxp0 da0da0fxp1fxp1

Page 23: QoS Support in Operating Systems

ReservfsReservfs

• Two types of reservation directories:Two types of reservation directories:• scheduler directoriesscheduler directories• queue directoriesqueue directories

• Scheduler directories are hierarchically Scheduler directories are hierarchically expandableexpandable

• Queue directories are not expandableQueue directories are not expandable

Page 24: QoS Support in Operating Systems

ReservfsReservfs

• Scheduler directory:Scheduler directory:• shareshare• newqueuenewqueue• newreservnewreserv• special queue: q0special queue: q0

• Queue directory:Queue directory:• shareshare• backlogbacklog

/reserv/reserv

cpucpu fxp0fxp0 ca0ca0fxp1fxp1

q0q0q0q0 q0q0r1r1

q0q0

q0q0 q1q1

share backlog

newqueue share newreserv

Page 25: QoS Support in Operating Systems

Reservfs Reservfs

Web Server Video Server

Application Interface:

Reservation file system

CPU schedulerLink scheduler Disk scheduler

CPU 1 Net 1 Net 2 Disk1 Disk2

Scheduler Interface:

Page 26: QoS Support in Operating Systems

Reservfs APIReservfs API

• Creation of a new queue/scheduler reservationCreation of a new queue/scheduler reservation

• fd=open(newqueue/newreserve,O_CREAT)fd=open(newqueue/newreserve,O_CREAT)

• fdfd of newly created share file of newly created share file

Page 27: QoS Support in Operating Systems

Creating Queue ReservationCreating Queue Reservation

backlogshare

q1

open(“newqueue”,O_CREAT)

da0

newqueueshare

newreservq0

fd=

/reserv/reserv

cpucpu fxp0fxp0 da0da0fxp1fxp1

q0q0q0q0 q0q0r1r1

q0q0

q0q0

q1q1

q1q1

Page 28: QoS Support in Operating Systems

Creating Scheduler ReservationCreating Scheduler Reservation

/reserv/reserv

cpucpu fxp0fxp0 fxp1fxp1

q0q0q0q0 q0q0r1r1

q0q0 q1q1

open(“newreserv”,O_CREAT)

fd=

da0

newqueueshare

newreservq0 q1

da0da0

q0q0 q1q1

r0sharenewqueuenewreserv

q0

r0r0

Page 29: QoS Support in Operating Systems

Reservfs APIReservfs API

• Changing QoS parametersChanging QoS parameters• writing a weight and min value to the share writing a weight and min value to the share

filefile

• Getting QoS parametersGetting QoS parameters• reading the share filereading the share file

• Getting/setting queue parametersGetting/setting queue parameters• reading/writing the backlog filereading/writing the backlog file

Page 30: QoS Support in Operating Systems

Reservfs APIReservfs API

killerbee$ cd /reservkillerbee$ ls -altotal 5dr-xr-xr-x 0 root wheel 512 Sep 15 11:37 .drwxr-xr-x 20 root wheel 512 Sep 12 21:54 ..dr-xr-xr-x 0 root wheel 512 Sep 15 11:37 cpudr-xr-xr-x 0 root wheel 512 Sep 15 11:37 fxp0dr-xr-xr-x 0 root wheel 512 Sep 15 11:37 fxp1

killerbee$ cd fxp0killerbee$ ls -alRtotal 6dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 .dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 ..-rw------- 1 root wheel 1 Sep 15 11:39 newqueue-rw------- 1 root wheel 1 Sep 15 11:39 newreservdr-xr-xr-x 0 root wheel 512 Sep 15 11:39 q0-r-------- 1 root wheel 1 Sep 15 11:39 share

./q0:total 4dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 .dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 ..-rw------- 1 root wheel 1 Sep 15 11:39 backlog-rw------- 1 root wheel 1 Sep 15 11:39 share

Command line output:

Page 31: QoS Support in Operating Systems

Reservfs APIReservfs API

killerbee$ cd q1killerbee$ ls -altotal 4dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 .dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 ..-rw------- 1 root wheel 1 Sep 15 11:39 share-rw------- 1 root wheel 1 Sep 15 11:39 backlogkillerbee$ cat share50 1000000killerbee$

killerbee$ cd r0killerbee$ ls -altotal 6dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 .dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 ..-rw------- 1 root wheel 1 Sep 15 11:39 newqueue-rw------- 1 root wheel 1 Sep 15 11:39 newreservdr-xr-xr-x 0 root wheel 512 Sep 15 11:39 q0-r-------- 1 root wheel 1 Sep 15 11:39 sharekillerbee$ echo “50 1000000” > newqueuekillerbee$ ls -altotal 6dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 .dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 ..-rw------- 1 root wheel 1 Sep 15 11:39 newqueue-rw------- 1 root wheel 1 Sep 15 11:39 newreservdr-xr-xr-x 0 root wheel 512 Sep 15 11:39 q0dr-xr-xr-x 0 root wheel 512 Sep 15 11:39 q1-r-------- 1 root wheel 1 Sep 15 11:39 share

Page 32: QoS Support in Operating Systems

Reservfs Reservfs

Web Server Video Server

Application Interface:

Reservation file system

CPU schedulerLink scheduler Disk scheduler

CPU 1 Net 1 Net 2 Disk1 Disk2

Scheduler Interface:

Page 33: QoS Support in Operating Systems

Reservfs Scheduler InterfaceReservfs Scheduler Interface

• Schedulers registers by providingSchedulers registers by providingthe following the following interface routinesinterface routines via via reservfs_registerreservfs_register():():

• init(priv)init(priv)• create(priv, parent, type) create(priv, parent, type) • start(priv, parent, type)start(priv, parent, type)• delete(priv, node)delete(priv, node)• get/set(priv, node, values, type)get/set(priv, node, values, type)

Page 34: QoS Support in Operating Systems

Reservfs ImplementationReservfs Implementation• Built via vnode/vfs interfaceBuilt via vnode/vfs interface• AA reserv{} reserv{} structure represents each reservfs filestructure represents each reservfs file• reserv{}reserv{} representing a directory contains a representing a directory contains a

pointer to the corresponding node at schedulerpointer to the corresponding node at scheduler• Scheduler independentScheduler independent• Implements garbage collection mechanismImplements garbage collection mechanism

Page 35: QoS Support in Operating Systems

Talk OutlineTalk Outline

• IntroductionIntroduction• SchedulersSchedulers• Reservation File System (reservfs)Reservation File System (reservfs)• TaggingTagging• Web Server ExperimentsWeb Server Experiments• Access Control and ProfilesAccess Control and Profiles• Eclipse/BSD StatusEclipse/BSD Status• Related WorkRelated Work• Future WorkFuture Work

Page 36: QoS Support in Operating Systems

Tagging Tagging

• A request arriving at a scheduler must be A request arriving at a scheduler must be associated with the appropriate reservationassociated with the appropriate reservation

• Each request is tagged with a pointer to a queue Each request is tagged with a pointer to a queue nodenode• mbuf{}, buf{} and proc{}mbuf{}, buf{} and proc{} are augmented are augmented

• How is a request tagged?How is a request tagged?

Page 37: QoS Support in Operating Systems

Tagging (contd.) Tagging (contd.) • For a file, its file descriptor is tagged with a disk For a file, its file descriptor is tagged with a disk

reservationreservation• For a connected socket, its file descriptor is For a connected socket, its file descriptor is

tagged with a network reservationtagged with a network reservation• For unconnected sockets, we provide a late For unconnected sockets, we provide a late

tagging mechanismtagging mechanism• Each process is tagged with a cpu reservationEach process is tagged with a cpu reservation• We associate reservations with references to We associate reservations with references to

objectsobjects

Page 38: QoS Support in Operating Systems

Default List of a Process Default List of a Process

• Default reservations of a process, one for each Default reservations of a process, one for each resourceresource

• A list of tags (pointers to queue directories)A list of tags (pointers to queue directories)• Used when a tag is otherwise not specifiedUsed when a tag is otherwise not specified• Two new files are added for each process Two new files are added for each process pid pid

in /in /proc/pidproc/pid• /proc/pid/default /proc/pid/default to represent the default listto represent the default list• /proc/pid/cdefault /proc/pid/cdefault to represent the child default to represent the child default

listlist

Page 39: QoS Support in Operating Systems

Default List of a Process (contd.)Default List of a Process (contd.)

• Reading these file returns the name of default Reading these file returns the name of default queue directories, e.g.,queue directories, e.g.,

/reserv/cpu/q1/reserv/cpu/q1/reserv/fxp0/r2/q1/reserv/fxp0/r2/q1/reserv/da0/r1/q3/reserv/da0/r1/q3

• A process, with the appropriate access rights, A process, with the appropriate access rights, can change the entries of default filescan change the entries of default files

Page 40: QoS Support in Operating Systems

Implicit Tagging Implicit Tagging • The file descriptor returned by The file descriptor returned by open(), accept()open(), accept() or or

connect()connect() is automatically tagged with default is automatically tagged with default

• The tag of the file descriptor of an unconnected The tag of the file descriptor of an unconnected socket is set to default at socket is set to default at sendto()sendto() and and sendmesg()sendmesg()

• When a process forks, the child process is tagged When a process forks, the child process is tagged with the default cpu reservationwith the default cpu reservation

Page 41: QoS Support in Operating Systems

Explicit TaggingExplicit Tagging

• The tag of a file descriptor can be set/read with The tag of a file descriptor can be set/read with new commands to new commands to fcntl():fcntl():

• F_SET_RESF_SET_RES• F_GET_RESF_GET_RES

• A new system callA new system call chcpures() chcpures() to change the cpu to change the cpu reservation of a processreservation of a process

Page 42: QoS Support in Operating Systems

Reservation DomainsReservation Domains

• Permissions of a process to use, create and Permissions of a process to use, create and manipulate manipulate reservationsreservations

• The reservation domain of a process is The reservation domain of a process is independent of its protection domainindependent of its protection domain

Page 43: QoS Support in Operating Systems

reserv Breserv A

cpu cycles0.8 0.2

network bandwidth

reserv Breserv A0.8 0.2

0.5 0.5 0.5 0.5

disk bandwidth

reserv Breserv A0.5 0.5

0.8 0.2

reserv 1 reserv 2 reserv 1 reserv 2 reserv 1 reserv2

Reservation domain

Reservation domain 2

Reservations and Reservation Reservations and Reservation DomainsDomains

1

reserv 1 reserv2

0.5 0.5

Page 44: QoS Support in Operating Systems

Reservfs Garbage CollectionReservfs Garbage Collection

• Based on reference countsBased on reference counts• every application that is using a specific node every application that is using a specific node

adds a reference on it (to the vnode)adds a reference on it (to the vnode)

• Triggered by the vnode layerTriggered by the vnode layer• when the last application finishes using the when the last application finishes using the

node this is garbage collectednode this is garbage collected

• fcntl()fcntl() available to maintain the node even if no available to maintain the node even if no references to it existreferences to it exist

Page 45: QoS Support in Operating Systems

SRP Input ProcessingSRP Input Processing

• Demultiples incoming packetsDemultiples incoming packets• before network and higher-level protocol before network and higher-level protocol

processingprocessing• Unprocesed input queue per socketUnprocesed input queue per socket• Processes input protocols in context of receiving Processes input protocols in context of receiving

processprocess• Drops packets when per-socket queue is fullDrops packets when per-socket queue is full• Avoids receive livelock Avoids receive livelock

Page 46: QoS Support in Operating Systems

Talk OutlineTalk Outline

• IntroductionIntroduction• SchedulersSchedulers• Reservation File System (reservfs)Reservation File System (reservfs)• TaggingTagging• Web Server ExperimentsWeb Server Experiments• Access Control and ProfilesAccess Control and Profiles• Eclipse/BSD StatusEclipse/BSD Status• Related WorkRelated Work• Future WorkFuture Work

Page 47: QoS Support in Operating Systems

QoS Support for Web ServerQoS Support for Web Server

• Virtual hosting with Apache server:Virtual hosting with Apache server:• separate Apache server for each virtual hostseparate Apache server for each virtual host• single Apache server for all virtual hostssingle Apache server for all virtual hosts

• Eclipse/BSD isolates and differentiates Eclipse/BSD isolates and differentiates performance of virtual hostsperformance of virtual hosts• multiple Apache servers----implicit taggingmultiple Apache servers----implicit tagging• single Apache server----explicit taggingsingle Apache server----explicit tagging

• We implemented an Apache module for We implemented an Apache module for explicit taggingexplicit tagging

Page 48: QoS Support in Operating Systems

Experimental SetupExperimental Setup• Apache Web Server:Apache Web Server:

• A multi-process serverA multi-process server• (Pre)spawns helper processes(Pre)spawns helper processes• A process handles one request at a time A process handles one request at a time • Each process calls Each process calls accept()accept() to service the next to service the next

connection requestconnection request

• HTTP clients run on five different machinesHTTP clients run on five different machines• Servers are running FreeBSD 2.2.8 or Eclipse/BSD 2.2.8 on a Servers are running FreeBSD 2.2.8 or Eclipse/BSD 2.2.8 on a

PC (266 MHz Pentium Pro, 64 MB RAM, 9 GB Seagate PC (266 MHz Pentium Pro, 64 MB RAM, 9 GB Seagate ST39173W fast wide SCSI disk)ST39173W fast wide SCSI disk)

• Machines are connected with a 10/100 Mbps Ethernet switchMachines are connected with a 10/100 Mbps Ethernet switch

Page 49: QoS Support in Operating Systems

Experiments Experiments

• Hosting two sites with two serversHosting two sites with two servers

/reserv/reserv

cpucpu fxp0fxp0 da0da0

q0q0 q1q1 qq22

q0q0 q1q1 qq22

q0q0 q1q1 qq22

Reservation domain of server 1

Reservation domain of server 2

Page 50: QoS Support in Operating Systems

CPU Intensive WorkloadCPU Intensive Workload

Page 51: QoS Support in Operating Systems

CPU Intensive WorkloadCPU Intensive Workload

Page 52: QoS Support in Operating Systems

Network Intensive WorkloadNetwork Intensive Workload

Page 53: QoS Support in Operating Systems

Disk Intensive WorkloadDisk Intensive Workload

Page 54: QoS Support in Operating Systems

Input Intensive WorkloadInput Intensive Workload

Page 55: QoS Support in Operating Systems

Input Intensive WorkloadInput Intensive Workload

Page 56: QoS Support in Operating Systems

Experiments Experiments

• Hosting virtual hosts with a single Apache serverHosting virtual hosts with a single Apache server• Four web sitesFour web sites

/reserv/reserv

cpucpu fxp0fxp0 da0da0

q0 q1 q2 q3 q4 q0 q1 q2 q3 q4 q0 q1 q2 q3 q4 q0 q1 q2 q3 q4 q0 q1 q2 q3 q0 q1 q2 q3 q4 q4

Page 57: QoS Support in Operating Systems

Apache Module for TaggingApache Module for Tagging

• Apache code not modified: module addedApache code not modified: module added

• Apache config defines which reservation to use Apache config defines which reservation to use based on “a rule”, e.g.,based on “a rule”, e.g.,• directory-baseddirectory-based• port-basedport-based

• Module uses Module uses fcntl()fcntl() and and chcpures()chcpures() for explicit for explicit tagging tagging

Page 58: QoS Support in Operating Systems

Isolating Web SitesIsolating Web Sites

Eclipse/BSD

Page 59: QoS Support in Operating Systems

Isolating Web SitesIsolating Web Sites

FreeBSD

Page 60: QoS Support in Operating Systems

Talk OutlineTalk Outline

• IntroductionIntroduction• Reservation File System (reservfs)Reservation File System (reservfs)• TaggingTagging• SchedulersSchedulers• Apache Web Server ExperimentsApache Web Server Experiments• Access Control and ProfilesAccess Control and Profiles• Eclipse/BSD StatusEclipse/BSD Status• Related WorkRelated Work• Future WorkFuture Work

Page 61: QoS Support in Operating Systems

Access ControlAccess Control

• Permissions of a process to use or modify the Permissions of a process to use or modify the objects belonging to the reservfsobjects belonging to the reservfs

• Currently, a process can use/modify Currently, a process can use/modify reservations “below” its default listreservations “below” its default list

• Soon, Eclipse/BSD will have more sophisticated Soon, Eclipse/BSD will have more sophisticated access control access control • process can have different permissions on a process can have different permissions on a

reservation (e.g., permission for tagging but reservation (e.g., permission for tagging but not for modifying)not for modifying)

• process can have permission on arbitrary set process can have permission on arbitrary set of reservationsof reservations

Page 62: QoS Support in Operating Systems

Multiple Default Lists: ProfilesMultiple Default Lists: Profiles

• Multiple default lists (profiles) simplifies explicit Multiple default lists (profiles) simplifies explicit taggingtagging

• Server applications typically serve different Server applications typically serve different entities (depending on client, content, etc.) with entities (depending on client, content, etc.) with different QoS assignments different QoS assignments

• Global list of system-wide profilesGlobal list of system-wide profiles• Profiles provide an easy way to manage and Profiles provide an easy way to manage and

share “default” reservations of different entitiesshare “default” reservations of different entities

Page 63: QoS Support in Operating Systems

Talk OutlineTalk Outline

• IntroductionIntroduction• Reservation File System (reservfs)Reservation File System (reservfs)• TaggingTagging• SchedulersSchedulers• Apache Web Server ExperimentsApache Web Server Experiments• Access Control and ProfilesAccess Control and Profiles• Eclipse/BSD StatusEclipse/BSD Status• Related WorkRelated Work• Future WorkFuture Work

Page 64: QoS Support in Operating Systems

• Derived from FreeBSD Derived from FreeBSD • 3.23.2• 2.2.82.2.8

• FreeBSD compatibleFreeBSD compatible

• Eclipse/BSD code is available at Eclipse/BSD code is available at http://www.bell-labs.com/project/eclipsehttp://www.bell-labs.com/project/eclipse including:including:• reservfsreservfs• hierarchical network schedulinghierarchical network scheduling• hierarchical disk schedulinghierarchical disk scheduling• hierarchical cpu schedulinghierarchical cpu scheduling• input schedulinginput scheduling• also, Apache module for tagging and other applications also, Apache module for tagging and other applications

Eclipse/BSD StatusEclipse/BSD Status

Page 65: QoS Support in Operating Systems

Related WorkRelated Work

• ALTQ ALTQ • good for routersgood for routers• not sufficient for QoS support in a general-not sufficient for QoS support in a general-

purpose OSpurpose OS• Resource ContainersResource Containers

• different from Reservation Domainsdifferent from Reservation Domains• limited (similar to our Profiles)limited (similar to our Profiles)• not flexible enough to specify a number of not flexible enough to specify a number of

useful provisioning needsuseful provisioning needs

Page 66: QoS Support in Operating Systems

Future workFuture work

• QoS on cluster of serversQoS on cluster of servers

• Support for fine-grained automatic taggingSupport for fine-grained automatic tagging

• More server applicationsMore server applications

• Supporting other QoS parametersSupporting other QoS parameters

• Other schedulersOther schedulers

Page 67: QoS Support in Operating Systems

Eclipse/Eclipse/BSDBSD::an Operating System with Quality of Service Supportan Operating System with Quality of Service Support

Banu Ö[email protected]