CS 1651Advanced Systems Software

Jack LangeAssistant Professor

University of Pittsburgh

Course Objectives

• Understand basic OS principals• Understand historical background of OS

design• Gain practical experience with current systems– x86 and Linux

• Learn how modern systems are designed and operated

• Gain some exposure to systems research

Administrivia

• Instructor: Jack Lange– Email: jacklange@cs.pitt.edu– Office: Sennott Square #5407– Office Hours: Weds 2-4PM

Communication• Course homepage

– http://www.cs.pitt.edu/~jacklange/teaching/cs1651-f13/– Announcements, clarifications, corrections– Additional resources for projects

• Google Group– http://groups.google.com/group/pitt-cs1651-f13– pitt-cs1651-f13@googlegroups.com – Private discussion group

• Open venue for class discussions and questions

– Based on email (Pitt addresses)• Email me if you want to use a different one

Class Details

• Inverse of other graduate courses– Lectures on Tuesdays• Provide background• Supplemental readings assigned (Check Schedule!)

– Presentations on Thursdays• Student presents selected research paper• Other students read and provide a review of the paper

before hand

Grading

• Grading– Participation (20%)– Final (30%)– 4 Projects (50%)

• Late policy– Submit by midnight of the due date– 10% penalty for every day late

Projects

• Work Individually– C is required

• Highly Recommended: OS or having some familiarity with Unix systems programming, preferably in C or C++– All projects are in C– BUILDING software is 50% of the grade of this

class

1-9

Textbooks

• None are “required” but highly recommended– Linux Device Drivers– Understanding the Linux Kernel

Reading papers and evaluating systems

• What are the most important ideas: perhaps a combination of their motivations, observations, interesting parts of the design, or clever parts of their implementation.

• What are the largest flaws; maybe an experiment was poorly designed or the main idea had a narrow scope applicability. Being able to assess weaknesses as well as strengths is an important skill for this course and beyond.

• What is the relevance of the ideas today, potential future research suggested by the article, etc.

What makes a good presentation

• What is the problem?• Why should people care about the problem?• How do you intend to address it (high level)?• How did you address it (w/ lowlevel details of

interesting components)?• Why is your approach superior to others?• What are the shortcomings of your approach?• Did your approach work?

What makes a bad presentation

• Lack of context• Reading from slides• Lots of text or data• Only focusing on technical details• Ignoring high level points

Why learn about Operating Systems?

• Tangible reasons– Build or modify a real operating system– Administer and use system well– Tune application performance

• Intangibles– Intrinsic curiosity

• Understand how much of you computer system works– Gain/apply knowledge in other areas of Computer Science

• Computer architecture and devices• Synchronization in programming languages• Data structures and algorithms• Performance analysis

– Challenge of designing large, complex systems

What is an operating System

• Not easy to define:Users

ApplicationsOperating System

Hardware• OS:

– Everything that isn’t an application or hardware• OS:

– Software that abstract hardware into a useful form for applications– Standard Library– Resource Coordinator

First Function: Standard Library

• Advantages of standard library– Allow applications to reuse common facilities– Make different devices look the same– Provide higher level abstractions

• Challenges– What are the right

abstractions?

Second Function: Resource Coordinator

• Resource: “Anything valuable” – (e.g. CPU, memory, disk, network)

• Advantages of resource coordinator– Virtualize resource so multiple users/applications can share– Protect applications from one another– Provide efficient and fair access to resources

• Challenges– What mechanisms?– What policies?

What Functionality in OS?• No single right answer– Desired functionality depends on outside factors

Users & Applications

OperatingSystem

Computer Architecture

TechnologyChanges

Expectations

• OS must adapt– Change abstractions provided to users– Change algorithms to implement those abstractions– Change low-level implementation to deal with hardware

• Current operating systems driven by its evolution• Two distinct cases in history

– Case 1: Computers are expensive– Case 2: Computers are cheap

History of the OS• Commercial systems (1950s) and HPC systems (early 1990s)

– Enormous and expensive• Goal: Get the system working

– Single operator/programmer/user runs and debugs at a time• OS functionality

– Standard library -> No coordination of resources– Monitor that is always resident; transfer control to programs

OS

User JobMemory

• Problem: Inefficient use of hardware– Performance Metrics

• Throughput and Utilization

Batch Processing• Batch: Group of jobs submitted to machine together

– Operator collects jobs; orders efficiently; runs one at a time

• Role of OS: Same as before• Advantages

– Amortize setup costs over many jobs– Keeps machine busy during a single users idle time

• Disadvantage– User must wait for results until batch collected and submitted

• If bug, receive memory and register dump; submit job again

• Improve system throughput and utilization, but lose interactivity

• Still the prevalent supercomputing model

Multiprogrammed Batch Systems

• Spooling provides pool of ready jobs– Keep multiple jobs resident in memory– OS chooses which job to run– When job waits for I/O, switch to another resident job

• New OS functionality– Job scheduling policies– Memory management and protection (virtual memory)

• Advantage: Improves throughput and utilization• Disadvantage: Still not interactive

OS

User Job 1 Memory

User Job 2

User Job 2

History of the OS: Phase 2• Introduction of inexpensive, fast devices

– Keyboards and monitors text editors and interactive debuggers– New set of performance trade-offs

• Goal: True interactivity (via improved response time)

• Time-sharing:Switch between jobs to give appearance of dedicated machine

• Advantage– Users easily submit jobs and get immediate feedback

• New OS functionality– More complex job scheduling, memory management– Concurrency control and synchronization

Personal Computers

• Computers became cheap (late 70s, early 80s)– Dedicated machine per user

• “Improved” functionality from OS– Remove time-sharing of multiple jobs– No protection– No virtual memory– OS becomes subroutine again

• Conclusion: OS functionality changes with hardware and users

OS

User JobMemory

State of current systems• Large

– 100k’s to millions of lines of code– 100-1000 person-years of work

• Complex– Performance is important– Conflicting needs of different users

• Poorly understood– System outlives any of its builders– Cannot remove all bugs– Behavior is hard to predict, tuning is done by guessing

• Current trends– Multiprocessors– Networked systems– Cloud based systems

Distributed System Architecture

Distributed System Architecture

Distributed Memory Architecture

Message Passing

• Explicit communication operations– Specify sender and receiver– Serialize data into a message– Transmit message over communication channel

• Communication Channels– Sockets

• E.g. Ethernet– RDMA (Remote direct memory access)

• E.g. Infiniband– Shared memory

• Becoming more prevalent on many-core architectures