FlashAn efficient and portable Web server

Today’s paper, FLASH

• Quite old (1999)

• Reading old papers gives us lessons• We can see which solution among all proposed solutions is

more widely adopted• We can understand how HW environment affects SW solutions,

by comparing old days and now days

SPED Web servers in 1999

• Logical flow of a typical Web server

• Single-process event-driven (SPED)• Excellent performance for in-memory cached files• Poor performance for files on disks

• File I/O blocks the CPU

MT / MP approach

• Multi-process (MP) or multi-threaded (MT) approach• Spawn multiple SPED instances to serve HTTP requests• One can work for a new request when the others are blocked

by file I/O

FLASH

• Asymmetric Multi-Process Event Driven (AMPED) ap-proach• Disk I/O is done by separated helper processes (or threads)• Helper processes communicate with main process via IPC (e.g.

pipe)

AMPED vs MT/MP

• Memory: AMPED << MT/MP• MT/MP approach creates a thread/process for each connection

• Concurrency: AMPED > MT/MP• MT/MP’s maximum concurrency is limited by Kernel’s process

limit• AMPED’s helper process can be shared by multiple connec-

tions

Limitations of AMPED

• IPC overhead• IPC needs assist of Kernel, which introduces unnecessary

mode switching overhead

• Context switching overhead• Running multiple processes/threads incurs frequent context

switching(Actually, AMPED is better than MT/MP but not enough)

Background of design decisions

• 16 yeas ago (1999),• No AIO

Spawn a processes only for disk I/O• CPU was relatively faster than network interface

IPC overhead or context switching overhead were not critical• Memory was expensive

AMPED requires much less memory compare to MT/MP• Familiar with single-CPU core environment

No multi-core considerationEvaluation setup

FLASH (1999) mTCP (2014) Difference

# of CPU cores 1 16 x16

CPU clock speed

333 MHz 2.9 GHz x8.7

Memory 128 MB 32 GB x256

Network BW 200 Mbps 10 Gbps x50

Asynchronous File I/O

• Non-blocking file I/O• Application only requests disk read• Kernel notifies completion of disk read to application• Enables completely non-blocking single-threaded pro-

gramming but increases programming difficulty

• First presented in 1997, (As far as I know)• Proactor: An Object Behavioral Pattern for Demultiplexing and

Dispatching Handlers for Asynchronous Events• 4th annual Pattern Languages of Programming conference

Nowadays

• Status• AIO is introduced (From 2003, Linux kernel 2.6.x)

Single thread can manage everything without blocking No IPC overhead, no context switching overhead• CPU is bottleneck• Memory is cheap• Multi-core scalability is very important

• AMPED vs AIO?• Most popular Web servers still stick on MP/MT!• Apache: MT, Nginx: MP

(Both Apache and Nginx support AIO, but disabled by default)