HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt...

HyLog: A High Performance Approach to Managing Disk

Layout

Wenguang Wang Yanping Zhao Rick Bunt

Department of Computer ScienceUniversity of Saskatchewan

Saskatoon, Canada

USENIX FAST 2004April 1, 2004 Wenguang Wang

April 1, 2004 USENIX FAST 2004 Wenguang Wang 2

Background

• The write performance of a storage system is impacted by– the disk characteristics

• Disk positioning time• Transfer bandwidth

– the strategy for writing• Overwrite• LFS (Log-structured File System)


Overwrite

• Idea: new data are overwritten on top of old data

• Problems: lots of time lost in disk arm positioning in workloads with small writes scattered over the disk


LFS

• Idea: new data are accumulated and written to new disk locations in large sequential transfers

• Assumptions of the disk characteristics: large sequential transfers are more efficient than small block transfers


LFS (cont.)

• Advantages– good write performance– no small write penalty on RAID-5– fast recovery– easy to support snapshot and versioning

(WAFL)

• Problems: segment cleaning is expensive– For a year 1991 disk, TPC-B workload, and 50%

disk space utilization, cleaning overhead reduces overall system throughput by 33% (Seltzer et al. USENIX’95)


Motivation

• Observation: disk sequential transfer bandwidth has improved 10x more than positioning time

• Question: how are Overwrite and LFS affected by this trend?

DEC RZ26(year 1991)

Cheetah X15 36LP

(year 2003)Diff.

Positioning time

15ms 5.6ms 2.7x

Transfer B/w 2.3MB/s 61MB/s 27x


Objective

• Revisit the performance of LFS under modern and future disks

• Evaluate the performance of LFS under disk arrays and concurrent users

• Attempt to perform better than LFS and Overwrite


Outline

• Background, Motivation, and Objective

• The analysis of LFS and Overwrite• The design of HyLog• Experimental methodology and

results• Conclusions and future work


Experimental Parameters

• Three SCSI disks– DEC RZ26 (year 1991)– Quantum atlas10k (year 1999)– Cheetah X15 36LP (year 2003)

• Page size: 8KB• Workload: uniformly distributed

random update (TPC-B)


• In Overwrite:

• In LFS:

• T1 > T2

• Segment I/O Efficiency = T1 / T2

Modeling Write Performance

Time to write N pages is T1

Time to write a segment containing N pages is T2


A Simple Scenario

• Assume the segments to be cleaned are always 80% utilized (cleaning space utilization = 80%)

• LFS requires 5 seg. reads and 4 seg. writes to reclaim a free segment

• LFS requires 10 seg. I/Os (9 seg. for cleaning, 1 seg. for new data) to write a segment

• If Segment I/O Efficiency > 10, LFS is still faster than Overwrite!


Segment I/O Efficiency

0

5

10

15

20

25

30

35

40

45

0 500 1000 1500 2000 2500 3000 3500 4000

Seg

men

t I/O

Eff

icie

ncy

Segment Size (KB)

Cheetah X15 36LP (year 2003)

Quantum atlas10k, (year 1999)

DEC RZ26, (year 1991)

33


0

5

10

15

20

25

30

35

0 0.2 0.4 0.6 0.8 1

Wri

te C

ost

Cleaning Space Utilization

Overwrite vs. LFS

Overwrite

LFScleaning

LFS hole-plugging

1999 disk

0.88


Overwrite vs. LFS

• The crossing point where LFS has the same performance as Overwrite

Year of DiskCleaning

Space Utilization

Disk Space Utilization

1991 0.52 0.74

1999 0.88 0.94

2003 0.94 0.97


Disk Access Characteristics

• In most workloads, most writes are to a small number of pages (hot pages)

• Impact of skewness on LFS performance– Most of the cleaning cost comes from

cold pages– Most of the good write performance

comes from accumulating the writes to hot pages


HyLog (Hybrid Log-structured

Approach)• Separates the disk into two

partitions: hot partition and cold partition

• Uses log-structured approach to manage the hot partition

• Uses overwrite to manage the cold partition


0

5

10

15

20

25

30

35

0 0.2 0.4 0.6 0.8 1

Proportion of Hot Pages

Disk Space Utilization 97%

0

5

10

15

20

25

30

35

0 0.2 0.4 0.6 0.8 1

Proportion of Hot Pages

Disk Space Utilization 90%

Performance Potential of HyLog

Wri

te

Co

st

Disk: year 2003, workload: 80% references are in 20% pages

Overwrite

LFS

HyLog

Overwrite

LFS

HyLog


Design of HyLog• Key design issue: page separating

algorithm– Collects page write frequencies – Finds the hot page proportion to minimize

expected write cost– Determines the threshold of write

frequency from the desired hot page proportion

– Uses the threshold to distinguish hot pages from cold pages


Evaluation Methodology

• Trace driven simulation– Overwrite, LFS, WOLF, and HyLog are

implemented– TPC-C, Email, and OLTP traces– Year 1999, 2003, and 2008 disk models – No think time between requests

• Metrics– Throughput: # I/O requests finished per

second


HyLog adjusts the hot page proportion between 35-45%

Results – HyLog Page Separating Algorithm

0

2

4

6

8

10

12

14

0 0.2 0.4 0.6 0.8 1

Thro

ughp

ut (x

1000

)

Hot Page Proportion

Fixed Hot Page ProportionHyLog

TPC-C trace with 20 users and 4 disks, 98% disk space utilization


Results – Disk Space Utilization

TPC-C trace with 20 users and 4 disks

0

10

20

30

40

50

60

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Thro

ughp

ut (x

1000

)


Overwrite (atlas10k - year'99 disk)HyLog (atlas10k - year'99 disk)

LFS (atlas10k - year'99 disk)

1999 disk


Results – Disk Type


0

10

20

30

40

50

60

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Thro

ughp

ut (x

1000

)




Overwrite (year’03 disk)

LFS/HyLog (year’03 disk)

1999 disk

2003 disk


Results – Disk Type (cont.)


0

10

20

30

40

50

60

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Thro

ughp

ut (x

1000

)








1999 disk

2003 disk

2008 disk


Results – Number of Users

TPC-C trace, disk space utilization 98%, year 1999 disk

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

5 10 15 20 25 30

Nor

mal

ized

Thr

ough

put

Number of Users

HyLog (8 disks)LFS (8 disks)


Results – Number of Disks


0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2 4 6 8 10 12 14

Nor

mal

ized

Thr

ough

put

Number of Disks

HyLog (7 users)LFS (7 users)


Results – RAID-5

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog

Thr

oug

hp

ut

RAID-0RAID-5

Disk Space Utilization 98%Disk Space Utilization 80%

TPC-C trace, year 1999 disk, 8-disk RAID-0, 9-disk RAID-5


Results – Other Traces

0

1

2

3

4

5

6

Overwrite LFS WOLF HyLog Overwrite LFS WOLF HyLog

Nor

mal

ized

Thr

ough

put

Disk Space Utilization=90%

Disk Space Utilization=98%

OLTP Email

Year 1999 disk


Conclusions

• On modern and future disks, LFS significantly outperforms Overwrite unless the disk space utilization is very high

• HyLog performs comparably to the best of Overwrite, LFS, and WOLF


Future Work

• Add fast recovery support in HyLog– All meta-data are considered as hot

pages

• Stabilize NetBSD LFS implementation and measure its performance

• Implement and evaluate HyLog in NetBSD


Nor

mal

ized

Thr

ough

put

5 10 15 20 25 30

2 4

6 8

10 12

14

0.4 0.6 0.8

1 1.2 1.4 1.6 1.8

Results – # Users and Disks


HyLogLFS

Number of Users

Number of Disks

HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt...

Documents

Transcript of HyLog: A High Performance Approach to Managing Disk Layout Wenguang Wang Yanping Zhao Rick Bunt...