RRDtool - how to make it sit up and beg?

Tobias Oetiker

OETIKER+PARTNER AG

GoOpen 09

about rrdtool

I a database with controlled memory lossI data is only an approximation of realityI quick setup and useI graphics included

designed for speed

HEADER

RRA 05 Minutes Average

RRA 160 Minutes Average

RRA 260 Minutes Maximum

RRA 310 Minutes Average

HEADER

RRA 05 Minutes Average

RRA 160 Minutes Average

RRA 260 Minutes Maximum

RRA 310 Minutes Average

HEADER

RRA 05 Minutes Average

RRA 160 Minutes Average

RRA 260 Minutes Maximum

RRA 310 Minutes Average

HEADER

RRA 05 Minutes Average

RRA 160 Minutes Average

RRA 260 Minutes Maximum

RRA 310 Minutes Average

10:45:00 10:50:00 10:55:00 11:00:00

diy performance improvements

I rrd is disk boundI stripingI nvramI tmpfs and copy

scientific approach

I Dave Plonka, LISA’07I vm optimization kills rrdI fix read-ahead with fadvise randomI batch updates of a single rrdI running 320k RRD updates every 5 Minutes

Linux 2.4Short RA

Linux 2.6Short RA

Linux 2.6FadviseRandom

buffer cache is kingI empirical evidence shows cache helpsI no-cache: double the workI worse: writes gets blocked by read

Disk Block

Disk Block

Read+WriteModify Disk Block Disk Block Disk Block

Disk Block

Cached Block

WriteModify Disk Block Disk Block Disk Block

active buffer cache management with fadvise

I fadvise and madviseI RANDOM - no read aheadI DONTNEED - drop synced data from cacheI highly implementation dependentI linux >= 2.6.18 gets it rightI keep only the hot blocks

Next Data Access Hot Blocks

features of RRDtool 1.3

I active buffer cache management with fadvise and madviseI memory mapped io (bernhard fischer)I holt winters with moving baseline (evan miller)I cairo and pango for graphicsI pdf, svg, eps outputI inline text formattingI anti-aliasing controllable

rrd cache daemon

I multiple updates to the same rrd are the fastestI cache daemon batches updatesI journal replay for crash caseI remote updates (no auth)

rrdtool update b.rrd N:24

rrdtool update c.rrd N:1234

rrdtool update a.rrd N:12

rrdtool update g.rrd N:212

rrdtool update b.rrd N:24

rrdtool update c.rrd N:1234

rrdtool update a.rrd N:12

rrdtool update g.rrd N:212

RRDCacheDaemon

Batch Updates

Journal

Disk

Disk

features of RRDtool 1.4

I beta out nowI rrd cache daemon (florian forster and kevin brintnall)I libdbi integration for instant db access (martin sperl)I graph prediction functions (martin sperl)I graph legend placement (melchior rabe)I inline text formatting

features of RRDtool 1.5

I portable data formatI remote graphing through rrd cache daemonI authentication for rrdcachedI ...

?

Tobi Oetiker <tobi@oetiker.ch>

Graph Critic - HTTP Cache Traffic

graph by Charles Glass

Graph Critic - RTT of MPLS VPN endpoints

graph by Bruno Ciscato

Graph Critic - System Information

graph by kmindi

Graph Critic - Energy Mix

graph by Lutz Schulze

Graph Critic - Thermostat Indoor / Outdoor

graph by Andres Brownworth