1

JIFL: JIT Instrumentation Framework for Linux

Marek OlszewskiAdam CzajkowskiKeir Mierle

University of Toronto

2

Instrumenting Operating Systems Operating systems are growing in complexity

Becoming harder to understand Kernel instrumentation is a well know method of

combating this problem Used for: debugging, profiling, monitoring, coverage

testing, security auditing... Dynamic instrumentation is especially useful

No recompilation & no reboot Good for debugging systemic problems Feasible in production settings

3

Dynamic Instrumentation All dynamic instrumentation tools for operating

systems are probe based Overwrite existing code with jump/trap instructions

Efficient on fixed length architectures Slow on variable length architectures

Must use trap instruction (and hash table lookup) JIT-based instrumentation can be more efficient

Proven itself for user space (Pin, Valgrind) Probe-based instrumentation is seldom used in user-

space

4

Probe-based InstrumentationOS Code Instrumentation

Code

InstrumentationCode

Trap Handler

1. Look up which instrumentation to call

2. Call instrumentation3. Emulate overwritten

instruction

5

JIT Instrumentation JIT instrumentation rewrites the OS code with calls

to instrumentation code Creates a duplicate instrumented copy of the OS Called the code-cache

Since instrumentation is dynamic, it is not feasible to rewrite the entire operating system up-front Instrumentation is performed just-in-time, basic

block by basic block, right before each new basic block is executed.

The resulting code is fast No hash table lookup required Though there is some cost in executing in the code-

cache

6

JIT InstrumentationOS Code Instrumentation

Code

InstrumentationCode

Duplicate Copy of OS

7

Software Architecture

KERNEL SPACE

JIFL (Loadable Kernel Module)

Code Cache

Runtime System

JIT compiler

DispatcherModifiedSyscall

JIFL Plugin(Loadable Kernel Module)

JIFL Instrumentation API

Linux KernelSystem Call

Code

USER SPACE

JIFL Plugin Starter

10

Performance Evaluation

24012241

2093

495

40

4641

0

1000

2000

3000

4000

5000

Baseline Jifl: NoInstrumentation

Jifl: Call Trace Jifl: BB Counting Kprobe: CallTrace

Kprobe: BBCounting

Req

uest

s / S

econ

d

Apache Web Server Throughput

11

Conclusions JIT instrumentation viable for operating systems Fine grained instrumentation now possible for

kernel space on variable length architectures Intel’s x86 AMD’s AMD64

Great performance Though it comes with a fixed cost