1

JIFL: JIT Instrumentation Framework for Linux

Marek OlszewskiAdam CzajkowskiKeir Mierle

University of Toronto

2

Instrumenting Operating Systems

Operating systems are growing in complexity

Becoming harder to understand

Kernel instrumentation is a well know method of combating this problem

Used for: debugging, profiling, monitoring, coverage testing, security auditing...

Dynamic instrumentation is especially useful

No recompilation & no reboot

Good for debugging systemic problems

Feasible in production settings

3

Dynamic Instrumentation

All dynamic instrumentation tools for operating systems are probe based

Overwrite existing code with jump/trap instructions

Efficient on fixed length architectures

Slow on variable length architectures

Must use trap instruction (and hash table lookup)

JIT-based instrumentation can be more efficient

Proven itself for user space (Pin, Valgrind)

Probe-based instrumentation is seldom used in user-space

4

Probe-based Instrumentation

OS Code InstrumentationCode

InstrumentationCode

Trap Handler

1. Look up which instrumentation to call

2. Call instrumentation3. Emulate overwritten

instruction

5

JIT Instrumentation

JIT instrumentation rewrites the OS code with calls to instrumentation code Creates a duplicate instrumented copy of the OS

Called the code-cache

Since instrumentation is dynamic, it is not feasible to rewrite the entire operating system up-front Instrumentation is performed just-in-time, basic

block by basic block, right before each new basic block is executed.

The resulting code is fast No hash table lookup required

Though there is some cost in executing in the code-cache

6

JIT Instrumentation

OS Code InstrumentationCode

InstrumentationCode

Duplicate Copy of OS

7

Software Architecture

KERNEL SPACE

JIFL (Loadable Kernel Module)

Code Cache

Runtime System

JIT compiler

Dispatcher

ModifiedSyscall

JIFL Plugin(Loadable Kernel Module)

JIFL Instrumentation API

Linux KernelSystem Call

Code

USER SPACE

JIFL Plugin Starter

10

Performance Evaluation

24012241

2093

495

40

4641

0

1000

2000

3000

4000

5000

Baseline Jifl: NoInstrumentation

Jifl: Call Trace Jifl: BB Counting Kprobe: CallTrace

Kprobe: BBCounting

Re

qu

est

s / S

eco

nd

Apache Web Server Throughput

11

Conclusions

JIT instrumentation viable for operating systems

Fine grained instrumentation now possible for kernel space on variable length architectures

Intel’s x86

AMD’s AMD64

Great performance

Though it comes with a fixed cost