A Formal Foundation for Trace-Based JIT Compilation

A Formal Foundation for Trace-Based JIT Compilers

Maarten Vandercammen* Jens Nicolay* Stefan Marr† Joeri De Koster*Theo D’Hondt* Coen De Roover*

*Vrije Universiteit Brussel, Belgium† Johannes Kepler University Linz, Austria

*firstname.lastname@vub.ac.be † stefan.marr@jku.at

Abstract

Trace-based JIT compilers identify frequently executed pro-gram paths at run-time and subsequently record, compileand optimize their execution. In order to improve the per-formance of the generated machine instructions, JIT com-pilers heavily rely on dynamic analysis of the code. Existingwork treats the components of a JIT compiler as a monolithicwhole, tied to particular execution semantics. We propose aformal framework that facilitates the design and implemen-tation of a tracing JIT compiler and its accompanying dy-namic analyses by decoupling the tracing, optimization, andinterpretation processes. This results in a framework that ismore configurable and extensible than existing formal trac-ing models. We formalize the tracer and interpreter as twoabstract state machines that communicate through a mini-mal, well-defined interface. Developing a tracing JIT com-piler becomes possible for arbitrary interpreters that imple-ment this interface. The abstract machines also provide thenecessary hooks to plug in custom analyses and optimiza-tions.

Categories and Subject Descriptors D.3.1 [Formal Defini-tions and Theory]: semantics; D.3.4 [Processors]: compil-ers, optimization

Keywords tracing JIT compilation, operational semantics,dynamic analysis

1. Introduction

Just-in-time (JIT) compilation is a technique where, insteadof statically compiling and optimizing an entire program up-front, an execution engine observes the program’s executionand a JIT compiler emits machine code at run-time. Doing so

allows the compiler to take into account specific character-istics of the program’s execution when generating machineinstructions, such as the values or types of the expressionsthat are executed. Dynamic analysis is therefore essential tothe process of JIT compilation, since JIT compilers use thesekinds of analyses to optimize the instructions that they gen-erate at run-time [6].

The few formal models that exist on tracing compilation[2, 5] are irreversibly tied to one particular execution modelfor one particular programming language and treat the dif-ferent components of a tracing JIT compiler – interpreter,tracer, compilers, and optimizers – as a monolithic wholewith strong coupling between them. Investigating differentexecution models requires extensive changes to the languagesemantics used by these models. They are geared more to-ward exploring soundness of trace optimizations instead ofenabling experiments in dynamic analysis.

We propose a formal framework that facilitates the de-sign and implementation of a tracing JIT compiler and thedynamic analyses on which it relies by decoupling the trac-ing, optimization and interpretation processes, resulting ina complete framework that is more configurable and exten-sible than existing formal tracing models. The main benefitof our model is that it enables applying tracing compilationto, and subsequent dynamic analysis of, any arbitrary inter-preter that satisfies a small, fixed interface (Section 3.2). Ex-cept for this minimal interface, the interpreter is otherwisetreated as a black box and no particular implementation isspecified. Our model also provides the necessary hooks toplug in custom analyses and optimizations. We do not defineany concrete trace optimizations, but because of the strongdecoupling of components, trace optimizations can be addedin a modular way without being tied to any particular traceror interpreter.

2. Trace-Based JIT Compilation

Trace-based JIT compilation is a variant of JIT compilationthat builds on two basic assumptions: most of the executiontime of a program is spent in loops, and several iterationsof the same loop are likely to take the same path throughthe program [1]. Starting from these two premises, tracing

Maarten Vandercammen, Jens Nicolay, Stefan Marr, Joeri De Koster,

Theo D’Hondt, Coen De Roover

mvdcamme@vub.ac.be

Trace-Based JIT Compilation

2

(define (fac n) (if (< n 2) 1 (* n (fac (- n 1)))))

(label ‘fac-loop) (literal-value 2) (save-val) (lookup-var 'x) (save-val) (lookup-var '<) (apply-native 2) (guard-false) (literal-value 1) (save-val) (lookup-var 'x) (save-val) (lookup-var '-) (apply-native 2) . . . (goto 'fac-loop)

Trace-Based JIT Compilation

2

(define (fac n) (if (< n 2) 1 (* n (fac (- n 1)))))

(label ‘fac-loop) (literal-value 2) (save-val) (lookup-var 'x) (save-val) (lookup-var '<) (apply-native 2) (guard-false) (literal-value 1) (save-val) (lookup-var 'x) (save-val) (lookup-var '-) (apply-native 2) . . . (goto 'fac-loop)

Traces are always linear!Use guards to protect against changes in control flow

(label ‘fac-loop) (literal-value 2) (save-val) (lookup-var 'x) (save-val) (lookup-var '<) (apply-native 2) (guard-false) (literal-value 1) (save-val) (lookup-var 'x) (save-val) (lookup-var '-) (apply-native 2) . . . (goto 'fac-loop)

Existing formalisms

3

Guo and Palsberg (2011)

Dissegna et al. (2014)Framework for proving soundness of optimisations

Existing formalisms

3

Guo and Palsberg (2011)

Dissegna et al. (2014)Framework for proving soundness of optimisations

But:• Tied to specific execution model • No general description of tracing • Implicit assumptions

Existing formalisms

3

Guo and Palsberg (2011)

Dissegna et al. (2014)Framework for proving soundness of optimisations

But:• Tied to specific execution model • No general description of tracing • Implicit assumptions

No general frameworks!

Research goal

4

Tracing of arbitrary execution model

Enable dynamic analysis

and

Enable reasoning about JIT execution

and

Research goal

5

Language semantics

(interpreter)

Traced language semantics

(TJIT Compiler)

Specialises in interpretation

Specialises in tracing semantics

+ Optimisation

Approach

6

Tracing machine

Interpreter machine

Approach

6

Tracing machine

Interpreter machine

Master

Tracer/Interpreter interface

7

step : ProgramState → InterpreterReturn restart : RestartPoint × ProgramState → ProgramState optimise : Trace → Trace

Interpreter as state machine (e.g. CEK machine)And

Tracing machine

8

Normal interpretation

Trace recording

Trace optimisation

Trace execution

Loop starts [untraced]

Loop ends

Loop starts [traced]

Guard failed

Store trace

Trace finished

Tracing machine

9

Normal interpretation

Trace recording

Trace optimisation

Trace execution

Loop starts [untraced]

Loop ends

Loop starts [traced]

Guard failed

Store trace

Trace finished

Tracing machine

10

Normal interpretation

Trace recording

Trace optimisation

Trace execution

Loop starts [untraced]

Loop ends

Loop starts [traced]

Guard failed

Store trace

Trace finished

Tracing machine

11

Normal interpretation

Trace recording

Trace optimisation

Trace execution

Loop starts [untraced]

Loop ends

Loop starts [traced]

Guard failed

Store trace

Trace finished

optimise : Trace → Trace

Tracing machine

12

Normal interpretation

Trace recording

Trace optimisation

Trace execution

Loop starts [untraced]

Loop ends

Loop starts [traced]

Guard failed

Store trace

Trace finished

Tracer/Interpreter interface

13

step : ProgramState → InterpreterReturn

Trace recording

Normal interpretation

Tracer/Interpreter interface

13

step : ProgramState → InterpreterReturn

InterpreterReturn: < State2, { LLT1, … , LLT3 }, Signal (loop or no loop) >

Trace recording

Normal interpretation

State 1 State 2

Intermediate state 1

Intermediate state 2

High-level transition

Low-level transition 1

LLT 2

LLT 3

Tracing machine

14

ς0 ς1 ς2 ς3 ς4

Executing trace = Replaying LLT’s on current program state

lookup-var(<) apply-native(2) guard-false(consequent) push-continuation(…)

. . . lookup-var(<) apply-native(2) guard-false((< n 2), consequent) push-continuation(…) . . .

Trace execution

(define (fac n) (if (< n 2) 1 (* n (fac (- n 1)))))

Guard failure

15

ς0 ς1 ς2

Executing trace = Replaying LLT’s on current program state

lookup-var(<) apply-native(2) ς3’guard-false(consequent)

Trace execution

(define (fac n) (if (< n 2) 1 (* n (fac (- n 1)))))

. . . lookup-var(<) apply-native(2) guard-false((< n 2), consequent) push-continuation(…) . . .

Guard failure

16

restart : RestartPoint × ProgramState → ProgramState

ς2guard-false((< n 2),

consequent) ς3ς3’

LLT : ProgramState → ProgramState | RestartPoint

Trace execution

(if (< n 2) 1 . . .)

Guard failure

16

restart : RestartPoint × ProgramState → ProgramState

ς3’ = restart(consequent, ς2)

ς2guard-false((< n 2),

consequent) ς3ς3’

LLT : ProgramState → ProgramState | RestartPoint

Trace execution

(if (< n 2) 1 . . .)

Implementation

17

Formal semantics Implementation

Tracingmachine

Experiment

18

Scheme-like language using CESK-style interpreter

Tracingmachine

Apply transformation on set of interpreters

(match program-state . . . ((ifk condition consequent alternative) ρ σ κ)) (if condition (interpreter-return program-state (ev consequent) #f (restore-env) (pop-continuation) (guard-true alternative)) . . .)

Experimentation

19

Tracingmachine

meta-interpreter

meta-tracing JIT compiler

Tracing framework

User program

Language interpreter

Underlying runtime

ContributionLanguage semantics

(interpreter)

Traced language semantics

(TJIT Compiler)

Recording/executing traces

Handling guard failures

Hook for dynamic analysis20

ContributionLanguage semantics

(interpreter)

Traced language semantics

(TJIT Compiler)

Recording/executing traces

Handling guard failures

Hook for dynamic analysis20

But… Transformation manual

Interpreter modelled as state machine

Future work

21

Automatic transformation?

Future work

21

Current:

Local optimisation: individual traces

Future:

Global optimisation: also optimise paths

between traces

Automatic transformation?

JIT compilation

Future work

21

Current:

Local optimisation: individual traces

Future:

Global optimisation: also optimise paths

between tracesCommon framework for expressing

execution of traces and untraced code

Automatic transformation?

JIT compilation

Conclusion

22

Motivation Approach Contribution