The LLVM Assembler and Machine Code Infrastructurellvm.org/devmtg/2010-11/Dunbar-MC.pdf · The LLVM...
Transcript of The LLVM Assembler and Machine Code Infrastructurellvm.org/devmtg/2010-11/Dunbar-MC.pdf · The LLVM...
Overview
• What?• Why?• How?• High-Level Design Goals• Architecture• Status
Monday, November 29, 2010
What?
• What is MC?– “Machine code”– Focus is working with “object files”
• Project started late 2009
Monday, November 29, 2010
What?
• What is MC?– “Machine code”– Focus is working with “object files”
• Project started late 2009– Enabled for production in LLVM 2.8 (Oct 2010)
Monday, November 29, 2010
Why?
• Direct object writing– Simplicity, correctness, and performance– Single source of truth
Monday, November 29, 2010
Why?
• Direct object writing– Simplicity, correctness, and performance– Single source of truth
• Advanced micro-arch optimizations
Monday, November 29, 2010
Why?
• Direct object writing– Simplicity, correctness, and performance– Single source of truth
• Advanced micro-arch optimizations• Platform for advancing low-level tools
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Code Generator
Assembly Printer (JIT)
Execution
Code (.c)
LLVM JIT
JIT Encoder
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Code Generator
Assembly Printer (JIT)
Execution
Code (.c)
LLVM JIT
JIT Encoder
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Code Generator
Assembly Printer (JIT)
Execution
Code (.c)
LLVM JIT
JIT Encoder
Duplicate Code!
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Code Generator
Assembly Printer (JIT)
Execution
Code (.c)
LLVM JIT
JIT Encoder
Duplicate Code!
No Public API!
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Standard Compiler
Code Generator
Assembly Printer (JIT)
Execution
Code (.c)
LLVM JIT
JIT Encoder
Duplicate Code!
No Public API!
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Modern Compiler
Monday, November 29, 2010
How?
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Modern Compiler
Monday, November 29, 2010
How?
Code Generator
MCized Printer
Code (.c)
Modern Compiler
MCStreamer
AsmPrinter ObjectWriter
Assembly (.s) Object File (.o)
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
– Test components in isolation
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
– Test components in isolation• Flexibility
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
– Test components in isolation• Flexibility
– Many uses for each MC component
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
– Test components in isolation• Flexibility
– Many uses for each MC component• Pluggable Targets
Monday, November 29, 2010
High-Level Design Goals
• Reuse• Performance
– No redundant effort• Testability
– Test components in isolation• Flexibility
– Many uses for each MC component• Pluggable Targets• Non-pluggable Object Formats
Monday, November 29, 2010
How is MC Used?
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Code Generator
JIT
Execution!
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!
It’s a JIT!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Code Generator
JIT
Execution!
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!
It’s a JIT!
(with inline asm support!)
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Code Generator
JIT
Execution!
Code (.c)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!
It’s a JIT!
(with inline asm support!)
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Code Generator
JIT
Execution!
Code (.c) Object File (.o)
Disassembler
Assembly Printer
Assembly (.s)
Monday, November 29, 2010
How is MC Used?
It’s an Assembler!
It’s a Compiler!
It’s a Compilassem
bler!
It’s a JIT!
(with inline asm support!)
It’s a Disassembler!
Object Writer
Assembler Parser
Assembly (.s)
Object File (.o)
Code Generator
Assembly Printer
Assembly (.s)
Code (.c)
Code Generator
Object Writer
Object File (.o)
Code (.c)
Code Generator
JIT
Execution!
Code (.c) Object File (.o)
Disassembler
Assembly Printer
Assembly (.s)
Monday, November 29, 2010
How is MC Used?
Assembler Parser
Assembly (.s)
Code Generator
Code (.c)
Code Generator
Code (.c)
Code Generator
Code (.c) Object File (.o)
Disassembler
Object Writer
Object File (.o)
Assembly Printer
Assembly (.s)
Object Writer
Object File (.o)
JIT
Execution!
Assembly Printer
Assembly (.s)
Monday, November 29, 2010
How is MC Used?
MCStreamer MCInst
Assembler Parser
Assembly (.s)
Code Generator
Code (.c)
Code Generator
Code (.c)
Code Generator
Code (.c) Object File (.o)
Disassembler
Object Writer
Object File (.o)
Assembly Printer
Assembly (.s)
Object Writer
Object File (.o)
JIT
Execution!
Assembly Printer
Assembly (.s)
Monday, November 29, 2010
How is MC Used?
MCStreamer MCInst
Assembler Parser
Assembly (.s)
Code Generator
Code (.c)
Code Generator
Code (.c)
Code Generator
Code (.c) Object File (.o)
Disassembler
Object Writer
Object File (.o)
Assembly Printer
Assembly (.s)
Object Writer
Object File (.o)
JIT
Execution!
Assembly Printer
Assembly (.s)
Monday, November 29, 2010
MCStreamer
• Core MC Component– Programmatic Assembler API– Best explained by example
Monday, November 29, 2010
MCStreamer
• Core MC Component– Programmatic Assembler API– Best explained by example
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... Out.SwitchSection(Ctx.getMachOSection(...));
...
}
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... Out.EmitSymbolAttribute(Ctx.LookupSymbol(“_main”), MCSymbolAttr::MCSA_Global); ...
}
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... Out.EmitValueToAlignment(4, 0x90);
...
}
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... Out.EmitLabel(Ctx.LookupSymbol(“_main”));
...
}
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... Out.EmitLabel(Ctx.LookupSymbol(“_main”));
...
}
Monday, November 29, 2010
MCStreamer
void foo(MCStreamer &Out, MCContext &Ctx) { ... MCInst I = { ??? }; Out.EmitInstruction(I); ...
}
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands– Operands:
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands– Operands:
– Registers
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands– Operands:
– Registers– Immediates (constants and expressions)
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands– Operands:
– Registers– Immediates (constants and expressions)– Floating point immediates
Monday, November 29, 2010
MCInst
• Second major MC abstraction• MCInst is a simple representation of a machine instruction
– Consists of opcode and operands– Operands:
– Registers– Immediates (constants and expressions)– Floating point immediates
– Affords simple C API
Monday, November 29, 2010
The llvm-mc tool
• llvm-mc is the command line tool for testing MC– Includes assembler, object file writer, and disassembler
Monday, November 29, 2010
The llvm-mc tool
• llvm-mc is the command line tool for testing MC– Includes assembler, object file writer, and disassembler
• Can use it to show encoding and MCInst structure
Monday, November 29, 2010
The llvm-mc tool
• llvm-mc is the command line tool for testing MC– Includes assembler, object file writer, and disassembler
• Can use it to show encoding and MCInst structure
$ llvm-mc --show-inst t.s! pushl! %ebp ## <MCInst #2044 PUSH32r ## <MCOperand Reg:44>>
Monday, November 29, 2010
The llvm-mc tool
• llvm-mc is the command line tool for testing MC– Includes assembler, object file writer, and disassembler
• Can use it to show encoding and MCInst structure
$ llvm-mc --show-inst t.s! pushl! %ebp ## <MCInst #2044 PUSH32r ## <MCOperand Reg:44>>
$ llvm-mc --show-encoding t.s! pushl! %ebp ## encoding: [0x55]
Monday, November 29, 2010
Instruction Matching
• Ties together the parsed instruction with target .td files
Monday, November 29, 2010
Instruction Matching
• Ties together the parsed instruction with target .td files• Uses a custom tblgen backend to generate match tables
Monday, November 29, 2010
Instruction Matching
• Ties together the parsed instruction with target .td files• Uses a custom tblgen backend to generate match tables
... { X86::PUSHF16, "pushfw", Convert, { }, 0 }, { X86::PUSH32r, "pushl", Convert__Reg1_0, { MCK_GR32 }, 0 }, { X86::PUSH32rmr, "pushl", Convert__Reg1_0, { MCK_GR32 }, 0 }, { X86::PUSHCS32, "pushl", Convert, { MCK_CS }, Feature_In32BitMode }, ...
Monday, November 29, 2010
Instruction Matching
• Ties together the parsed instruction with target .td files• Uses a custom tblgen backend to generate match tables
... { X86::PUSHF16, "pushfw", Convert, { }, 0 }, { X86::PUSH32r, "pushl", Convert__Reg1_0, { MCK_GR32 }, 0 }, { X86::PUSH32rmr, "pushl", Convert__Reg1_0, { MCK_GR32 }, 0 }, { X86::PUSHCS32, "pushl", Convert, { MCK_CS }, Feature_In32BitMode }, ...
Monday, November 29, 2010
Current Status
• Integrated assembler is default for X86 for Darwin• Lots of testing and qualification for X86
Monday, November 29, 2010
Current Status
• Integrated assembler is default for X86 for Darwin• Lots of testing and qualification for X86• ELF/X86-64 support is done
– On by default in Clang top-of-tree
Monday, November 29, 2010
Current Status
• Integrated assembler is default for X86 for Darwin• Lots of testing and qualification for X86• ELF/X86-64 support is done
– On by default in Clang top-of-tree• COFF support is well underway
– Passes many programs in LLVM test-suite repository
Monday, November 29, 2010
Current Status
• Integrated assembler is default for X86 for Darwin• Lots of testing and qualification for X86• ELF/X86-64 support is done
– On by default in Clang top-of-tree• COFF support is well underway
– Passes many programs in LLVM test-suite repository• ARM support is ongoing
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
i386.c insn-emit.c combine.c insn-recog.c0
1
2
3
4
Com
pile
Wal
l Tim
e (s
)
System Assembler Integrated Assembler
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
i386.c insn-emit.c combine.c insn-recog.c0
1
2
3
4
0.790.91Com
pile
Wal
l Tim
e (s
)
System Assembler Integrated Assembler
15%
Fast
er!
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
i386.c insn-emit.c combine.c insn-recog.c0
1
2
3
4
0.790.91 1.001.22
Com
pile
Wal
l Tim
e (s
)
System Assembler Integrated Assembler
15%
Fast
er!
22%
Fast
er!
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
i386.c insn-emit.c combine.c insn-recog.c0
1
2
3
4
0.790.91 1.001.22
0.790.89Com
pile
Wal
l Tim
e (s
)
System Assembler Integrated Assembler
15%
Fast
er!
22%
Fast
er!
13%
Fast
er!
Monday, November 29, 2010
Current Status: Performance
• Example numbers from SPECCPU’s 403.gcc– clang with and without -integrated-as– Using -O0 -g for i386
i386.c insn-emit.c combine.c insn-recog.c0
1
2
3
4
0.790.91 1.001.22
0.790.89
2.37
3.02
Com
pile
Wal
l Tim
e (s
)
System Assembler Integrated Assembler
15%
Fast
er!
22%
Fast
er!
13%
Fast
er!
27%
Fast
er!
Monday, November 29, 2010
Summary
• Good compile-time improvements• Reduced system complexity• Many new tools and opportunities
Monday, November 29, 2010
Summary
• Good compile-time improvements• Reduced system complexity• Many new tools and opportunities• What’s next?
Monday, November 29, 2010
Summary
• Good compile-time improvements• Reduced system complexity• Many new tools and opportunities• What’s next?
– JIT needs to be converted
Monday, November 29, 2010
Summary
• Good compile-time improvements• Reduced system complexity• Many new tools and opportunities• What’s next?
– JIT needs to be converted– User-level disassembler
Monday, November 29, 2010