Efficient Code Cache Management for Dynamic Multi-Tiered...
Transcript of Efficient Code Cache Management for Dynamic Multi-Tiered...
![Page 1: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/1.jpg)
Efficient Code Cache Management for Dynamic Multi-Tiered Compilation SystemsTobias Hartmann, ETH Zurich, Oracle Corp.Albert Noll, Oracle CorporationThomas R. Gross, ETH Zurich
![Page 2: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/2.jpg)
2
Introduction
Optimized code
Instrumented code
FreeCodeCache
![Page 3: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/3.jpg)
3
Outline
Hotspot™ JVM
Design
Implementation
Evaluation
Conclusion
![Page 4: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/4.jpg)
4
Hotspot™ JVM
![Page 5: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/5.jpg)
5
Dynamic compilation in the JVM
![Page 6: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/6.jpg)
6
History
JDK 6 JDK 7 / 8
compiled code
VM internals
non-profiled code
profiled code
JDK 9 / Future
GPU code
… ?
CodeCache
...
...
CodeCache
Sweeper AOT code
CodeCache
![Page 7: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/7.jpg)
7
Code cache
Central component
Continuous chunk of memory Fixed size Bump pointer allocation with free list
CodeCache
Compilerthreads
GC
Sweeper
Runtime
Serviceability Debugging
![Page 8: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/8.jpg)
8
Challenges
With tiered compilation amount of code increased by 2-4 X
All code in one cache Different types and characteristics Access to specific code: full iteration
Code cache fragmentation
![Page 9: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/9.jpg)
9
Challenges
With tiered compilation amount of code increased by 2-4 X
All code in one cache Different types and characteristics Access to specific code: full iteration
Code cache fragmentation
Solution: Segmented Code Cache
![Page 10: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/10.jpg)
10
Properties of compiled code
Lifetime
Size
Cost of generation
![Page 11: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/11.jpg)
11
Types of compiled code
Non-method code
Profiled method code Instrumented (C1) Limited lifetime
Non-profiled method code Highly optimized (C2) Long lifetime
![Page 12: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/12.jpg)
12
Code cache fragmentation
profiled code
non-profiled code
freeCodeCache
![Page 13: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/13.jpg)
13
Hotness of code
CodeCache
profiled code
non-profiled code
free
![Page 14: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/14.jpg)
14
Segmented code cache
Dividing code cache into distinct segments
![Page 15: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/15.jpg)
15
Dynamic resizing
Allowing the segments to resize
![Page 16: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/16.jpg)
16
Implementation
Two prototype implementations Fully functional With and without resizing
Corner cases Small code cache sizes Different compiler configurations Code cache sweeper
Several optimizations possible
![Page 17: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/17.jpg)
17
Code cache fragmentation
profiled code
non-profiled code
free
![Page 18: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/18.jpg)
18
Hotness of code
non-profiled codeprofiled code
![Page 19: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/19.jpg)
19
Performance evaluation
Segmented code cache
Segmented code cache with dynamic resizing
Hardware setup 4 Intel Xeon E7-4830 CPUs at 2.13 GHz with 24 MB cache 64 GB main memory
![Page 20: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/20.jpg)
20
Instruction TLB
![Page 21: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/21.jpg)
21
Instruction TLB- 19 %
![Page 22: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/22.jpg)
22
Instruction TLB (long running)
![Page 23: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/23.jpg)
23
Instruction TLB (long running)- 44 %
![Page 24: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/24.jpg)
24
Instruction cache
![Page 25: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/25.jpg)
25
Instruction cache- 14 %
![Page 26: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/26.jpg)
26
Sweep time
![Page 27: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/27.jpg)
27
Sweep time- 46 %
![Page 28: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/28.jpg)
28
Execution time
Octane Specjbb2005 Specjvm2008 Javac0
1
2
3
4
5
6
7
Benchmark
Improvement in %
![Page 29: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/29.jpg)
29
Evaluation summary
Performance improvement for regular sizes Execution time: up to 6% Sweep time: up to 46% Fragmentation: up to 98% iTLB and iCache miss rates: up to 44%, 14%
Resizing does not pay off
Only enable segmentation with Tiered compilation Large code cache (> 240 MB)
![Page 30: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/30.jpg)
30
Conclusion
Organization of code cache important Code locality Fragmentation
Impact on overall performance
Fully integrated into latest version Including tool support Integration into JDK 9 in process
![Page 31: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/31.jpg)
31
Future work
Separation of code and metadata
Fine grained sweeping Sweep profiled code heap more often
Code heap partitioning
Heterogeneous code More code heaps
![Page 32: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/32.jpg)
32
Thank you for your attention!
http://openjdk.java.net/jeps/197
![Page 33: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/33.jpg)
33
Related work
Java Virtual Machines Jikes RVM Maxine JVM Dalvik JVM
Dynamic Binary Translators Generational code cache [Hazelwood and Smith]
Garbage collectors
![Page 34: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/34.jpg)
34
Resizing of code heaps
![Page 35: Efficient Code Cache Management for Dynamic Multi-Tiered ...cr.openjdk.java.net/~thartmann/talks/2014-Efficient_Code...Efficient Code Cache Management for Dynamic Multi-Tiered Compilation](https://reader033.fdocuments.us/reader033/viewer/2022060823/609c7796356f730f0261369a/html5/thumbnails/35.jpg)
35
Resizing of code heaps