Bear Market Pauses, While Credit Markets Continue to Crumble
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory
-
Upload
c2b2-consulting -
Category
Technology
-
view
811 -
download
1
description
Transcript of Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory
![Page 1: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/1.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
@author Jaromir Hamala, [email protected] @author(type=Twitter) @jerrinot
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the
Memory Heap
![Page 2: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/2.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Who Am I?
■ Jaromir Hamala
■ A curious developer
■ Who likes to break play with stuff
![Page 3: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/3.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Safe Harbor
This talk may contain non-sense!
![Page 4: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/4.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Facts
• Memory is Cheap
• We have a lot of data
• Data in Memory == Fastest Possible Data (apart from CPU
caches)
http://www.npr.org/blogs/thetwo-way/2012/06/06/154416480/ho-hum-dull-and-boring-are-now-a-pair
![Page 5: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/5.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
On-Heap Allocation
• CoolObject cool = new CoolObject()
• Great:
– It’s easy and usually *very* fast! • Allocation can be actually faster than in C
– GC goodies!
• Not so great:
– GC leads to Stop-the-World Pauses
– Bigger Heap -> Longer Pauses
![Page 6: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/6.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
![Page 7: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/7.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
![Page 8: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/8.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Why to go Off-Heap?
• Latency caused by GC
• Locality
• Sharing with non-Java
http://digboston.com/boston-lulz/2013/08/trolley-trollop-first-comes-love/attachment/why-god-why-kitten1/
![Page 9: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/9.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
GC Implementations
• (Old Gen) GC in HotSpot (Oracle/Sun JDK, OpenJDK)
– ParOld – STW pauses by design!
– CMS – fragmentation -> STW pause
– G1 – Immature (yet?), not fulfilling the expectations -> occasional STW pauses
![Page 10: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/10.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Latency can kill you!
![Page 11: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/11.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
What is Off-Heap?
• Area of memory not maintained by GC
– Manual memory control
• No Notion of Java object
– serialization
http://nickshell1983.wordpress.com/2010/06/29/taking-a-god-nudged-leap-of-faith/
![Page 12: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/12.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Off-Heap Options?
• Direct Memory Buffer
• sun.misc.Unsafe
• JNI
• Memory Mapped File
• ….
http://en.wikipedia.org/wiki/File:Road_to_Hell_One_Sheet.jpg
![Page 13: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/13.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Direct Memory Buffer
• ByteBuffer directBuf = ByteBuffer.allocateDirect(int noBytes);
• Part of Java NIO
• Max. 2GB / buffer
• Pure Java = Portable
• getX()/putX() methods
![Page 14: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/14.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
sun.misc.Unsafe
– long addr = unsafe.allocateMemory(noBytes)
– Not a standardized part of JDK!
– Implemented by most JVM vendors
• Sun / Oracle / IBM
– It can disappear from JDK without warning!
![Page 15: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/15.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Cassandra I.
public class NativeAllocator implements IAllocator
{
static final Unsafe unsafe;
static {
try {
Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (sun.misc.Unsafe) field.get(null);
}
catch (Exception e) {
throw new AssertionError(e);
}
}
public long allocate(long size) {
return unsafe.allocateMemory(size);
}
public void free(long peer) {
unsafe.freeMemory(peer);
}
}
![Page 16: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/16.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
JNI
• Java Native Interface
• Native library means worst portability between platforms
• JNI Overhead
• Can re-use already existing (native) allocator
![Page 17: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/17.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Cassandra II.
• Java Native Interface via JNA public class JEMallocAllocator implements IAllocator {
public interface JEMLibrary extends Library {
long malloc(long size);
void free(long pointer);
}
private final JEMLibrary library;
public JEMallocAllocator() {
library = (JEMLibrary) Native.loadLibrary("jemalloc", JEMLibrary.class);
}
public long allocate(long size) {
return library.malloc(size);
}
public void free(long peer) {
library.free(peer);
}
}
![Page 18: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/18.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Memory Mapped File
• Part of NIO
• Pure Java -> Portable
• Allows sharing data between processes
• Persisted by OS
![Page 19: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/19.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
private static int count = 1010241024; //10 MB
public static void main(String[] args) throws Exception {
RandomAccessFile memoryMappedFile = new RandomAccessFile("largeFile.txt", "rw");
//Mapping a file into memory
MappedByteBuffer out = memoryMappedFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, count);
//Writing into Memory Mapped File
for (int i = 0; i < count; i++) {
out.put((byte) 'A');
}
System.out.println("Writing to Memory Mapped File is completed");
//reading from memory file in Java
for (int i = 0; i < 10 ; i++) {
System.out.print((char) out.get(i));
}
System.out.println("Reading from Memory Mapped File is completed");
}
Javin Paul,
http://javarevisited.blogspot.com/2012/01/memorymapped-file-and-io-in-java.html
![Page 20: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/20.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Use cases for Off-Heap?
• Caches / Data Grids
– A lot of data
– Objects with well defined lifecycle
• (Extremely) Latency-Sensitive tasks
– High-Frequency trading…
• Sharing Data with non-Java
![Page 21: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/21.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Apache DirectMemory
• “…is a off-heap cache for the Java Virtual Machine”
• Easy to integrate
• Friendly License
• Experimental
• http://directmemory.apache.org/
![Page 22: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/22.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
DirectMemory Usage
CacheService<Object, Object> cacheService =
new DirectMemory<Object, Object>()
.setNumberOfBuffers( 10 )
.setSize( 1000 )
.setInitialCapacity( 100000 )
.setConcurrencyLevel( 4 )
.newCacheService();
put( K key, V value )
put( K key, V value, int expiresIn )
![Page 23: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/23.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Is Off-Heap The Future of Java?
• Oh God! Please no!
• R&D should be aimed to more effective GC
– Zing JVM (based on HotSpot, C4 - the fully concurrent GC)
– RedHat Shenandoah?
![Page 24: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/24.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Final Warning(s)
• Stay On-Heap unless you have no other choice!
• If you think you have no other choice, think twice
• Do NOT re-invent the wheel! (Memory Management)
• Measure, measure, measure!
![Page 25: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/25.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Further Sources
• http://mechanical-sympathy.blogspot.com/ – Martin Thompson. There is a brilliant mailing list as well!
• http://psy-lob-saw.blogspot.com – Nitsan Wakart
• http://vanillajava.blogspot.com – Peter Lawrey
• http://insightfullogic.com/blog/ – Richard Warburton
• http://ashkrit.blogspot.com – Ashkrit
![Page 26: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/26.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
![Page 27: Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory](https://reader034.fdocuments.us/reader034/viewer/2022052523/555c265dd8b42a09438b4cc8/html5/thumbnails/27.jpg)
© C2B2 Consulting Limited 2013
All Rights Reserved
Thank you