Beyond JSONMaxim Zaks

@iceX33

Agenda• War Stories

• And very technical details

2007

2007

JSON vs. XML• much more human readable

• no schema

• no streaming parser

• too early for financial sector

2015

Simulation Games

Deterministic simulaiton running on mobile device

And validated on the BackEnd

Share code between BackEnd & FrontEnd

Stateless Worker

Parsing a JSON config (400kb) and game state (30kb)

takes 60 ms

Did we gave up?

No!We Google

FlatBuffersis an efficient cross platform serialization

library for C++, Java, C#, Go, Python and JavaScript (C,

PHP & Ruby in progress).

How does it work?

Define a schematable Person {

name: string;

age : int;

}

root_type Person;

Call code generator./flatc -n person01.fbs

Use generated code for readingpublic sealed class Person : Table {

public static Person GetRootAsPerson(ByteBuffer _bb) {...}

public static Person GetRootAsPerson(ByteBuffer _bb, Person obj) {...}

public string Name { get { ... } }

public int Age { get { ... } }

};

byte[] data = ...;

Person person = Person.GetRootAsPerson(new ByteBuffer(data));

Suffer through generated code for writingpublic sealed class Person : Table { public static Offset<Person> CreatePerson(FlatBufferBuilder builder, StringOffset name = default(StringOffset), int age = 0) { ... }

public static void FinishPersonBuffer(FlatBufferBuilder builder, Offset<Person> offset) { ... }};

FlatBufferBuilder fbb = new FlatBufferBuilder(1);var nameOffset = fbb.CreateString("maxim");var personOffset = Person.CreatePerson(fbb, nameOffset, 34);Person.FinishPersonBuffer(fbb, personOffset);byte[] data = fbb.SizedByteArray();

What is underneath?

Why is it so fast to readAnd cumbersome to write

Call FlatBuffers compiler./flatc -b person01.fbs person01.json

And if you want binary to JSON./flatc -t person01.fbs -- person01.bin --raw-binary --strict-json

={

"name": "maxim",

"age": 34

}

FlatBuffers has evolution strategy

table Person {

name: string;

age : int;

}

root_type Person;

->table Person {

name: string;

age : int (deprecated);

birthday : double;

}

root_type Person;

table Person {

name: string;

age : int;

}

root_type Person;

->table Person {

name: string (id: 0);

birthday : double (id: 2);

age : int (deprecated, id: 1);

}

root_type Person;

It's backwards and forwards

compatiblenew code can read old data

&old code can read new data

I wanted to make writing more

usablespecially for unit testing

Eager serializationpublic sealed class Person : Table {

public string name;

public int age;

public static Person FromByteArray(byte[] data) {...}

public byte[] ToByteArray(){...}

}

// var person = new Person();

byte[] data = ...;

var person = Person.FromByteArray(data);

person.name = "Alex";

byte[] data2 = person.ToByteArray();

let person = Person(name: "Maxim", age: 34)

let fbData = person.toByteArray

NSData( bytes: UnsafePointer<UInt8>(fbData),

length: fbData.count)

.writeToFile("person.bin", atomically: true)

Facebook is using FlatBuffers !

FlatBuffersSwift benchmark• 65.015 mseconds for parsing JSON (3.6MB)

• 36.478 mseconds for eager FlatBuffers (2.1MB)

• 00.002 mseconds for lazy FlatBuffers (2.1MB)

Some rain on FlatBuffers parade• Zip JSON: 3.6MB -> 739KB

• Zip FB: 2.1MB -> 1.1MB

Outlook for FlatBuffers• real Graph not a Tree

• could support streaming

• is easy to encrypt

• could be very memory efficient with custom compression

Questions?

Thank you!@iceX33

https://google.github.io/flatbuffershttps://github.com/mzaks/FlatBuffersSchemaEditor

https://github.com/mzaks/FlatBuffersSwift