Story Writing Byte Serializer in Golang

36
Story writing byte Serializer in go @dxhuy @huydx

Transcript of Story Writing Byte Serializer in Golang

Page 1: Story Writing Byte Serializer in Golang

Story writing byte Serializer in go

@dxhuy @huydx

Page 2: Story Writing Byte Serializer in Golang

Byte serializer

Page 3: Story Writing Byte Serializer in Golang

Problem space• Writting buffer control that need to persist struct

data in disk • Struct data is simple (will not change in near

future) • Program needs

• Low memory footprint • Low CPU usage

Page 4: Story Writing Byte Serializer in Golang

Bunches of options

• encoding/gob (base on encoding/binary) • gogoprotobuf • capnproto (glycerine/go-capnproto) • ugorji/go/codec • mgo.v2/bson • .....

Page 5: Story Writing Byte Serializer in Golang

Some problems • Some are overcomplex

• Cryptic error message • Some are fast, but not support all datastructure (map)

• flatbuffer (could use vector instead, but look up is not O(1))

• All libraries does some abstraction, make it hard to debug • Write to disk failed at the middle, some bytes are written,

some are not • Using library lack of fine-grained control

• You want some special behaviours for some special field • You want some special behaviours when it failed

Page 6: Story Writing Byte Serializer in Golang

So let's write your own

Page 7: Story Writing Byte Serializer in Golang

Serializer anatomy

Page 8: Story Writing Byte Serializer in Golang

type A struct { Name string BirthDay time.Time Phone string Siblings int Spouse bool Money float64 }

Example struct

Page 9: Story Writing Byte Serializer in Golang

Struct layout mattertype A struct { Name string BirthDay time.Time Phone string Siblings int Spouse bool Money float64 }

size + order

Page 10: Story Writing Byte Serializer in Golang

In general, there 2 types• Dynamic layout: just pass struct, serializer will do

everything for you • encoding/gob, encoding/json

• Library have to figure out "what type" first, than serialize later

• Fix layout: you have to tell serializer about your struct first • protobuf, capnproto, messagepack...

• Library already know type, just using code-gen to serialize

Page 11: Story Writing Byte Serializer in Golang

Dynamic layout Fix layout

Advantages

- Easy to use - Easy support nested struct... - No additional step

- Easy to optimize - Managable protocol file (.proto or .flatbuffer)

Disadvantages

- Harder to optimize - Need reflection (performance downgrade)

- Needs code generation

Page 12: Story Writing Byte Serializer in Golang

What should we use• What we have

• Fix protocol • Need low memory footprint / low CPU usage

• So I decided to have serialization method which is • Fix layout in code • But without codegen

Page 13: Story Writing Byte Serializer in Golang

func MarshalRaw(a *A, buf *bytes.Buffer) { encodeString(a.Name, buf) encodeUint64(uint64(a.BirthDay.UnixNano()), buf) encodeString(a.Phone, buf) encodeUint64(uint64(a.Siblings), buf) encodeBool(a.Spouse, buf) encodeFloat64(a.Money, buf) }

Your struct field order is fixed in code

NameBirthday

Phone

...

Page 14: Story Writing Byte Serializer in Golang

Implementation

Page 15: Story Writing Byte Serializer in Golang

First try• Using encoding/binary to convert type to byte

array • Write byte array to buffer • For dynamic size struct (vector, map..)

• Write size first as int and than write payload • When decode, read size first, and than read

payload

Page 16: Story Writing Byte Serializer in Golang

uint64func encodeUint64(v uint64, w io.Writer) error { b := [64 / 8]byte{} binary.LittleEndian.PutUint64(b[:], v) _, err := w.Write(b[:]) return err }

func decodeUint64(r io.Reader) (uint64, error) { var l uint64 err := binary.Read(r, binary.LittleEndian, &l) if err != nil { return 0, err } return l, nil }

Page 17: Story Writing Byte Serializer in Golang

stringfunc encodeString(v string, w io.Writer) error { l := len(v) err := encodeUint16(uint16(l), w) if err != nil { return err } _, err = w.Write([]byte(v)) return err }

func decodeString(r io.Reader) (string, error) { l, err := decodeUint16(r) if err != nil { return "", err } b := make([]byte, l) _, err = r.Read(b) return string(b), err }

Page 18: Story Writing Byte Serializer in Golang

float64func encodeFloat64(v float64, w io.Writer) error { var zeroByte byte b := [64 / 8]byte{}

bs := math.Float64Bits(v) binary.LittleEndian.PutUint64(b[:], bs) _, err := w.Write(b[:]) return err }

func decodeFloat64(r io.Reader) (float64, error) { var l uint64 err := binary.Read(r, binary.LittleEndian, &l) if err != nil { return 0, nil } return math.Float64fromBits(l), nil }

Page 19: Story Writing Byte Serializer in Golang

It's so simple And does nothing special

It must be fast!

Page 20: Story Writing Byte Serializer in Golang

Let's benchmark• Using

• https://github.com/alecthomas/go_serialization_benchmarks

• Add our own serialization method and compare with another • Call it `raw`

• Let's see result

Page 21: Story Writing Byte Serializer in Golang

BenchmarkMsgpMarshal-8 10000000 161 ns/op 128 B/op 1 allocs/op BenchmarkMsgpUnmarshal-8 5000000 307 ns/op 112 B/op 3 allocs/op BenchmarkVmihailencoMsgpackMarshal-8 1000000 1840 ns/op 368 B/op 6 allocs/op BenchmarkVmihailencoMsgpackUnmarshal-8 1000000 1874 ns/op 384 B/op 13 allocs/op BenchmarkRawMarshaller-8 2000000 826 ns/op 384 B/op 13 allocs/op BenchmarkRawUnmarshaller-8 2000000 710 ns/op 338 B/op 17 allocs/op BenchmarkJsonMarshal-8 500000 2804 ns/op 1232 B/op 10 allocs/op BenchmarkJsonUnmarshal-8 500000 2999 ns/op 464 B/op 7 allocs/op BenchmarkEasyJsonMarshal-8 1000000 1223 ns/op 784 B/op 5 allocs/op BenchmarkEasyJsonUnmarshal-8 1000000 1351 ns/op 160 B/op 4 allocs/op BenchmarkBsonMarshal-8 1000000 1405 ns/op 392 B/op 10 allocs/op BenchmarkBsonUnmarshal-8 1000000 1869 ns/op 248 B/op 21 allocs/op BenchmarkGobMarshal-8 2000000 903 ns/op 48 B/op 2 allocs/op BenchmarkGobUnmarshal-8 2000000 913 ns/op 112 B/op 3 allocs/op BenchmarkXdrMarshal-8 1000000 1553 ns/op 456 B/op 21 allocs/op BenchmarkXdrUnmarshal-8 1000000 1392 ns/op 240 B/op 11 allocs/op BenchmarkUgorjiCodecMsgpackMarshal-8 1000000 2190 ns/op 2753 B/op 8 allocs/op BenchmarkUgorjiCodecMsgpackUnmarshal-8 500000 2207 ns/op 3008 B/op 6 allocs/op BenchmarkUgorjiCodecBincMarshal-8 1000000 2070 ns/op 2785 B/op 8 allocs/op BenchmarkUgorjiCodecBincUnmarshal-8 500000 2386 ns/op 3168 B/op 9 allocs/op BenchmarkSerealMarshal-8 500000 2563 ns/op 912 B/op 21 allocs/op BenchmarkSerealUnmarshal-8 500000 3068 ns/op 1008 B/op 34 allocs/op BenchmarkBinaryMarshal-8 1000000 1221 ns/op 256 B/op 16 allocs/op BenchmarkBinaryUnmarshal-8 1000000 1389 ns/op 335 B/op 22 allocs/op BenchmarkFlatBuffersMarshal-8 5000000 345 ns/op 0 B/op 0 allocs/op BenchmarkFlatBuffersUnmarshal-8 5000000 259 ns/op 112 B/op 3 allocs/op BenchmarkCapNProtoMarshal-8 3000000 423 ns/op 56 B/op 2 allocs/op BenchmarkCapNProtoUnmarshal-8 5000000 384 ns/op 200 B/op 6 allocs/op BenchmarkCapNProto2Marshal-8 2000000 695 ns/op 244 B/op 3 allocs/op BenchmarkCapNProto2Unmarshal-8 2000000 859 ns/op 320 B/op 6 allocs/op BenchmarkHproseMarshal-8 1000000 1033 ns/op 479 B/op 8 allocs/op BenchmarkHproseUnmarshal-8 2000000 1028 ns/op 319 B/op 10 allocs/op BenchmarkProtobufMarshal-8 2000000 885 ns/op 200 B/op 7 allocs/op BenchmarkProtobufUnmarshal-8 2000000 641 ns/op 192 B/op 10 allocs/op BenchmarkGoprotobufMarshal-8 3000000 447 ns/op 312 B/op 4 allocs/op BenchmarkGoprotobufUnmarshal-8 3000000 592 ns/op 432 B/op 9 allocs/op BenchmarkGogoprotobufMarshal-8 10000000 131 ns/op 64 B/op 1 allocs/op BenchmarkGogoprotobufUnmarshal-8 10000000 222 ns/op 96 B/op 3 allocs/op BenchmarkColferMarshal-8 10000000 123 ns/op 64 B/op 1 allocs/op BenchmarkColferUnmarshal-8 10000000 181 ns/op 112 B/op 3 allocs/op BenchmarkGencodeMarshal-8 10000000 153 ns/op 80 B/op 2 allocs/op BenchmarkGencodeUnmarshal-8 10000000 172 ns/op 112 B/op 3 allocs/op BenchmarkGencodeUnsafeMarshal-8 20000000 98.2 ns/op 48 B/op 1 allocs/op BenchmarkGencodeUnsafeUnmarshal-8 10000000 142 ns/op 96 B/op 3 allocs/op BenchmarkXDR2Marshal-8 10000000 151 ns/op 64 B/op 1 allocs/op BenchmarkXDR2Unmarshal-8 10000000 145 ns/op 32 B/op 2 allocs/op BenchmarkGoAvroMarshal-8 500000 2291 ns/op 1032 B/op 33 allocs/op BenchmarkGoAvroUnmarshal-8 300000 5388 ns/op 3440 B/op 89 allocs/op

Not bad But slow as 1/10 compare to

BenchmarkGencode

Page 22: Story Writing Byte Serializer in Golang

What we did wrong?

Page 23: Story Writing Byte Serializer in Golang

Slow pattern• Use GODEBUG=allocfreetrace=1 to find

redundant allocation pattern

func encodeUint64(v uint64, w io.Writer) error {

b := [64 / 8]byte{} binary.LittleEndian.PutUint64(b[:], v) _, err := w.Write(b[:]) return err }

func encodeString(v string, w io.Writer) error { l := len(v) err := encodeUint16(uint16(l), w) if err != nil { return err }

_, err = w.Write([]byte(v)) return err }

func rawbyteslice(size int) (b []byte) { cap := roundupsize(uintptr(size)) p := mallocgc(cap, nil, false) if cap != uintptr(size) { memclrNoHeapPointers(add(p, uintptr(size)), cap-uintptr(size)) }

*(*slice)(unsafe.Pointer(&b)) = slice{p, size, int(cap)} return }

Page 24: Story Writing Byte Serializer in Golang

Slow pattern• Took a look at some fast serialization

• Just byte copying around, no alloc

• And in our case, we write to file write after encode, so we do not need each serialization buffer, we just need global one

Page 25: Story Writing Byte Serializer in Golang

Second try

• Prepare a global buffer • Grow if needed • Clear buffer each run

• Just copy byte around, no more allocation

Page 26: Story Writing Byte Serializer in Golang

var bufferByte = make([]byte, DEFAULT_BUFFER_CAP)

func (rs Raw2Serializer) Marshal(o interface{}) []byte { a := o.(*A)

cleanBuffer()

idx := 0 idx += WriteString(idx, a.Name) idx += WriteUint64(idx, uint64(a.BirthDay.UnixNano())) idx += WriteString(idx, a.Phone) idx += WriteUint64(idx, uint64(a.Siblings)) idx += WriteBool(idx, a.Spouse) idx += WriteFloat64(idx, a.Money)

// copy from a to bufferByte return bufferByte[0:idx] }

small different, need index control to know where we need to copy and need to clean Buffer for each run

Page 27: Story Writing Byte Serializer in Golang

func WriteUint64(idx int, n uint64) int { if (idx + 8) > currentCap {

growBufferIfneeded() }

for i := uint(idx); i < uint(8); i++ { bufferByte[i] = byte(n >> (i*8)) } return 8 }

func WriteString(idx int, s string) int { l := len(s)

if (idx + l) > currentCap { growBufferIfneeded()

}

n := WriteUint64(idx, uint64(l))

// NOTE: copy works without conversion copy(bufferByte[idx+n:idx+l], s) return l+n }

Page 28: Story Writing Byte Serializer in Golang

BenchmarkMsgpMarshal-8 10000000 158 ns/op 128 B/op 1 allocs/op BenchmarkMsgpUnmarshal-8 5000000 335 ns/op 112 B/op 3 allocs/op BenchmarkVmihailencoMsgpackMarshal-8 1000000 1764 ns/op 368 B/op 6 allocs/op BenchmarkVmihailencoMsgpackUnmarshal-8 1000000 1779 ns/op 384 B/op 13 allocs/op BenchmarkRawMarshaller-8 2000000 806 ns/op 384 B/op 13 allocs/op

BenchmarkRawUnmarshaller-8 2000000 706 ns/op 338 B/op 17 allocs/op BenchmarkRaw2Marshaller-8 20000000 83.6 ns/op 0 B/op 0 allocs/op

10 times faster ! As fast as fastest andyleap/gencode

bench again

Page 29: Story Writing Byte Serializer in Golang

What I learned• Hidden allocation reduce performance • Serialization to file pitfalls

• Need thread-safe implement to prevent dirty file • Need versioning (write version first, than payload

later) for backward compatibility • Checksum matter

• You can calculate checksum directly from struct, no need to calculate from bytes • Using fnv to hash all fields and add up together,

instead of using CRC32 to calculate the whole byte arrays

Page 30: Story Writing Byte Serializer in Golang

Interesting techniques of other serialization methods

Page 31: Story Writing Byte Serializer in Golang

varint (protobuf)• Available in many softwares (protobuf, sqlite,

webassemlby (LEB128 of LLVM), golang encoding/binary)

• Compressed positive integer (negative number with 2-complement will take more bits)

• Idea: • most of integer in our app is small ("not very big") • Use as little number of bits as possible • 7 bit per byte, MSB bit as "continuation bit"

• Cons: • CPU cost • decoding is a bit complex

Page 32: Story Writing Byte Serializer in Golang

varint (protobuf) t := uint64(l)

for t >= 0x80 { buf[i+8] = byte(t) | 0x80 t >>= 7 i++ }

Many variant (group varint encoding, prefix varint encoding etc ... )

Page 33: Story Writing Byte Serializer in Golang

zigzag encoding (protobuf)• varint works only with positive number • Zigzag encoding encode negative number as

nearest (in absolute) positive number0 0 -1 1 1 2 -2 3 2147483647 4294967294 -2147483648 4294967295

zigzag = (n << 1) ^ (n >> (BIT_WIDTH - 1) Remember that arithmetic shift replicates the sign bit (n >> (BIT_WIDTH - 1) -> 11111...1 for negative (n >> (BIT_WIDTH - 1) -> 00000...0 for positive So when XOR with negative n, a lot of 1 will be eliminate

Page 34: Story Writing Byte Serializer in Golang

float reverse (gob)// floatBits returns a uint64 holding the bits of a floating-point number. // Floating-point numbers are transmitted as uint64s holding the bits // of the underlying representation. They are sent byte-reversed, with // the exponent end coming out first, so integer floating point numbers // (for example) transmit more compactly. This routine does the // swizzling. func floatBits(f float64) uint64 { u := math.Float64bits(f) var v uint64 for i := 0; i < 8; i++ { v <<= 8 v |= u & 0xFF u >>= 8 } return v }

Page 35: Story Writing Byte Serializer in Golang

unsafe (andyleap/gencode)

v := *(*uint64)(unsafe.Pointer(&(d.Height)))

Unmarshal number without copy or allocation

You could use same technique for string too http://qiita.com/mattn/items/176459728ff4f854b165

Page 36: Story Writing Byte Serializer in Golang

Finally• Write your own serialization is not hard, and fun

• You can learn a lot from existence method • There are tons of techniques could be used to

enhance performance • When there are no much preferences, let's use

fix-layout type serialization • Version control proto file • High performance