Benchmarks of Go serialization methods

This is a test suite for benchmarking various Go serialization methods.

Tested serialization methods

• encoding/gob
• encoding/json
• github.com/alecthomas/binary
• github.com/davecgh/go-xdr/xdr
• github.com/Sereal/Sereal/Go/sereal
• github.com/ugorji/go/codec
• gopkg.in/vmihailenco/msgpack.v2
• github.com/youtube/vitess/go/bson (using the bsongen code generator)
• labix.org/v2/mgo/bson
• github.com/philhofer/msgp (code generator for msgpack)
• github.com/golang/protobuf (generated code)
• github.com/gogo/protobuf (generated code, optimized version of goprotobuf)
• github.com/DeDiS/protobuf (reflection based)
• github.com/google/flatbuffers

Running the benchmarks

go get -u -t
go test -bench='.*' ./

Shameless plug: I use pawk to format the table:

go test -bench='.*' ./ | pawk -F'\t' '"%-40s %10s %10s %s %s" % f'

Recommendation

If performance, correctness and interoperability are the most important factors, gogoprotobuf is currently the best choice. It does require a pre-processing step (eg. via Go 1.4's "go generate" command).

But as always, make your own choice based on your requirements.

Data

The data being serialized is the following structure with randomly generated values:

type A struct {
    Name     string
    BirthDay time.Time
    Phone    string
    Siblings int
    Spouse   bool
    Money    float64
}

Results

Results with Go 1.3 on an Intel i7-3930K:

benchmark                                  iter        time/iter   bytes alloc          allocs
---------                                  ----        ---------   -----------          ------
BenchmarkVmihailencoMsgpackMarshal          1000000       2376 ns/op      352 B/op        5 allocs/op
BenchmarkVmihailencoMsgpackUnmarshal        1000000       2380 ns/op      347 B/op       13 allocs/op
BenchmarkJsonMarshal                         500000       3501 ns/op      584 B/op        7 allocs/op
BenchmarkJsonUnmarshal                       300000       5553 ns/op      447 B/op        8 allocs/op
BenchmarkBsonMarshal                         500000       2347 ns/op      504 B/op       19 allocs/op
BenchmarkBsonUnmarshal                       500000       2596 ns/op      296 B/op       22 allocs/op
BenchmarkVitessBsonMarshal                  1000000       1735 ns/op     1168 B/op        4 allocs/op
BenchmarkVitessBsonUnmarshal                1000000       1036 ns/op      224 B/op        4 allocs/op
BenchmarkGobMarshal                          200000      10662 ns/op     1688 B/op       35 allocs/op
BenchmarkGobUnmarshal                         30000      49289 ns/op    17493 B/op      377 allocs/op
BenchmarkXdrMarshal                          500000       2821 ns/op      520 B/op       24 allocs/op
BenchmarkXdrUnmarshal                       1000000       2163 ns/op      271 B/op       12 allocs/op
BenchmarkUgorjiCodecMsgpackMarshal           500000       4096 ns/op     1905 B/op       10 allocs/op
BenchmarkUgorjiCodecMsgpackUnmarshal         500000       3770 ns/op     1840 B/op       14 allocs/op
BenchmarkUgorjiCodecBincMarshal              300000       4066 ns/op     1938 B/op       10 allocs/op
BenchmarkUgorjiCodecBincUnmarshal            500000       4004 ns/op     2000 B/op       17 allocs/op
BenchmarkSerealMarshal                       300000       4857 ns/op     1360 B/op       26 allocs/op
BenchmarkSerealUnmarshal                     500000       4287 ns/op      972 B/op       37 allocs/op
BenchmarkBinaryMarshal                      1000000       2440 ns/op      408 B/op       19 allocs/op
BenchmarkBinaryUnmarshal                    1000000       2339 ns/op      416 B/op       24 allocs/op
BenchmarkMsgpMarshal                        3000000        447 ns/op      144 B/op        1 allocs/op
BenchmarkMsgpUnmarshal                      3000000        584 ns/op      112 B/op        3 allocs/op
BenchmarkGoprotobufMarshal                  2000000        971 ns/op      312 B/op        4 allocs/op
BenchmarkGoprotobufUnmarshal                1000000       1220 ns/op      432 B/op        9 allocs/op
BenchmarkGogoprotobufMarshal               10000000        229 ns/op       64 B/op        1 allocs/op
BenchmarkGogoprotobufUnmarshal              5000000        323 ns/op      112 B/op        3 allocs/op
BenchmarkFlatbuffersMarshal                 3000000        523 ns/op        0 B/op        0 allocs/op
BenchmarkFlatBuffersUnmarshal              30000000       53.4 ns/op        0 B/op        0 allocs/op
BenchmarkProtobufMarshal                    1000000       1452 ns/op      232 B/op        9 allocs/op
BenchmarkProtobufUnmarshal                  1000000       1111 ns/op      192 B/op       10 allocs/op

Note: the gob results are not really representative of normal performance, as gob is designed for serializing streams or vectors of a single type, not individual values.

Issues

The benchmarks can also be run with validation enabled.

VALIDATE=1 go test -bench='.*' ./

Unfortunately, several of the serializers exhibit issues:

1. (minor) BSON drops sub-microsecond precision from time.Time.
2. (minor) Vitess BSON drops sub-microsecond precision from time.Time.
3. (minor) Ugorji Binc Codec drops the timezone name (eg. "EST" -> "-0500") from time.Time.

BenchmarkVmihailencoMsgpackMarshal   1000000          1770 ns/op         408 B/op          8 allocs/op
BenchmarkVmihailencoMsgpackUnmarshal      500000          3073 ns/op         512 B/op         15 allocs/op
BenchmarkJsonMarshal      500000          3292 ns/op         584 B/op          7 allocs/op
BenchmarkJsonUnmarshal    200000          6077 ns/op         543 B/op         10 allocs/op
BenchmarkBsonMarshal     1000000          2302 ns/op         504 B/op         19 allocs/op
BenchmarkBsonUnmarshal  --- FAIL: BenchmarkBsonUnmarshal
    serialization_benchmarks_test.go:299: unmarshaled object differed:
        &{36f9dfc8311e1efd 2014-12-20 08:28:56.697060419 +1100 AEDT bce6c01982 0 true 0.9078469433531421}
        &{36f9dfc8311e1efd 2014-12-20 08:28:56.697 +1100 AEDT bce6c01982 0 true 0.9078469433531421}
BenchmarkVitessBsonMarshal   1000000          1644 ns/op        1168 B/op          4 allocs/op
BenchmarkVitessBsonUnmarshal    --- FAIL: BenchmarkVitessBsonUnmarshal
    serialization_benchmarks_test.go:299: unmarshaled object differed:
        &{d22635208bfe823f 2014-12-20 08:28:58.370539555 +1100 AEDT 6c6e6d4ef8 2 true 0.2026063151373092}
        &{d22635208bfe823f 2014-12-19 21:28:58.37 +0000 UTC 6c6e6d4ef8 2 true 0.2026063151373092}
BenchmarkGobMarshal   200000          9770 ns/op        1688 B/op         35 allocs/op
BenchmarkGobUnmarshal      30000         47695 ns/op       17590 B/op        379 allocs/op
BenchmarkXdrMarshal   500000          2559 ns/op         519 B/op         24 allocs/op
BenchmarkXdrUnmarshal     500000          3016 ns/op         352 B/op         14 allocs/op
BenchmarkUgorjiCodecMsgpackMarshal    500000          3662 ns/op        1464 B/op         24 allocs/op
BenchmarkUgorjiCodecMsgpackUnmarshal      300000          4533 ns/op        1232 B/op         29 allocs/op
BenchmarkUgorjiCodecBincMarshal   500000          3745 ns/op        1480 B/op         24 allocs/op
BenchmarkUgorjiCodecBincUnmarshal   --- FAIL: BenchmarkUgorjiCodecBincUnmarshal
    serialization_benchmarks_test.go:299: unmarshaled object differed:
        &{94fd6da7835a0346 2014-12-20 08:29:10.50608751 +1100 AEDT 7749978493 0 true 0.09633537464986415}
        &{94fd6da7835a0346 2014-12-20 08:29:10.50608751 +1100 +1100 7749978493 0 true 0.09633537464986415}
BenchmarkSerealMarshal    300000          4572 ns/op        1360 B/op         26 allocs/op
BenchmarkSerealUnmarshal      300000          4950 ns/op        1068 B/op         39 allocs/op
BenchmarkBinaryMarshal   1000000          2168 ns/op         408 B/op         19 allocs/op
BenchmarkBinaryUnmarshal      500000          3249 ns/op         512 B/op         26 allocs/op
BenchmarkGoprotobufMarshal   2000000           928 ns/op         312 B/op          4 allocs/op
BenchmarkGoprotobufUnmarshal     1000000          1201 ns/op         432 B/op          9 allocs/op
BenchmarkProtobufMarshal     1000000          1342 ns/op         232 B/op          9 allocs/op
BenchmarkProtobufUnmarshal   1000000          2128 ns/op         288 B/op         12 allocs/op

All other fields are correct however.