Marshal fields in alphabetical order

[ivanjaros]

When changes are made to struct fields(added/removed/reordered) in Go code, the order of fields is preserved in marshalled content. This causes issues when we want to compare marshalled content, for example old and new code, while we have the same values, yet we end up with different result. Or if we have json from external source, comparing the json itself will always fail to match.

Hence, I think that struct fields should be alphabetically ordered and always marshalled that way to avoid messing with results when in reality the order of fields defined in Go should have no impact on the end-result.

Maps are handled that way, i see no reason why struct fields themselves could not behave the same.

package main

import (
	"github.com/goccy/go-json"
)

type Foo struct {
	One   string `json:"one"`
	Two   string `json:"two"`
	Three string `json:"three"`
}

type Bar struct {
	Three string `json:"three"`
	Two   string `json:"two"`
	One   string `json:"one"`
}

func main() {
	f := Foo{
		One:   "1",
		Two:   "2",
		Three: "3",
	}

	data, _ := json.Marshal(f)
	println(string(data))

	b := Bar{
		One:   "1",
		Two:   "2",
		Three: "3",
	}

	data, _ = json.Marshal(b)
	println(string(data))
}

result:
{"one":"1","two":"2","three":"3"}
{"three":"3","two":"2","one":"1"}

This is especially problematic when we're using json as storage format and some time passes, code changes, and we want to compare with some old records and suddenly we get different json. That forces us to unmarshal the old json and either marshal it again with new struct and compare them or simply compare unmarshalled struct..instead of just marshalling new struct and comparing bytes with old json, not having to do anything with it.

Not to mention that any hash will also change and won't match the old hash even though the content is the same, but is not put/marshalled in the same order.

That is also why other formats, like protocol buffers use ordering for their fields so the format does not change if new fields are added or order of fields defined in she schema is mixed.

[ivanjaros]

The solution is trivial.

in encoder, turn this

func (c *Compiler) typeToStructTags(typ *runtime.Type) runtime.StructTags {
	tags := runtime.StructTags{}
	fieldNum := typ.NumField()
	for i := 0; i < fieldNum; i++ {
		field := typ.Field(i)
		if runtime.IsIgnoredStructField(field) {
			continue
		}
		tags = append(tags, runtime.StructTagFromField(field))
	}
	return tags
}

into this:

func (c *Compiler) typeToStructTags(typ *runtime.Type) runtime.StructTags {
	tags := runtime.StructTags{}
	fieldNum := typ.NumField()
	for i := 0; i < fieldNum; i++ {
		field := typ.Field(i)
		if runtime.IsIgnoredStructField(field) {
			continue
		}
		tags = append(tags, runtime.StructTagFromField(field))
	}
	sort.Slice(tags, func(i, j int) bool {         <-------
		return tags[i].Key < tags[j].Key       <-------
	})                                             <-------
	return tags
}

which will produce:

{"one":"1","three":"3","two":"2"}
{"one":"1","three":"3","two":"2"}

One caveat might be sorting json tags that start with integer, so instead of 1foo, 2foo, 3foo, 11foo, 12foo it would produce 1foo, 11foo, 12foo, 2foo, 3foo. But that can be mitigated by better sorting library than the native or just dedicated comparator.

[ivanjaros]

🤨