Non-determinism: leading space characters in diff occasionally turn into non-breaking space

[mejedi]

go 1.22.5
github.com/google/go-cmp v0.6.0

I noticed that cmp.Diff alternates between regular and non-breaking space in +- column. One is way more likely to get regular space.

This behavior creates issues when output is compared verbatim, e.g. in testable examples:

func ExampleCmpRepr() {
    type foo struct{ B, A int }
    fmt.Printf("%#v\n", cmp.Diff(foo{A: 1}, foo{A: 2}))
    // Output: "  testnet.foo{\n  \tB: 0,\n- \tA: 1,\n+ \tA: 2,\n  }\n"
}

It fails occasionally with the following output. Using go test -bench=. somehow makes it more likely to manifest.

--- FAIL: ExampleCmpRepr (0.00s)
got:
"\u00a0\u00a0testnet.foo{\n\u00a0\u00a0\tB: 0,\n-\u00a0\tA: 1,\n+\u00a0\tA: 2,\n\u00a0\u00a0}\n"
want:
"  testnet.foo{\n  \tB: 0,\n- \tA: 1,\n+ \tA: 2,\n  }\n"

[dsnet]

Hi @mejedi, sorry to hear about the issue, but this is working as intended. The documentation on Diff says:

Do not depend on this output being stable. If you need the ability to programmatically interpret the difference, consider using a custom Reporter.

The output of Diff has in the past and will continue to change because it is intended to evolve to be more and more humanly readable. Codifying it's output in something like an example test would mean that the test is brittle and breaks whenever Diff's output changes again. The instability of the output was done to deliberately make it harder to accidentally assume that the output was stable.

[mejedi]

Hi @dsnet, thank you for the prompt reply.

At the moment, I use go-cmp for testing custom low-level network code. Go-cmp turned out great to verify packets. However, for an outsider, it is not immediately apparent what does it mean to diff network packets and how the output looks like (object representation is being diffed, not raw bytes, so that individual mismatching fields or entire layers show up).

What is the best way to showcase cmp.Diff output in docs?

Re intentional non-determinism: I agree with the idea. The current implementation however is rather hostile. Go test results are cached, AND the probabilities of different outcomes are very skewed, AND the difference in the output is hard to tell with a naked eye.

[dsnet]

What is the best way to showcase cmp.Diff output in docs?

Unfortunately there is no answer at the moment, but the ability to evolve the output is currently deemed a greater benefit than stability.

I've long thought about forking the current reporter implementation and codifying it as "stable". This way we can continue to evolve the default output, and also provide a stable output for use cases that need it. Output in example docs is a legitimate reason to want a stable format.