Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shares: NamespacePaddingShare can take advantage of Go's memory guarantees to zero memory and use make([]byte, LEN) instead of bytes.Repeat([]byte{0}, LEN) which is less efficient #41

Closed
4 tasks
odeke-em opened this issue Feb 29, 2024 · 0 comments · Fixed by #42
Assignees
Labels
enhancement New feature or request external

Comments

@odeke-em
Copy link
Contributor

Summary of Bug

While auditing this repository I noticed these code snippets

padding := bytes.Repeat([]byte{0}, FirstSparseShareContentSize)

and

padByte := []byte{0}
padding := bytes.Repeat(padByte, missingBytes)

Insights

Those code snippets try to create zero byte slices of LEN. Go guarantees that all memory shall be zeroed per https://go.dev/ref/spec#The_zero_value and even better, make is made much more efficient than bytes.Repeat and quick benchmarks can prove this

Benchmarks

diff --git a/shares/padding.go b/shares/padding.go
index e0d905c..3c8d010 100644
--- a/shares/padding.go
+++ b/shares/padding.go
@@ -1,7 +1,6 @@
 package shares
 
 import (
-	"bytes"
 	"errors"
 
 	"github.com/celestiaorg/go-square/namespace"
@@ -20,7 +19,7 @@ func NamespacePaddingShare(ns namespace.Namespace, shareVersion uint8) (Share, e
 	if err := b.WriteSequenceLen(0); err != nil {
 		return Share{}, err
 	}
-	padding := bytes.Repeat([]byte{0}, FirstSparseShareContentSize)
+	padding := make([]byte, FirstSparseShareContentSize)
 	b.AddData(padding)
 
 	share, err := b.Build()

with this benchmark

package shares

import (
	"bytes"
	"testing"

	"github.com/celestiaorg/go-square/namespace"
)

var sink any = nil

func pad(tb testing.TB) any {
	ns1 = namespace.MustNewV0(bytes.Repeat([]byte{1}, namespace.NamespaceVersionZeroIDSize))
	got, err := NamespacePaddingShare(ns1, ShareVersionZero)
	if err != nil {
		tb.Fatal(err)
	}
	return got
}

func BenchmarkPaddingVsRepeat(b *testing.B) {
	b.ReportAllocs()

	for i := 0; i < b.N; i++ {
		sink = pad(b)
	}

	if sink == nil {
		b.Fatal("benchmark did not run!")
	}
}

which shows the improvements

$ benchstat before.txt after.txt 
name               old time/op    new time/op    delta
PaddingVsRepeat-8     674ns ± 1%     538ns ± 0%  -20.22%  (p=0.000 n=9+9)

name               old alloc/op   new alloc/op   delta
PaddingVsRepeat-8    1.31kB ± 0%    0.83kB ± 0%  -36.59%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
PaddingVsRepeat-8      12.0 ± 0%      11.0 ± 0%   -8.33%  (p=0.000 n=10+10)

/cc @elias-orijtech @liamsi

Version

4135f35

Steps to Reproduce

Please see the details above

For Admin Use

  • Not duplicate issue
  • Appropriate labels applied
  • Appropriate contributors tagged
  • Contributor assigned/self-assigned
@odeke-em odeke-em added the bug Something isn't working label Feb 29, 2024
odeke-em added a commit to orijtech/go-square that referenced this issue Feb 29, 2024
…, N)

This change uses Go's memory model guarantees that the zero value memory
shall be zeroed and hence to create a slice of 0 bytes aka padding,
we can simply use make([]byte, N) which is much more efficient than
bytes.Repeat([]byte{0}, N) and the benchmarks show this improvement:

```shell
$ benchstat before.txt after.txt
name               old time/op    new time/op    delta
PaddingVsRepeat-8     674ns ± 1%     538ns ± 0%  -20.22%  (p=0.000 n=9+9)

name               old alloc/op   new alloc/op   delta
PaddingVsRepeat-8    1.31kB ± 0%    0.83kB ± 0%  -36.59%  (p=0.000 n=10+10)

name               old allocs/op  new allocs/op  delta
PaddingVsRepeat-8      12.0 ± 0%      11.0 ± 0%   -8.33%  (p=0.000 n=10+10)
```

Fixes celestiaorg#41
@rootulp rootulp added enhancement New feature or request and removed bug Something isn't working labels Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request external
Projects
None yet
2 participants