Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement rhp4 basic client and server #96

Merged
merged 75 commits into from
Oct 28, 2024
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
aee1098
rhp4: implement rhp4 server and basic client
n8maninger Sep 13, 2024
ca55d0f
deps: update core
n8maninger Sep 13, 2024
2ea0695
deps: remove replace directive
n8maninger Sep 13, 2024
570b9fb
rhp4: combine client and server packages
n8maninger Sep 13, 2024
d2cc526
rhp4: minify form and renew RPC
n8maninger Sep 13, 2024
c36795c
rhp4: remove debug logging
n8maninger Sep 13, 2024
1ac1b5c
rhp4: fix lint
n8maninger Sep 13, 2024
bfb7aa0
chain: extend test, add panic to encode functions
n8maninger Sep 13, 2024
612ff08
chain: use larger length prefix
n8maninger Sep 13, 2024
2d36531
rhp4: check offset is a multiple of LeafSize
n8maninger Sep 13, 2024
c0236b2
rhp4: add context to client methods
n8maninger Sep 13, 2024
ba11351
rhp4,testutil: fix mismatched muxes
n8maninger Sep 13, 2024
a1ae24b
deps: update mux
n8maninger Sep 13, 2024
b924d57
deps: update mux
n8maninger Sep 14, 2024
e76f65d
rhp/v4: Option->ServerOption
n8maninger Sep 17, 2024
9fbe148
rhp/v4: add basis diff comment
n8maninger Sep 17, 2024
5bed8f9
rhp/v4: address comments
n8maninger Sep 17, 2024
ec60cba
rhp/v4: address comments
n8maninger Sep 17, 2024
10ec12b
deps: update core and mux
n8maninger Sep 17, 2024
2bad9a0
implement basic sia mux client with context interupt
n8maninger Sep 17, 2024
cce2d6b
rhp/v4: RPC read sector write to writer
n8maninger Sep 17, 2024
a911c83
rhp/v4: use request validate
n8maninger Sep 19, 2024
46eb1e4
chain,rhp4: add type safe supported protocols
n8maninger Sep 26, 2024
d91a33d
rhp4: add invalid root error
n8maninger Sep 26, 2024
f7c5de1
rhp4: remove incremented revision number. It was moved to PayWithCont…
n8maninger Sep 26, 2024
6391156
rpc: move validate up
n8maninger Sep 27, 2024
161f86e
rhp4: add verify sector rpc
n8maninger Sep 27, 2024
8e75f20
Merge remote-tracking branch 'origin/master' into nate/rhp4
n8maninger Sep 27, 2024
24d8518
rho: add verify sector test
n8maninger Sep 27, 2024
e331259
Merge remote-tracking branch 'origin/master' into nate/rhp4
n8maninger Sep 27, 2024
cf28a22
rhp4: fix lint
n8maninger Sep 27, 2024
2e81244
rhp/v4: fix modify sectors test
n8maninger Sep 27, 2024
ab3a7bd
rhp4: add result types
n8maninger Sep 27, 2024
48da7a3
rhp4: address comments
n8maninger Sep 28, 2024
19326ed
rhp4: make input validation clearer
n8maninger Sep 28, 2024
4977cf4
rhp4,chain: remove slashes in protocol
n8maninger Sep 28, 2024
a80d6d9
rhp4: period in docstring
n8maninger Sep 28, 2024
9dc122d
rhp4: switch RPCWrite to streaming
n8maninger Sep 28, 2024
c1dddb9
rhp4: use clear for zero reader (Chris)
n8maninger Sep 28, 2024
3543c96
rhp4: add proof validation comment
n8maninger Sep 28, 2024
44117d3
rhp4: address review comments
n8maninger Sep 28, 2024
4592625
rhp4: add append sectors rpc
n8maninger Oct 2, 2024
aad5fd7
rhp4: fix lint
n8maninger Oct 2, 2024
1406d3d
rpc: set miner fee
n8maninger Oct 3, 2024
15e898d
rhp4,chain: simplify renew, move proof updates to chain manager (#102)
n8maninger Oct 8, 2024
c7e1c25
Merge branch 'master' into nate/rhp4
n8maninger Oct 8, 2024
56f89a2
rhp4 RPC Refresh (#104)
n8maninger Oct 8, 2024
e240c9c
rhp4: address comments
n8maninger Oct 8, 2024
a799184
deps: update core
n8maninger Oct 9, 2024
02dcd60
rhp/v4: Implement Merkle proofs
lukechampine Oct 10, 2024
322d7a3
rhp4: update interfaces
n8maninger Oct 19, 2024
da3e94e
rhp4: rename interface
n8maninger Oct 20, 2024
5a9fa81
rhp4: add UpgradeConn helper
n8maninger Oct 21, 2024
c00c7f7
Merge pull request #112 from SiaFoundation/nate/rhp4-upgrade-helper
n8maninger Oct 21, 2024
fac5183
rhp4: update frand
n8maninger Oct 22, 2024
9eb661e
Merge remote-tracking branch 'origin/master' into nate/rhp4
n8maninger Oct 22, 2024
17d2b72
chain: fix deadlock
n8maninger Oct 22, 2024
8bf8e6c
deps: update core
n8maninger Oct 21, 2024
c8966b7
rhp4: implement RPC remove
n8maninger Oct 21, 2024
b81a3e5
deps: upgrade core
n8maninger Oct 21, 2024
c14ef51
rhp4: use core usage
n8maninger Oct 21, 2024
37ab2f5
rhp4: add usage to form contract
n8maninger Oct 21, 2024
d816b0a
rhp4: RPC remove -> RPC free
n8maninger Oct 22, 2024
1a78cf7
Merge pull request #111 from SiaFoundation/nate/rhp4-rpc-remove
n8maninger Oct 23, 2024
50bdd90
deps: update core
n8maninger Oct 23, 2024
61581aa
rhp4: reduce test verbosity
n8maninger Oct 23, 2024
7e89429
rhp/v4: Fix failing Merkle proof tests
lukechampine Oct 24, 2024
288722c
Merge remote-tracking branch 'origin/master' into nate/rhp4
n8maninger Oct 24, 2024
1e5ac3f
deps: update core
n8maninger Oct 24, 2024
3f86f99
deps: update core
n8maninger Oct 24, 2024
f2fbcd6
rhp4: fix RPC refresh renter signatures
n8maninger Oct 24, 2024
2d0fd5b
reduce blocks mined for tests
n8maninger Oct 24, 2024
6741532
chain: explicit return values
n8maninger Oct 25, 2024
1d05f0a
rhp4: sector pointer
n8maninger Oct 26, 2024
5a00184
deps: update core
n8maninger Oct 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 12 additions & 18 deletions chain/chain_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"go.sia.tech/core/types"
"go.sia.tech/coreutils"
"go.sia.tech/coreutils/chain"
"go.sia.tech/coreutils/testutil"
"lukechampine.com/frand"
)

Expand Down Expand Up @@ -113,11 +114,7 @@ func newMemState() *memState {
}

func TestV2Attestations(t *testing.T) {
n, genesisBlock := chain.TestnetZen()

n.InitialTarget = types.BlockID{0xFF}
n.HardforkV2.AllowHeight = 2
n.HardforkV2.RequireHeight = 3
n, genesisBlock := testutil.V2Network()

policy := types.AnyoneCanSpend()
addr := policy.Address()
Expand All @@ -144,7 +141,7 @@ func TestV2Attestations(t *testing.T) {
ms := newMemState()

// mine until a utxo is spendable
mineBlocks(t, cm, 150)
mineBlocks(t, cm, int(n.MaturityDelay)+1)
ms.Sync(t, cm)

txn := types.V2Transaction{
Expand Down Expand Up @@ -179,13 +176,12 @@ func TestV2Attestations(t *testing.T) {
ms := newMemState()

// mine until a utxo is spendable
mineBlocks(t, cm, 150)
mineBlocks(t, cm, int(n.MaturityDelay)+1)
ms.Sync(t, cm)

sk := types.GeneratePrivateKey()
ann := chain.HostAnnouncement{
NetAddress: "foo.bar:1234",
PublicKey: sk.PublicKey(),
ann := chain.V2HostAnnouncement{
{Address: "foo.bar:1234", Protocol: "tcp"},
}
se := ms.SpendableElement(t)
txn := types.V2Transaction{
Expand Down Expand Up @@ -229,13 +225,12 @@ func TestV2Attestations(t *testing.T) {
ms := newMemState()

// mine until a utxo is spendable
mineBlocks(t, cm, 150)
mineBlocks(t, cm, int(n.MaturityDelay)+1)
ms.Sync(t, cm)

sk := types.GeneratePrivateKey()
ann := chain.HostAnnouncement{
NetAddress: "foo.bar:1234",
PublicKey: sk.PublicKey(),
ann := chain.V2HostAnnouncement{
{Address: "foo.bar:1234", Protocol: "tcp"},
}
txn := types.V2Transaction{
ArbitraryData: frand.Bytes(16),
Expand Down Expand Up @@ -274,13 +269,12 @@ func TestV2Attestations(t *testing.T) {
ms := newMemState()

// mine until a utxo is spendable
mineBlocks(t, cm, 150)
mineBlocks(t, cm, int(n.MaturityDelay)+1)
ms.Sync(t, cm)

sk := types.GeneratePrivateKey()
ann := chain.HostAnnouncement{
NetAddress: "foo.bar:1234",
PublicKey: sk.PublicKey(),
ann := chain.V2HostAnnouncement{
{Address: "foo.bar:1234", Protocol: "tcp"},
}
se := ms.SpendableElement(t)
minerFee := types.Siacoins(1)
Expand Down
73 changes: 54 additions & 19 deletions chain/hostannouncement.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package chain

import (
"bytes"
"errors"

"go.sia.tech/core/consensus"
"go.sia.tech/core/types"
Expand All @@ -11,36 +12,66 @@ const attestationHostAnnouncement = "HostAnnouncement"

var specifierHostAnnouncement = types.NewSpecifier("HostAnnouncement")

// A HostAnnouncement represents a signed announcement of a host's network
// address. Announcements may be made via arbitrary data (in a v1 transaction)
// or via attestation (in a v2 transaction).
type HostAnnouncement struct {
PublicKey types.PublicKey `json:"publicKey"`
NetAddress string `json:"netAddress"`
type (
// A HostAnnouncement represents a signed announcement of a host's network
// address. Announcements may be made via arbitrary data (in a v1 transaction)
// or via attestation (in a v2 transaction).
HostAnnouncement struct {
PublicKey types.PublicKey `json:"publicKey"`
NetAddress string `json:"netAddress"`
}

// A Protocol is a string identifying a network protocol that a host may be
// reached on.
Protocol string
n8maninger marked this conversation as resolved.
Show resolved Hide resolved

// A NetAddress is a pair of protocol and address that a host may be reached on
NetAddress struct {
Protocol Protocol `json:"protocol"`
Address string `json:"address"`
}

// A V2HostAnnouncement lists all the network addresses a host may be reached on
V2HostAnnouncement []NetAddress
n8maninger marked this conversation as resolved.
Show resolved Hide resolved
)

// EncodeTo implements types.EncoderTo.
func (na NetAddress) EncodeTo(e *types.Encoder) {
e.WriteString(string(na.Protocol))
e.WriteString(na.Address)
}

// DecodeFrom implements types.DecoderFrom.
func (na *NetAddress) DecodeFrom(d *types.Decoder) {
na.Protocol = Protocol(d.ReadString())
na.Address = d.ReadString()
}

// ToAttestation encodes a host announcement as an attestation.
func (ha HostAnnouncement) ToAttestation(cs consensus.State, sk types.PrivateKey) types.Attestation {
if ha.PublicKey != sk.PublicKey() {
panic("key mismatch") // developer error
func (ha V2HostAnnouncement) ToAttestation(cs consensus.State, sk types.PrivateKey) types.Attestation {
buf := bytes.NewBuffer(nil)
e := types.NewEncoder(buf)
types.EncodeSlice(e, ha)
if err := e.Flush(); err != nil {
panic(err) // should never happen
}
a := types.Attestation{
PublicKey: ha.PublicKey,
PublicKey: sk.PublicKey(),
Key: attestationHostAnnouncement,
Value: []byte(ha.NetAddress),
Value: buf.Bytes(),
}
a.Signature = sk.SignHash(cs.AttestationSigHash(a))
return a
}

// FromAttestation decodes a host announcement from an attestation.
func (ha *HostAnnouncement) FromAttestation(a types.Attestation) bool {
func (ha *V2HostAnnouncement) FromAttestation(a types.Attestation) error {
if a.Key != attestationHostAnnouncement {
return false
return errors.New("not a host announcement")
}
ha.PublicKey = a.PublicKey
ha.NetAddress = string(a.Value)
return true
d := types.NewBufDecoder(a.Value)
types.DecodeSlice(d, (*[]NetAddress)(ha))
lukechampine marked this conversation as resolved.
Show resolved Hide resolved
return d.Err()
}

// ToArbitraryData encodes a host announcement as arbitrary data.
Expand Down Expand Up @@ -93,11 +124,15 @@ func ForEachHostAnnouncement(b types.Block, fn func(HostAnnouncement)) {
}
}
}
}

// ForEachV2HostAnnouncement calls fn on each v2 host announcement in a block.
func ForEachV2HostAnnouncement(b types.Block, fn func(types.PublicKey, []NetAddress)) {
for _, txn := range b.V2Transactions() {
for _, a := range txn.Attestations {
var ha HostAnnouncement
if ha.FromAttestation(a) {
fn(ha)
var ha V2HostAnnouncement
if err := ha.FromAttestation(a); err == nil {
fn(a.PublicKey, ha)
}
}
}
Expand Down
58 changes: 53 additions & 5 deletions chain/hostannouncement_test.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
package chain

import (
"encoding/binary"
"math"
"testing"

"go.sia.tech/core/consensus"
"go.sia.tech/core/types"
"lukechampine.com/frand"
)

func TestForEachHostAnnouncement(t *testing.T) {
Expand All @@ -17,11 +20,6 @@ func TestForEachHostAnnouncement(t *testing.T) {
Transactions: []types.Transaction{
{ArbitraryData: [][]byte{ha.ToArbitraryData(sk)}},
},
V2: &types.V2BlockData{
Transactions: []types.V2Transaction{
{Attestations: []types.Attestation{ha.ToAttestation(consensus.State{}, sk)}},
},
},
}
ForEachHostAnnouncement(b, func(a HostAnnouncement) {
if a.PublicKey != sk.PublicKey() {
Expand All @@ -31,3 +29,53 @@ func TestForEachHostAnnouncement(t *testing.T) {
}
})
}

func TestForEachV2HostAnnouncement(t *testing.T) {
sk := types.GeneratePrivateKey()
ha := V2HostAnnouncement([]NetAddress{
{Protocol: "tcp", Address: "foo.bar:1234"},
{Protocol: "tcp6", Address: "baz.qux:5678"},
{Protocol: "webtransport", Address: "quux.corge:91011"},
})
randomAttestation := types.Attestation{
PublicKey: sk.PublicKey(),
Key: "foo",
Value: frand.Bytes(60),
}
cs := consensus.State{}
randomAttestation.Signature = sk.SignHash(cs.AttestationSigHash(randomAttestation))

invalidData := make([]byte, 100)
binary.LittleEndian.PutUint64(invalidData, math.MaxUint64)
extraBigAttestation := types.Attestation{
PublicKey: sk.PublicKey(),
Key: attestationHostAnnouncement,
Value: invalidData,
}
extraBigAttestation.Signature = sk.SignHash(cs.AttestationSigHash(extraBigAttestation))

b := types.Block{
V2: &types.V2BlockData{
Transactions: []types.V2Transaction{
{Attestations: []types.Attestation{
randomAttestation,
extraBigAttestation,
ha.ToAttestation(consensus.State{}, sk),
}},
},
},
}
ForEachV2HostAnnouncement(b, func(pk types.PublicKey, addresses []NetAddress) {
if pk != sk.PublicKey() {
t.Error("pubkey mismatch")
} else if len(addresses) != len(ha) {
t.Error("length mismatch")
} else {
for i := range addresses {
if addresses[i] != ha[i] {
t.Error("address mismatch:", addresses[i], ha[i])
}
}
}
})
}
Loading
Loading