Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wctl refactoring: Concrete data structures and code cleanup #319

Closed
wants to merge 155 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
155 commits
Select commit Hold shift + click to select a range
07e1ffe
Remove gossip mechanism. (#276)
Oct 15, 2019
e78a1ec
[WIP] Remove transaction creator and graph. (#277)
Oct 16, 2019
ce9e286
ledger: fix a nil pointer error
Oct 16, 2019
82ae4ab
Implement pulling transactions with bloom filter (#278)
Oct 17, 2019
6572405
ledger, graph: remove graph, and nop
Oct 17, 2019
0353f41
ledger: remove FinalizedRounds()
Oct 17, 2019
8d65d95
wavelet: implement block proposal
Oct 17, 2019
e731f91
wavelet: fix bloom filter rebuild
Oct 17, 2019
6f7ceea
utils: remove unused method
Oct 17, 2019
2cf75cf
ledger, collapse, mempool: move BloomFilter into Mempool, refactor Me…
Oct 17, 2019
5955b0a
testutil: temporarily comment out test code
Oct 17, 2019
6444707
protocol: fix response not using mempool
Oct 17, 2019
9153d02
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 17, 2019
a60938c
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 17, 2019
de831af
all: replace [blake2b.Size256]byte with BlockID or TransactionID
Oct 17, 2019
80159bb
collapse, ledger: replace [32]byte with TransactionID
Oct 17, 2019
b91ff99
round: remove files related to round vote: remove unused import
Oct 17, 2019
a98fb15
block: replace variadic with slice
Oct 17, 2019
9138c32
wavelet: fix state syncing to use mempool and blocks
Oct 17, 2019
057d548
wavelet: fix rebase
Oct 17, 2019
27da56a
wavelet: improve mempool pruning
Oct 17, 2019
699671f
protocol, rpc: update CheckOutOfSync protocol
Oct 17, 2019
6f07f28
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 17, 2019
c9469f1
wavelet: remove unused variable
Oct 17, 2019
3017126
ledger, round, rounds, vote, protocol, collapse, common: remove entir…
iwasaki-kenta Oct 17, 2019
d006449
Merge remote-tracking branch 'origin/himitsu' into himitsu
iwasaki-kenta Oct 17, 2019
cb205bd
ledgeR: rename RoundIndex to BlockIndex for OutOfSyncRequest
iwasaki-kenta Oct 17, 2019
efb83f2
ledger: rename RoundIndex to BlockIndex for OutOfSyncRequest
iwasaki-kenta Oct 17, 2019
c0d08b1
ledger, vote: refactor query to extract the tally calculation
Oct 17, 2019
35fede1
block, genesis, ledger: replace variadic parameter with slice, fix co…
Oct 17, 2019
0accdd3
vote: remove commented function
Oct 17, 2019
1b8ccae
tx, log, api: remove parent ids, parent seeds, seed, seed length, pre…
iwasaki-kenta Oct 17, 2019
567ca4f
Merge remote-tracking branch 'origin/himitsu' into himitsu
iwasaki-kenta Oct 17, 2019
0b735cc
cmd/wavelet, ledger, mempool: cleanup mempool and implement/document …
iwasaki-kenta Oct 17, 2019
59a687b
ledger: remove LatestBlockID and LatestBlockIndex funcs, and instead …
iwasaki-kenta Oct 17, 2019
1ba2717
wavelet: add comments to mempool
Oct 17, 2019
e972c63
wavelet: refactor mempool
Oct 17, 2019
08ab6ce
wavelet: remove redundant method
Oct 17, 2019
5650cec
ledger, protocol, rpc: rename DownloadMissingTx to PullTransactions
iwasaki-kenta Oct 17, 2019
fb8144c
Merge remote-tracking branch 'origin/himitsu' into himitsu
iwasaki-kenta Oct 17, 2019
b8b8742
ledger: fix deadlock in FinalizeBlocks
iwasaki-kenta Oct 17, 2019
6fc0add
protocol: simplify bloom filter instantiation to prevent allocs
iwasaki-kenta Oct 17, 2019
64ce2da
tx: remove AttachSenderToTransaction and modify NewTransaction to tak…
iwasaki-kenta Oct 17, 2019
19a48d5
tx: add check that the tag is correct on unmarshal
iwasaki-kenta Oct 17, 2019
3b3f46c
metrics, tx/json, ledger, protocol, collapse, conf, cmd/wavelet: repl…
iwasaki-kenta Oct 17, 2019
f4df3b7
ledger: remove further trace of rounds
iwasaki-kenta Oct 17, 2019
f14fa04
tx/applier, db, collapse, cmd/wavelet: replace further traces of roun…
iwasaki-kenta Oct 17, 2019
61d6d82
ledger, protocol, snowball, vote: make two different snowball impleme…
Oct 17, 2019
57410c9
wavelet: implement block finalization
Oct 17, 2019
3179a5c
vote, snowball, protocol, ledger: refactor snowball to support both s…
Oct 17, 2019
b15aa8a
api, cmd/wavelet: fix build errors
Oct 18, 2019
c5c73b4
conf/test: fix tests
iwasaki-kenta Oct 18, 2019
b8e822b
ledger: check if block proposed is nil
iwasaki-kenta Oct 18, 2019
f9bbe2f
tx, protocol: fix logging message intent, and also fix transaction un…
iwasaki-kenta Oct 18, 2019
470ad1a
ledger: fix deadlock
Oct 18, 2019
23058e3
ledger: collapse transactions when proposing and querying block
Oct 18, 2019
39aa4ef
ledger, protocol, snowball, vote: refactor snowball and vote calculat…
Oct 18, 2019
7846aab
ledger: fix mempool is reshuffled before collapse. only call collapse…
Oct 18, 2019
a245a95
mempool: fix reshuffle does not delete the block's transactions
Oct 18, 2019
95d98dc
mempool: update reshuffle comment
Oct 18, 2019
50dfb9a
tx, actions: add block index to transaction
iwasaki-kenta Oct 18, 2019
a7c9389
api: fix compile error
Oct 18, 2019
1301951
ledger, mempool, protocol: don't delete tx in the map during shufflin…
Oct 18, 2019
fe634c4
mempool, ledger: rename some of the methods to make it more descriptive
Oct 18, 2019
2ff4db6
ledger: as an experiment, have the entire mempool proposed as a block
iwasaki-kenta Oct 18, 2019
a7d21cf
ledger: fix filter finalization vote
Oct 18, 2019
521f167
vote: add comments
Oct 18, 2019
5b2e3a0
vote: fix voteID of syncVote
Oct 18, 2019
a5a2282
ledger: calculate merkle root when proposing block
Oct 18, 2019
036b139
ledger: check merkle root before ticking snowball
Oct 18, 2019
dae7d2e
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 18, 2019
cddb1c8
block: fix block marshaling
Oct 18, 2019
c4e1af0
protocol: refactor query response for readability
Oct 18, 2019
80d75a7
wavelet: remove ignored tx count
Oct 18, 2019
446b6b9
wavelet: keep track of actual tx count in block
Oct 18, 2019
798fb33
block: preallocate the buffer
Oct 18, 2019
40234ac
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 18, 2019
d3a13ac
ledger: add debug logs
Oct 18, 2019
d801b81
ledger: fix wrong block index check
Oct 18, 2019
aba340a
ledger: add missing checks after collapse
Oct 18, 2019
ba9ec0b
block: increase preallocated buffer
Oct 18, 2019
c0ac1cc
collapse: add context to error missing transaction
Oct 18, 2019
3e62db3
ledger, vote: use pointer type for finalizationVote
Oct 18, 2019
1f3c40e
ledger, vote: use pointer type for syncVote
Oct 18, 2019
8624d3a
snowball: modify snowball to be same as simulator
Oct 18, 2019
fb8e4f3
ledger: temporarily disable sync
Oct 18, 2019
16d6a91
ledger: add more debug logs
Oct 18, 2019
2d6d2ab
wavelet: reenable syncing
Oct 18, 2019
471edd0
ledger: temporarily disable pruning
Oct 18, 2019
acecd85
protocol: don't log zero transactions
Oct 18, 2019
3a898ff
vote/test: fix tests
Oct 18, 2019
ded13e7
Merge branch 'himitsu' of github.com:perlin-network/wavelet into himitsu
Oct 18, 2019
0f82171
ledger: remove unused code
Oct 18, 2019
cc29d11
ledger: change type finalizationVote to pointer. remove unused import
Oct 18, 2019
1cf43be
cmd/wavelet: uncomment preferred id
Oct 19, 2019
daefd7c
wavelet, conf: fix transactions not being pulled
Oct 19, 2019
6e56d4a
cmd/benchmark, wavelet: add block_finalized metric
Oct 19, 2019
c2ad888
ledger: use first quarter of mempool for block proposal
Oct 19, 2019
73849fa
ledger: fix goroutine not stopped
Oct 19, 2019
da85900
ledger: fix missing waitgroup add
Oct 19, 2019
10f53b8
ledger: use SelectPeers for PullTransactions, and propose only 1/3rd …
iwasaki-kenta Oct 20, 2019
259434f
conf: increase snowball beta to 150
iwasaki-kenta Oct 20, 2019
c0c127d
[himitsu] Refactor transactions in Mempool (#282)
Oct 21, 2019
d070f01
ledger: increase period in which transactions are pulled
iwasaki-kenta Oct 21, 2019
97bafe2
cmd/benchmark: only send 1 transaction
iwasaki-kenta Oct 21, 2019
cf747ba
actions: initial workflow
iwasaki-kenta Oct 21, 2019
1f69de3
actions; rename to yml
iwasaki-kenta Oct 21, 2019
b47f985
db, tx: improve bytes.Buffer performance by setting the buffer capaci…
Oct 21, 2019
342b8f3
Send list of tx id instead of bloom filter for pulling transactions (…
Oct 21, 2019
b9ed376
ledger: refactor query() to make it easier to understand, and remove …
Oct 21, 2019
c7b7143
ledger: reduce the calling of conf.GetSnowballK()
Oct 21, 2019
819dcfc
ledger: fix PullTransactions() to pull from K peers instead of just one
Oct 21, 2019
675324b
Minor fix for a panic on failed pulls (#285)
rkeene Oct 22, 2019
5a0ce05
Nonce checking and support. (#286)
iwasaki-kenta Oct 22, 2019
2f14556
cmd/benchmark: remove local command
iwasaki-kenta Oct 22, 2019
fbd96a9
Remove mempool, and cleanup locks around transactions with helper met…
iwasaki-kenta Oct 22, 2019
19c3eb9
tx/test: fix test errors (#289)
iwasaki-kenta Oct 22, 2019
a514455
api/msg: remove redundant type checks for SendTransactionRequest unma…
iwasaki-kenta Oct 22, 2019
414adff
transactions: implement a missing transactions index, which indexes i…
iwasaki-kenta Oct 23, 2019
60a2c49
ledger, transactions: have tx manager keep track of the latest block …
iwasaki-kenta Oct 23, 2019
46154db
transactions: create off-by-one error tests for pruning, tests for pr…
iwasaki-kenta Oct 23, 2019
634f2d4
Initial work for having client SDKs maintain local nonce and block he…
Oct 23, 2019
8c3dd2a
Properly increment nonce and block height in wctl client (#294)
Oct 24, 2019
3f88743
cmd/wavelet: fix build errors
Oct 24, 2019
63cf42d
Add more debug info into cmd/wavelet, and make what transactions are…
iwasaki-kenta Oct 24, 2019
2b9cb54
wavelet: update noise version (#298)
Oct 24, 2019
46f5421
Minor cleanup over transactions, and fix off-by-one error in wctl. (#…
iwasaki-kenta Oct 24, 2019
9ac8d68
Logging fixes. (#300)
a-urth Oct 24, 2019
65ed06d
cmd/benchmark: add comment to easiyl switch from benchmarking stake t…
iwasaki-kenta Oct 24, 2019
8808768
events, wctl, ledger: log num transactions pruned upon finalizing a b…
iwasaki-kenta Oct 24, 2019
aa52ab1
Refactor Snowball and add unit tests (#304)
Oct 24, 2019
fba3936
Merge API refactor commit
diamondburned Oct 8, 2019
824deca
Minor refactor
diamondburned Oct 25, 2019
09fd9bf
WIP api refactor
diamondburned Oct 25, 2019
e04daf6
Refactored API
diamondburned Oct 26, 2019
cc966e9
WIP
diamondburned Oct 26, 2019
e8fbb01
WIP Loggable interface, finished UnmarshalValue
diamondburned Oct 28, 2019
98a7d2c
WIP logget
diamondburned Oct 29, 2019
b4f33ec
WIP
diamondburned Oct 30, 2019
ffbd851
WIP
diamondburned Nov 1, 2019
98d48be
Added MiniLog, added Value marshals, WIP wctl callbacks
diamondburned Nov 28, 2019
13c0228
Simplified even more the logging API, removed old chunky callback system
diamondburned Nov 29, 2019
fc1c436
Improvements, now compile
diamondburned Nov 29, 2019
f3e0b05
Everything except cmd/wavelet builds
diamondburned Nov 30, 2019
ffca591
Stable-ish state
diamondburned Nov 30, 2019
bb86bf9
Fixed ValueHex
diamondburned Dec 11, 2019
72fb3fb
WIP merge
diamondburned Jan 11, 2020
c322527
made wavelet compile again, merged fully with master
diamondburned Jan 11, 2020
ccd1325
Completed merging, CLI test mostly passes
diamondburned Jan 12, 2020
7e3505c
Integration test for CLI now fully passes
diamondburned Jan 12, 2020
27ab9da
API test now passes
diamondburned Jan 12, 2020
b97aa36
Cleaned up wctl even further
diamondburned Jan 13, 2020
9e5b44a
Fixed api unit tests
diamondburned Jan 13, 2020
88ee1bb
Fixed api integration tests
diamondburned Jan 13, 2020
cd14617
Removed redundant things
diamondburned Jan 13, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions accounts_events.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
package wavelet

import (
"github.com/perlin-network/wavelet/log"
"github.com/rs/zerolog"
"github.com/valyala/fastjson"
)

// Balance Update

// balance_updated
type AccountBalanceUpdated struct {
AccountID AccountID `json:"account_id"`
Balance uint64 `json:"balance"`
}

var _ log.Loggable = (*AccountBalanceUpdated)(nil)

func (a AccountBalanceUpdated) MarshalEvent(ev *zerolog.Event) {
marshalAccountIDandUint64(ev, a.AccountID, "balance", a.Balance, "balance")
}

func (a *AccountBalanceUpdated) UnmarshalValue(v *fastjson.Value) error {
return unmarshalAccountIDandUint64(v, a.AccountID, "balance", &a.Balance)
}

// Gas Balance update

// gas_balance_updated
type AccountGasBalanceUpdated struct {
AccountID AccountID `json:"account_id"`
GasBalance uint64 `json:"gas_balance"`
}

var _ log.Loggable = (*AccountGasBalanceUpdated)(nil)

func (a AccountGasBalanceUpdated) MarshalEvent(ev *zerolog.Event) {
marshalAccountIDandUint64(ev, a.AccountID, "gas_balance", a.GasBalance, "gas balance")
}

func (a *AccountGasBalanceUpdated) UnmarshalValue(v *fastjson.Value) error {
return unmarshalAccountIDandUint64(v, a.AccountID, "gas_balance", &a.GasBalance)
}

// Num Pages update

// num_pages_updated
type AccountNumPagesUpdated struct {
AccountID AccountID `json:"account_id"`
NumPages uint64 `json:"num_pages"`
}

var _ log.Loggable = (*AccountNumPagesUpdated)(nil)

func (a AccountNumPagesUpdated) MarshalEvent(ev *zerolog.Event) {
marshalAccountIDandUint64(ev, a.AccountID, "num_pages", a.NumPages, "pages")
}

func (a *AccountNumPagesUpdated) UnmarshalValue(v *fastjson.Value) error {
return unmarshalAccountIDandUint64(v, a.AccountID, "num_pages", &a.NumPages)
}

// Stake update

// stake_updated
type AccountStakeUpdated struct {
AccountID AccountID `json:"account_id"`
Stake uint64 `json:"stake"`
}

var _ log.Loggable = (*AccountStakeUpdated)(nil)

func (a AccountStakeUpdated) MarshalEvent(ev *zerolog.Event) {
marshalAccountIDandUint64(ev, a.AccountID, "stake", a.Stake, "stake")
}

func (a *AccountStakeUpdated) UnmarshalValue(v *fastjson.Value) error {
return unmarshalAccountIDandUint64(v, a.AccountID, "stake", &a.Stake)
}

// Reward update

// reward_updated
type AccountRewardUpdated struct {
AccountID AccountID `json:"account_id"`
Reward uint64 `json:"reward"`
}

var _ log.Loggable = (*AccountRewardUpdated)(nil)

func (a AccountRewardUpdated) MarshalEvent(ev *zerolog.Event) {
marshalAccountIDandUint64(ev, a.AccountID, "reward", a.Reward, "reward")
}

func (a *AccountRewardUpdated) UnmarshalValue(v *fastjson.Value) error {
return unmarshalAccountIDandUint64(v, a.AccountID, "reward", &a.Reward)
}

func marshalAccountIDandUint64(ev *zerolog.Event, id AccountID, ukey string, u uint64, updated string) {
ev.Hex("account_id", id[:])
ev.Uint64(ukey, u)
ev.Msg("Updated account " + updated + ".")
}

// yes, I know this is a terrible function name, it can't be helped
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think this comment is necessary.

func unmarshalAccountIDandUint64(v *fastjson.Value, id AccountID, ukey string, u *uint64) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it definitely should be renamed at least to something containing amount instead of uint64.
also maybe 2 separate functions might be used - one for account and another one for amount? even though there will be two function calls, intention and naming will be much cleaner.

if err := log.ValueHex(v, id[:], "account_id"); err != nil {
return err
}

*u = v.GetUint64(ukey)
return nil
}
Loading