Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC API compatibility test #227

Merged
merged 23 commits into from
Sep 30, 2024
Merged

RPC API compatibility test #227

merged 23 commits into from
Sep 30, 2024

Conversation

piersy
Copy link

@piersy piersy commented Sep 18, 2024

Adds a test that is segregated from normal compilation by a build tag which compares the RPC API outputs of a celo-blockchain node and an op-geth node using a migrated version of the datadir used by the celo node.

The intent is to ensure that our cel2 nodes will return rpc api responses for historical data that are consistent with the celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url `<celo rpc url>`  -op-geth-url `<op-geth rpc url>`

The build tag disables static analysis in vscode so if you still want that you need to add the build tag to your preferences:

{
  ...
  "go.buildTags": "compat_test",
  ...
}

Closes https://github.com/celo-org/celo-blockchain-planning/issues/363

@piersy piersy requested review from karlb and alecps September 18, 2024 13:51
Copy link

@karlb karlb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just some typo fixes and thoughts.

I skipped objdiff during the review.

compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
for _, tx := range r.celoTxs {
// This sets the y value to be a big number where abs is nul rather than a zero length array if the numebr is zero.
// It doesn't change the number but it does change the representation.
types.SetYNullStyleBigIfZero(tx)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow I don't see where SetYNullStyleBigIfZero is defined.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, when I enabled the build tag it disabled static analysis for the test, meaning when I checked for usages of SetYNullStyleBigIfZero I found none, when in fact the test was calling it. Will reinstate.

compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
compat_test/compat_test.go Outdated Show resolved Hide resolved
@piersy
Copy link
Author

piersy commented Sep 19, 2024

@karlb, unfortunately celo8 pulled in a change which has broken the test.

It's the use of new atomic types such as atomic.Pointer inside blocks and transactions. Atomic pointer values that share the same underlying values are not considered equal by deep equal. I tried adding some admittedly hacky workaround in the EqualObjects method, but it's not working for all cases. You can see some discussion on how to handle atomic.Pointer here google/go-cmp#326

My next approach would be to write some manual comparison functions for transactions blocks and headers, unless you have any better ideas.

@karlb
Copy link

karlb commented Sep 20, 2024

My next approach would be to write some manual comparison functions for transactions blocks and headers, unless you have any better ideas.

Disappointingly, I don't see a better way to do it. Explicit comparison functions seem to be the way to go.

@piersy piersy requested a review from karlb September 24, 2024 15:31
@piersy
Copy link
Author

piersy commented Sep 24, 2024

Hi @karlb I've updated the test now to use manual comparison functions for blocks and transactions. Also I added a "fix" for json unmarshaling transactions received from the celo node. That "fix" can be removed once we have done all the migrations we need to do.

We need to be able to support json decoding celo legacy transactions
from a celo node to allow the comparison test to fetch and compare
transactions from both celo and op-geth nodes. Unfortunately due to C
compilation issues it's not feasible to import and use the celo
ethclient.Client for this.
@piersy piersy force-pushed the piersy/rpc-api-compatibility-test branch from b3354ec to 9b18b54 Compare September 24, 2024 20:24
@piersy piersy merged commit 13f6cb6 into celo8 Sep 30, 2024
7 checks passed
@piersy piersy deleted the piersy/rpc-api-compatibility-test branch September 30, 2024 12:54
piersy added a commit that referenced this pull request Oct 3, 2024
Adds a test which compares the RPC API outputs of a celo-blockchain node
and an op-geth node using a migrated version of the datadir used by the
celo node.

The test is segregated from normal compilation by the build tag
'compat_test'.

The intent of this test is to ensure that our cel2 nodes will return rpc
api responses for historical data that are consistent with the
celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url '<celo rpc url>'  -op-geth-url '<op-geth rpc url>'

The build tag disables static analysis in vscode so if you still want
that you need to add the build tag to your preferences:
{
  ...
  "go.buildTags": "compat_test",
  ...
}

Also updates json transaction unmarshaling to be able to handle
unmarshaling from a celo-node, and adds a utility function in the types
package to help make transactions comparable. We'll want to remove these
at a later date since they exist solely support running this test.

Co-authored-by: Karl Bartel <[email protected]>
karlb added a commit that referenced this pull request Oct 11, 2024
Adds a test which compares the RPC API outputs of a celo-blockchain node
and an op-geth node using a migrated version of the datadir used by the
celo node.

The test is segregated from normal compilation by the build tag
'compat_test'.

The intent of this test is to ensure that our cel2 nodes will return rpc
api responses for historical data that are consistent with the
celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url '<celo rpc url>'  -op-geth-url '<op-geth rpc url>'

The build tag disables static analysis in vscode so if you still want
that you need to add the build tag to your preferences:
{
  ...
  "go.buildTags": "compat_test",
  ...
}

Also updates json transaction unmarshaling to be able to handle
unmarshaling from a celo-node, and adds a utility function in the types
package to help make transactions comparable. We'll want to remove these
at a later date since they exist solely support running this test.

Co-authored-by: Karl Bartel <[email protected]>
piersy added a commit that referenced this pull request Oct 15, 2024
Adds a test which compares the RPC API outputs of a celo-blockchain node
and an op-geth node using a migrated version of the datadir used by the
celo node.

The test is segregated from normal compilation by the build tag
'compat_test'.

The intent of this test is to ensure that our cel2 nodes will return rpc
api responses for historical data that are consistent with the
celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url '<celo rpc url>'  -op-geth-url '<op-geth rpc url>'

The build tag disables static analysis in vscode so if you still want
that you need to add the build tag to your preferences:
{
  ...
  "go.buildTags": "compat_test",
  ...
}

Also updates json transaction unmarshaling to be able to handle
unmarshaling from a celo-node, and adds a utility function in the types
package to help make transactions comparable. We'll want to remove these
at a later date since they exist solely support running this test.

Co-authored-by: Karl Bartel <[email protected]>
piersy added a commit that referenced this pull request Oct 15, 2024
Adds a test which compares the RPC API outputs of a celo-blockchain node
and an op-geth node using a migrated version of the datadir used by the
celo node.

The test is segregated from normal compilation by the build tag
'compat_test'.

The intent of this test is to ensure that our cel2 nodes will return rpc
api responses for historical data that are consistent with the
celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url '<celo rpc url>'  -op-geth-url '<op-geth rpc url>'

The build tag disables static analysis in vscode so if you still want
that you need to add the build tag to your preferences:
{
  ...
  "go.buildTags": "compat_test",
  ...
}

Also updates json transaction unmarshaling to be able to handle
unmarshaling from a celo-node, and adds a utility function in the types
package to help make transactions comparable. We'll want to remove these
at a later date since they exist solely support running this test.

Co-authored-by: Karl Bartel <[email protected]>
karlb added a commit that referenced this pull request Dec 13, 2024
Adds a test which compares the RPC API outputs of a celo-blockchain node
and an op-geth node using a migrated version of the datadir used by the
celo node.

The test is segregated from normal compilation by the build tag
'compat_test'.

The intent of this test is to ensure that our cel2 nodes will return rpc
api responses for historical data that are consistent with the
celo-blockchain node.

Run it like this:

go test -v ./compat_test -tags compat_test -celo-url '<celo rpc url>'  -op-geth-url '<op-geth rpc url>'

The build tag disables static analysis in vscode so if you still want
that you need to add the build tag to your preferences:
{
  ...
  "go.buildTags": "compat_test",
  ...
}

Also updates json transaction unmarshaling to be able to handle
unmarshaling from a celo-node, and adds a utility function in the types
package to help make transactions comparable. We'll want to remove these
at a later date since they exist solely support running this test.

Co-authored-by: Karl Bartel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants