CCIP Load Test connected to crib (not yet working) #15404

0xAustinWang · 2024-11-25T15:43:07Z

https://smartcontract-it.atlassian.net/browse/CCIP-4107

github-actions · 2024-11-25T15:56:41Z

AER Report: CI Core

aer_workflow , commit , Detect Changes , Clean Go Tidy & Generate , Scheduled Run Frequency , Flakeguard Root Project / Get Tests To Run , Core Tests (go_core_tests) , Core Tests (go_core_tests_integration) , lint , Core Tests (go_core_ccip_deployment_tests) , Core Tests (go_core_race_tests) , Core Tests (go_core_fuzz) , Flakeguard Deployment Project / Get Tests To Run , Flakeguard Root Project / Run Tests , Flakeguard Root Project / Report , Flakeguard Deployment Project / Run Tests , Flakeguard Deployment Project / Report , Flakey Test Detection , SonarQube Scan

1. Not enough arguments in call to env.GetConfig: [go_core_ccip_deployment_tests]

Source of Error:

environment/crib/env_test.go:13:12: not enough arguments in call to env.GetConfig
	have ()
	want (map[uint64]string)

**Why**: The function `env.GetConfig` is being called without the required arguments. The function signature expects a `map[uint64]string` as an argument, but none is provided.

Suggested fix: Update the call to env.GetConfig to include the required map[uint64]string argument.

2. Assignment mismatch: 1 variable but env.GetConfig returns 2 values: [go_core_ccip_deployment_tests]

Source of Error:

environment/crib/env_test.go:13:12: assignment mismatch: 1 variable but env.GetConfig returns 2 values

**Why**: The function `env.GetConfig` returns two values, but only one variable is used to capture the return values.

Suggested fix: Update the assignment to capture both return values from env.GetConfig.

3. Execution reverted: [go_core_tests]

Source of Error:

{
  "run.ID": 5,
  "executionID": "400411a5-1fb1-4e00-8975-4da234a77d0a",
  "specID": 1,
  "jobID": 1,
  "jobName": "",
  "run.PipelineTaskRuns": [
    {
      "id": "2782c0ef-714a-4ced-a65b-32656823c1cb",
      "type": "ethcall",
      "output": null,
      "error": "execution reverted",
      "createdAt": "2024-12-18T10:54:14.788759497Z",
      "finishedAt": "2024-12-18T10:54:14.78941958Z",
      "index": -1,
      "dotId": "check_upkeep_tx"
    }
  ],
  "run.Errors": [
    "check_upkeep_tx(ethcall); execution reverted"
  ]
}

**Why**: The `ethcall` task in the pipeline encountered an "execution reverted" error, indicating that the Ethereum transaction failed during execution.

Suggested fix: Investigate the smart contract logic and ensure that the conditions for the transaction are met. Check for sufficient gas, correct input parameters, and contract state.

4. Task run cancelled (fail early): [go_core_tests]

Source of Error:

{
  "run.ID": 5,
  "executionID": "400411a5-1fb1-4e00-8975-4da234a77d0a",
  "specID": 1,
  "jobID": 1,
  "jobName": "",
  "run.PipelineTaskRuns": [
    {
      "id": "00000000-0000-0000-0000-000000000000",
      "type": "ethtx",
      "output": null,
      "error": "task run cancelled (fail early)",
      "createdAt": "2024-12-18T10:54:14.7894303Z",
      "finishedAt": "2024-12-18T10:54:14.7894303Z",
      "index": -1,
      "dotId": "perform_upkeep_tx"
    }
  ],
  "run.Errors": [
    "perform_upkeep_tx(ethtx); task run cancelled (fail early)"
  ]
}

**Why**: The `ethtx` task was cancelled early due to a preceding task failure in the pipeline, specifically the `ethcall` task.

Suggested fix: Resolve the issue causing the ethcall task to fail. Once the ethcall task succeeds, the subsequent tasks should proceed without early cancellation.

5. Timed out after 20.001s: [go_core_tests]

Source of Error:

integration_test.go:263: 
Timed out after 20.001s.
Expected
<[]uint8 | len:0, cap:0>: 
to equal
<[]uint8 | len:2, cap:2>: [18, 52]

**Why**: The test case timed out after 20 seconds, indicating that the expected condition was not met within the allotted time.

Suggested fix: Increase the timeout duration if the operation is expected to take longer. Alternatively, optimize the code to ensure it completes within the expected time frame.

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

AnieeG · 2024-12-03T16:45:03Z

deployment/environment.go

@@ -35,6 +35,7 @@ type OnchainClient interface {
 	bind.DeployBackend
 	BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)
 	NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
+	BlockByNumber(ctx context.Context, number *big.Int) (*types.Block, error)


We use HeaderByNumber(context, nil) to get the latest block. Can you use that instead?

AnieeG · 2024-12-03T16:49:06Z

integration-tests/load/ccip/helpers.go

+	SequenceNumber uint64    `json:"sequence_number"`
+}
+
+func GetAddressFromTypeAndVersion(ab deployment.AddressBook, cs uint64, tv string) (common.Address, error) {


is it being used anywhere?

AnieeG · 2024-12-03T16:50:16Z

integration-tests/load/ccip/helpers.go

+		"sourceSelector":      fmt.Sprintf("%d", src),
+		"destinationSelector": fmt.Sprintf("%d", dst),


would it be easier to use names/chain id instead? Labelling by selector can create confusion as selector is a large number

AnieeG · 2024-12-03T16:50:52Z

integration-tests/load/ccip/helpers.go

+
+func SendMetricsToLoki(l zerolog.Logger, lc *wasp.LokiClient, updatedLabels map[string]string, metrics *LokiMetric) {
+	if err := lc.HandleStruct(wasp.LabelsMapToModel(updatedLabels), time.Now(), metrics); err != nil {
+		l.Error().Err(err).Msg(ErrLokiPush)


do you not want the test to fail if it fails to push?

AnieeG · 2024-12-03T16:52:50Z

integration-tests/load/ccip/destination_gun.go

+
+	return router.ClientEVM2AnyMessage{
+		Receiver:     rcv,
+		Data:         common.Hex2Bytes("hello world"),


How about using some identifier here like msg with id to distinguish.

AnieeG · 2024-12-03T17:02:46Z

integration-tests/load/ccip/destination_gun.go

+		m.env.Chains[src].DeployerKey.Value = fee
+		defer func() { m.env.Chains[src].DeployerKey.Value = nil }()
+	}
+	_, err = r.CcipSend(


Are you approving router to spend fee tokens in the beginning?

AnieeG · 2024-12-03T17:03:06Z

integration-tests/load/ccip/destination_gun.go

+		m.chainSelector,
+		msg)
+	if err != nil {
+		return &wasp.Response{Error: err.Error(), Group: waspGroup, Failed: true}


same comment about using deployment.MayBeDataError(err)

AnieeG · 2024-12-03T17:04:05Z

integration-tests/load/ccip/destination_gun.go

+	}
+
+	lokiLabels := setLokiLabels(src, m.chainSelector)
+	SendMetricsToLoki(m.l, m.loki, lokiLabels, &LokiMetric{


is this a blocking call?

AnieeG · 2024-12-03T17:05:22Z

integration-tests/load/ccip/ccip_test.go

+// Parse all events from the simulated chains, send to Loki
+// step 4: teardown
+// Stop the chains, cleanup the environment
+func TestCCIPLoad_RPS(t *testing.T) {


A lot of code in this can be extracted to common functions. You might need more than one load test in future

integration-tests/load/ccip/ccip_test.go

0xAustinWang requested review from a team as code owners November 25, 2024 15:43

0xAustinWang requested review from winder, rstout, asoliman92, makramkd, dimkouv, mateusz-sekara and 0xnogo November 25, 2024 15:43

AnieeG reviewed Dec 3, 2024

View reviewed changes

0xAustinWang force-pushed the aw/CCIPLoad branch from 1eee96f to 2b94462 Compare December 11, 2024 11:14

0xAustinWang requested a review from a team as a code owner December 11, 2024 11:14

0xAustinWang marked this pull request as draft December 11, 2024 11:18

0xAustinWang added 6 commits December 17, 2024 15:21

add new ccip load test, compiles

cb1f984

fix build

b38d7da

make test runnable

af85019

rename

648bf68

in progress

1d87264

build

4f21404

0xAustinWang force-pushed the aw/CCIPLoad branch from 2b94462 to 4f21404 Compare December 17, 2024 08:51

0xAustinWang added 2 commits December 17, 2024 19:36

load test transmitting

8c01cc2

load test, have transaction, need to use event watcher

44152cb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CCIP Load Test connected to crib (not yet working) #15404

CCIP Load Test connected to crib (not yet working) #15404

0xAustinWang commented Nov 25, 2024

github-actions bot commented Nov 25, 2024 •

edited

Loading

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

AnieeG Dec 3, 2024

		"sourceSelector": fmt.Sprintf("%d", src),
		"destinationSelector": fmt.Sprintf("%d", dst),

CCIP Load Test connected to crib (not yet working) #15404

Are you sure you want to change the base?

CCIP Load Test connected to crib (not yet working) #15404

Conversation

0xAustinWang commented Nov 25, 2024

github-actions bot commented Nov 25, 2024 • edited Loading

AER Report: CI Core

1. Not enough arguments in call to env.GetConfig: [go_core_ccip_deployment_tests]

2. Assignment mismatch: 1 variable but env.GetConfig returns 2 values: [go_core_ccip_deployment_tests]

3. Execution reverted: [go_core_tests]

4. Task run cancelled (fail early): [go_core_tests]

5. Timed out after 20.001s: [go_core_tests]

AER Report: Operator UI CI ran successfully ✅

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 25, 2024 •

edited

Loading