From 814538b321fed7558206946fe0f32a690df70ffd Mon Sep 17 00:00:00 2001 From: Ilja Pavlovs <5300706+iljapavlovs@users.noreply.github.com> Date: Thu, 12 Sep 2024 18:37:46 +0300 Subject: [PATCH] DEVSVCS-134: adding readme on how to run VRF CTF tests (#14409) * DEVSVCS-134: adding readme on how to run VRF CTF tests * DEVSVCS-134: minor updates --- integration-tests/docs/VRF.md | 65 +++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 integration-tests/docs/VRF.md diff --git a/integration-tests/docs/VRF.md b/integration-tests/docs/VRF.md new file mode 100644 index 00000000000..0e74d66da93 --- /dev/null +++ b/integration-tests/docs/VRF.md @@ -0,0 +1,65 @@ +# How To Run VRF Tests +* All test configs should be placed in the [integration-tests/testconfig](integration-tests/testconfig) folder +* All test configs for running tests in live testnets should be under [integration-tests/testconfig/vrfv2plus/overrides](integration-tests/testconfig/vrfv2plus/overrides) folder + + +## Functional Tests +### In CI - using On Demand Workflows +```bash +gh workflow run "on-demand-vrfv2plus-smoke-tests.yml" \ +--ref develop \ +-f=test_secrets_override_key= \ +-f test_config_override_path= \ +-f test_suite="Selected Tests" \ # Optional, Options - "All Tests", "Selected Tests". Default is "All Tests". If "Selected Tests" is selected, then `test_list_regex` should be provided +-f test_list_regex="" \ # Optional, default is "TestVRFv2Plus$/(Link_Billing|Native_Billing|Direct_Funding)|TestVRFV2PlusWithBHS" which are P0 tests +-f chainlink_version="<>" # Optional, default is image created from develop branch. Not needed if you run tests against existing environment +-f notify_user_id_on_failure= # Optional, default is empty. If provided, will notify the user on slack if the tests fail +``` + +#### Examples: +Run P0 tests against existing environment (Staging) on Arbitrum Sepolia +```bash +gh workflow run "on-demand-vrfv2plus-smoke-tests.yml" \ +--ref develop \ +-f=test_secrets_override_key= \ +-f test_config_override_path=integration-tests/testconfig/vrfv2plus/overrides/staging/arbitrum_sepolia_staging_test_config.toml \ +-f test_suite="Selected Tests" +``` + +Run all tests deploying all contracts, CL nodes with `2.15.0` version on Base Sepolia +```bash +gh workflow run "on-demand-vrfv2plus-smoke-tests.yml" \ +--ref develop \ +-f=test_secrets_override_key= \ +-f test_config_override_path=integration-tests/testconfig/vrfv2plus/overrides/new_env/base_sepolia_new_env_test_config.toml \ +-f test_suite="All Tests" \ +-f chainlink_version="2.15.0" +``` + +### Locally +```bash +cd integration-tests +TEST_LOG_LEVEL=debug \ +BASE64_CONFIG_OVERRIDE=$(cat | base64) \ +go test -v -timeout 15m -run "" ./smoke +``` + +## Performance Tests +```bash +gh workflow run "on-demand-vrfv2plus-performance-test.yml" \ +--ref develop \ +-f=test_secrets_override_key= \ +-f test_config_override_path= \ +-f performanceTestType=“Smoke” # Options - "Smoke", "Soak", "Stress", "Load". +-f test_list_regex="" # Optional, default is "TestVRFV2PlusPerformance" +``` + +#### Examples: +Run SOAK tests against existing environment (Staging) on Base Sepolia +```bash +gh workflow run "on-demand-vrfv2plus-performance-test.yml" \ +--ref develop \ +-f=test_secrets_override_key= \ +-f test_config_override_path=integration-tests/testconfig/vrfv2plus/overrides/staging/base_sepolia_staging_test_config.toml \ +-f performanceTestType=“Soak” +```