Skip to content

Commit

Permalink
Merge branch 'main' into self-signing-config
Browse files Browse the repository at this point in the history
  • Loading branch information
iansuvak authored Nov 21, 2024
2 parents 8dfb194 + 90ad76f commit d4da0a2
Show file tree
Hide file tree
Showing 7 changed files with 148 additions and 17 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ jobs:
with:
go-version-file: 'go.mod'

- name: Install AvalancheGo Release
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh

- name: Set subnet-evm version
run: |
source ./scripts/versions.sh
Expand All @@ -38,9 +41,6 @@ jobs:
repository: ava-labs/subnet-evm
ref: ${{ env.SUBNET_EVM_VERSION }}

- name: Install AvalancheGo Release
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh

- name: Build Subnet-EVM Plugin Binary
run: ./scripts/build.sh /tmp/e2e-test/avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy

Expand Down
14 changes: 6 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,27 +20,25 @@ Applications in this repository depend on the following upstream repositories, b
> [!NOTE]
> We require any commits referenced in our `main` branch to be present in the default branches of the repositories above, but during active development it might be useful to work against changes in progress that are still on feature branches.
When developing such features that require updates to one or more of the above, care must be taken to understand where the relevant code comes from. The binaries of applications built in this repo are built against versions referenced in the `go.mod` file. The E2E tests run against a simulated network running locally that is started by calling a separately compiled `avalanchego` binary as well as its plugins. These are compiled based on the values of `AVALANCHEGO_VERSION` in the local checkout of `subnet-evm` when running the tests locally and directly in this repository's `./scripts/versions.sh` when running E2E tests remotely through GitHub actions.
When developing such features that require updates to one or more of the above, care must be taken to understand where the relevant code comes from. The binaries of applications built in this repo are built against versions referenced in the `go.mod` file. The E2E tests run against a simulated network running locally that is started by calling a separately compiled `avalanchego` binary as well as its plugins. These are compiled based on the values of `AVALANCHEGO_VERSION` in this repository's `./scripts/versions.sh`.

`avalanchego` and `coreth` have a direct circular dependency and this repository is only indirectly dependent on `coreth` but directly dependent on `avalanchego`. Therefore if any updates are required from the `coreth` side, a corresponding `avalanchego` commit referencing those changes is required. On the other hand `subnet-evm` just depends directly on `avalanchego`.

> [!NOTE]
> It's possible that a `subnet-evm` version is compatible with multiple different `avalanchego` versions and not limited to the one listed in `subnet-evm`'s `/scripts/versions.sh`
### Example dependency update flow

The most complicated example case that can arise above is that a feature depends on a new change in `coreth`. And the steps below outline the necessary commits:
The most complicated example case that can arise above is that a feature depends on a new change in `coreth` and all versions are mutually incompatible. The steps below outline the necessary commits:

1. If an `avalanchego` commit referencing this change in its `go.mod` file doesn't exist yet then it needs to be added.
2. Add a commit in `subnet-evm` that references the `avalanchego` commit from above in both its `go.mod` file as well as its `scripts/versions.sh` file.
3. Create a new commit in this repository referencing `avalanchego` and `subnet-evm` directly and `coreth` indirectly as well as update references in the `scripts/version.sh` file for both `AVALANCHEGO_VERSION` and `SUBNET_EVM_VERSION`.

Publishing all of the commits mentioned above to GitHub branches will enable running E2E tests through the CI.

Running the tests locally doesn't require publishing the `subnet-evm` commit since `./scripts/e2e_test.sh` takes a flag specifying local checkout of `subnet-evm` repository.

> [!NOTE]
> Locally running E2E tests using local checkout of `subnet-evm` will install `avalanchego` version specified by the `AVALANCHEGO_VERSION` in that working tree's `./scripts/versions.sh`.
> [!TIP]
> Using the local checkout it's possible to run tests against a `tmpnet` consisting of nodes using a different version of `avalanchego` than the application being tested which might be helpful when troubleshooting.
> Running the tests locally doesn't require publishing the `subnet-evm` commit since `./scripts/e2e_test.sh` takes a flag specifying a local checkout of the `subnet-evm` repository.
## Releases
GoReleaser is used to build the binaries of the services and also Docker images with those binaries. The monorepo feature of GoReleaser Pro is used to automate the release flow in response to tags like `signature-aggregator/v0.0.0`. The release actions in .github/workflows automate this, but the release build can also be run locally. Be sure to install the "pro" distribution of the command line utility, so that it can parse the `monorepo` key. For example:
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/kms v1.37.6
github.com/ethereum/go-ethereum v1.13.14
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/onsi/ginkgo/v2 v2.21.0
github.com/onsi/ginkgo/v2 v2.22.0
github.com/onsi/gomega v1.35.1
github.com/pingcap/errors v0.11.4
github.com/pkg/errors v0.9.1
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -523,8 +523,8 @@ github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vv
github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=
github.com/onsi/ginkgo/v2 v2.21.0 h1:7rg/4f3rB88pb5obDgNZrNHrQ4e6WpjonchcpuBRnZM=
github.com/onsi/ginkgo/v2 v2.21.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo=
github.com/onsi/ginkgo/v2 v2.22.0 h1:Yed107/8DjTr0lKCNt7Dn8yQ6ybuDRQoMGrNFKzMfHg=
github.com/onsi/ginkgo/v2 v2.22.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo=
github.com/onsi/gomega v1.4.1/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
Expand Down
2 changes: 1 addition & 1 deletion scripts/e2e_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ if [ "$LOCAL" = true ]; then
exit 1
fi
cwd=$PWD
cd $SUBNET_EVM_PATH
BASEDIR=$DATA_DIRECTORY AVALANCHEGO_BUILD_PATH=$DATA_DIRECTORY/avalanchego ./scripts/install_avalanchego_release.sh
cd $SUBNET_EVM_PATH
./scripts/build.sh $DATA_DIRECTORY/avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy

cd $cwd
Expand Down
133 changes: 133 additions & 0 deletions scripts/install_avalanchego_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
# Copyright (C) 2024, Ava Labs, Inc. All rights reserved.
# See the file LICENSE for licensing terms.

# Following script is adapted from https://github.com/ava-labs/subnet-evm/blob/master/scripts/install_avalanchego_release.sh
set -e

# Load the versions
ICM_OFFCHAIN_CONTRACTS_PATH=$(
cd "$(dirname "${BASH_SOURCE[0]}")"
cd .. && pwd
)
source "$ICM_OFFCHAIN_CONTRACTS_PATH"/scripts/versions.sh

# Load the constants
source "$ICM_OFFCHAIN_CONTRACTS_PATH"/scripts/constants.sh

############################
# download avalanchego
# https://github.com/ava-labs/avalanchego/releases
GOARCH=$(go env GOARCH)
GOOS=$(go env GOOS)
BASEDIR=${BASEDIR:-"/tmp/avalanchego-release"}
AVALANCHEGO_BUILD_PATH=${AVALANCHEGO_BUILD_PATH:-${BASEDIR}/avalanchego}

mkdir -p ${BASEDIR}

AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${AVALANCHEGO_VERSION}/avalanchego-linux-${GOARCH}-${AVALANCHEGO_VERSION}.tar.gz
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-linux-${GOARCH}-${AVALANCHEGO_VERSION}.tar.gz

if [[ ${GOOS} == "darwin" ]]; then
AVAGO_DOWNLOAD_URL=https://github.com/ava-labs/avalanchego/releases/download/${AVALANCHEGO_VERSION}/avalanchego-macos-${AVALANCHEGO_VERSION}.zip
AVAGO_DOWNLOAD_PATH=${BASEDIR}/avalanchego-macos-${AVALANCHEGO_VERSION}.zip
fi

BUILD_DIR=${AVALANCHEGO_BUILD_PATH}-${AVALANCHEGO_VERSION}

extract_archive() {
mkdir -p ${BUILD_DIR}

if [[ ${AVAGO_DOWNLOAD_PATH} == *.tar.gz ]]; then
tar xzvf ${AVAGO_DOWNLOAD_PATH} --directory ${BUILD_DIR} --strip-components 1
elif [[ ${AVAGO_DOWNLOAD_PATH} == *.zip ]]; then
unzip ${AVAGO_DOWNLOAD_PATH} -d ${BUILD_DIR}
mv ${BUILD_DIR}/build/* ${BUILD_DIR}
rm -rf ${BUILD_DIR}/build/
fi
}

# first check if we already have the archive
if [[ -f ${AVAGO_DOWNLOAD_PATH} ]]; then
# if the download path already exists, extract and exit
echo "found avalanchego ${AVALANCHEGO_VERSION} at ${AVAGO_DOWNLOAD_PATH}"

extract_archive
else
# try to download the archive if it exists
if curl -s --head --request GET ${AVAGO_DOWNLOAD_URL} | grep "302" > /dev/null; then
echo "${AVAGO_DOWNLOAD_URL} found"
echo "downloading to ${AVAGO_DOWNLOAD_PATH}"
curl -L ${AVAGO_DOWNLOAD_URL} -o ${AVAGO_DOWNLOAD_PATH}

extract_archive
else
# else the version is a git commit (or it's invalid)
GIT_CLONE_URL=https://github.com/ava-labs/avalanchego.git
GIT_CLONE_PATH=${BASEDIR}/avalanchego-repo/

# check to see if the repo already exists, if not clone it
if [[ ! -d ${GIT_CLONE_PATH} ]]; then
echo "cloning ${GIT_CLONE_URL} to ${GIT_CLONE_PATH}"
git clone --no-checkout ${GIT_CLONE_URL} ${GIT_CLONE_PATH}
fi

# check to see if the commitish exists in the repo
WORKDIR=$(pwd)

cd ${GIT_CLONE_PATH}

git fetch

echo "checking out ${AVALANCHEGO_VERSION}"

set +e
# try to checkout the branch
git checkout origin/${AVALANCHEGO_VERSION} > /dev/null 2>&1
CHECKOUT_STATUS=$?
set -e

# if it's not a branch, try to checkout the commit
# Try to checkout the branch. If it fails, try the commit.
if ! git checkout "origin/${AVALANCHEGO_VERSION}" > /dev/null 2>&1; then
if ! git checkout "${AVALANCHEGO_VERSION}" > /dev/null 2>&1; then
# If the version is in the format of tag-commit, try to extract the commit and checkout.
AVALANCHEGO_VERSION=$(extract_commit "${AVALANCHEGO_VERSION}")
if ! git checkout "${AVALANCHEGO_VERSION}" > /dev/null 2>&1; then
echo
echo "'${AVALANCHEGO_VERSION}' is not a valid release tag, commit hash, or branch name"
exit 1
fi
fi
fi

COMMIT=$(git rev-parse HEAD)

# use the commit hash instead of the branch name or tag
BUILD_DIR=${AVALANCHEGO_BUILD_PATH}-${COMMIT}

# if the build-directory doesn't exist, build avalanchego
if [[ ! -d ${BUILD_DIR} ]]; then
echo "building avalanchego ${COMMIT} to ${BUILD_DIR}"
./scripts/build.sh
mkdir -p ${BUILD_DIR}

mv ${GIT_CLONE_PATH}/build/* ${BUILD_DIR}/
fi

cd $WORKDIR
fi
fi

AVALANCHEGO_PATH=${AVALANCHEGO_BUILD_PATH}/avalanchego
AVALANCHEGO_PLUGIN_DIR=${AVALANCHEGO_BUILD_PATH}/plugins

mkdir -p ${AVALANCHEGO_BUILD_PATH}
mkdir -p ${AVALANCHEGO_PLUGIN_DIR}

cp ${BUILD_DIR}/avalanchego ${AVALANCHEGO_PATH}


echo "Installed AvalancheGo release ${AVALANCHEGO_VERSION}"
echo "AvalancheGo Path: ${AVALANCHEGO_PATH}"
echo "Plugin Dir: ${AVALANCHEGO_PLUGIN_DIR}"
4 changes: 2 additions & 2 deletions signature-aggregator/aggregator/aggregator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -241,10 +241,10 @@ func TestCreateSignedMessageRetriesAndFailsWithoutP2PResponses(t *testing.T) {
).Times(maxRelayerQueryAttempts)

_, err = aggregator.CreateSignedMessage(msg, nil, subnetID, 80)
require.ErrorContains(
require.ErrorIs(
t,
err,
"failed to collect a threshold of signatures",
errNotEnoughSignatures,
)
}

Expand Down

0 comments on commit d4da0a2

Please sign in to comment.