Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify logic around dev builds and rebuilding #72

Merged
merged 7 commits into from
Sep 4, 2024
+102 −49
Merged

Clarify logic around dev builds and rebuilding #72

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
720fbb9
Disable rebuild of dev docker imgs w --build false
Tristan-Wilson Aug 7, 2024
26ffe7d
Clarify logic around dev builds and rebuilding
Tristan-Wilson Aug 12, 2024
0e22925
No skip_build
Tristan-Wilson Aug 12, 2024
aae904b
Contracts and node images should rebuild on --init
Tristan-Wilson Aug 13, 2024
4732c5d
Declare variable before use for -u flag
Tristan-Wilson Aug 13, 2024
4c489a7
Change contracts to utils, add --force-build-utils
Tristan-Wilson Aug 14, 2024
a7c9f53
Merge branch 'master' into skip-rebuild-opt
Tristan-Wilson Sep 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 91 additions & 48 deletions test-node.bash
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash

set -e
set -eu

NITRO_NODE_VERSION=offchainlabs/nitro-node:v3.0.1-cf4b74e-dev
BLOCKSCOUT_VERSION=offchainlabs/blockscout:v1.0.0-c8db5b1
Expand Down Expand Up @@ -37,22 +37,30 @@ else
fi

run=true
force_build=false
validate=false
detach=false
blockscout=false
tokenbridge=false
l3node=false
consensusclient=false
redundantsequencers=0
dev_build_nitro=false
dev_build_blockscout=false
l3_custom_fee_token=false
l3_token_bridge=false
batchposters=1
devprivkey=b6b15c8cb491557369f3c7d2c287b053eb229daa9c22138887752191c9520659
l1chainid=1337
simple=true

# Use the dev versions of nitro/blockscout
dev_nitro=false
dev_blockscout=false

# Rebuild docker images
build_dev_nitro=false
build_dev_blockscout=false
build_contracts=false
build_node_images=false

while [[ $# -gt 0 ]]; do
case $1 in
--init)
Expand All @@ -61,6 +69,8 @@ while [[ $# -gt 0 ]]; do
read -p "are you sure? [y/n]" -n 1 response
if [[ $response == "y" ]] || [[ $response == "Y" ]]; then
force_init=true
build_contracts=true
build_node_images=true
echo
else
exit 0
Expand All @@ -70,28 +80,68 @@ while [[ $# -gt 0 ]]; do
;;
--init-force)
force_init=true
build_contracts=true
build_node_images=true
shift
;;
--dev)
simple=false
shift
if [[ $# -eq 0 || $1 == -* ]]; then
# If no argument after --dev, set both flags to true
dev_build_nitro=true
dev_build_blockscout=true
dev_nitro=true
build_dev_nitro=true
dev_blockscout=true
build_dev_blockscout=true
else
while [[ $# -gt 0 && $1 != -* ]]; do
if [[ $1 == "nitro" ]]; then
dev_build_nitro=true
dev_nitro=true
build_dev_nitro=true
elif [[ $1 == "blockscout" ]]; then
dev_build_blockscout=true
dev_blockscout=true
build_dev_blockscout=true
fi
shift
done
fi
;;
--build)
force_build=true
build_dev_nitro=true
build_dev_blockscout=true
build_contracts=true
build_node_images=true
shift
;;
--no-build)
build_dev_nitro=false
build_dev_blockscout=false
build_contracts=false
build_node_images=false
shift
;;
--build-dev-nitro)
build_dev_nitro=true
shift
;;
--no-build-dev-nitro)
build_dev_nitro=false
shift
;;
--build-dev-blockscout)
build_dev_blockscout=true
shift
;;
--no-build-dev-blockscout)
build_dev_blockscout=false
shift
;;
--build-contracts)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the solution.

Small comment:
This isn't just contracts, but also scripts and tokenbridge. Can we rename it? build-iternal? build-utils? something similar..

Complicating things:
contracts and tokenbridge are weird dockers. They both fetch external code and build it according to NITRO_CONTRACTS_BRANCH, TOKEN_BRIDGE_BRANCH env variables. It means that it's very possible that rebuild is necessary even though docker doesn't recognize it. So we need an option to force rebuilding them. (docker build --force? I'm not sure how to do it)

I think a manual --build-contracts/internal/other should force rebuilding, but --init and --init-force should just do a normal build.
So if someone wants to force rebuild they'll do --init --build-contracts

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the option name could have force inside, like --force-build-utils or something

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took your suggestions. The way to force rebuild with docker is --no-cache

build_contracts=true
shift
;;
--no-build-contracts)
build_contracts=false
shift
;;
--validate)
Expand Down Expand Up @@ -178,6 +228,7 @@ while [[ $# -gt 0 ]]; do
echo
echo OPTIONS:
echo --build rebuild docker images
echo --no-build don\'t rebuild docker images
echo --dev build nitro and blockscout dockers from source instead of pulling them. Disables simple mode
echo --init remove all data, rebuild, deploy new rollup
echo --pos l1 is a proof-of-stake chain \(using prysm for consensus\)
Expand All @@ -194,28 +245,18 @@ while [[ $# -gt 0 ]]; do
echo --no-tokenbridge don\'t build or launch tokenbridge
echo --no-run does not launch nodes \(useful with build or init\)
echo --no-simple run a full configuration with separate sequencer/batch-poster/validator/relayer
echo --build-dev-nitro rebuild dev nitro docker image
echo --no-build-dev-nitro don\'t rebuild dev nitro docker image
echo --build-dev-blockscout rebuild dev blockscout docker image
echo --no-build-dev-blockscout don\'t rebuild dev blockscout docker image
echo --build-contracts rebuild contracts related docker images
echo --no-build-contracts rebuild contracts related docker images
echo
echo script runs inside a separate docker. For SCRIPT-ARGS, run $0 script --help
exit 0
esac
done

if $force_init; then
force_build=true
fi

if $dev_build_nitro; then
if [[ "$(docker images -q nitro-node-dev:latest 2> /dev/null)" == "" ]]; then
force_build=true
fi
fi

if $dev_build_blockscout; then
if [[ "$(docker images -q blockscout:latest 2> /dev/null)" == "" ]]; then
force_build=true
fi
fi

NODES="sequencer"
INITIAL_SEQ_NODES="sequencer"

Expand Down Expand Up @@ -255,51 +296,52 @@ fi
if $blockscout; then
NODES="$NODES blockscout"
fi
if $force_build; then
echo == Building..
if $dev_build_nitro; then
if ! [ -n "${NITRO_SRC+set}" ]; then
NITRO_SRC=`dirname $PWD`
fi
if ! grep ^FROM "${NITRO_SRC}/Dockerfile" | grep nitro-node 2>&1 > /dev/null; then
echo nitro source not found in "$NITRO_SRC"
echo execute from a sub-directory of nitro or use NITRO_SRC environment variable
exit 1
fi
docker build "$NITRO_SRC" -t nitro-node-dev --target nitro-node-dev


if $dev_nitro && $build_dev_nitro; then
echo == Building Nitro
if ! [ -n "${NITRO_SRC+set}" ]; then
NITRO_SRC=`dirname $PWD`
fi
if $dev_build_blockscout; then
if $blockscout; then
docker build blockscout -t blockscout -f blockscout/docker/Dockerfile
fi
if ! grep ^FROM "${NITRO_SRC}/Dockerfile" | grep nitro-node 2>&1 > /dev/null; then
echo nitro source not found in "$NITRO_SRC"
echo execute from a sub-directory of nitro or use NITRO_SRC environment variable
exit 1
fi
docker build "$NITRO_SRC" -t nitro-node-dev --target nitro-node-dev
fi
if $dev_blockscout && $build_dev_blockscout; then
if $blockscout; then
echo == Building Blockscout
docker build blockscout -t blockscout -f blockscout/docker/Dockerfile
fi
fi

if $build_contracts; then
LOCAL_BUILD_NODES="scripts rollupcreator"
if $tokenbridge || $l3_token_bridge; then
LOCAL_BUILD_NODES="$LOCAL_BUILD_NODES tokenbridge"
fi
docker compose build --no-rm $LOCAL_BUILD_NODES
fi

if $dev_build_nitro; then
if $dev_nitro; then
docker tag nitro-node-dev:latest nitro-node-dev-testnode
else
docker pull $NITRO_NODE_VERSION
docker tag $NITRO_NODE_VERSION nitro-node-dev-testnode
fi

if $dev_build_blockscout; then
if $blockscout; then
if $blockscout; then
if $dev_blockscout; then
docker tag blockscout:latest blockscout-testnode
fi
else
if $blockscout; then
else
docker pull $BLOCKSCOUT_VERSION
docker tag $BLOCKSCOUT_VERSION blockscout-testnode
fi
fi

if $force_build; then
if $build_node_images; then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we need this as a separate step?
isn't everything this can do covered by previous builds?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only place we're building $NODES that I could find in the code.

docker compose build --no-rm $NODES scripts
fi

Expand Down Expand Up @@ -419,6 +461,7 @@ if $force_init; then
echo l3owneraddress $l3owneraddress
docker compose run scripts --l2owner $l3owneraddress write-l3-chain-config

EXTRA_L3_DEPLOY_FLAG=""
if $l3_custom_fee_token; then
echo == Deploying custom fee token
nativeTokenAddress=`docker compose run scripts create-erc20 --deployer user_fee_token_deployer --mintTo user_token_bridge_deployer --bridgeable $tokenbridge | tail -n 1 | awk '{ print $NF }'`
Expand Down
Loading