Deploy specific github branch #67

phaleth · 2021-10-26T17:22:40Z

phaleth
Oct 26, 2021

With the Dockerfile bellow I'm trying to deploy a specific branch from this repo. I guess the main branch will throw the same error on the cargo install perseus-cli call. Not really sure what's missing. Is there a way to deploy directly from the repo?

Dockerfile

# get the base image
FROM rust:1.55-slim AS build

# install build dependencies
RUN apt update \
  && apt install -y --no-install-recommends lsb-release apt-transport-https \
  build-essential curl git

# prepare root project dir
WORKDIR /app

# download the target for wasm
RUN rustup target add wasm32-unknown-unknown

# install wasm-pack
RUN cargo install wasm-pack

# retrieve the src dir
RUN curl https://codeload.github.com/arctic-hen7/perseus-size-opt/tar.gz/main | tar -xz --strip=2 perseus-size-opt-main/examples/simple

# go to src dir
WORKDIR /app/simple

# install perseus-cli
RUN cargo install perseus-cli --git "https://github.com/arctic-hen7/perseus" --branch "feat-feature-gating"

# clean and prep app
RUN perseus clean && perseus prep

# specify deps in app config
RUN sed -i s'/perseus = .*/perseus = { git = "https:\/\/github.com\/arctic-hen7\/perseus", branch = "feat-feature-gating" }/' ./Cargo.toml \
  && sed -i s'/perseus-size-opt = .*/perseus-size-opt = { git = "https:\/\/github.com\/arctic-hen7\/perseus-size-opt", branch = "main" }/' ./Cargo.toml \
  && cat ./Cargo.toml

# modify lib.rs
RUN sed -i s'/SizeOpts::default()/SizeOpts { wee_alloc: true, lto: false, opt_level: "s".to_string(), codegen_units: 1, }/' ./src/lib.rs \
  && sed -i s'/"Hello World!"/"世界您好 !"/' ./src/lib.rs \
  && cat ./src/lib.rs

# run plugin(s) to adjust app
RUN perseus tinker \
  && cat .perseus/Cargo.toml \
  && cat ./src/lib.rs

# deploy app
RUN perseus deploy

# prepare deployment image
FROM bitnami/minideb:buster

WORKDIR /app

COPY --from=build /app/simple/pkg /app/

ENV PERSEUS_STANDALONE=true

ENV HOST=0.0.0.0

CMD ["./server"]

Docker build output

...
Step 8/19 : RUN cargo install perseus-cli --git "https://github.com/arctic-hen7/perseus" --branch "feat-feature-gating"
 ---> Running in c135707b175b
    Updating git repository `https://github.com/arctic-hen7/perseus`
  Installing perseus-cli v0.3.0-beta.13 (https://github.com/arctic-hen7/perseus?branch=feat-feature-gating#854c614f)
    Updating crates.io index
 Downloading crates ...
  Downloaded clap v3.0.0-beta.5
  Downloaded console v0.14.1
  Downloaded fmterr v0.1.1
  Downloaded include_dir v0.6.2
  Downloaded glob v0.3.0
  Downloaded include_dir_impl v0.6.2
  Downloaded fs_extra v1.2.0
  Downloaded anyhow v1.0.44
  Downloaded textwrap v0.14.2
  Downloaded number_prefix v0.4.0
  Downloaded indicatif v0.17.0-beta.1
  Downloaded os_str_bytes v4.2.0
  Downloaded clap_derive v3.0.0-beta.5
  Downloaded cargo_toml v0.9.2
  Downloaded proc-macro-hack v0.5.19
  Downloaded strsim v0.10.0
   Compiling proc-macro2 v1.0.30
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.80
   Compiling version_check v0.9.3
   Compiling libc v0.2.105
   Compiling serde v1.0.130
   Compiling anyhow v1.0.44
   Compiling memchr v2.4.1
   Compiling proc-macro-hack v0.5.19
   Compiling autocfg v1.0.1
   Compiling unicode-segmentation v1.8.0
   Compiling lazy_static v1.4.0
   Compiling unicode-width v0.1.9
   Compiling regex-syntax v0.6.25
   Compiling ryu v1.0.5
   Compiling serde_derive v1.0.130
   Compiling once_cell v1.8.0
   Compiling hashbrown v0.11.2
   Compiling serde_json v1.0.68
   Compiling strsim v0.10.0
   Compiling number_prefix v0.4.0
   Compiling glob v0.3.0
   Compiling bitflags v1.3.2
   Compiling itoa v0.4.8
   Compiling termcolor v1.1.2
   Compiling fs_extra v1.2.0
   Compiling fmterr v0.1.1
   Compiling proc-macro-error-attr v1.0.4
   Compiling proc-macro-error v1.0.4
   Compiling unicase v2.6.0
   Compiling textwrap v0.14.2
   Compiling heck v0.3.3
   Compiling indexmap v1.7.0
   Compiling regex v1.5.4
   Compiling os_str_bytes v4.2.0
   Compiling terminal_size v0.1.17
   Compiling atty v0.2.14
   Compiling quote v1.0.10
   Compiling console v0.15.0
   Compiling console v0.14.1
   Compiling indicatif v0.17.0-beta.1
   Compiling toml v0.5.8
   Compiling clap_derive v3.0.0-beta.5
   Compiling thiserror-impl v1.0.30
   Compiling include_dir_impl v0.6.2
   Compiling include_dir v0.6.2
   Compiling thiserror v1.0.30
   Compiling clap v3.0.0-beta.5
   Compiling cargo_toml v0.9.2
   Compiling perseus-cli v0.3.0-beta.13 (/usr/local/cargo/git/checkouts/perseus-7ea6a66d4aa78f48/854c614/packages/perseus-cli)
error: proc-macro derive panicked
  --> packages/perseus-cli/src/prepare.rs:16:24
   |
16 | const SUBCRATES: Dir = include_dir!("./.perseus");
   |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = help: message: "/usr/local/cargo/git/checkouts/perseus-7ea6a66d4aa78f48/854c614/packages/perseus-cli/./.perseus" doesn't exist
   = note: this error originates in the macro `include_dir` (in Nightly builds, run with -Z macro-backtrace for more info)

error: cannot find macro `proc_macro_call` in this scope
  --> packages/perseus-cli/src/prepare.rs:16:24
   |
16 | const SUBCRATES: Dir = include_dir!("./.perseus");
   |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: this error originates in the macro `include_dir` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `perseus-cli` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `perseus-cli v0.3.0-beta.13 (https://github.com/arctic-hen7/perseus?branch=feat-feature-gating#854c614f)`, intermediate artifacts can be found at `/tmp/cargo-installv8mhcB`

Caused by:
  build failed
The command '/bin/sh -c cargo install perseus-cli --git "https://github.com/arctic-hen7/perseus" --branch "feat-feature-gating"' returned a non-zero code: 101

Answered by arctic-hen7

Oct 26, 2021

Have a look at the Bonnie scripts, the repo itself is designed to be used in a self-contained manner (i.e. using its own bleeding edge version of thr CLI). Try bonnie setup and then bonnie dev example basic deploy. You'll then find a production-ready pkg/ in examples/basic/. You can install Bonnie with cargo install bonnie, and you can see the commands available for this repo with bonnie help.

With regards to the exact reason, installing the CLI needs the .perseus/ directory to actually be inlined, which is a special part of the deployment process. I may be able to refactor it to work like this though, but that may not be possible...

View full answer

arctic-hen7 · 2021-10-26T20:14:57Z

arctic-hen7
Oct 26, 2021
Maintainer

Have a look at the Bonnie scripts, the repo itself is designed to be used in a self-contained manner (i.e. using its own bleeding edge version of thr CLI). Try bonnie setup and then bonnie dev example basic deploy. You'll then find a production-ready pkg/ in examples/basic/. You can install Bonnie with cargo install bonnie, and you can see the commands available for this repo with bonnie help.

With regards to the exact reason, installing the CLI needs the .perseus/ directory to actually be inlined, which is a special part of the deployment process. I may be able to refactor it to work like this though, but that may not be possible...

6 replies

arctic-hen7 Oct 27, 2021
Maintainer

Yes, the perseus repo has its own version, but then perseus-size-opt is independent, so it uses the normal Perseus CLI that's deployed on crates.io. Also I don't think copying the example into the main repo will work, because there'd be a version mismatch between the plugin itself and the bleeding edge versions of Perseus. Also, there have been some breaking changes on that branch that would stuff things up. Unfortunately, you're going to have to wait for now if you want to get the size optimizations plugin to work with feature-gating, but I'll release the latest versions of both Perseus and that plugin soon!

phaleth Nov 7, 2021
Author

With the following Dockerfile where I use bonnie setup I also got perseus tinker command commented out, because the Perseus CLI on it's own, without Bonnie, does not work. Is there a way to run perseus tinker with Bonnie?

Also the build with this Dockerfile throws an error at the bonnie dev example simple deploy call.

# get the base image
FROM rust:1.55-slim AS build

# install build dependencies
RUN apt update \
  && apt install -y --no-install-recommends lsb-release apt-transport-https \
  build-essential curl

# vars
ENV PERSEUS_REPO=main \
    PERSEUS_VERSION=0.3.0-beta.17 \
    PERSEUS_SIZE_OPT_VERSION=0.1.2

# prepare root project dir
WORKDIR /app

# download the target for wasm
RUN rustup target add wasm32-unknown-unknown

# install wasm-pack
RUN cargo install wasm-pack

# retrieve the repo dir
RUN curl https://codeload.github.com/arctic-hen7/perseus/tar.gz/${PERSEUS_REPO} | tar -xz

# go to repo dir
WORKDIR /app/perseus-${PERSEUS_REPO}

# install bonnie
RUN cargo install bonnie

# install perseus-cli from repo
RUN bonnie setup

# go back to root project dir
WORKDIR /app

# retrieve the src dir
RUN curl https://codeload.github.com/arctic-hen7/perseus-size-opt/tar.gz/v${PERSEUS_SIZE_OPT_VERSION} | tar -xz --strip=2 perseus-size-opt-${PERSEUS_SIZE_OPT_VERSION}/examples/simple

# move simple example to repo examples
RUN mv /app/simple /app/perseus-${PERSEUS_REPO}/examples/simple

# go to src dir
WORKDIR /app/perseus-${PERSEUS_REPO}/examples/simple

# specify deps in app config
RUN sed -i s"/perseus = .*/perseus = \"${PERSEUS_VERSION}\"/" ./Cargo.toml \
  && sed -i s"/perseus-size-opt = .*/perseus-size-opt = \"${PERSEUS_SIZE_OPT_VERSION}\"/" ./Cargo.toml \
  && cat ./Cargo.toml

# modify lib.rs
RUN sed -i s'/SizeOpts::default()/SizeOpts { wee_alloc: true, lto: false, opt_level: "s".to_string(), codegen_units: 1, }/' ./src/lib.rs \
  && sed -i s'/"Hello World!"/"世界您好 !"/' ./src/lib.rs \
  && cat ./src/lib.rs

# run plugin(s) to adjust app
#RUN perseus tinker \
#  && cat .perseus/Cargo.toml \
#  && cat ./src/lib.rs

# go back to repo dir
WORKDIR /app/perseus-${PERSEUS_REPO}

# single-threaded perseus CLI mode
ENV PERSEUS_CLI_SEQUENTIAL=true

# deploy app
RUN bonnie dev example simple deploy

# prepare deployment image
FROM bitnami/minideb:buster

WORKDIR /app

COPY --from=build /app/simple/pkg /app/

ENV PERSEUS_STANDALONE=true

ENV HOST=0.0.0.0

CMD ["./server"]

arctic-hen7 Nov 7, 2021
Maintainer

Yeah, bonnie dev example <example-name> can be followed by anything you'd pass to Perseus. Just use bonnie dev example tiny tinker, that should work.

arctic-hen7 Nov 7, 2021
Maintainer

With that Dockerfile though, you'll need to set the Perseus version to be local, ../../packages/perseus. And I'm not sure that will be compatible with the size optimizations plugin...

phaleth Nov 8, 2021
Author

Let's just conclude the following for now. I wasn't able to find out how to deploy from repo with optimizations. Bellow is a Dockerfile that deploys the main branch. It produces a quite large docker image at the first build step. I guess it's good enough for quick tests.

# get the base image
FROM rust:1.55-slim AS build

# install build dependencies
RUN apt update \
  && apt install -y --no-install-recommends lsb-release apt-transport-https \
  build-essential curl

# vars
ENV PERSEUS_REPO=main

# prepare root project dir
WORKDIR /app

# download the target for wasm
RUN rustup target add wasm32-unknown-unknown

# install wasm-pack
RUN cargo install wasm-pack

# install bonnie
RUN cargo install bonnie

# retrieve the repo dir
RUN curl https://codeload.github.com/arctic-hen7/perseus/tar.gz/${PERSEUS_REPO} | tar -xz

# go to repo dir
WORKDIR /app/perseus-${PERSEUS_REPO}

# install perseus-cli from repo
RUN bonnie setup

# clean app
RUN bonnie dev example tiny clean

# go to src dir
WORKDIR /app/perseus-${PERSEUS_REPO}/examples/tiny

# modify lib.rs
RUN sed -i s'/"Hello World!"/"世界您好 !"/' ./src/lib.rs \
  && cat ./src/lib.rs

# go to repo dir
WORKDIR /app/perseus-${PERSEUS_REPO}

# single-threaded perseus CLI mode
#ENV PERSEUS_CLI_SEQUENTIAL=true

# deploy app
RUN bonnie dev example tiny deploy

# move repo dir
RUN mv /app/perseus-${PERSEUS_REPO} /app/perseus-repo

# prepare deployment image
FROM bitnami/minideb:buster

WORKDIR /app

COPY --from=build /app/perseus-repo/examples/tiny/pkg /app/

ENV PERSEUS_STANDALONE=true

ENV HOST=0.0.0.0

CMD ["./server"]

REPOSITORY          TAG               IMAGE ID       CREATED          SIZE
perseus-simple      2021-11-08        c1e833d71a48   17 seconds ago   78.9MB
<none>              <none>            a08c6ebaec96   20 seconds ago   9.86GB

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy specific github branch #67

{{title}}

Replies: 1 comment 6 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Deploy specific github branch #67

phaleth Oct 26, 2021

Replies: 1 comment · 6 replies

arctic-hen7 Oct 26, 2021 Maintainer

arctic-hen7 Oct 27, 2021 Maintainer

phaleth Nov 7, 2021 Author

arctic-hen7 Nov 7, 2021 Maintainer

arctic-hen7 Nov 7, 2021 Maintainer

phaleth Nov 8, 2021 Author

phaleth
Oct 26, 2021

Replies: 1 comment 6 replies

arctic-hen7
Oct 26, 2021
Maintainer

arctic-hen7 Oct 27, 2021
Maintainer

phaleth Nov 7, 2021
Author

arctic-hen7 Nov 7, 2021
Maintainer

arctic-hen7 Nov 7, 2021
Maintainer

phaleth Nov 8, 2021
Author