Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix conda errors in release container #1750

Merged
merged 7 commits into from
Jun 25, 2024
Merged

Fix conda errors in release container #1750

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
17ca622
Ensure the conda-bld dir is initialized as an empty channel, avoids e…
dagardner-nv Jun 17, 2024
805c6ff
Ensure /opt/conda/conda-bld is a valid conda channel, preventing a co…
dagardner-nv Jun 18, 2024
0f81aa8
Switch from mamba to conda with libmamba
dagardner-nv Jun 18, 2024
5de3f2b
Remove 'conda index' for development target, not needed
dagardner-nv Jun 18, 2024
f07277e
Merge branch 'branch-24.06' into david-docker-compose-1744
dagardner-nv Jun 18, 2024
ed2ec5e
Merge branch 'branch-24.06' of github.com:nv-morpheus/Morpheus into d…
dagardner-nv Jun 24, 2024
2a469e7
Remove repeated comment
dagardner-nv Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 28 additions & 17 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
# ============== LEGEND =================================================== #
# [conda_bld_morpheus]
# /
# [base] -> [conda_env] -> [base_extended] -> [runtime]
# [base] -> [conda_env] -> [base_extended] -> [runtime_conda_create] -> [runtime]
# \
# [conda_env_dev] -> [development] -> [development_pydbg]
#
Expand Down Expand Up @@ -218,23 +218,14 @@ RUN --mount=type=cache,id=workspace_cache,target=/workspace/.cache,sharing=locke
cd ${MORPHEUS_ROOT_HOST} &&\
MORPHEUS_PYTHON_BUILD_STUBS=OFF CONDA_BLD_PATH=/opt/conda/conda-bld ./ci/conda/recipes/run_conda_build.sh morpheus

# ============ Stage: runtime ============
# Setup container for runtime environment
FROM base_extended as runtime

# ============ Stage: runtime_conda_create ============
# Setup conda for the runtime environment
FROM base_extended as runtime_conda_create
ARG MORPHEUS_ROOT_HOST
ARG CUDA_MAJOR_VER
ARG CUDA_MINOR_VER

# Only copy specific files/folders over that are necessary for runtime
COPY "${MORPHEUS_ROOT_HOST}/conda/environments/*.yaml" "./conda/environments/"
COPY "${MORPHEUS_ROOT_HOST}/docker" "./docker"
COPY "${MORPHEUS_ROOT_HOST}/docs" "./docs"
COPY "${MORPHEUS_ROOT_HOST}/examples" "./examples"
COPY "${MORPHEUS_ROOT_HOST}/models" "./models"
COPY "${MORPHEUS_ROOT_HOST}/scripts" "./scripts"
COPY "${MORPHEUS_ROOT_HOST}/*.md" "./"
COPY "${MORPHEUS_ROOT_HOST}/LICENSE" "./"
COPY "${MORPHEUS_ROOT_HOST}/conda/environments/runtime_cuda-${CUDA_MAJOR_VER}${CUDA_MINOR_VER}_arch-x86_64.yaml" "./conda/environments/"

# Mount Morpheus conda package build in `conda_bld_morpheus`
RUN --mount=type=bind,from=conda_bld_morpheus,source=/opt/conda/conda-bld,target=/opt/conda/conda-bld \
Expand All @@ -254,10 +245,30 @@ RUN --mount=type=bind,from=conda_bld_morpheus,source=/opt/conda/conda-bld,target
-c nvidia/label/dev \
-c pytorch \
-c defaults \
morpheus &&\
morpheus && \
/opt/conda/bin/conda env update --solver=libmamba -n morpheus --file \
conda/environments/runtime_cuda-${CUDA_MAJOR_VER}${CUDA_MINOR_VER}_arch-x86_64.yaml

/opt/conda/bin/conda env update --solver=libmamba -n morpheus --file conda/environments/runtime_cuda-${CUDA_MAJOR_VER}${CUDA_MINOR_VER}_arch-x86_64.yaml && \
conda clean -afy
# ============ Stage: runtime ============
dagardner-nv marked this conversation as resolved.
Show resolved Hide resolved
# Setup container for runtime environment
FROM runtime_conda_create as runtime

ARG MORPHEUS_ROOT_HOST

# Only copy specific files/folders over that are necessary for runtime
COPY "${MORPHEUS_ROOT_HOST}/conda/environments/*.yaml" "./conda/environments/"
COPY "${MORPHEUS_ROOT_HOST}/docker" "./docker"
COPY "${MORPHEUS_ROOT_HOST}/docs" "./docs"
COPY "${MORPHEUS_ROOT_HOST}/examples" "./examples"
COPY "${MORPHEUS_ROOT_HOST}/models" "./models"
COPY "${MORPHEUS_ROOT_HOST}/scripts" "./scripts"
COPY "${MORPHEUS_ROOT_HOST}/*.md" "./"
COPY "${MORPHEUS_ROOT_HOST}/LICENSE" "./"

# Ensure the conda-bld directory is indexed even if empty
dagardner-nv marked this conversation as resolved.
Show resolved Hide resolved
RUN /opt/conda/bin/conda clean -afy && \
# Ensure the conda-bld directory is indexed even if empty
/opt/conda/bin/conda index /opt/conda/conda-bld

# Use morpheus by default
CMD [ "morpheus" ]
Expand Down
4 changes: 2 additions & 2 deletions examples/digital_fingerprinting/production/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ WORKDIR /workspace/examples/digital_fingerprinting/

# Install DFP dependencies
RUN source activate morpheus \
&& mamba env update -n morpheus --file /workspace/conda/environments/examples_cuda-121_arch-x86_64.yaml
&& /opt/conda/bin/conda env update --solver=libmamba -n morpheus --file /workspace/conda/environments/examples_cuda-121_arch-x86_64.yaml

# Set the tracking URI for mlflow
ENV MLFLOW_TRACKING_URI="http://mlflow:5000"
Expand All @@ -53,7 +53,7 @@ FROM base as jupyter

# Install the jupyter specific requirements
RUN source activate morpheus &&\
mamba install -y -c conda-forge \
/opt/conda/bin/conda install --solver=libmamba -y -c conda-forge \
ipywidgets \
jupyter_contrib_nbextensions \
# notebook v7 is incompatible with jupyter_contrib_nbextensions
Expand Down
Loading