Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added docker-file and edited the instructions for it #50

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Added docker-file and edited the instructions for it #50

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
325394f
Create Dockerfile
rpj09 Mar 16, 2023
021ee4a
Update README.md
rpj09 Mar 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Base image
FROM ubuntu:20.04
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use the Nvidia container toolkit so we can leverage NVIDIA GPUs

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure i will take care of that


# Set working directory
WORKDIR /app

# Update and install required packages
RUN apt-get update && \
apt-get install -y git-lfs wget && \
rm -rf /var/lib/apt/lists/*

# Download and install Miniconda
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
bash Miniconda3-latest-Linux-x86_64.sh -b -p /opt/conda && \
rm Miniconda3-latest-Linux-x86_64.sh

# Set conda to automatically activate base environment on login
RUN echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
echo "conda activate base" >> ~/.bashrc

# Create OpenChatKit environment
COPY environment.yml .
RUN conda env create -f environment.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should use mamba here instead. This can take a very long time to build the image.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This takes a while (forever) for me. Meanwhile it doesn't recognize conda yet, have to use /opt/conda/bin/conda instead of conda.


# Install Git LFS
RUN git lfs install

# Copy OpenChatKit code
COPY . .

# Prepare GPT-NeoX-20B model
RUN python pretrained/GPT-NeoX-20B/prepare.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be pretty large. It would be better to bind mount a directory and have the entrypoint call prepare.py.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @csris i am quite new to this project and community. can you help me out by explaining the project structure and whats the requirement in detail. I need a little bit of guidance regarding this project.


# Set entrypoint to bash shell
ENTRYPOINT ["/bin/bash"]