Jupyterlab guide and Docker update

JUPYTERLAB_GUIDE.md

@@ -0,0 +1,203 @@
+# JupyterLab Guide
+
+This guide contains tips and FAQs which help to avoid common issues that arise when using JupyterLab.  
+
+# Table of Contents
+### Tips
+1. [Open multiple notebook tabs](#open-multiple-notebook-tabs)
+2. [Check solutions or feedback](#check-solutions-or-feedback)
+3. [Check memory usage](#check-memory-usage)
+4. [Get a fresh copy of a notebook](#get-a-fresh-copy-of-a-notebook)
+5. [View hidden files](#view-hidden-files)
+6. [Clone a GitHub repo](#clone-a-github-repo)
+7. [Get a copy of everything](#get-a-copy-of-everything)
+
+### FAQs
+1. [Why did I get a zero/low-mark when I submitted everything on time?](#why-did-i-get-a-zerolow-mark-when-i-submitted-everything-on-time)
+2. [Why are none of my cells running?](#why-are-none-of-my-cells-running)
+3. [Why does my kernel keep dying?](#why-does-my-kernel-keep-dying)
+4. [Why did I get a zero on a question when all the tests passed?](#why-did-i-get-a-zero-on-a-question-when-all-the-tests-passed)
+5. [Why did I get a low mark on a question when all the tests passed?](#why-did-i-get-a-low-mark-on-a-question-when-all-the-tests-passed)
+6. [I made a typo in my code, can I still get marks for the question?](#i-made-a-typo-in-my-code-can-i-still-get-marks-for-the-question)
+7. [I am getting a 400 (Bad Request) or 404 (Not Found) error, how do I fix it?](#i-am-getting-a-400-bad-request-or-404-not-found-error-how-do-i-fix-it)
+8. [How do I get a blank worksheet or tutorial for practice?](#how-do-i-get-a-blank-worksheet-or-tutorial-for-practice)
+
+## Tips
+
+## Open multiple notebook tabs
+
+When working on a notebook, you might want to have multiple tabs of the same notebook open to easily reference previous questions.
+
+**DO NOT** open another browser tab, create another instance of JupyterLab, and open the same notebook. Doing so may overwrite the notebook you are working on with an empty or half-completed version.
+
+Instead, use JupyterLab's built in tab functionality to create another view for the notebook.
+
+To do so in JupyterLab, right click the tab that contains the notebook in question.
+
+On the dropdown menu, click **New View for Notebook**:    
+
+![Alt text](images/image1.png)
+
+This will create a split screen, showing a copy of the notebook on half of the screen.
+
+## Check solutions or feedback
+
+Solutions and feedback are uploaded to your home directory. This is a different directory to where assignments are stored.
+
+To get to your home directory, open JupyterLab and then click the small grey folder icon on top of the menu to the left that shows files and folders.
+
+![Alt text](images/image2.png)
+
+Then that same menu should show assignment solutions and feedback. Solutions are typically released a few days after the due date, feedback is released after assignments are graded (this may take a couple weeks as tutorials are manually graded).
+
+## Check memory usage
+
+Student JupyterLab instances are limited to 2GB (2048MB) of memory usage, if that limit is reached by running a notebook then the kernel will be killed and cells will stop running.
+
+To check how much memory JupyterLab is using, there is a indicator on the bottom left of the window:
+
+![Alt text](images/image3.png)
+
+The maximum amount of memory usage a notebook should use is around 1GB (1024MB). If you find that JupyterLab is hitting the limit very often, here are some steps you can take:
+
+### Restart your JupyterLab server
+
+On the top left, go to **File -> Hub Control Panel** and it will open a window with a red button with text **Stop My Server**, click the button and then click the **Log Out** button on the top right. 
+
+Close the browser tab and then open JupyterLab again from Canvas.
+
+### Check your code
+
+If restarting your server does not work, then there may be an issue with your code.
+
+Run your notebook one cell at a time. If you hit a cell that causes memory usage to hit the 2GB limit, examine the code in the cell to make sure there aren't any errors e.g. running a function on unfiltered data.
+
+## Get a fresh copy of a notebook
+
+If you need a new copy of a notebook, for example if you accidentally deleted a grading cell or want one for practice, first rename the notebook you have in your account currently.
+
+For example if you need a new copy of a `worksheet_wrangling.ipynb`, right-click it in the file menu and choose the **Rename** option and rename it to `worksheet_wrangling_old.ipynb`.
+
+Then close JupyterLab and then open the assignment again on Canvas. A new copy of the assignment should be created in place of the renamed assignment.
+
+## View hidden files
+
+*Warning*: Only delete hidden files that you have created, other hidden files may be used by the server
+
+To view hidden files, on the top left go to **View** and on the dropdown click **Show Hidden Files**
+
+## Clone a GitHub repo
+
+To clone a GitHub repo for collaborative work, follow these steps:
+
+1. Find the HTTPS repo link from GitHub and copy it to the clipboard
+
+![alt text](images/image5.png)
+
+2. Go to your JupyterLab and make sure you're in the home directory by clicking the small grey folder icon on the top left of the menu that shows your files and folders
+
+![Alt text](images/image2.png)
+
+3. Go to the JupyterLab git extension menu by clicking its tab on the left of JupyterLab:
+
+![alt text](images/image6.png)
+
+4. Click "Clone a Repository" and then paste the repo link from step 1 into the box and click Clone.
+
+![alt text](images/image7.png)
+
+The repo should now be in your home directory in a folder titled with the name of your repo.
+
+To get to your cloned repo, remember to navigate to the home directory as shown in step 2.
+
+## Get a copy of everything
+
+To get a zipped copy of all of your work at the end of term:
+
+1. Open a terminal by clicking the terminal icon when you first open JupyterLab from the Canvas homepage or in a new JupyterLab tab.
+
+![Alt text](images/image4.png)
+
+2. Enter the following command into the terminal:
+```
+cd /home/jovyan/work/ && zip -r backupjupyterlab.zip *
+```
+If you received a `Disk quota exceeded` error, then there are too many files. Delete any files or folders that you don't need (especially large .csv files) and try the command again.
+
+3. Go to your home folder [as described here](#check-solutions-or-feedback) and there should be a zip file. Right-click the file and click Download to download a copy onto your device. 
+
+## FAQs
+
+### Why did I get a zero/low-mark when I submitted everything on time?
+
+If you are aware of the above, then make sure that you are opening another notebook tab [in the correct way](#open-multiple-notebook-tabs). If you opened another browser tab with JupyterLab, then there is a chance your notebook was overwritten.
+
+Lastly, do not rename the assignment notebook file or any folders. The autograder relies on those names to collect submissions.
+
+### Why are none of my cells running?
+
+Before anything else, try restarting JupyterHub as described [here](#restart-your-jupyterlab-server).
+
+Make sure you're using a compatible browser. We recommend Chrome, Edge, or Firefox. Also update your browser to the latest version.
+
+Update your device's operating system to the latest version and reboot your device.
+
+Disable any extensions to check if they are affecting JupyterLab.
+
+Try opening JupyterLab in an incognito browser session.
+
+### Why does my kernel keep dying?
+
+Please read [Check memory usage](#check-memory-usage).
+
+### Why did I get a zero on a question when all the tests passed?
+
+You may have deleted a grading cell accidentally (grading cells are cells that contain test functions e.g. `test_3.2()`).
+
+The autograder relies on these cells to give students marks. Creating a new cell and then copying the test function into that cell will not fix the issue.
+
+If you do accidentally delete a grading cell, you should [get a fresh copy of the assignment notebook](#get-a-fresh-copy-of-a-notebook). Then, copy your work into the fresh notebook. 
+
+### Why did I get a low mark on a question when all the tests passed?
+
+Some questions have hidden tests that are not shown in the student version of the assignment.
+
+Your answer may have passed the visible tests but failed the hidden tests when it was being marked.
+
+Examine the feedback for the question carefully to see if hidden tests were run on your code. Instructions for accessing feedback are [here](#check-solutions-or-feedback).
+
+### I made a typo in my code, can I still get marks for the question?
+
+We will not be making exceptions for typos in autograded coding questions.
+
+Read the questions carefully as some questions request answers of a certain type (e.g. String).
+
+The questions include tests that give instant feedback on your code. Run those tests frequently to ensure that you haven't made any typos.
+
+### I am getting a 400 (Bad Request) or 404 (Not Found) error, how do I fix it?
+
+Before continuing with the instructions below, try JupyterLab again after restarting your browser.
+
+The errors could be due to your browser's cache and it may need to be cleared.
+
+Instructions:
+
+[For Edge](https://www.microsoft.com/en-us/edge/learning-center/how-to-manage-and-clear-your-cache-and-cookies?form=MA13I2)
+
+[For Chrome](https://support.google.com/accounts/answer/32050?hl=en&co=GENIE.Platform%3DDesktop)
+
+[For Firefox](https://support.mozilla.org/en-US/kb/how-clear-firefox-cache)
+
+After clearing your browser's cache, restart the browser and try opening JupyterLab again.
+
+If it still has the same error, try using Incognito or Private Browsing on your browser.
+
+Lastly, disable any browser extensions as they could interfere with JupyterLab. Also check that your browser is updated to its latest version.
+
+### How do I get a blank worksheet or tutorial for practice?
+
+Follow the instructions [here](#get-a-fresh-copy-of-a-notebook) to get a blank copy of a tutorial or worksheet.
+
+
+
+

dockerfiles/r-stat-201-grading/Dockerfile

@@ -1,17 +1,15 @@
 # Copyright (c) UBC-STAT Development Team.
 # Distributed under the terms of the Modified BSD License.
-FROM ubcstat/r-stat-201:latest
+# TODO: For testing purposes only, change to ubcstat/r-stat-201:latest
+FROM ubcstatit/r-stat-201-student:latest
 
-LABEL maintainer="Tiffany Timbers <tiffany.timbers@stat.ubc.ca>"
+LABEL maintainer="Brian Kim <brian.kim@stat.ubc.ca>"
 
-# install the rise slides extension
-RUN mamba install --quiet --yes "rise"
+# install rise slides extension, nbgrader, clean cache
+RUN mamba install --quiet --yes \
+    'nbgrader=0.8.2' && \
+    mamba clean --all -f -y
 
-# install nbgrader
-RUN mamba install --quiet --yes "nbgrader=0.7.1" \
-    && jupyter nbextension install --sys-prefix --py nbgrader --overwrite \
-    && jupyter nbextension enable --sys-prefix --py nbgrader \
-    && jupyter serverextension enable --sys-prefix --py nbgrader 
+# re-upgrade jupyter-server (mamba downgrades it)
+RUN pip install --upgrade jupyter-server
 
-# Clean the cache
-RUN mamba clean --all -f -y 

dockerfiles/r-stat-201/Dockerfile

@@ -1,47 +1,66 @@
 # Copyright (c) UBC-DSCI Development Team.
 # Distributed under the terms of the Modified BSD License.
-FROM jupyter/r-notebook
+FROM jupyter/minimal-notebook
 
-LABEL maintainer="Tiffany Timbers <[email protected]>"
+LABEL maintainer="Brian Kim <[email protected]>"
+
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 
 USER root
 
-# install vim
-RUN apt-get update && apt-get install -y vim
+# R pre-requisites
+RUN apt-get update --yes && \
+    apt-get install --yes --no-install-recommends \
+    fonts-dejavu \
+    unixodbc \
+    unixodbc-dev \
+    r-cran-rodbc \
+    gfortran \
+    gcc \
+    vim && \
+    apt-get clean && rm -rf /var/lib/apt/lists/*
+
+USER ${NB_UID}
+
+# Change workdir to $HOME/work so config files are preserved when bind mounting
+WORKDIR "${HOME}/work"
+
+# disable warnings that pollute build logs; seems to be related to the update to python 3.11
+# https://discourse.jupyter.org/t/debugger-warning-it-seems-that-frozen-modules-are-being-used-python-3-11-0/16544/12
+ENV PYDEVD_DISABLE_FILE_VALIDATION=1
+
+COPY conda-linux-64.lock /tmp/conda-linux-64.lock
+
+# Install R packages from lock file.
+RUN mamba update --quiet --file /tmp/conda-linux-64.lock && \
+	mamba clean --all -y -f && \
+	fix-permissions "${CONDA_DIR}" && \
+	fix-permissions "/home/${NB_USER}" && \
+	Rscript -e "devtools::install_github('UBC-MDS/[email protected]')" && \
+	Rscript -e "devtools::install_github('UBC-MDS/[email protected]')" 
 
+# Install pexpect from main branch to fix asyncio issue with jupyterlab_git
+RUN pip install --ignore-installed git+https://github.com/pexpect/pexpect.git@master
+
+# Disable the cell toolbar (which ignores metadata and students often accidentally click + delete grading cells)
+RUN jupyter labextension disable @jupyterlab/cell-toolbar-extension
+
+# Disable announcement pop-up on start
+RUN jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
+
+# Configure shortcuts-extension to remove Shift-M cell merge shortcut
+COPY config/shortcuts.jupyterlab-settings /home/${NB_USER}/.jupyter/lab/user-settings/\@jupyterlab/shortcuts-extension/shortcuts.jupyterlab-settings
+
+# Copy jupyter_server_config.py which allows students to see and delete hidden files
+COPY config/jupyter_server_config.py /home/${NB_USER}/.jupyter
+
+# Copy gitconfig that sets global default pull strategy to rebase
+COPY config/.gitconfig /home/${NB_USER}/
+
+# Make sure everything in the home folder is owned by NB_USER for running docker image locally.
+USER root
+RUN chown -R ${NB_USER} /home/${NB_USER}
 USER ${NB_UID}
 
-WORKDIR "${HOME}"
-
-# remove the "work/" directory added in an earlier layer...
-RUN rm -rf work
-
-# Install R packages
-RUN mamba install --quiet --yes \
-	'r-cowplot' \
-	'r-ggally' \
-	'r-gridextra' \
-	'r-infer' \
-	'r-kknn' \
-	'r-testthat' \
-	'r-rpostgres' \
-	'r-memoise' \
-	&& mamba clean --all -f -y \
-	&& fix-permissions "${CONDA_DIR}" \
-	&& fix-permissions "/home/${NB_USER}" \
-	&& Rscript -e "remotes::install_github('allisonhorst/[email protected]')" \
-	&& Rscript -e "install.packages('ISLR', repos='http://cran.us.r-project.org')" \
-	&& Rscript -e "devtools::install_github('UBC-MDS/[email protected]')" \
-	&& Rscript -e "devtools::install_github('UBC-MDS/[email protected]')" 
-
-# Install nbgitpuller, jupytext, jlab-git
-RUN pip install git+https://github.com/data-8/nbgitpuller@c23461d4f7ce47ec4e0d2bcfc3b2fde4939b80f8 \
-  && jupyter serverextension enable --sys-prefix nbgitpuller \
-  && pip install jupyterlab-git \
-  && pip install jupytext --upgrade \
-  && jupyter labextension install @techrah/text-shortcuts \
-  && jupyter lab build  
-
-# Install the extension to remove the shift+M merge shortcut 
-RUN mkdir -p /home/${NB_USER}/.jupyter/lab/user-settings/\@jupyterlab/shortcuts-extension
-COPY rm-merge-shortcut.py /home/${NB_USER}/.jupyter/lab/user-settings/\@jupyterlab/shortcuts-extension/shortcuts.jupyterlab-settings
+# Disable HEALTHCHECK for performance reasons
+HEALTHCHECK NONE