diff --git a/DevOps/frankfurter/README.md b/DevOps/frankfurter/README.md deleted file mode 100644 index 83c29b8d..00000000 --- a/DevOps/frankfurter/README.md +++ /dev/null @@ -1,101 +0,0 @@ -# frankfurter -Various tools and scripts to make the deployment life easier. - -## Before starting... -Note that since all Beaglebone blacks have the same static IP settings by default, after logging -into one (which happens implicitly when you run any of the make targets described below) it is -necessary to remove the corresponding entry in `.ssh/known_hosts` before logging a different Beaglebone -is possible. - -## Setting up a Beaglebone Black - -### Base Installation - -The preferred way of bringing up a Beaglebone is: - -1. Get a prebuilt micro SD card from the DevOps box (currently, they are labeled "Frank III"). -2. Insert the card into the Beaglebone you wish to flash while the Beaglebone is off. -3. Hold down the `S2` button, and power on the Beaglebone. -4. The Beaglebone will have booted up the image on the card instead of the one in flash memory. `ssh ubuntu@192.168.7.2` in. -5. To commit the image to flash memory, run `sudo /opt/scripts/tools/eMMC/bbb-eMMC-flasher-eewiki-ext4.sh` ([see here](https://github.com/RobertCNelson/boot-scripts/tree/master/tools/eMMC)). -6. Wait about 15 minutes for the image to be flashed. -7. Power down with `sudo shutdown now`. -8. Remove the card. -9. Continue to the [team-specific configuration instructions](#team-specific-configuration). - -Alternatively, - -1. Download a [stock Ubuntu prebuilt image](http://elinux.org/BeagleBoardUbuntu#eMMC:_All_BeagleBone_Varients_with_eMMC) (preferably 16.04). -2. Follow the instructions given by the link in step 1 to bring up a Beaglebone running stock Ubuntu. -3. Be sure you can `ssh ubuntu@192.168.7.2`. See the [networking section](#networking) for setup instructions. -4. Connect the Beaglebone to the internet. You should be able to get a response from `ping 8.8.8.8`, but not necessarily from `ping google.com`. One way to do this is to run `frankfurter/scripts/tools/usb-fwd.sh`, possibly as `sudo`. -5. On your machine, run `frankfurter/scripts/install/start_frank_install.sh`. Type in `ubuntu`'s password as many times as is necessary. -6. `ssh` in, and run `start_tmux_job.sh`. Again, enter `ubuntu`'s password as necessary. -7. After the kernel is upgraded (to ensure `linux-headers-$(uname -r)` is available), the Beaglebone will automatically reboot. As instructed, reconnect the Beaglebone to the internet after the reboot, `ssh` in, and run `tmux new-session -d '~/PieCentral/DevOps/frankfurter/scripts/install/master_frank_setup.sh'`. -8. Wait about one hour for the Beaglebone to configure and install all dependencies (most of this time is used to install the `linux-headers` package and build the wireless dongle driver). You can check on the status of your installation with `tmux attach`. -9. Power down with `sudo shutdown now`. -10. Continue to the [team-specific configuration instructions](#team-specific-configuration). - -### Team-Specific Configuration - -1. Power on the Beaglebone. -2. Navigate to `frankfurter/resources`. -3. Run `python3 interfaces.py`. Follow the prompts. This configures the Beaglebone to connect to a team-specific router. -4. Navigate to `frankfurter/scripts/update`. -5. Acquire or build an update zipped tarball that is placed in `frankfurter/build`, and run `./deploy_update.sh`. -6. Navigate to `frankfurter/scripts/tools`. -7. Run `./usb-fix.sh`. The Beaglebone should reboot. - -## `runtime` and `memcached` - -`runtime` is started on boot by `runtime.service` by `systemd` (see pioneers/PieCentral#100, installed to `/lib/systemd/system`). - -To interact with the runtime service, run `sudo systemctl runtime.service`, where `` can be: -* `start` -* `restart` -* `stop` -* `status` - -`memcached` has been configured to run on port `12357`. See `resources/memcached.conf`. - -## Installed Executables - -After setup, the following executables are available in `/home/ubuntu/bin`: - -| Name | Purpose | Notes | -| ------------ | ------------------------------------------------------------------------------------------------------------------ | ----- | -| `mac.py` | Print the MAC address of a network interface, which is passed as the first argument (see `--help` for details) | None | -| `runtime.sh` | Modify `PYTHONPATH` to include the `runtime` and `hibike` directories and execute `runtime` | Invoked by `runtime.service` (added to `systemd`) | -| `update.sh` | Extract all the tarballs in `/home/ubuntu/updates` and installs `hibike` and `runtime` in `/home/ubuntu/PieCentral` | Backs up the last version of `PieCentral` to `PieCentral.backup` | - -## Update creation -The `create_update` make target downloads the newest versions of runtime, hibike, etc, and packages them -into a tarball. From here, we can take the files created (they're placed in the -`frankfurter/build` directory) and distribute them to teams, or push them [online](https://pioneers.github.io/dawn/) for Dawn to see. - -## Update deployment -`deploy_update` assumes that you're USB tethered to a Beaglebone and then uploads and installs an update -to the connected beaglebone, creating a new update using `create_update` if the `frankfurter/build` directory does not exist. - -## Networking - -### [Unsupported] USB - -**With the implementation of the USB fix, `ssh` over USB is now disabled. These instructions remain for working with pre-USB-fix Beaglebones.** - -To `ssh` over USB: - -1. Power on the Beaglebone with a mini-USB cable. -2. Try `ssh ubuntu@192.168.7.2`. If your computer can see the Beaglebone (that is, you get a `Connection refused` error or receive a prompt for `ubuntu`'s password), stop. Otherwise, continue. -3. Run `ifconfig` and identify the name of the interface shared with the Beaglebone. -4. Run `ifconfig 192.168.7.1`. This may require `sudo`. -5. Once connected, the Beaglebone will be available at `192.168.7.2`. - -### FTDI USB to Serial Adapter - -See [these instructions](https://dave.cheney.net/2013/09/22/two-point-five-ways-to-access-the-serial-console-on-your-beaglebone-black). - -## Some Notes -1. Beaglebones are set-up using the master branch of PieCentral repository at the time of set-up. -2. When we issue updates we will package both the runtime and hibike directory in a tarball hosted on GitHub pages which will then be extracted into the correct locations on the Beaglebone. -3. Student code is stored in `~/PieCentral/runtime/studentCode.py`. diff --git a/DevOps/frankfurter/resources/runtime.service b/DevOps/frankfurter/resources/runtime.service deleted file mode 100644 index af043982..00000000 --- a/DevOps/frankfurter/resources/runtime.service +++ /dev/null @@ -1,15 +0,0 @@ -[Unit] -Description=Robot runtime daemon -After=sysinit.target -Wants=update.service -AssertPathExists=/home/ubuntu/PieCentral/runtime -AssertPathExists=/home/ubuntu/PieCentral/hibike - -[Service] -Type=simple -WorkingDirectory=/home/ubuntu/PieCentral/runtime -Environment="PYTHONPATH=/home/ubuntu/PieCentral/hibike:/home/ubuntu/PieCentral/runtime" -ExecStart=/usr/bin/env python3 runtime.py - -[Install] -WantedBy=sysinit.target diff --git a/DevOps/frankfurter/resources/update.service b/DevOps/frankfurter/resources/update.service deleted file mode 100644 index 9e338c84..00000000 --- a/DevOps/frankfurter/resources/update.service +++ /dev/null @@ -1,12 +0,0 @@ -[Unit] -Description=Robot software updater -After=sysinit.target -AssertPathExists=/home/ubuntu/bin/update.sh - -[Service] -Type=oneshot -ExecStart=/home/ubuntu/bin/update.sh - -[Install] -WantedBy=sysinit.target -WantedBy=runtime.service diff --git a/DevOps/frankfurter/scripts/install/bootstrap b/DevOps/frankfurter/scripts/install/bootstrap deleted file mode 100755 index afdf8071..00000000 --- a/DevOps/frankfurter/scripts/install/bootstrap +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/bash - -# bootstrap -- Raspberry Pi-side script that bootstraps the installation process. -# -# Runs interactively within an SSH client. -# Designed to exit quickly so that setup is largely autonomous. - -# Enable passwordless sudo for the default user -echo "$USER ALL=(ALL) NOPASSWD:ALL" | sudo tee --append "/etc/sudoers.d/$USER" -sudo chmod 0440 "/etc/sudoers.d/$USER" - -# Install installation service -mkdir -p "$HOME/.config/systemd/user" -mv "$HOME/install.service" "$HOME/.config/systemd/user" -systemctl --user daemon-reload && systemctl --user enable install.service - -# Remove settings that cause network unavailability on boot -sudo sed -i "/auto eth0/d" /etc/network/interfaces -sudo sed -i "/iface eth0 inet dhcp/d" /etc/network/interfaces - -# Add Google's DNS server -echo "nameserver 8.8.8.8" | sudo tee --append /etc/resolvconf/resolv.conf.d/head -echo "nameserver 8.8.4.4" | sudo tee --append /etc/resolvconf/resolv.conf.d/head - -sudo reboot diff --git a/DevOps/frankfurter/scripts/install/master_setup b/DevOps/frankfurter/scripts/install/master_setup deleted file mode 100755 index d628bf00..00000000 --- a/DevOps/frankfurter/scripts/install/master_setup +++ /dev/null @@ -1,90 +0,0 @@ -#!/bin/bash - -# master_setup -- Configures a generic frankfuter robot -# -# Team-specific configuration (e.g. networking) is handled separately. -# -# This script assumes the Raspberry Pi initially runs vanilla Ubuntu and has an -# internet connection through its host (see 'usb_net_fwd'). -# -# Should not be used in production (too slow). Images should instead be replicated by -# pulling a base image off a microSD card. - -wifi_driver_src="$HOME/wifi-driver-src" - -function download_wifi_driver { - git clone "https://github.com/xtknight/mt7610u-linksys-ae6000-wifi-fixes.git" "$wifi_driver_src" -} - -function upgrade_kernel { - sudo /opt/scripts/tools/update_kernel.sh -} - -function install_wifi_driver { - cd "$wifi_driver_src" - make clean - make - sudo make install - echo 'mt7610u_sta' | sudo tee --append /etc/modules -} - -function install_apt_packages { - sudo apt update -y - sudo apt upgrade -y - # FIXME: small issue with `/etc/issue` needing to be replaced - sudo apt install -y man-db make build-essential gcc git vim tmux htop curl zip unzip linux-headers-$(uname -r) - sudo apt install -y python3 python3-dev python3-pip # Python dependencies - sudo apt install -y devmem2 - sudo apt clean -y - sudo apt autoremove -y -} - -function disable_apache { - sudo systemctl stop apache2 - sudo systemctl disable apache2 - sudo systemctl daemon-reload -} - -function init_homedir { - mkdir -p "$HOME/bin" "$HOME/updates" "$HOME/student-code" - git init "$HOME/student-code" - mv "$HOME/update.sh" "$HOME/bin" && chmod +x "$HOME/bin/update.sh" - - mv "$HOME/update.service" "$HOME/.config/systemd/user" - systemctl --user daemon-reload && systemctl --user enable update.service -} - -stage_flags="$HOME/install-flags" -mkdir -p "$stage_flags" - -kernel_upgraded_flag="$stage_flags/kernel-upgraded" -wifi_driver_flag="$stage_flags/wifi-driver" -completed_flag="$stage_flags/completed" - -sudo /sbin/route add default gw "192.168.7.1" -if [ ! -f "$kernel_upgraded_flag" ]; then - upgrade_kernel - touch "$kernel_upgraded_flag" -elif [ ! -f "$wifi_driver_flag" ]; then - install_apt_packages - download_wifi_driver - install_wifi_driver - touch "$wifi_driver_flag" -else - # Set date and time to PST - sudo timedatectl set-timezone America/Los_Angeles - disable_apache - - # Install Python packages - sudo pip3 install pipenv - # FIXME: can fail sometimes with duplicate dependencies that conflict - sudo pip3 install -r "$HOME/requirements.txt" - - sudo usermod -a -G dialout "$USER" - sudo mv "$HOME/interfaces" "/etc/network/interfaces" - init_homedir - - touch "$completed_flag" -fi - -sudo reboot diff --git a/DevOps/frankfurter/scripts/install/start_install b/DevOps/frankfurter/scripts/install/start_install deleted file mode 100755 index 16e4372b..00000000 --- a/DevOps/frankfurter/scripts/install/start_install +++ /dev/null @@ -1,38 +0,0 @@ -#!/bin/bash - -# start_install -- Staff-run script to begin customizing an Ubuntu image for PiE. -# -# The Beaglebone must have internet access (e.g. through USB using `usb_net_fwd`). - -source "$(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env" -echo "Starting frankfurter installation ..." -echo -e $warning"Remember to enable wifi on the Beaglebone with 'usb_net_fwd'."$clear - -ssh-copy-id $ssh_options "$user@$default_ip" 2>/dev/null - -echo "Copying scripts ..." -resources=( - "$frankfurter/resources/interfaces" - "$frankfurter/resources/update.sh" - "$frankfurter/resources/install.service" - "$frankfurter/resources/update.service" - "$frankfurter/scripts/install/master_setup" - "$frankfurter/scripts/install/bootstrap" -) -for resource in ${resources[@]}; do - scp "$resource" "$user@$default_ip:~" 2> /dev/null -done - -# Export Pipfiles into merged requirements.txt -echo "Building consolidated 'requirements.txt' ..." -requirements="$tmp_dir/requirements.txt" -make_requirements > "$requirements" -scp "$requirements" "$user@$default_ip:~" > /dev/null -rm "$requirements" - -ssh -t $ssh_options "$user@$default_ip" "/home/$user/bootstrap" 2>/dev/null -echo -e $success"Installation process now entering autonomous mode."$clear -echo "Your Beaglebone should be ready in about 15 minutes." -echo -echo -e "You can verify your Beaglebone is ready by checking whether the file '~/install-flags/completed' exists." -echo -e $warning"The Beaglebone may periodically reboot during installation."$clear diff --git a/DevOps/frankfurter/scripts/tools/add_team_config b/DevOps/frankfurter/scripts/tools/add_team_config deleted file mode 100755 index e0a2972b..00000000 --- a/DevOps/frankfurter/scripts/tools/add_team_config +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/bash - -# add_team_config -- Team-specific configuration -# -# 1. Updates Runtime on the robot. -# 2. Deploys team-specific network configuration. - -source "$(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env" - -deploy_update -make_team_net diff --git a/DevOps/frankfurter/scripts/tools/env b/DevOps/frankfurter/scripts/tools/env deleted file mode 100755 index 4c2805cd..00000000 --- a/DevOps/frankfurter/scripts/tools/env +++ /dev/null @@ -1,62 +0,0 @@ -#!/bin/bash - -# env -- A collection of useful variables -# -# To use (from PieCentral): -# $ source "$(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env" - -piecentral=$(git rev-parse --show-toplevel) -protos="$piecentral/ansible-protos" -runtime="$piecentral/runtime" -hibike="$piecentral/hibike" -dawn="$piecentral/dawn" -api_docs="$dawn/static/website-robot-api-master" -frankfurter="$piecentral/DevOps/frankfurter" -frank_resources="$frankfurter/resources" - -user=ubuntu -eth_ip=192.168.6.2 -usb_ip=192.168.7.2 -if [ -d /tmp ]; then - tmp_dir=/tmp -else - tmp_dir=$(pwd) -fi -default_ip="$usb_ip" -ssh_options="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" - -ifs=% # Disable whitespace truncation -indent=' ' - -black="\e[1;30m" -red="\e[1;31m" -green="\e[1;32m" -yellow="\e[1;33m" -blue="\e[1;34m" -purple="\e[1;35m" -cyan="\e[1;36m" -white="\e[1;37m" -clear="\e[0m" - -error="$red" -warning="$yellow" -success="$green" -info="$blue" - -export PATH="$PATH:$frankfurter/scripts/install" -export PATH="$PATH:$frankfurter/scripts/tools" -export PATH="$PATH:$frankfurter/scripts/update" - -function prompt { - read -p "$1 (y/N) " -n 1 -e choice - if [ "$choice" == 'y' ]; then - echo "$choice" - fi -} - -function make_requirements { - cd "$runtime" && pipenv lock --python 3.7 --requirements > "$tmp_dir/runtime-requirements.txt" 2>/dev/null - cd "$hibike" && pipenv lock --python 3.7 --requirements > "$tmp_dir/hibike-requirements.txt" 2>/dev/null - cat "$tmp_dir/"*"-requirements.txt" | sort -u > "$tmp_dir/requirements.txt" - rm "$tmp_dir/"*"-requirements.txt" -} diff --git a/DevOps/frankfurter/scripts/tools/frank-docker b/DevOps/frankfurter/scripts/tools/frank-docker deleted file mode 100755 index 2540cfbd..00000000 --- a/DevOps/frankfurter/scripts/tools/frank-docker +++ /dev/null @@ -1,9 +0,0 @@ -#!/bin/bash - -# frank-docker -- Run Frankfurter as a docker image - -source $(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env -set -e - -docker build -t frankfurter "$piecentral" -docker run -d --memory="512m" -p 1234:1234 -p 1235:1235 -p 1236:1236 frankfurter diff --git a/DevOps/frankfurter/scripts/tools/make_team_net b/DevOps/frankfurter/scripts/tools/make_team_net deleted file mode 100755 index 6cbb707a..00000000 --- a/DevOps/frankfurter/scripts/tools/make_team_net +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash - -# make_team_net -- Build a network configuration for a particular team. - -source "$(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env" - -build_dir="$tmp_dir/net-build" -mkdir -p "$build_dir" - -read -e -p 'Enter the team number: ' team_number -if [[ -z $(echo "$team_number" | grep -E '^[0-9]+$') || ("$team_number" -gt 50) ]]; then - echo -e $error"Invalid team number: '$team_number'"$clear - echo "Team number must be integer between 0 and 50, inclusive." - exit 1 -fi -read -e -p 'Enter the team router password: ' router_psk - -# Copy config file templates -cp "$frankfurter/resources/interfaces" "$build_dir" -cp "$frankfurter/resources/wpa_supplicant.conf" "$build_dir" - -# Perform substitutions -sed -i "s/address 192.168.128.200/address 192.168.128.$(( team_number + 200 ))/" "$build_dir/interfaces" -sed -i "s/address 192.168.0.200/address 192.168.0.$(( team_number + 200 ))/" "$build_dir/interfaces" -sed -i "9s/ssid=\"\"/ssid=\"Team$team_number\"/" "$build_dir/wpa_supplicant.conf" -sed -i "10s/psk=\"\"/psk=\"$router_psk\"/" "$build_dir/wpa_supplicant.conf" - -# Copy and move files -scp "$build_dir/interfaces" "$build_dir/wpa_supplicant.conf" "$user@$default_ip:~" -ssh -t "$user@$default_ip" "sudo mv ~/interfaces /etc/network && sudo mv ~/wpa_supplicant.conf /etc/wpa_supplicant" - -rm -rf "$build_dir" diff --git a/DevOps/frankfurter/scripts/tools/usb_net_fwd b/DevOps/frankfurter/scripts/tools/usb_net_fwd deleted file mode 100755 index d48c7630..00000000 --- a/DevOps/frankfurter/scripts/tools/usb_net_fwd +++ /dev/null @@ -1,64 +0,0 @@ -#!/bin/bash - -# usb_fwd -- Enable internet on a Beaglebone over USB with request forwarding. - -set -e -source "$(git rev-parse --show-toplevel)/DevOps/frankfurter/scripts/tools/env" -host_ip=192.168.7.1 -ip_fwd=/proc/sys/net/ipv4/ip_forward - -usage="Enable internet on a Beaglebone with request forwarding.\n" -usage=$usage"Usage: $(basename $0) [-hHc] [command]\n\n" -usage=$usage"Options:\n" -usage=$usage$indent"-h Show this help message.\n" -usage=$usage$indent"-i Accept requests from this network interface " -usage=$usage"(required for the 'enable' command, ignored otherwise).\n" -usage=$usage$indent"-o Forward requests to this network interface (required for 'enable').\n\n" -usage=$usage"Commands: enable, disable\n\n" -usage=$usage"Use 'ifconfig' to identify interface names.\n" -usage=$usage$warning"May require sudo to write to '$ip_fwd'."$clear - -command="${@: -1}" -iface="" -oface="" -while getopts ":hi:o:" opt; do - case $opt in - h) - echo -e "$usage" >&2 - exit - ;; - i) iface="$OPTARG";; - o) oface="$OPTARG";; - \?) - echo -e $error"Invalid option: '-$OPTARG'"$clear >&2 - echo -e "Use '-h' to see usage." >&2 - exit 1 - ;; - esac -done - -if [[ "$command" = "enable" && ( -z "$iface" || -z "$oface" ) ]]; then - echo -e $error"The 'enable' command requires both output and input network interfaces."$clear >&2 - echo -e "Use '-h' to see usage." >&2 - exit 1 -fi - -case $command in - enable) - ssh -t $ssh_options "$user@$default_ip" "sudo /sbin/route add default gw $host_ip" - iptables --table nat --append POSTROUTING --out-interface "$oface" -j MASQUERADE - iptables --append FORWARD --in-interface "$iface" -j ACCEPT - echo 1 > "$ip_fwd" - echo -e $success"Forwarding enabled."$clear - ;; - disable) - ssh -t $ssh_options "$user@$default_ip" "sudo /sbin/route delete default gw $host_ip" - # TODO: undo `iptables` modification on host - echo 0 > "$ip_fwd" - echo -e $success"Forwarding disabled."$clear - ;; - \?) - echo -e $error"Invalid command: '$command'"$clear >&2 - echo -e "Use '-h' to see usage." >&2 - exit 1 -esac diff --git a/foundry/netbridge.sh b/foundry/netbridge.sh index 5de5dc0d..3d6b4e41 100755 --- a/foundry/netbridge.sh +++ b/foundry/netbridge.sh @@ -1,6 +1,11 @@ #!/bin/bash -# netbridge.sh -- Host-side script +# netbridge.sh -- Allows this machine to serve as a network bridge for another host +# +# 1. This machine (the bridge) will be configured to accept network requests +# from one network interface and forward them to another. +# 2. The remote machine will be configured to route all of its network traffic +# to the bridge. set -e ip link @@ -24,4 +29,9 @@ iptables -A FORWARD -i "${forward}" -o "${accept}" -m state --state RELATED,ESTA iptables -A FORWARD -i "${accept}" -o "${forward}" -j ACCEPT sysctl -w net.ipv4.ip_forward=1 -ssh "$remote_user"@"$remote_ip" "sudo ip route add default via ${host_ip}" +ssh "${remote_user}"@"${remote_ip}" "sudo ip route add default via ${host_ip}" + +# To undo (without rebooting both machines): +# 1. Remote: route delete default gw "$host_ip" +# 2. Bridge: sysctl -w net.ipv4.ip_forward=0 +# 3. Bridge: undo `iptables` entries diff --git a/foundry/provisioning/raspbian.yml b/foundry/provisioning/raspbian.yml index f96d139f..5575c771 100644 --- a/foundry/provisioning/raspbian.yml +++ b/foundry/provisioning/raspbian.yml @@ -1,9 +1,10 @@ --- +# No need to update the default 'pi' user password from 'raspberry', since we +# want to help staff debug student robots. - name: provision raspberry pi hosts: all roles: - role: base - when: '"team" not in ansible_run_tags' - role: raspbian vars: ansible_user: pi diff --git a/foundry/provisioning/roles/raspbian/files/bin/commit-code b/foundry/provisioning/roles/raspbian/files/bin/commit-code new file mode 100755 index 00000000..c1b2377c --- /dev/null +++ b/foundry/provisioning/roles/raspbian/files/bin/commit-code @@ -0,0 +1,5 @@ +#!/bin/bash + +cd ~/storage +git add *.py +git commit -m "[$(date --utc +%F-%H-%M-%S)] Student code update"