Skip to content

Commit

Permalink
Updated Mlucas install script to determine CPU cache size.
Browse files Browse the repository at this point in the history
  • Loading branch information
tdulcet committed Apr 8, 2024
1 parent 7b8781b commit 480c286
Show file tree
Hide file tree
Showing 14 changed files with 84 additions and 61 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ jobs:
- uses: actions/download-artifact@v4
- name: Install
run: |
sudo apt-get -yqq update
sudo apt-get -yqq install lftp
sudo apt-get update -y
sudo apt-get install -y lftp
- name: Script
run: |
set -x
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ jobs:
- uses: actions/checkout@v4
- name: Install
run: |
sudo apt-get -yqq update
sudo apt-get -yqq install ocl-icd-opencl-dev pocl-opencl-icd
sudo apt-get update -y
sudo apt-get install -y ocl-icd-opencl-dev pocl-opencl-icd
- name: Before script
run: |
sed -i '/^GPU=/,/^fi/ s/^/# /' gpuowl.sh
Expand Down Expand Up @@ -175,7 +175,7 @@ jobs:
- uses: actions/checkout@v4
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
# python3 -m pip install --upgrade pip
python3 -m pip install requests
- name: Script
run: |
Expand Down
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
[![Build Status](https://travis-ci.com/tdulcet/Distributed-Computing-Scripts.svg?branch=master)](https://travis-ci.com/tdulcet/Distributed-Computing-Scripts)
[![Actions Status](https://github.com/tdulcet/Distributed-Computing-Scripts/workflows/CI/badge.svg?branch=master)](https://github.com/tdulcet/Distributed-Computing-Scripts/actions)
[![Actions Status](https://github.com/tdulcet/Distributed-Computing-Scripts/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/tdulcet/Distributed-Computing-Scripts/actions/workflows/ci.yml)

# Distributed Computing Scripts
Distributed Computing Scripts for GIMPS, BOINC and Folding@home
Expand All @@ -24,15 +23,15 @@ To run Prime95 for Stress/Torture Testing, see the [Testing and Benchmarking](ht

### CUDALucas

Downloads, builds, sets up and runs [CUDALucas](https://sourceforge.net/p/cudalucas/code/HEAD/tree/trunk/). Downloads, sets up and runs our [PrimeNet Python script](#primenet) for automated PrimeNet assignments.
Downloads, builds, sets up and runs [CUDALucas](https://sourceforge.net/p/cudalucas/code/HEAD/tree/trunk/). Downloads, sets up and runs our [PrimeNet Python program](#primenet) for automated PrimeNet assignments.

```
wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/cudalucas.sh -qO - | bash -s -- [PrimeNet User ID] [Computer name] [Type of work] [Idle time to run (mins)]
```

### Mlucas

Downloads, builds, sets up and runs [Mlucas](https://www.mersenneforum.org/mayer/README.html#download1). Downloads, sets up and runs our [PrimeNet Python script](#primenet) for automated PrimeNet assignments. Supports x86 Intel and AMD and ARM CPUs, but only recommended for ARM CPUs, which [Prime95/MPrime](#prime95mprime) does not support. Prime95/MPrime is faster than Mlucas on x86 CPUs. Run: `wget https://raw.github.com/tdulcet/Linux-System-Information/master/info.sh -qO - | bash -s` to output your system information, including CPU and architecture.
Downloads, builds, sets up and runs [Mlucas](https://www.mersenneforum.org/mayer/README.html#download1). Downloads, sets up and runs our [PrimeNet Python program](#primenet) for automated PrimeNet assignments. Supports x86 Intel and AMD and ARM CPUs, but only recommended for ARM CPUs, which [Prime95/MPrime](#prime95mprime) does not support. Prime95/MPrime is faster than Mlucas on x86 CPUs. Run: `wget https://raw.github.com/tdulcet/Linux-System-Information/master/info.sh -qO - | bash -s` to output your system information, including CPU and architecture.
This script follows the recommended instructions on the [Mlucas README](https://www.mersenneforum.org/mayer/README.html) for each architecture and CPU.

```
Expand All @@ -41,7 +40,7 @@ wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/mlucas.

### GpuOwl

Downloads, builds, sets up and runs the latest version of [GpuOwl](https://github.com/preda/gpuowl) for PRP tests, version 7.2-112 for PRP tests with combined P-1 and the [v6 branch](https://github.com/preda/gpuowl/tree/v6) for LL and standalone P-1 tests. Downloads, sets up and runs our [PrimeNet Python script](#primenet) for automated PrimeNet assignments. Creates wrapper script to run the correct version of GpuOwl based on the next assignment. Supports Nvidia, AMD and Intel GPUs supporting OpenCL. Note that [GpuOwl uses C++20](https://github.com/preda/gpuowl#build) and thus requires at least the GNU C++ compiler 8. Run: `g++ --version` to output your version.
Downloads, builds, sets up and runs the latest version of [GpuOwl](https://github.com/preda/gpuowl) for PRP tests, version 7.2-112 for PRP tests with combined P-1 and the [v6 branch](https://github.com/preda/gpuowl/tree/v6) for LL and standalone P-1 tests. Downloads, sets up and runs our [PrimeNet Python program](#primenet) for automated PrimeNet assignments. Creates wrapper script to run the correct version of GpuOwl based on the next assignment. Supports Nvidia, AMD and Intel GPUs supporting OpenCL. Note that [GpuOwl uses C++20](https://github.com/preda/gpuowl#build) and thus requires at least the GNU C++ compiler 8. Run: `g++ --version` to output your version.

```
wget https://raw.github.com/tdulcet/Distributed-Computing-Scripts/master/gpuowl.sh -qO - | bash -s -- [PrimeNet User ID] [Computer name] [Type of work] [Idle time to run (mins)]
Expand Down Expand Up @@ -307,7 +306,7 @@ These scripts should work on Ubuntu, Debian and any Linux distribution that can

Pull requests welcome! Ideas for contributions:

PrimeNet script:
PrimeNet program/script:
* Support more GIMPS programs.
* Support setting more of the program options.
* Support setting per thread options.
Expand Down Expand Up @@ -339,7 +338,7 @@ General:

## License

The scripts are all MIT licensed, except the PrimeNet script which is GPLv2.
The scripts are all MIT licensed, except the PrimeNet program which is GPLv2.

Thanks to [Daniel Connelly](https://github.com/Danc2050) for updating the PrimeNet Python script from Mlucas to eliminate the password requirement by getting assignments using the [PrimeNet API](http://v5.mersenne.org/v5design/v5webAPI_0.97.html) and to support reporting the assignment results and progress for CUDALucas using the PrimeNet API, for porting the Prime95 script to Python and for helping create and test the Google Colab Jupyter Notebooks!

Expand Down
8 changes: 4 additions & 4 deletions cudalucas.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,13 @@ if ! command -v svn >/dev/null; then
echo -e "Installing Apache Subversion"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install subversion -y
sudo apt-get install -y subversion
fi
if ! command -v nvcc >/dev/null; then
echo -e "Installing the Nvidia CUDA Toolkit"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install nvidia-cuda-toolkit -y
sudo apt-get install -y nvidia-cuda-toolkit
fi
if [[ -n $CC ]] && ! command -v "$CC" >/dev/null; then
echo "Error: $CC is not installed." >&2
Expand All @@ -75,7 +75,7 @@ echo -e "Downloading CUDALucas\n"
svn checkout https://svn.code.sf.net/p/cudalucas/code/trunk "$DIR"
cd "$DIR"
DIR=$PWD
echo -e "\nDownloading the PrimeNet script\n"
echo -e "\nDownloading the PrimeNet program\n"
if [[ -e ../primenet.py ]]; then
cp -v ../primenet.py .
else
Expand Down Expand Up @@ -176,7 +176,7 @@ cat <<EOF >CUDALucas.sh
#!/bin/bash
# Copyright © 2020 Teal Dulcet
# Start CUDALucas and the PrimeNet script if the computer has not been used in the specified idle time and stop it when someone uses the computer
# Start CUDALucas and the PrimeNet program if the computer has not been used in the specified idle time and stop it when someone uses the computer
# ${DIR@Q}/CUDALucas.sh
if who -s | awk '{ print \$2 }' | (cd /dev && xargs -r stat -c '%U %X') | awk '{if ('"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2<$TIME) { print \$1"\t"'"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2; ++count }} END{if (count>0) { exit 1 }}' >/dev/null; then
Expand Down
10 changes: 5 additions & 5 deletions cudalucas2.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,13 @@ if ! command -v svn >/dev/null; then
echo -e "Installing Apache Subversion"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install subversion -y
sudo apt-get install -y subversion
fi
if ! command -v nvcc >/dev/null; then
echo -e "Installing the Nvidia CUDA Toolkit"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install nvidia-cuda-toolkit -y
sudo apt-get install -y nvidia-cuda-toolkit
fi
if [[ -n $CC ]] && ! command -v "$CC" >/dev/null; then
echo "Error: $CC is not installed." >&2
Expand Down Expand Up @@ -127,9 +127,9 @@ EOF
fi
DIR=$PWD
if [[ -f "primenet.py" ]]; then
echo -e "The PrimeNet script is already downloaded\n"
echo -e "The PrimeNet program is already downloaded\n"
else
echo -e "\nDownloading the PrimeNet script\n"
echo -e "\nDownloading the PrimeNet program\n"
if [[ -e ../primenet.py ]]; then
cp -v ../primenet.py .
else
Expand Down Expand Up @@ -189,7 +189,7 @@ cat <<EOF >CUDALucas.sh
#!/bin/bash
# Copyright © 2020 Teal Dulcet
# Start CUDALucas and the PrimeNet script if the computer has not been used in the specified idle time and stop it when someone uses the computer
# Start CUDALucas and the PrimeNet program if the computer has not been used in the specified idle time and stop it when someone uses the computer
# ${DIR@Q}/CUDALucas.sh
if who -s | awk '{ print \$2 }' | (cd /dev && xargs -r stat -c '%U %X') | awk '{if ('"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2<$TIME) { print \$1"\t"'"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2; ++count }} END{if (count>0) { exit 1 }}' >/dev/null; then
Expand Down
2 changes: 1 addition & 1 deletion google-colab/Colab GPU CUDALucas.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"#@markdown #### ↖️ Click the ▶️ button after deciding on the options below.\n",
"#@markdown #### 🔌 Make sure the GPU is enabled under *Runtime→Change runtime type*\n",
"#@markdown #### 💡 Keep each notebook **open** in the browser to prevent disconnection. 📌 [the tab](https://support.mozilla.org/kb/pinned-tabs-keep-favorite-websites-open) or move them to a dedicated window for easy access.\n",
"#@markdown #### ℹ️ This notebook uses both our CUDALucas and Prime95 [Bash install scripts](https://github.com/tdulcet/Distributed-Computing-Scripts#organizations) and our [PrimeNet Python script](https://github.com/tdulcet/Distributed-Computing-Scripts#primenet).\n",
"#@markdown #### ℹ️ This notebook uses both our CUDALucas and Prime95 [Bash install scripts](https://github.com/tdulcet/Distributed-Computing-Scripts#organizations) and our [PrimeNet Python program](https://github.com/tdulcet/Distributed-Computing-Scripts#primenet).\n",
"#@markdown #### 📜 Please see the [documentation](https://github.com/tdulcet/Distributed-Computing-Scripts/tree/master/google-colab) for more information and to ❤️ support us.\n",
"#@markdown #### 🤷 Optionally, create a GIMPS/PrimeNet account [here](https://www.mersenne.org/update/) and [join](https://www.mersenne.org/jteam/) the “Portland State University” team!\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion google-colab/Colab GPU GpuOwl.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"#@markdown #### ↖️ Click the ▶️ button after deciding on the options below.\n",
"#@markdown #### 🔌 Make sure the GPU is enabled under *Runtime→Change runtime type*\n",
"#@markdown #### 💡 Keep each notebook **open** in the browser to prevent disconnection. 📌 [the tab](https://support.mozilla.org/kb/pinned-tabs-keep-favorite-websites-open) or move them to a dedicated window for easy access.\n",
"#@markdown #### ℹ️ This notebook uses both our GpuOwl and Prime95 [Bash install scripts](https://github.com/tdulcet/Distributed-Computing-Scripts#organizations) and our [PrimeNet Python script](https://github.com/tdulcet/Distributed-Computing-Scripts#primenet).\n",
"#@markdown #### ℹ️ This notebook uses both our GpuOwl and Prime95 [Bash install scripts](https://github.com/tdulcet/Distributed-Computing-Scripts#organizations) and our [PrimeNet Python program](https://github.com/tdulcet/Distributed-Computing-Scripts#primenet).\n",
"#@markdown #### 📜 Please see the [documentation](https://github.com/tdulcet/Distributed-Computing-Scripts/tree/master/google-colab) for more information and to ❤️ support us.\n",
"#@markdown #### 🤷 Optionally, create a GIMPS/PrimeNet account [here](https://www.mersenne.org/update/) and [join](https://www.mersenne.org/jteam/) the “Portland State University” team!\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion google-colab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Though each GPU works well and will complete most assignments in a matter of day
## Acknowledgements
We acknowledge the following projects, which enabled and encouraged us to create these notebooks:
* [Our Bash install scripts](/../../#organizations), which automatically download, build and setup the respective GIMPS programs on Colab.
* [Our PrimeNet Python script](/../../#primenet), which automatically gets assignments, reports assignment results and progress for CUDALucas and GpuOwl using the PrimeNet API. Adapted from the PrimeNet Python script from [Mlucas](https://www.mersenneforum.org/mayer/README.html#download2) by Loïc Le Loarer and Ernst W. Mayer.
* [Our PrimeNet Python program](/../../#primenet), which automatically gets assignments, reports assignment results and progress for CUDALucas and GpuOwl using the PrimeNet API. Adapted from the PrimeNet Python script from [Mlucas](https://www.mersenneforum.org/mayer/README.html#download2) by Loïc Le Loarer and Ernst W. Mayer.
* The [GPU72 notebook](https://github.com/chalsall/GPU72_CoLab), whose work encouraged us to also use the [form](https://colab.research.google.com/notebooks/forms.ipynb) format.
* The [Linux System Information script](https://github.com/tdulcet/Linux-System-Information), which outputs the system information for the Colab VMs.

Expand Down
14 changes: 7 additions & 7 deletions gpuowl-wrapper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ GPUOWL_LL=3
# Standalone P-1 tests
GPUOWL_PM1=3

# worktodo and results files for the PrimeNet script
# worktodo and results files for the PrimeNet program/script
WorkFile="worktodo.ini"
ResultsFile="results.ini"

Expand Down Expand Up @@ -122,7 +122,7 @@ fi
if [[ -n $TOTAL_GPU_MEM ]]; then
# echo -e -n "\tGPU Memory (RAM):\t"
# for i in "${!TOTAL_GPU_MEM[@]}"; do
# echo -n "$([[ $i -gt 0 ]] && echo ", ")$(printf "%'d" "${TOTAL_GPU_MEM[i]}") MiB ($(numfmt --from=iec --to=iec-i "${TOTAL_GPU_MEM[i]}M")B)"
# echo -n "$( ((i)) && echo ", ")$(printf "%'d" "${TOTAL_GPU_MEM[i]}") MiB ($(numfmt --from=iec --to=iec-i "${TOTAL_GPU_MEM[i]}M")B)"
# done
# echo
echo -e "\tGPU Memory (RAM):\t$(printf "%'d" "${TOTAL_GPU_MEM[DEVICE]}") MiB ($(numfmt --from=iec --to=iec-i "${TOTAL_GPU_MEM[DEVICE]}M")B)"
Expand Down Expand Up @@ -152,11 +152,11 @@ trap 'trap - INT; kill -INT "$$"' INT
while true; do
date

if [[ -r $aResultsFile ]] && mapfile -t aresults <"$aResultsFile" && [[ ${#aresults[*]} -gt 0 ]]; then
if [[ -r $aResultsFile ]] && mapfile -t aresults <"$aResultsFile" && ((${#aresults[*]})); then
printf "Found %'d new result(s) in %s. Moving to %s.\n" ${#aresults[*]} "${aResultsFile@Q}" "${ResultsFile@Q}"
for result in "${aresults[@]}"; do
if echo "$result" | grep -q 'gpuowl'; then
if [[ -r $WorkFile ]] && mapfile -t worktodo <"$WorkFile" && [[ ${#worktodo[*]} -gt 0 ]]; then
if [[ -r $WorkFile ]] && mapfile -t worktodo <"$WorkFile" && ((${#worktodo[*]})); then
if command -v jq >/dev/null; then
exponent=$(echo "$result" | jq -r '.exponent')
worktype=$(echo "$result" | jq -r '.worktype')
Expand Down Expand Up @@ -208,13 +208,13 @@ while true; do
echo "No results found."
fi

if ! [[ -r $aWorkFile ]] || ! mapfile -t aworktodo <"$aWorkFile" || ! [[ ${#aworktodo[*]} -gt 0 ]]; then
if [[ -r $WorkFile ]] && mapfile -t worktodo <"$WorkFile" && [[ ${#worktodo[*]} -gt 0 ]]; then
if ! [[ -r $aWorkFile ]] || ! mapfile -t aworktodo <"$aWorkFile" || ! ((${#worktodo[*]})); then
if [[ -r $WorkFile ]] && mapfile -t worktodo <"$WorkFile" && ((${#worktodo[*]})); then
printf "Found %'d work to do(s) in the %s file. Copying the first one to %s.\n" ${#worktodo[*]} "${WorkFile@Q}" "${aWorkFile@Q}"
printf '%s\n' "${worktodo[0]}" >>"$aWorkFile"
mapfile -t aworktodo <"$aWorkFile"
else
echo "Error: No work to do. Please run the PrimeNet script or manually add some work to the ${WorkFile@Q} file." >&2
echo "Error: No work to do. Please run the PrimeNet program/script or manually add some work to the ${WorkFile@Q} file." >&2
exit 1
fi
else
Expand Down
12 changes: 6 additions & 6 deletions gpuowl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ if ! command -v make >/dev/null || ! command -v g++ >/dev/null; then
echo -e "Installing Make and the GNU C++ compiler"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install build-essential -y
sudo apt-get install -y build-essential
fi
if [[ -n $CXX ]] && ! command -v "$CXX" >/dev/null; then
echo "Error: $CXX is not installed." >&2
Expand All @@ -86,14 +86,14 @@ if ! ldconfig -p | grep -iq 'libgmp\.' || ! ldconfig -p | grep -iq 'libgmpxx\.'
echo -e "Installing the GNU Multiple Precision (GMP) library"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
sudo apt-get install libgmp-dev -y
sudo apt-get install -y libgmp-dev
fi
if ! ldconfig -p | grep -iq 'libOpenCL\.'; then
echo -e "Installing the OpenCL library"
echo -e "Please enter your password if prompted.\n"
sudo apt-get update -y
# sudo apt-get install ocl-icd-* opencl-headers clinfo -y
sudo apt-get install ocl-icd-opencl-dev clinfo -y
# sudo apt-get install -y ocl-icd-* opencl-headers clinfo
sudo apt-get install -y ocl-icd-opencl-dev clinfo
fi
TIME=$(echo "$TIME" | awk '{ printf "%g", $1 * 60 }')
mkdir "$DIR"
Expand Down Expand Up @@ -158,7 +158,7 @@ else
sed -i 's/`git describe --tags --long --dirty --always`/'"${name}${behind_by:+-${behind_by}${sha:+-g${sha::7}}}"'/' $DIR1/Makefile
fi
fi
echo -e "\nDownloading the PrimeNet script\n"
echo -e "\nDownloading the PrimeNet program\n"
if [[ -e ../primenet.py ]]; then
cp -v ../primenet.py .
else
Expand Down Expand Up @@ -263,7 +263,7 @@ cat <<EOF >gpuowl.sh
#!/bin/bash
# Copyright © 2020 Teal Dulcet
# Start GpuOwl and the PrimeNet script if the computer has not been used in the specified idle time and stop it when someone uses the computer
# Start GpuOwl and the PrimeNet program if the computer has not been used in the specified idle time and stop it when someone uses the computer
# ${DIR@Q}/gpuowl.sh
if who -s | awk '{ print \$2 }' | (cd /dev && xargs -r stat -c '%U %X') | awk '{if ('"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2<$TIME) { print \$1"\t"'"\${EPOCHSECONDS:-\$(date +%s)}"'-\$2; ++count }} END{if (count>0) { exit 1 }}' >/dev/null; then
Expand Down
Loading

0 comments on commit 480c286

Please sign in to comment.