[FEAT] Add ability to run arbitrary command on a set working directory (

#3404)

# Overview
- this PR will add the ability to run some arbitrary command on the ray
cluster that the workflow has booted up
- the working directory can be specified (defaults to `.github/assets`)
- the command can be specified (must be provided; furthermore, empty
strings will result in a failure of the workflow)

.github/workflows/run-cluster.yaml

@@ -5,13 +5,22 @@ on:
     inputs:
       daft_version:
         type: string
-        description: The wheel artifact to use
+        description: The version of daft to install
         required: false
       python_version:
         type: string
         description: The version of python to use
         required: false
         default: "3.9"
+      command:
+        type: string
+        description: The command to run on the cluster
+        required: true
+      working_dir:
+        type: string
+        description: The working directory to submit to the cluster
+        required: false
+        default: .github/working-dir
 
 jobs:
   run-command:
@@ -65,7 +74,11 @@ jobs:
     - name: Submit job to ray cluster
       run: |
         source .venv/bin/activate
-        ray job submit --address http://localhost:8265 -- python -c "print('Hello, world!')"
+        if [[ -z '${{ inputs.command }}' ]]; then
+          echo 'Invalid command submitted; command cannot be empty'
+          exit 1
+        fi
+        ray job submit --working-dir ${{ inputs.working_dir }} --address http://localhost:8265 -- ${{ inputs.command }}
     - name: Spin down ray cluster
       if: always()
       run: |

.github/working-dir/README.md

@@ -0,0 +1,23 @@
+# Working dir for submission
+
+This an example of a working directory which can be included inside of runs of the ray-cluster in a GitHub Actions workflow.
+
+## Usage
+
+In order to submit your own script, create a file in this directory, add+commit+push it to GitHub, and submit a GitHub Actions workflow request by specifying the path to this directory and a python command to execute the file.
+
+## Example
+
+First create the file:
+
+```bash
+touch .github/working-dir/my_script.py
+echo "print('Hello, world!')" >> .github/working-dir/my_script.py
+```
+
+Then submit the request to execute the workflow to run this file on a ray-cluster.
+You can either do this via the GitHub Actions UI or by running the GitHub CLI:
+
+```bash
+gh workflow run run-cluster.yaml --ref $MY_BRANCH -f command="python my_script.py"
+```

.github/working-dir/simple.py

@@ -0,0 +1,5 @@
+import daft
+
+df = daft.from_pydict({"nums": [1, 2, 3]})
+df = df.with_column("result", daft.col("nums").cbrt()).collect()
+df.show()