Merge pull request #2 from mobiusml/automatic_endpoints

Automate Endpoint Generation and Basic Build System

.github/workflows/python-package.yml

@@ -0,0 +1,38 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://docs.github.com/en/ actions/automating-builds-and-tests/building-and-testing-python
+
+name: Python package
+
+on:
+  push:
+    branches:
+      - '*'  # Runs on push to any branch
+  pull_request:
+    branches:
+      - '*'  # Runs on pull requests to any branch
+  workflow_dispatch:  # Allows for manual triggering
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10"]
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+      with:
+        token: ${{ secrets.GH_MOBIUS_PIPELINE_TOKEN }}
+        submodules: recursive
+    - name: Bootstrap poetry
+      run: |
+        curl -sSL https://install.python-poetry.org | python - -y
+    - name: Update PATH
+      run: echo "$HOME/.local/bin" >> $GITHUB_PATH
+    - name: Install dependencies
+      run: poetry install
+    - name: Test with pytest
+      run: poetry run pytest 

.vscode/settings.json

@@ -2,5 +2,10 @@
     "[python]": {
         "editor.defaultFormatter": "ms-python.black-formatter"
     },
-    "python.formatting.provider": "none"
+    "python.formatting.provider": "none",
+    "python.testing.pytestArgs": [
+        "aana"
+    ],
+    "python.testing.unittestEnabled": false,
+    "python.testing.pytestEnabled": true
 }

Dockerfile

@@ -23,11 +23,15 @@ COPY . /app
 # Install the package with poetry
 RUN sh install.sh
 
+# Prepare the startup script
+RUN chmod +x startup.sh
+
 # Disable buffering for stdout and stderr to get the logs in real time
 ENV PYTHONUNBUFFERED=1
 
 # Expose the desired port
 EXPOSE 8000
 
-# Set the command to run the SDK when the container starts
-CMD ["poetry", "run", "serve", "run", "--port", "8000", "--host", "0.0.0.0", "aana.main:server"]
+# Run the startup script, TARGET can be set with environment variables
+ENV TARGET=llama2
+CMD ["/app/startup.sh"]

README.md

@@ -1,3 +1,5 @@
+[![Python package](https://github.com/mobiusml/aana_sdk/actions/workflows/python-package.yml/badge.svg)](https://github.com/mobiusml/aana_sdk/actions/workflows/python-package.yml)
+
 # Aana
 
 Aana is a multi-model SDK for deploying and serving machine learning models.
@@ -28,9 +30,11 @@ sh install.sh
 5. Run the SDK.
 
 ```bash
-CUDA_VISIBLE_DEVICES=0 poetry run serve run --port 8000 --host 0.0.0.0 aana.main:server
+CUDA_VISIBLE_DEVICES=0 poetry run aana --port 8000 --host 0.0.0.0 --target llama2
 ```
 
+The target parameter specifies the set of endpoints to deploy.
+
 The first run might take a while because the models will be downloaded from Google Drive and cached.
 
 Once you see `Deployed Serve app successfully.` in the logs, the server is ready to accept requests.
@@ -66,9 +70,11 @@ docker build -t aana:0.1.0 .
 4. Run the Docker container.
 
 ```bash
-docker run --rm --init -p 8000:8000 --gpus all -e CUDA_VISIBLE_DEVICES=0 -v aana_cache:/root/.aana -v aana_hf_cache:/root/.cache/huggingface --name aana_instance aana:0.1.0
+docker run --rm --init -p 8000:8000 --gpus all -e TARGET="llama2" -e CUDA_VISIBLE_DEVICES=0 -v aana_cache:/root/.aana -v aana_hf_cache:/root/.cache/huggingface --name aana_instance aana:0.1.0
 ```
 
+Use the environment variable TARGET to specify the set of endpoints to deploy.
+
 The first run might take a while because the models will be downloaded from Google Drive and cached. The models will be stored in the `aana_cache` volume. The HuggingFace models will be stored in the `aana_hf_cache` volume. If you want to remove the cached models, remove the volume.
 
 Once you see `Deployed Serve app successfully.` in the logs, the server is ready to accept requests.