Initial Release Intelligence Layer SDK

Enjoy the first step to our journey towards evaluation driven AI development. Co-authored-by: Rutger Cappendijk <[email protected]> Co-authored-by: Volker Stampa <[email protected]> Co-authored-by: Niklas Finken <[email protected]> Co-authored-by: Piotr Mazurek <[email protected]> Co-authored-by: Pit Neitemeier <[email protected]> Co-authored-by: Johannes Wesch <[email protected]> Co-authored-by: Ben Brandt <[email protected]> Co-authored-by: Valentina Galata <[email protected]> Co-authored-by: Felix Fehse <[email protected]> Co-authored-by: Niklas Köhnecke <[email protected]> Co-authored-by: Julian Wiesler <[email protected]> Co-authored-by: Merlin Kallenborn <[email protected]> Co-authored-by: Florian Schepers <[email protected]> Co-authored-by: Sebastian Niehus <[email protected]> Co-authored-by: Ivo Schaper <[email protected]> Co-authored-by: Sebastian Zett <[email protected]> Co-authored-by: Martin Reinhardt <[email protected]> Co-authored-by: Samuel Weinbach <[email protected]>
Aleph-Alpha · Apr 30, 2024 · 1f898bd · 1f898bd
commit 1f898bd
Show file tree

Hide file tree

Showing 215 changed files with 34,093 additions and 0 deletions.
diff --git a/.darglint2 b/.darglint2
@@ -0,0 +1,3 @@
+[darglint2]
+ignore=DAR003,DAR201,DAR202,DAR301,DAR401
+docstring_style=google
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,34 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!-- IMPORTANT: By posting an issue, your username will be PUBLIC to everyone who has access to this repository. If you don't want to expose this to other users, feel free to email us at [email protected] -->
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**System (please complete the following information):**
+ - OS: [e.g. Mac]
+ - Version [e.g. 10.14]
+ - Intelligence Layer Version [e.g. 0.1.0]
+
+**Additional context**
+Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,22 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!-- IMPORTANT: By posting an issue, your username will be PUBLIC to everyone who has access to this repository. If you don't want to expose this to other users, feel free to email us at [email protected] -->
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
@@ -0,0 +1,37 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    groups:
+      minor:
+        update-types:
+          - minor
+          - patch
+
+  - package-ecosystem: "github-actions"
+    # Workflow files stored in the default location of `.github/workflows`. (You don't need to specify `/.github/workflows` for `directory`. You can use `directory: "/"`.)
+    directory: "/"
+    schedule:
+      interval: "daily"
+
+  - package-ecosystem: "npm"
+    directory: "/trace-viewer/"
+    schedule:
+      interval: "daily"
+    groups:
+      minor:
+        update-types:
+          - minor
+          - patch
+
+  - package-ecosystem: "docker"
+    directory: "/trace-viewer/"
+    schedule:
+      interval: "daily"
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -0,0 +1,12 @@
+# Description
+No description.
+
+## Before Merging
+ - [ ] Review the code changes
+    - Unused print / comments / TODOs
+    - Missing docstrings for functions that should have them
+    - Consistent variable names
+    - ...
+ - [ ] Update `changelog.md` if necessary
+ - [ ] Commit messages should contain a semantic [label](https://gist.github.com/joshbuchea/6f47e86d2510bce28f8e7f42ae84c716) and the ticket number
+   - Consider squashing if this is not the case
diff --git a/.github/workflows/artifactory.yml b/.github/workflows/artifactory.yml
@@ -0,0 +1,56 @@
+name: Artifactory Deployment of PyPi and trace-viewer
+
+on:
+  workflow_dispatch: {}
+  release:
+    types: [published]
+
+env:
+  ARTIFACTORY_URL: https://alephalpha.jfrog.io
+  ARTIFACTORY_PYPI_REPOSITORY: "intelligence-layer"
+  ARTIFACTORY_DOCKER_REGISTRY: alephalpha.jfrog.io/intelligence-layer-images
+
+jobs:
+  build-and-push-pypi:
+    permissions:
+      contents: read
+      id-token: write
+    runs-on: ubuntu-latest
+    steps:
+      - name: Get Identity Token From Github
+        run: |
+          ID_TOKEN=$(curl -sLS -H "User-Agent: actions/oidc-client" -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
+          "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=https://alephalpha.jfrog.io" | jq .value | tr -d '"')
+          echo "ID_TOKEN=${ID_TOKEN}" >> $GITHUB_ENV
+      - name: Get Jfrog Access Token with Github Identity Token
+        env:
+          ID_TOKEN: ${{ env.ID_TOKEN }}
+        run: |
+          JFROG_ACCESS_TOKEN=$(curl \
+            -X POST \
+            -H "Content-type: application/json" \
+            $ARTIFACTORY_URL/access/api/v1/oidc/token \
+            -d \
+            "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"$ID_TOKEN\", \"provider_name\": \"github\"}" \
+          | jq .access_token -r)
+          echo "JFROG_ACCESS_TOKEN=${JFROG_ACCESS_TOKEN}" >> $GITHUB_ENV
+      - name: Checkout
+        uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: Install and configure Poetry
+        uses: snok/install-poetry@v1
+        with:
+          virtualenvs-create: true
+          virtualenvs-in-project: true
+          installer-parallel: true
+      - name: Build package and push to Artifactory
+        env:
+          JFROG_ACCESS_TOKEN: ${{ env.JFROG_ACCESS_TOKEN }}
+        run: |
+          poetry build
+          poetry config repositories.artifactory $ARTIFACTORY_URL/artifactory/api/pypi/$ARTIFACTORY_PYPI_REPOSITORY
+          JFROG_ACCESS_TOKEN_SUBJECT=$(echo $JFROG_ACCESS_TOKEN | awk -F'.' '{print $2}'  | sed 's/.\{1,3\}$/&==/' | base64 -d | jq '.sub' -r)
+          poetry config http-basic.artifactory "$JFROG_ACCESS_TOKEN_SUBJECT" "$JFROG_ACCESS_TOKEN"
+          poetry publish -r artifactory
diff --git a/.github/workflows/daily.yml b/.github/workflows/daily.yml
@@ -0,0 +1,88 @@
+name: "os-support-tests"
+
+on:
+  workflow_dispatch:
+  # Scheduled workflows will only run on the default branch.
+  schedule:
+    - cron: '0 0 * * *' # runs once a day at midnight in the timezone of your GitHub repository
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  test:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [macos-latest, windows-latest]
+    runs-on: ${{matrix.os}}
+    # difference to regular test: no docker, no venv caching
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install and configure Poetry
+        uses: snok/install-poetry@v1
+        with:
+          virtualenvs-create: true
+          virtualenvs-in-project: true
+          installer-parallel: true
+      - name: Install dependencies
+        run: |
+          poetry config installer.max-workers 10
+          poetry install --no-interaction
+
+      - name: Run pytest
+        env:
+          AA_TOKEN: ${{ secrets.AA_TOKEN }}
+          HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
+          ARGILLA_API_URL: "http://localhost:6900/"
+          ARGILLA_API_KEY: "argilla.apikey"
+          CLIENT_URL: "https://api.aleph-alpha.com"
+        run: |
+          poetry run python -c "import nltk; nltk.download('punkt')"
+          poetry run pytest -n 10 -m "not docker"
+  run-notebooks:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [macos-latest, windows-latest]
+    runs-on: ${{matrix.os}}
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: Install and configure Poetry
+        uses: snok/install-poetry@v1
+        with:
+          virtualenvs-create: true
+          virtualenvs-in-project: true
+          installer-parallel: true
+      - name: Install dependencies
+        run: |
+          poetry config installer.max-workers 10
+          poetry install --no-interaction
+      - name: Configure Poetry for notebooks and run
+        env:
+          AA_TOKEN: ${{ secrets.AA_TOKEN }}
+          HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
+          ARGILLA_API_URL: "http://localhost:6900/"
+          ARGILLA_API_KEY: "argilla.apikey"
+          CLIENT_URL: "https://api.aleph-alpha.com"
+        run: |
+          [ -f .env ] && source .env
+          export AA_TOKEN
+          # Find all .ipynb files in the directory and pass them to xargs for parallel execution
+          rm -rf src/documentation/.ipynb_checkpoints
+          rm -rf src/documentation/how_tos/.ipynb_checkpoints
+
+          find src/documentation -name "*.nbconvert.ipynb" -type f -delete
+          find src/documentation -name "*.ipynb" ! -name "performance_tips.ipynb" ! -name "human_evaluation.ipynb" ! -name "how_to_human_evaluation_via_argilla.ipynb" | xargs -n 1 -P 6 poetry run jupyter nbconvert --to notebook --execute
+          find src/documentation -name "*.nbconvert.ipynb" -type f -delete
diff --git a/.github/workflows/on-push.yml b/.github/workflows/on-push.yml
@@ -0,0 +1,14 @@
+name: Intelligence Layer SDK Tests
+
+on:
+  push:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+jobs:
+  python-tests:
+    uses: ./.github/workflows/sdk-tests.yml
+    with:
+      runner: 'ubuntu-latest'
+    secrets: inherit