diff --git a/README.md b/README.md index f4bb3312a..f1ae727fd 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ The key features of the Intelligence Layer are: - [License](#license) - [For Developers](#for-developers) - [Python: Naming Conventions](#python-naming-conventions) + - [Tests](#tests) # Installation @@ -203,7 +204,7 @@ The tutorials aim to guide you through implementing several common use-cases wit | 6 | Document Index | Connect your proprietary knowledge base | [document_index.ipynb](./src/examples/document_index.ipynb) | | 7 | Human Evaluation | Connect to Argilla for manual evaluation | [human_evaluation.ipynb](./src/examples/human_evaluation.ipynb) | | 8 | Performance tips | Contains some small tips for performance | [performance_tips.ipynb](./src/examples/performance_tips.ipynb) | -| 9 | Deployment | Shows how to deploy a Task in a minimal FastAPI app. | [fastapi_tutorial.md](./src/examples/fastapi_tutorial.md) | +| 9 | Deployment | Shows how to deploy a Task in a minimal FastAPI app. | [fastapi_tutorial.ipynb](./src/examples/fastapi_tutorial.ipynb) | | 10 | Issue Classification | In-depth tutorial about implementing and evaluating an email classificator | [issue_classification_user_journy.ipynb](./src/examples/issue_classification_user_journey.ipynb) ## How-Tos diff --git a/src/examples/fastapi_tutorial.ipynb b/src/examples/fastapi_tutorial.ipynb new file mode 100644 index 000000000..475ee5964 --- /dev/null +++ b/src/examples/fastapi_tutorial.ipynb @@ -0,0 +1,323 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Tutorial: Extending a FastAPI App with the Aleph-Alpha Intelligence Layer\n", + "\n", + "In this tutorial, a basic [FastAPI](https://fastapi.tiangolo.com) app is extended with a new route at which a summary for a given text can be retrieved, using the _Aleph-Alpha Intelligence Layer_, and it's _Luminous_ control models.\n", + "\n", + "The full source code for this example app can be found at the end of this tutorial and in [src/examples/fastapi_example.py](./fastapi_example.py).\n", + "\n", + "## Basic FastAPI App\n", + "\n", + "The foundation for this tutorial is a minimal [FastAPI](https://fastapi.tiangolo.com) application with a root endpoint:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from http import HTTPStatus\n", + "\n", + "from fastapi import FastAPI, Response\n", + "\n", + "app = FastAPI()\n", + "\n", + "\n", + "@app.get(\"/\")\n", + "def root() -> Response:\n", + " return Response(content=\"Hello World\", status_code=HTTPStatus.OK)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This application can be started from the command line with the [Hypercorn](https://github.com/pgjones/hypercorn/) server as follows:\n", + "\n", + "```bash\n", + "hypercorn fastapi_example:app --bind localhost:8000\n", + "```\n", + "\n", + "If the start-up was successful, you should see a message similar to\n", + "```cmd\n", + "[2024-03-07 14:00:55 +0100] [6468] [INFO] Running on http://:8000 (CTRL + C to quit)\n", + "```\n", + "\n", + "Now that the server is running, we can perform a `GET` request via `cURL`:\n", + "```bash\n", + "curl -X GET http://localhost:8000\n", + "```\n", + "You should get\n", + "```\n", + "Hello World\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After successfully starting the basic FastAPI app, the next step is to add a route that makes use of the Intelligence Layer.\n", + "\n", + "## Adding the Intelligence Layer to the application\n", + "\n", + "The building blocks of the Intelligence Layer for applications are `Tasks`. In general, a task implements the `Task`\n", + "interface and defines an `Input` and an `Output`. Multiple tasks can be chained to create more complex applications.\n", + "Here, we will make use of the pre-built task `SteerableSingleChunkSummarize`. This task defines `SingleChunkSummarizeInput`\n", + "as it's input, and `SummarizeOutput` as it's output.\n", + "Like many other tasks, the `SteerableSingleChunkSummarize` task makes use of a `ControlModel`. The\n", + "`ControlModel` itself needs access to the Aleph-Alpha backend via a `AlephAlphaClientProtocol` client.\n", + "In short, the hierarchy is as follows:\n", + "\n", + "![task_dependencies.drawio.svg](task_dependencies.drawio.svg)\n", + "\n", + "We make use of the built-in [Dependency Injection](https://fastapi.tiangolo.com/reference/dependencies/) of FastAPI to\n", + "resolve this hierarchy automatically. In this framework, the defaults for the parameters are dynamically created with\n", + "the `Depends(func)` annotation, where `func` is a function that returns the default value.\n", + "\n", + "So, first, we define our client-generating function. For that, we provide the host URL and a valid Aleph-Alpha token,\n", + "which are stored in an `.env`-file.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "from aleph_alpha_client import Client\n", + "from dotenv import load_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "\n", + "def client() -> Client:\n", + " return Client(\n", + " token=os.environ[\"AA_TOKEN\"],\n", + " host=os.getenv(\"AA_CLIENT_BASE_URL\", \"https://api.aleph-alpha.com\"),\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we create a `ControlModel`. In this case, we make use of the `LuminousControlModel`, which takes\n", + "an `AlephAlphaClientProtocol` that we let default to the previously defined `client`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Annotated\n", + "\n", + "from fastapi import Depends\n", + "\n", + "from intelligence_layer.connectors import AlephAlphaClientProtocol\n", + "from intelligence_layer.core import LuminousControlModel\n", + "\n", + "\n", + "def default_model(app_client: Annotated[AlephAlphaClientProtocol, Depends(client)]):\n", + " return LuminousControlModel(client=app_client)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we create the actual `Task`. For our example, we choose the `SteerableSingleChunkSummarize`.\n", + "The `Input` of this task is a `SingleChunkSummarizeInput`, consisting of the text to summarize as the `chunk` field,\n", + "and the desired `Language` as the `language` field.\n", + "The `Output` of this task is a `SummarizeOutput` and contains the `summary` as text,\n", + "and number of generated tokens for the `summary` as the `generated_tokens` field." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from intelligence_layer.use_cases import SteerableSingleChunkSummarize\n", + "\n", + "\n", + "def summary_task(\n", + " model: Annotated[LuminousControlModel, Depends(default_model)],\n", + ") -> SteerableSingleChunkSummarize:\n", + " return SteerableSingleChunkSummarize(model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can then provide a `POST` endpoint on `/summary` to run the task.\n", + "The default for `task` will be set by `summary_task`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from intelligence_layer.core import NoOpTracer, Task\n", + "from intelligence_layer.use_cases import SingleChunkSummarizeInput, SummarizeOutput\n", + "\n", + "\n", + "@app.post(\"/summary\")\n", + "def summary_task_route_without_permissions(\n", + " input: SingleChunkSummarizeInput,\n", + " task: Annotated[\n", + " Task[SingleChunkSummarizeInput, SummarizeOutput], Depends(summary_task)\n", + " ],\n", + ") -> SummarizeOutput:\n", + " return task.run(input, NoOpTracer())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This concludes the addition of an Intelligence-Layer task to the FastAPI app. After restarting the server, we can call\n", + "our newly created endpoint via a command such as the following:\n", + "```bash\n", + "\n", + "curl -X POST http://localhost:8000/summary -H \"Content-Type: application/json\" -d '{\"chunk\": \"\", \"language\": {\"iso_639_1\": \"en\"}}'\n", + "```\n", + "\n", + "## Add Authorization to the Routes\n", + "\n", + "Typically, authorization is needed to control access to endpoints.\n", + "Here, we will give a minimal example of how a per-route authorization system could be implemented in the minimal example app.\n", + "\n", + "The authorization system makes use of two parts: An `AuthService` that checks whether the user is allowed to access a\n", + "given site, and a `PermissionsChecker` that is called on each route access and in turn calls the `AuthService`.\n", + "\n", + "For this minimal example, the `AuthService` is simply a stub. You will want to implement a concrete authorization service\n", + "tailored to your needs." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Sequence\n", + "\n", + "from fastapi.datastructures import URL\n", + "\n", + "\n", + "class AuthService:\n", + " def is_valid_token(self, token: str, permissions: Sequence[str], url: URL):\n", + " # Add your authentication logic here\n", + " print(f\"Checking permission for route: {url.path}\")\n", + " return True" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With this `PermissionsChecker`, `permissions` can be passed in to define which roles, e.g. \"user\" or \"admin\",\n", + "are allowed to access which endpoints. The `PermissionsChecker` implements the `__call__` function, so that it can be\n", + "used as a function in the `dependencies` argument of each route via `Depends`. For more details see the extended\n", + "definition of the `summary_task_route` further below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from fastapi import HTTPException, Request\n", + "\n", + "\n", + "class PermissionChecker:\n", + " def __init__(self, permissions: Sequence[str] = []):\n", + " self.permissions = permissions\n", + "\n", + " def __call__(\n", + " self,\n", + " request: Request,\n", + " auth_service: Annotated[AuthService, Depends(AuthService)],\n", + " ) -> None:\n", + " token = request.headers.get(\"Authorization\")\n", + " try:\n", + " if not auth_service.is_valid_token(token, self.permissions, request.url):\n", + " raise HTTPException(HTTPStatus.UNAUTHORIZED)\n", + " except RuntimeError:\n", + " raise HTTPException(HTTPStatus.INTERNAL_SERVER_ERROR)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A specific `PermissionChecker` with `\"User\"` permissions is created which will be called for the `/summary` route to check, whether a \"User\" is allowed to access it.\n", + "\n", + "The permission checker can then be added to any route via the `dependencies` argument in the decorator. Here, we add it to the `summary_task_route`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "permission_checker_for_user = PermissionChecker([\"User\"])\n", + "\n", + "\n", + "@app.post(\"/summary\", dependencies=[Depends(permission_checker_for_user)])\n", + "def summary_task_route(\n", + " input: SingleChunkSummarizeInput,\n", + " task: Annotated[\n", + " Task[SingleChunkSummarizeInput, SummarizeOutput], Depends(summary_task)\n", + " ],\n", + ") -> SummarizeOutput:\n", + " return task.run(input, NoOpTracer())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Complete Source\n", + "The final source can be found in the [accompanying python file](./fastapi_example.py)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "intelligence-layer-rp3__H-P-py3.11", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/src/examples/fastapi_tutorial.md b/src/examples/fastapi_tutorial.md deleted file mode 100644 index f8af0648f..000000000 --- a/src/examples/fastapi_tutorial.md +++ /dev/null @@ -1,307 +0,0 @@ -# Tutorial: Extending a FastAPI App with the Aleph-Alpha Intelligence Layer - -In this tutorial, a basic [FastAPI](https://fastapi.tiangolo.com) app is extended with a new route at which a summary for a given text can be retrieved, using the _Aleph-Alpha Intelligence Layer_, and it's _Luminous_ control models. - -The full source code for this example app can be found at the end of this tutorial and in [src/examples/fastapi_example.py](./fastapi_example.py). - -## Basic FastAPI App - -The foundation for this tutorial is a minimal [FastAPI](https://fastapi.tiangolo.com) application with a root endpoint: - -```python -from fastapi import FastAPI, Response -from http import HTTPStatus - - -app = FastAPI() - -@app.get("/") -def root() -> Response: - return Response(content="Hello World", status_code=HTTPStatus.OK) -``` - -This application can be started from the command line with the [Hypercorn](https://github.com/pgjones/hypercorn/) server as follows: - -```bash -hypercorn fastapi_example:app --bind localhost:8000 -``` - -If the start-up was successful, you should see a message similar to -```cmd -[2024-03-07 14:00:55 +0100] [6468] [INFO] Running on http://:8000 (CTRL + C to quit) -``` - -Now that the server is running, we can perform a `GET` request via `cURL`: -```bash -curl -X GET http://localhost:8000 -``` -You should get -``` -Hello World -``` - -After successfully starting the basic FastAPI app, the next step is to add a route that makes use of the Intelligence Layer. - -## Adding the Intelligence Layer to the application - -The building blocks of the Intelligence Layer for applications are `Tasks`. In general, a task implements the `Task` -interface and defines an `Input` and an `Output`. Multiple tasks can be chained to create more complex applications. -Here, we will make use of the pre-built task `SteerableSingleChunkSummarize`. This task defines `SingleChunkSummarizeInput` -as it's input, and `SummarizeOutput` as it's output. -Like many other tasks, the `SteerableSingleChunkSummarize` task makes use of a `ControlModel`. The -`ControlModel` itself needs access to the Aleph-Alpha backend via a `AlephAlphaClientProtocol` client. -In short, the hierarchy is as follows: - -![task_dependencies.drawio.svg](task_dependencies.drawio.svg) - -We make use of the built-in [Dependency Injection](https://fastapi.tiangolo.com/reference/dependencies/) of FastAPI to -resolve this hierarchy automatically. In this framework, the defaults for the parameters are dynamically created with -the `Depends(func)` annotation, where `func` is a function that returns the default value. - -So, first, we define our client-generating function. For that, we provide the host URL and a valid Aleph-Alpha token, -which are stored in an `.env`-file. - -```python -import os -from aleph_alpha_client import Client -from dotenv import load_dotenv - -load_dotenv() - -def client() -> Client: - return Client( - token=os.environ["AA_TOKEN"], - host=os.getenv("AA_CLIENT_BASE_URL", "https://api.aleph-alpha.com") - ) -``` - -Next, we create a `ControlModel`. In this case, we make use of the `LuminousControlModel`, which takes -an `AlephAlphaClientProtocol` that we let default to the previously defined `client`. - -```python -from typing import Annotated -from fastapi import Depends -from intelligence_layer.connectors import AlephAlphaClientProtocol -from intelligence_layer.core import LuminousControlModel - -def default_model(app_client: Annotated[AlephAlphaClientProtocol, Depends(client)]): - return LuminousControlModel(client=app_client) -``` - - -Finally, we create the actual `Task`. For our example, we choose the `SteerableSingleChunkSummarize`. -The `Input` of this task is a `SingleChunkSummarizeInput`, consisting of the text to summarize as the `chunk` field, -and the desired `Language` as the `language` field. -The `Output` of this task is a `SummarizeOutput` and contains the `summary` as text, -and number of generated tokens for the `summary` as the `generated_tokens` field. - -```python -from intelligence_layer.use_cases import SteerableSingleChunkSummarize -from intelligence_layer.core import LuminousControlModel - -def summary_task( - model: Annotated[LuminousControlModel, Depends(default_model)], -) -> SteerableSingleChunkSummarize: - return SteerableSingleChunkSummarize(model) -``` -We can then provide a `POST` endpoint on `/summary` to run the task. -The default for `task` will be set by `summary_task`. - -```python -from intelligence_layer.use_cases import ( - SingleChunkSummarizeInput, - SummarizeOutput, -) -from intelligence_layer.core import NoOpTracer - -@app.post("/summary") -def summary_task_route( - input: SingleChunkSummarizeInput, - task: Annotated[ - Task[SingleChunkSummarizeInput, SummarizeOutput], - Depends(summary_task) - ], -) -> SummarizeOutput: - return task.run(input, NoOpTracer()) -``` - -This concludes the addition of an Intelligence-Layer task to the FastAPI app. After restarting the server, we can call -our newly created endpoint via a command such as the following: -```bash - -curl -X POST http://localhost:8000/summary -H "Content-Type: application/json" -d '{"chunk": "", "language": {"iso_639_1": "en"}}' -``` - -## Add Authorization to the Routes - -Typically, authorization is needed to control access to endpoints. -Here, we will give a minimal example of how a per-route authorization system could be implemented in the minimal example app. - -The authorization system makes use of two parts: An `AuthService` that checks whether the user is allowed to access a -given site, and a `PermissionsChecker` that is called on each route access and in turn calls the `AuthService`. - -For this minimal example, the `AuthService` is simply a stub. You will want to implement a concrete authorization service -tailored to your needs. - -```python -from typing import Sequence -from fastapi.datastructures import URL - -class AuthService: - def is_valid_token( - self, - token: str, - permissions: Sequence[str], - url: URL - ): - # Add your authentication logic here - print(f"Checking permission for route: {url.path}") - return True -``` - -With this `PermissionsChecker`, `permissions` can be passed in to define which roles, e.g. "user" or "admin", -are allowed to access which endpoints. The `PermissionsChecker` implements the `__call__` function, so that it can be -used as a function in the `dependencies` argument of each route via `Depends`. For more details see the extended -definition of the `summary_task_route` further below. - -```python -from fastapi import HTTPException, Request - -class PermissionChecker: - def __init__(self, permissions: Sequence[str] = []): - self.permissions = permissions - - def __call__( - self, - request: Request, - auth_service:Annotated[AuthService, Depends(AuthService)], - ) -> None: - token = request.headers.get("Authorization") - try: - if not auth_service.is_valid_token(token, self.permissions, request.url): - raise HTTPException(HTTPStatus.UNAUTHORIZED) - except RuntimeError: - raise HTTPException(HTTPStatus.INTERNAL_SERVER_ERROR) -``` - -A specific `PermissionChecker` with `"User"` permissions is created which will be called for the `/summary` route to check, whether a "User" is allowed to access it. - -```python -permission_checker_for_user = PermissionChecker(["User"]) -``` - -The permission checker can be added to any route via the `dependencies` argument in the decorator. Here, we add it to the `summary_task_route`: - -```python -@app.post("/summary", dependencies=[Depends(permission_checker_for_user)]) -def summary_task_route( - input: SingleChunkSummarizeInput, - task: Annotated[ - Task[SingleChunkSummarizeInput, SummarizeOutput], - Depends(summary_task) - ], -) -> SummarizeOutput: - return task.run(input, NoOpTracer()) - -``` - - -## Complete Source - -```python -import http -import os -from http import HTTPStatus -from typing import Annotated, Sequence - -from aleph_alpha_client import Client -from dotenv import load_dotenv -from fastapi import Depends, FastAPI, HTTPException, Request, Response -from fastapi.datastructures import URL - -from intelligence_layer.connectors import AlephAlphaClientProtocol -from intelligence_layer.core import LuminousControlModel, NoOpTracer, Task -from intelligence_layer.use_cases import ( - SingleChunkSummarizeInput, - SteerableSingleChunkSummarize, - SummarizeOutput, -) - -# Minimal FastAPI app ########################################################## - -app = FastAPI() - - -@app.get("/") -def root() -> Response: - return Response(content="Hello World", status_code=HTTPStatus.OK) - - -# Authentication ############################################################### - - -class AuthService: - def is_valid_token(self, token: str, permissions: Sequence[str], url: URL) -> bool: - # Add your authentication logic here - print(f"Checking permission for route: {url.path}") - return True - - -class PermissionChecker: - def __init__(self, permissions: Sequence[str] = []): - self.permissions = permissions - - def __call__( - self, - request: Request, - auth_service: Annotated[AuthService, Depends(AuthService)], - ) -> None: - token = request.headers.get("Authorization") or "" - try: - if not auth_service.is_valid_token(token, self.permissions, request.url): - raise HTTPException(HTTPStatus.UNAUTHORIZED) - except RuntimeError: - raise HTTPException(HTTPStatus.INTERNAL_SERVER_ERROR) - - -permission_checker_for_user = PermissionChecker(["User"]) - - -# Intelligence Layer Task ###################################################### - -load_dotenv() - - -def client() -> Client: - return Client( - token=os.environ["AA_TOKEN"], - host=os.getenv("AA_CLIENT_BASE_URL", "https://api.aleph-alpha.com"), - ) - - -def default_model( - app_client: Annotated[AlephAlphaClientProtocol, Depends(client)] -) -> LuminousControlModel: - return LuminousControlModel(client=app_client) - - -def summary_task( - model: Annotated[LuminousControlModel, Depends(default_model)], -) -> SteerableSingleChunkSummarize: - return SteerableSingleChunkSummarize(model) - - -@app.post( - "/summary", - dependencies=[Depends(PermissionChecker(["User"]))], - status_code=http.HTTPStatus.OK, -) -def summary_task_route( - input: SingleChunkSummarizeInput, - task: Annotated[ - Task[SingleChunkSummarizeInput, SummarizeOutput], Depends(summary_task) - ], -) -> SummarizeOutput: - return task.run(input, NoOpTracer()) - -```