Skip to content

Commit

Permalink
docs: fastapi tutorial is now an ipynb
Browse files Browse the repository at this point in the history
  • Loading branch information
@NiklasKoehneckeAA
NiklasKoehneckeAA committed Apr 25, 2024
1 parent b08d282 commit 2c46184
Show file tree
Hide file tree
Showing 3 changed files with 325 additions and 308 deletions.
3 changes: 2 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The key features of the Intelligence Layer are:
- [License](#license)
- [For Developers](#for-developers)
- [Python: Naming Conventions](#python-naming-conventions)
- [Tests](#tests)

# Installation

Expand Down Expand Up @@ -203,7 +204,7 @@ The tutorials aim to guide you through implementing several common use-cases wit
| 6 | Document Index | Connect your proprietary knowledge base | [document_index.ipynb](./src/examples/document_index.ipynb) |
| 7 | Human Evaluation | Connect to Argilla for manual evaluation | [human_evaluation.ipynb](./src/examples/human_evaluation.ipynb) |
| 8 | Performance tips | Contains some small tips for performance | [performance_tips.ipynb](./src/examples/performance_tips.ipynb) |
| 9 | Deployment | Shows how to deploy a Task in a minimal FastAPI app. | [fastapi_tutorial.md](./src/examples/fastapi_tutorial.md) |
| 9 | Deployment | Shows how to deploy a Task in a minimal FastAPI app. | [fastapi_tutorial.ipynb](./src/examples/fastapi_tutorial.ipynb) |
| 10 | Issue Classification | In-depth tutorial about implementing and evaluating an email classificator | [issue_classification_user_journy.ipynb](./src/examples/issue_classification_user_journey.ipynb)

## How-Tos
Expand Down
323 changes: 323 additions & 0 deletions src/examples/fastapi_tutorial.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,323 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial: Extending a FastAPI App with the Aleph-Alpha Intelligence Layer\n",
"\n",
"In this tutorial, a basic [FastAPI](https://fastapi.tiangolo.com) app is extended with a new route at which a summary for a given text can be retrieved, using the _Aleph-Alpha Intelligence Layer_, and it's _Luminous_ control models.\n",
"\n",
"The full source code for this example app can be found at the end of this tutorial and in [src/examples/fastapi_example.py](./fastapi_example.py).\n",
"\n",
"## Basic FastAPI App\n",
"\n",
"The foundation for this tutorial is a minimal [FastAPI](https://fastapi.tiangolo.com) application with a root endpoint:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from http import HTTPStatus\n",
"\n",
"from fastapi import FastAPI, Response\n",
"\n",
"app = FastAPI()\n",
"\n",
"\n",
"@app.get(\"/\")\n",
"def root() -> Response:\n",
" return Response(content=\"Hello World\", status_code=HTTPStatus.OK)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This application can be started from the command line with the [Hypercorn](https://github.com/pgjones/hypercorn/) server as follows:\n",
"\n",
"```bash\n",
"hypercorn fastapi_example:app --bind localhost:8000\n",
"```\n",
"\n",
"If the start-up was successful, you should see a message similar to\n",
"```cmd\n",
"[2024-03-07 14:00:55 +0100] [6468] [INFO] Running on http://<your ip>:8000 (CTRL + C to quit)\n",
"```\n",
"\n",
"Now that the server is running, we can perform a `GET` request via `cURL`:\n",
"```bash\n",
"curl -X GET http://localhost:8000\n",
"```\n",
"You should get\n",
"```\n",
"Hello World\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After successfully starting the basic FastAPI app, the next step is to add a route that makes use of the Intelligence Layer.\n",
"\n",
"## Adding the Intelligence Layer to the application\n",
"\n",
"The building blocks of the Intelligence Layer for applications are `Tasks`. In general, a task implements the `Task`\n",
"interface and defines an `Input` and an `Output`. Multiple tasks can be chained to create more complex applications.\n",
"Here, we will make use of the pre-built task `SteerableSingleChunkSummarize`. This task defines `SingleChunkSummarizeInput`\n",
"as it's input, and `SummarizeOutput` as it's output.\n",
"Like many other tasks, the `SteerableSingleChunkSummarize` task makes use of a `ControlModel`. The\n",
"`ControlModel` itself needs access to the Aleph-Alpha backend via a `AlephAlphaClientProtocol` client.\n",
"In short, the hierarchy is as follows:\n",
"\n",
"![task_dependencies.drawio.svg](task_dependencies.drawio.svg)\n",
"\n",
"We make use of the built-in [Dependency Injection](https://fastapi.tiangolo.com/reference/dependencies/) of FastAPI to\n",
"resolve this hierarchy automatically. In this framework, the defaults for the parameters are dynamically created with\n",
"the `Depends(func)` annotation, where `func` is a function that returns the default value.\n",
"\n",
"So, first, we define our client-generating function. For that, we provide the host URL and a valid Aleph-Alpha token,\n",
"which are stored in an `.env`-file.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from aleph_alpha_client import Client\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv()\n",
"\n",
"\n",
"def client() -> Client:\n",
" return Client(\n",
" token=os.environ[\"AA_TOKEN\"],\n",
" host=os.getenv(\"AA_CLIENT_BASE_URL\", \"https://api.aleph-alpha.com\"),\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we create a `ControlModel`. In this case, we make use of the `LuminousControlModel`, which takes\n",
"an `AlephAlphaClientProtocol` that we let default to the previously defined `client`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from typing import Annotated\n",
"\n",
"from fastapi import Depends\n",
"\n",
"from intelligence_layer.connectors import AlephAlphaClientProtocol\n",
"from intelligence_layer.core import LuminousControlModel\n",
"\n",
"\n",
"def default_model(app_client: Annotated[AlephAlphaClientProtocol, Depends(client)]):\n",
" return LuminousControlModel(client=app_client)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, we create the actual `Task`. For our example, we choose the `SteerableSingleChunkSummarize`.\n",
"The `Input` of this task is a `SingleChunkSummarizeInput`, consisting of the text to summarize as the `chunk` field,\n",
"and the desired `Language` as the `language` field.\n",
"The `Output` of this task is a `SummarizeOutput` and contains the `summary` as text,\n",
"and number of generated tokens for the `summary` as the `generated_tokens` field."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intelligence_layer.use_cases import SteerableSingleChunkSummarize\n",
"\n",
"\n",
"def summary_task(\n",
" model: Annotated[LuminousControlModel, Depends(default_model)],\n",
") -> SteerableSingleChunkSummarize:\n",
" return SteerableSingleChunkSummarize(model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can then provide a `POST` endpoint on `/summary` to run the task.\n",
"The default for `task` will be set by `summary_task`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intelligence_layer.core import NoOpTracer, Task\n",
"from intelligence_layer.use_cases import SingleChunkSummarizeInput, SummarizeOutput\n",
"\n",
"\n",
"@app.post(\"/summary\")\n",
"def summary_task_route_without_permissions(\n",
" input: SingleChunkSummarizeInput,\n",
" task: Annotated[\n",
" Task[SingleChunkSummarizeInput, SummarizeOutput], Depends(summary_task)\n",
" ],\n",
") -> SummarizeOutput:\n",
" return task.run(input, NoOpTracer())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This concludes the addition of an Intelligence-Layer task to the FastAPI app. After restarting the server, we can call\n",
"our newly created endpoint via a command such as the following:\n",
"```bash\n",
"\n",
"curl -X POST http://localhost:8000/summary -H \"Content-Type: application/json\" -d '{\"chunk\": \"<your text to summarize here>\", \"language\": {\"iso_639_1\": \"en\"}}'\n",
"```\n",
"\n",
"## Add Authorization to the Routes\n",
"\n",
"Typically, authorization is needed to control access to endpoints.\n",
"Here, we will give a minimal example of how a per-route authorization system could be implemented in the minimal example app.\n",
"\n",
"The authorization system makes use of two parts: An `AuthService` that checks whether the user is allowed to access a\n",
"given site, and a `PermissionsChecker` that is called on each route access and in turn calls the `AuthService`.\n",
"\n",
"For this minimal example, the `AuthService` is simply a stub. You will want to implement a concrete authorization service\n",
"tailored to your needs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from typing import Sequence\n",
"\n",
"from fastapi.datastructures import URL\n",
"\n",
"\n",
"class AuthService:\n",
" def is_valid_token(self, token: str, permissions: Sequence[str], url: URL):\n",
" # Add your authentication logic here\n",
" print(f\"Checking permission for route: {url.path}\")\n",
" return True"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this `PermissionsChecker`, `permissions` can be passed in to define which roles, e.g. \"user\" or \"admin\",\n",
"are allowed to access which endpoints. The `PermissionsChecker` implements the `__call__` function, so that it can be\n",
"used as a function in the `dependencies` argument of each route via `Depends`. For more details see the extended\n",
"definition of the `summary_task_route` further below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from fastapi import HTTPException, Request\n",
"\n",
"\n",
"class PermissionChecker:\n",
" def __init__(self, permissions: Sequence[str] = []):\n",
" self.permissions = permissions\n",
"\n",
" def __call__(\n",
" self,\n",
" request: Request,\n",
" auth_service: Annotated[AuthService, Depends(AuthService)],\n",
" ) -> None:\n",
" token = request.headers.get(\"Authorization\")\n",
" try:\n",
" if not auth_service.is_valid_token(token, self.permissions, request.url):\n",
" raise HTTPException(HTTPStatus.UNAUTHORIZED)\n",
" except RuntimeError:\n",
" raise HTTPException(HTTPStatus.INTERNAL_SERVER_ERROR)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A specific `PermissionChecker` with `\"User\"` permissions is created which will be called for the `/summary` route to check, whether a \"User\" is allowed to access it.\n",
"\n",
"The permission checker can then be added to any route via the `dependencies` argument in the decorator. Here, we add it to the `summary_task_route`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"permission_checker_for_user = PermissionChecker([\"User\"])\n",
"\n",
"\n",
"@app.post(\"/summary\", dependencies=[Depends(permission_checker_for_user)])\n",
"def summary_task_route(\n",
" input: SingleChunkSummarizeInput,\n",
" task: Annotated[\n",
" Task[SingleChunkSummarizeInput, SummarizeOutput], Depends(summary_task)\n",
" ],\n",
") -> SummarizeOutput:\n",
" return task.run(input, NoOpTracer())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Complete Source\n",
"The final source can be found in the [accompanying python file](./fastapi_example.py)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "intelligence-layer-rp3__H-P-py3.11",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading

0 comments on commit 2c46184

Please sign in to comment.