Skip to content

Commit

Permalink
WIP: docs: Add elo how to
Browse files Browse the repository at this point in the history
TASK: IL-502
  • Loading branch information
MerlinKallenbornAA committed May 16, 2024
1 parent 0a32634 commit fb83968
Showing 1 changed file with 23 additions and 0 deletions.
23 changes: 23 additions & 0 deletions src/documentation/how_tos/hot_to_implement_elo_evaluations.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to implement elo evaluation runs\n",
"0. Run your tasks on the datasets you want to evaluate (see [here](./how_to_run_a_task_on_a_dataset.ipynb))\n",
" - When evaluating multiple runs, all of them need the same data types \n",
"2. Initialize all necessary repositories for the `IncrementalEvaluator`, and an `IncrementalEvaluationLogic`.\n",
"3. Run the evaluator to evaluate all examples and create a single `EvaluationOverview`\n",
"4. (Optional) Save the evaluation id for later use"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

0 comments on commit fb83968

Please sign in to comment.