diff --git a/docs/sentence_transformer/training_overview.md b/docs/sentence_transformer/training_overview.md index f22fdf1d3..af0d7b7f0 100644 --- a/docs/sentence_transformer/training_overview.md +++ b/docs/sentence_transformer/training_overview.md @@ -284,9 +284,10 @@ Here are the implemented Evaluators that come with Sentence Transformers: ======================================================================== =========================================================================================================================== Evaluator Required Data ======================================================================== =========================================================================================================================== -:class:`~sentence_transformers.evaluation.BinaryClassificationEvaluator` Pairs with class labels -:class:`~sentence_transformers.evaluation.EmbeddingSimilarityEvaluator` Pairs with similarity scores -:class:`~sentence_transformers.evaluation.InformationRetrievalEvaluator` Queries (qid => question), Corpus (cid => document), and relevant documents (qid => set[cid]) +:class:`~sentence_transformers.evaluation.BinaryClassificationEvaluator` Pairs with class labels. +:class:`~sentence_transformers.evaluation.EmbeddingSimilarityEvaluator` Pairs with similarity scores. +:class:`~sentence_transformers.evaluation.InformationRetrievalEvaluator` Queries (qid => question), Corpus (cid => document), and relevant documents (qid => set[cid]). +:class:`~sentence_transformers.evaluation.NanoBEIREvaluator` No data required. :class:`~sentence_transformers.evaluation.MSEEvaluator` Source sentences to embed with a teacher model and target sentences to embed with the student model. Can be the same texts. :class:`~sentence_transformers.evaluation.ParaphraseMiningEvaluator` Mapping of IDs to sentences & pairs with IDs of duplicate sentences. :class:`~sentence_transformers.evaluation.RerankingEvaluator` List of ``{'query': '...', 'positive': [...], 'negative': [...]}`` dictionaries. @@ -363,6 +364,27 @@ Sometimes you don't have the required evaluation data to prepare one of these ev # You can run evaluation like so: # dev_evaluator(model) +.. tab:: NanoBEIREvaluator + + .. raw:: html + +
+ + :: + + from sentence_transformers.evaluation import NanoBEIREvaluator + + # Initialize the evaluator. Unlike most other evaluators, this one loads the relevant datasets + # directly from Hugging Face, so there's no mandatory arguments + dev_evaluator = NanoBEIREvaluator() + # You can run evaluation like so: + # dev_evaluator(model) + .. warning:: When using `Distributed Training