From e0773ec26d06ba738384307310db2d9205ca2e6a Mon Sep 17 00:00:00 2001 From: Seonglae Cho Date: Tue, 3 Dec 2024 08:22:05 +0900 Subject: [PATCH 1/2] docs: streaming description from @xenova's suggestion Co-authored-by: Joshua Lochner --- docs/source/pipelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pipelines.md b/docs/source/pipelines.md index 62081e915..bec5abadb 100644 --- a/docs/source/pipelines.md +++ b/docs/source/pipelines.md @@ -150,7 +150,7 @@ It adds depth and richness without being overpowering its taste buds alone ### Streaming -Some pipelines such as `text-generation` or whisper `speech-to-text` models support streaming output. For example, when using a chat model like `Qwen2.5-Coder-0.5B-Instruct`, you can specify a callback function that will be called with each generated token text. +Some pipelines such as `text-generation` or `automatic-speech-recognition` support streaming output. This is achieved using the `TextStreamer` class. For example, when using a chat model like `Qwen2.5-Coder-0.5B-Instruct`, you can specify a callback function that will be called with each generated token text (if unset, new tokens will be printed to the console). ```js import { pipeline, TextStreamer } from "@huggingface/transformers"; From 9a0a110b71033bbe8d2ad7b58edd745d49edae22 Mon Sep 17 00:00:00 2001 From: Seonglae Cho Date: Tue, 3 Dec 2024 08:22:26 +0900 Subject: [PATCH 2/2] fix: streaming example from @xenova's suggestion Co-authored-by: Joshua Lochner --- docs/source/pipelines.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/source/pipelines.md b/docs/source/pipelines.md index bec5abadb..7ee32ddff 100644 --- a/docs/source/pipelines.md +++ b/docs/source/pipelines.md @@ -171,7 +171,8 @@ const messages = [ // Create text streamer const streamer = new TextStreamer(generator.tokenizer, { skip_prompt: true, - callback_function: (text) => process.stdout.write(text), // Stream to command line + // Optionally, do something with the text (e.g., write to a textbox) + // callback_function: (text) => { /* Do something with text */ }, }) // Generate a response