Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
  • Loading branch information
seonglae committed Dec 2, 2024
2 parents d50f26d + 9a0a110 commit abafa51
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions docs/source/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ It adds depth and richness without being overpowering its taste buds alone

### Streaming

Some pipelines such as `text-generation` or whisper `speech-to-text` models support streaming output. For example, when using a chat model like `Qwen2.5-Coder-0.5B-Instruct`, you can specify a callback function that will be called with each generated token text.
Some pipelines such as `text-generation` or `automatic-speech-recognition` support streaming output. This is achieved using the `TextStreamer` class. For example, when using a chat model like `Qwen2.5-Coder-0.5B-Instruct`, you can specify a callback function that will be called with each generated token text (if unset, new tokens will be printed to the console).

```js
import { pipeline, TextStreamer } from "@huggingface/transformers";
Expand All @@ -171,7 +171,8 @@ const messages = [
// Create text streamer
const streamer = new TextStreamer(generator.tokenizer, {
skip_prompt: true,
callback_function: (text) => process.stdout.write(text), // Stream to command line
// Optionally, do something with the text (e.g., write to a textbox)
// callback_function: (text) => { /* Do something with text */ },
})

// Generate a response
Expand Down

0 comments on commit abafa51

Please sign in to comment.