Skip to content

Commit

Permalink
Improve quick tour docs
Browse files Browse the repository at this point in the history
  • Loading branch information
xenova committed Oct 22, 2024
1 parent a8be5b9 commit a1a668f
Showing 1 changed file with 30 additions and 3 deletions.
33 changes: 30 additions & 3 deletions docs/snippets/1_quick-tour.snippet
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ out = pipe('I love transformers!')
import { pipeline } from '@huggingface/transformers';
// Allocate a pipeline for sentiment-analysis
let pipe = await pipeline('sentiment-analysis');
const pipe = await pipeline('sentiment-analysis');
let out = await pipe('I love transformers!');
const out = await pipe('I love transformers!');
// [{'label': 'POSITIVE', 'score': 0.999817686}]
```

Expand All @@ -40,5 +40,32 @@ let out = await pipe('I love transformers!');
You can also use a different model by specifying the model id or path as the second argument to the `pipeline` function. For example:
```javascript
// Use a different model for sentiment-analysis
let pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment');
const pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment');
```

By default, when running in the browser, the model will be run on your CPU (via WASM). If you would like
to run the model on your GPU (via WebGPU), you can do this by setting `device: 'webgpu'`, for example:
```javascript
// Run the model on WebGPU
const pipe = await pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', {
device: 'webgpu'
});
```

For more information, check out the [WebGPU guide](./guides/webgpu).

> [!NOTE]
> The WebGPU API is still experimental in many browsers, so if you run into any issues, please file a bug report
> [here](https://github.com/huggingface/transformers.js/issues/new?title=[WebGPU]%20Error%20running%20MODEL_ID_GOES_HERE&assignees=&labels=bug,webgpu&projects=&template=1_bug-report.yml).

In resource-constrained environments, such as web browsers, it is advisable to use a quantized version of
the model to lower bandwidth and optimize performance. This can be achieved by adjusting the `dtype` option,
which allows you to select the appropriate data type for your model. While the available options may vary
depending on the specific model, typical choices include `"fp32"` (default for WebGPU), `"fp16"`, `"q8"`
(default for WASM), and `"q4"`. For more information, check out the [quantization guide](./guides/dtypes).
```javascript
// Run the model at 4-bit quantization
const pipe = await pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', {
dtype: 'q4',
});
```

0 comments on commit a1a668f

Please sign in to comment.