add tip

huggingface · May 29, 2024 · ee9a498 · ee9a498
1 parent c95c07a
commit ee9a498
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/docs/source/en/llm_optims.md b/docs/source/en/llm_optims.md
@@ -181,6 +181,9 @@ text
 </hfoption>
 </hfoptions>
 
+> [!TIP]
+> Regardless of the strategy used with `torch.compile`, through `forward` or [`~GenerationMixin.generate`], you can avoid shape-related recompilations if you left-pad your LLM inputs to a limited set of values. The [`pad_to_multiple_of` tokenizer flag](https://huggingface.co/docs/transformers/main_classes/tokenizer#transformers.PreTrainedTokenizer.__call__.pad_to_multiple_of) is your friend!
+
 ## Speculative decoding
 
 > [!TIP]