helper #31152
Merged
helper #31152
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Co-authored-by: amyeroberts <[email protected]>
…s into add-patch-helper
itazap
pushed a commit
that referenced
this pull request
May 31, 2024
* helper * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * updates * more doc --------- Co-authored-by: amyeroberts <[email protected]>
vasqu
pushed a commit
to vasqu/transformers
that referenced
this pull request
Jun 1, 2024
* helper * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * updates * more doc --------- Co-authored-by: amyeroberts <[email protected]>
vasqu
added a commit
to vasqu/transformers
that referenced
this pull request
Jun 1, 2024
commit bf6ea14 Merge: b3261f5 96eb062 Author: Vasqu <[email protected]> Date: Sat Jun 1 02:49:53 2024 +0200 Merge remote-tracking branch 'origin/main' commit b3261f5 Author: Arthur <[email protected]> Date: Fri May 31 18:37:43 2024 +0200 Diff converter v2 (huggingface#30868) * current working example! * commit regex and result file * update * nit * push the conversion file * oups * roadmap and nits * attempt diffs for 3 files * persimmon * nit * add diff file that is the same as the modeling_llama.py * fix rope nits * updates * updates with converted versions * give some breathing space to the code * delete * update * update * push the actual result * update regex patterns * update regex patterns * fix some issues * fix some issues * fix some issues * updates * updates * updates * updates * updates * revert changes done to llama * updates * update gemma * updates * oups * current state * current state * update * ouiiii * nit * clear diffs * nit * fixup * update * doc 🚀 * 🔥 * for now use gemma * deal with comments * style * handle funtions * deal with assigns * todos * process inheritage * keep decorators? * 🤗 * deal with duplicates * fixup * correctly remove duplicate code * run ruff post script * ruff deals pretty well with imports, let's leave it to him * ah maybe not lol * for now remove all imports from child. * nit * conversion of llama * okay * convert starcoder2 * synch with main * update llama diff * updates * https://docs.astral.sh/ruff/rules/redefined-while-unused/ fixes the imports, bit needs later version of ruff * updates * okay actual state * non zero exit * update! * revert unrelated * remove other diff files * updates * cleanup * update * less diff! * stash * current updates * updates * No need for call * finished fining deps * update * current changes * current state * current state * new status * nit * finally * fixes * nits * order is now expected * use logger info instead of prints * fixup * up * nit * update * nits * update * correct merge * update * update * update * add warning * update caution message * update * better merging strategy * copy class statements :wink * fixups * nits * update * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * nits * smaller header * do cleanup some stuff * even simpler header? * fixup * updates * ruff * update examples * nit * TODO * state * OUUUUUUF * current state * nits * final state * add a readme * fixup * remove diff llama * fix * nit * dummy noy funny * ruff format tests src utils --check * everless diffs * less diffs and fix test * fixes * naming nit? * update converter and add supper example * nits * updated for function signatures * update * update * add converted dummies * autoformat * single target assign fix * fixup * fix some imports * fixes * don't push them * `# noqa: F841` --------- Co-authored-by: amyeroberts <[email protected]> commit ba34b39 Author: Vallepu Vamsi Krishna <[email protected]> Date: Fri May 31 21:53:11 2024 +0530 Added description of quantization_config (huggingface#31133) * Description of quantization_config Added missing description about quantization_config in replace_with_bnb_linear for better readability. * Removed trailing spaces commit 2a2ec42 Author: Pavel Iakubovskii <[email protected]> Date: Fri May 31 16:56:17 2024 +0100 Instance segmentation examples (huggingface#31084) * Initial setup * Metrics * Overfit on two batches * Train 40 epochs * Memory leak debugging * Trainer fine-tuning * Draft * Fixup * Trained end-to-end * Add requirements * Rewrite evaluator * nits * Add readme * Add instance-segmentation to the table * Support void masks * Remove sh * Update docs * Add pytorch test * Add accelerate test * Update examples/pytorch/instance-segmentation/README.md * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation_no_trainer.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation_no_trainer.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py * Fix consistency oneformer * Fix imports * Fix imports sort * Apply suggestions from code review Co-authored-by: NielsRogge <[email protected]> * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py Co-authored-by: Sangbum Daniel Choi <[email protected]> * Add resources to docs * Update examples/pytorch/instance-segmentation/README.md Co-authored-by: amyeroberts <[email protected]> * Update examples/pytorch/instance-segmentation/README.md Co-authored-by: amyeroberts <[email protected]> * Remove explicit model_type argument * Fix tests * Update readme * Note about other models --------- Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sangbum Daniel Choi <[email protected]> Co-authored-by: amyeroberts <[email protected]> commit 3231ed4 Author: Aymeric Roucher <[email protected]> Date: Fri May 31 14:16:23 2024 +0200 Add streaming, various fixes (huggingface#30838) * Implement streaming run in ReAct agents * Allow additional imports in code agents * Python interpreter: support classes and exceptions, fixes commit 899d73f Author: Marc Sun <[email protected]> Date: Fri May 31 12:44:20 2024 +0200 [trainer] add sanity evaluation option (huggingface#31146) * add sanity evaluation * fix * Apply suggestions from code review Co-authored-by: Zach Mueller <[email protected]> * fix --------- Co-authored-by: Zach Mueller <[email protected]> commit 09daece Author: Younes Belkada <[email protected]> Date: Fri May 31 12:36:46 2024 +0200 Quantization: Enhance bnb error message (huggingface#31160) enhance error message commit 390c9f4 Author: Asif Ajrof <[email protected]> Date: Fri May 31 16:34:29 2024 +0600 Update sam.md (huggingface#31130) `mask` variable is not defined. probably a writing mistake. it should be `segmentation_map`. `segmentation_map` should be a `1` channel image rather than `RGB`. [on a different note, the `mask_url` is the same as `raw_image`. could provide a better example. commit a6967c0 Author: Marc Sun <[email protected]> Date: Fri May 31 12:08:55 2024 +0200 Fix quantized cache output (huggingface#31143) commit aa2e1d4 Author: Yih-Dar <[email protected]> Date: Fri May 31 10:35:54 2024 +0200 pytest -rsfE (huggingface#31140) Co-authored-by: ydshieh <[email protected]> commit 6c33f18 Author: Arthur <[email protected]> Date: Fri May 31 08:49:33 2024 +0200 helper (huggingface#31152) * helper * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * updates * more doc --------- Co-authored-by: amyeroberts <[email protected]> commit adb74a2 Author: Younes Belkada <[email protected]> Date: Thu May 30 17:21:10 2024 +0200 Workflow: Remove `IS_GITHUB_CI` (huggingface#31147) remove `IS_GITHUB_CI` commit 3553184 Author: Younes Belkada <[email protected]> Date: Thu May 30 16:47:35 2024 +0200 Docs / Quantization: Replace all occurences of `load_in_8bit` with bnb config (huggingface#31136) Replace all occurences of `load_in_8bit` with bnb config commit e6dcdfd Author: zspo <[email protected]> Date: Thu May 30 22:25:43 2024 +0800 fix get_scheduler when name is warmup_stable_decay (huggingface#31128) fix get_scheduler args commit 9d8b6ea Author: Younes Belkada <[email protected]> Date: Thu May 30 11:45:03 2024 +0200 FIX / Quantization: Add extra validation for bnb config (huggingface#31135) add validation for bnb config commit 7fc432f Author: Yih-Dar <[email protected]> Date: Wed May 29 19:43:51 2024 +0200 Cleanup docker build (huggingface#31119) * remove * build --------- Co-authored-by: ydshieh <[email protected]> commit c350b52 Author: Dhruv Pai <[email protected]> Date: Wed May 29 07:20:59 2024 -0700 Add on_optimizer_step to callback options (huggingface#31095) * Modified test * Added on_optimizer_step to callbacks * Move callback after step is called * Added on optimizer step callback commit 545d7ca Author: Joao Gante <[email protected]> Date: Wed May 29 15:17:14 2024 +0100 Add VLM generation default contributor (huggingface#31115) * add Raushan * add Raushan commit 296c546 Author: Younes Belkada <[email protected]> Date: Wed May 29 15:56:28 2024 +0200 FIX / Docs: Fix GPTQ expected number of bits (huggingface#31111) Update overview.md commit b643801 Author: Yih-Dar <[email protected]> Date: Wed May 29 15:42:39 2024 +0200 Fix nightly circleci (huggingface#31114) * fix * fix --------- Co-authored-by: ydshieh <[email protected]> commit 89261a1 Author: Zach Mueller <[email protected]> Date: Wed May 29 09:35:37 2024 -0400 Rm maintainer + migrate (huggingface#31089) commit 0e3643c Author: Matt <[email protected]> Date: Wed May 29 13:33:26 2024 +0100 Fix faulty rstrip in module loading (huggingface#31108) commit a41deea Author: Matt <[email protected]> Date: Wed May 29 13:20:36 2024 +0100 Fix env.py in cases where torch is not present (huggingface#31113) * Fix env.py in cases where torch is not present * Simplify the fix (and avoid some issues) commit 61f854a Author: Huazhong Ji <[email protected]> Date: Wed May 29 18:57:54 2024 +0800 Improve `transformers-cli env` reporting (huggingface#31003) * Improve `transformers-cli env` reporting * move the line `"Using GPU in script?": "<fill in>"` to in if conditional statement * same option for npu commit 40ed3a8 Author: Lucain <[email protected]> Date: Wed May 29 12:55:43 2024 +0200 Use `HF_HUB_OFFLINE` + fix has_file in offline mode (huggingface#31016) * Fix has_file in offline mode * harmonize env variable for offline mode * Switch to HF_HUB_OFFLINE * fix test * revert test_offline to test TRANSFORMERS_OFFLINE * Add new offline test * merge conflicts * docs commit 300d03c Author: Younes Belkada <[email protected]> Date: Wed May 29 11:43:54 2024 +0200 FEAT: Add mistral v3 conversion script (huggingface#30981) * add mistral v3 conversion script * Update src/transformers/models/mistral/convert_mistral_weights_to_hf.py Co-authored-by: Arthur <[email protected]> * fixup --------- Co-authored-by: Arthur <[email protected]> commit 524d7bf Author: Raushan Turganbay <[email protected]> Date: Wed May 29 14:25:44 2024 +0500 Quantized KV cache: update quanto (huggingface#31052) * quanto latest version was refactored * add error msg * incorrect compare sign * Update src/transformers/cache_utils.py Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]> commit 9f98c9c Author: amyeroberts <[email protected]> Date: Tue May 28 18:07:07 2024 +0100 Deprecate low use models (huggingface#30781) * Deprecate models - graphormer - time_series_transformer - xlm_prophetnet - qdqbert - nat - ernie_m - tvlt - nezha - mega - jukebox - vit_hybrid - x_clip - deta - speech_to_text_2 - efficientformer - realm - gptsan_japanese * Fix up * Fix speech2text2 imports * Make sure message isn't indented * Fix docstrings * Correctly map for deprecated models from model_type * Uncomment out * Add back time series transformer and x-clip * Import fix and fix-up * Fix up with updated ruff commit 1cb30f0 Author: Younes Belkada <[email protected]> Date: Tue May 28 18:29:22 2024 +0200 Docs / Quantization: Redirect deleted page (huggingface#31063) Update _redirects.yml commit 1ed4924 Author: Younes Belkada <[email protected]> Date: Tue May 28 18:29:11 2024 +0200 TST: Fix instruct-blip tests (huggingface#31088) * fix flan t5 tests * better format commit 2a08fd3 Author: Jonny Li <[email protected]> Date: Tue May 28 12:25:15 2024 -0400 Fix DeepSpeed compatibility with weight_norm (huggingface#30881) (huggingface#31018) commit b5f4ec6 Author: Albert Villanova del Moral <[email protected]> Date: Tue May 28 17:47:35 2024 +0200 Fix PretrainedConfig docstring with deprecated resume_download (huggingface#31014) commit 454cbe0 Author: Yih-Dar <[email protected]> Date: Tue May 28 17:44:52 2024 +0200 skip `test_multi_gpu_data_parallel_forward` for `vit` and `deit` (huggingface#31086) fix Co-authored-by: ydshieh <[email protected]> commit e70c2ea Author: Younes Belkada <[email protected]> Date: Tue May 28 17:06:00 2024 +0200 FIX / OPT: Fix OPT multi-GPU training for `OPTForQuestionAnswering` (huggingface#31092) Update modeling_opt.py commit 6560e25 Author: Younes Belkada <[email protected]> Date: Tue May 28 17:05:44 2024 +0200 FIX: Add `accelerate` as a hard requirement (huggingface#31090) add accelerate commit 9bf05ec Author: Sigbjørn Skjæret <[email protected]> Date: Tue May 28 16:02:51 2024 +0200 Render chat template tojson filter as unicode (huggingface#31041) * Render chat template tojson filter as unicode * ruff-- commit e405f2b Author: Younes Belkada <[email protected]> Date: Tue May 28 15:04:43 2024 +0200 Docs / PEFT: Add PEFT API documentation (huggingface#31078) * add peft references * add peft references * Update docs/source/en/peft.md * Update docs/source/en/peft.md commit 5237955 Author: Raushan Turganbay <[email protected]> Date: Tue May 28 17:07:42 2024 +0500 Watermark: fix tests (huggingface#30961) * fix tests * style * Update tests/generation/test_utils.py Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]> commit f2a7f7c Author: Lysandre Debut <[email protected]> Date: Tue May 28 13:34:23 2024 +0200 Fix failing tokenizer tests (huggingface#31083) * Fix failing tokenizer tests * Use small tokenizer * Fix remaining reference commit 0e1935b Author: NielsRogge <[email protected]> Date: Tue May 28 13:22:06 2024 +0200 [SuperPoint, PaliGemma] Update docs (huggingface#31025) * Update docs * Add PaliGemma resources * Address comment * Update docs commit 2fe8356 Author: Sina Taslimi <[email protected]> Date: Tue May 28 13:09:32 2024 +0200 Fix typo in trainer.py (huggingface#31048) commit b74960c Author: Pavel Iakubovskii <[email protected]> Date: Tue May 28 11:06:06 2024 +0000 Fix OWLv2 post_process_object_detection for multiple images (huggingface#31082) * Add test for multiple images * [run slow] owlv2 * Fix box rescaling * [run slow] owlv2 commit 3e3599d Author: Pavel Iakubovskii <[email protected]> Date: Tue May 28 10:41:40 2024 +0000 Remove float64 cast for OwlVit and OwlV2 to support MPS device (huggingface#31071) Remove float64 commit 48d33da Author: oOraph <[email protected]> Date: Tue May 28 11:56:05 2024 +0200 fix from_pretrained in offline mode when model is preloaded in cache (huggingface#31010) * Unit test to verify fix Signed-off-by: Raphael Glon <[email protected]> * fix from_pretrained in offline mode when model is preloaded in cache Signed-off-by: Raphael Glon <[email protected]> * minor: fmt Signed-off-by: Raphael Glon <[email protected]> --------- Signed-off-by: Raphael Glon <[email protected]> Co-authored-by: Raphael Glon <[email protected]> commit 7c472e6 Author: Hengwen Tong <[email protected]> Date: Tue May 28 17:52:47 2024 +0800 Remove redundant backend checks in training_args.py (huggingface#30999) * Remove backend checks in training_args.py * Expilicit initialize the device --------- Co-authored-by: tonghengwen <[email protected]> commit 46b606e Author: AP <[email protected]> Date: Tue May 28 09:50:45 2024 +0000 Update quicktour.md to fix broken link to Glossary (huggingface#31072) Update quicktour.md to fix broken link Missing '/' in attention mask link in the transformers quicktour commit 580f464 Author: Clint Adams <[email protected]> Date: Tue May 28 05:48:23 2024 -0400 fix "piano" typo (huggingface#31027) commit 5e211d5 Author: Yih-Dar <[email protected]> Date: Tue May 28 11:36:26 2024 +0200 Remove `ninja` from docker image build (huggingface#31080) fix Co-authored-by: ydshieh <[email protected]> commit 8b91c20 Author: Yih-Dar <[email protected]> Date: Tue May 28 10:53:28 2024 +0200 use `@main` (huggingface#31065) use main Co-authored-by: ydshieh <[email protected]> commit 04440a0 Author: Yih-Dar <[email protected]> Date: Mon May 27 18:36:39 2024 +0200 skip `test_model_parallelism` for 2 model test classes (huggingface#31067) skip Co-authored-by: ydshieh <[email protected]> commit f803e2b Author: Yoach Lacombe <[email protected]> Date: Mon May 27 16:09:05 2024 +0200 Fix pad_to_max_length Whisper (huggingface#30787) * fix pad_to_max_length Whisper * add tests * make style commit b6eb29b Author: Marc Sun <[email protected]> Date: Mon May 27 15:53:45 2024 +0200 Fix quanto tests (huggingface#31062) fix quanto tests commit e581213 Author: amyeroberts <[email protected]> Date: Mon May 27 14:16:47 2024 +0100 Update feature request label in template (huggingface#30940) commit 05eff71 Author: Eitan Turok <[email protected]> Date: Mon May 27 08:57:43 2024 -0400 Follow up: Fix link in dbrx.md (huggingface#30514) * Fix link in dbrx.md * remove "though this may not be up to date" --------- Co-authored-by: Lysandre Debut <[email protected]> commit d5aa839 Author: Yih-Dar <[email protected]> Date: Mon May 27 13:47:47 2024 +0200 unpin uv (huggingface#31055) [push-ci-image] Co-authored-by: ydshieh <[email protected]> commit 165bd7a Author: Aymeric Roucher <[email protected]> Date: Mon May 27 10:34:14 2024 +0200 Redirect transformers_agents doc to agents (huggingface#31054) commit 6df5028 Author: Pablo Montalvo <[email protected]> Date: Fri May 24 19:02:55 2024 +0200 Paligemma- fix devices and dtype assignments (huggingface#31008) * fix devices and dtype assignments * [run-slow]paligemma commit 61f1d47 Author: Ita Zaporozhets <[email protected]> Date: Fri May 24 17:38:58 2024 +0200 Add split special tokens (huggingface#30772) * seems like `split_special_tokens` is used here * split special token * add new line at end of file * moving split special token test to common tests * added assertions * test * fixup * add co-author * passing rest of args to gptsan_japanese, fixing tests * removing direct comparison of fast and slow models * adding test support for UDOP and LayoutXLM * ruff fix * readd check if slow tokenizer * modify test to handle bos tokens * removing commented function * trigger build * applying review feedback - updated docstrings, var names, and simplified tests * ruff fixes * Update tests/test_tokenization_common.py Co-authored-by: Arthur <[email protected]> * applying feedback, comments * shutil temp directory fix --------- Co-authored-by: Arthur Zucker <[email protected]> Co-authored-by: Ita Zaporozhets <[email protected]> Co-authored-by: itazap <[email protected]> Co-authored-by: Arthur <[email protected]> Co-authored-by: Ita Zaporozhets <[email protected]> commit e2b9913 Author: BHUVAN M <[email protected]> Date: Fri May 24 20:50:09 2024 +0530 added interpolation for vitmae model in pytorch as well as tf. (huggingface#30732) * added interpolation for vitmae model in pytorch as well as tf. * Update modeling_vit_mae.py irreugalr import fixed * small changes and proper formatting * changes suggested in review. * modified decoder interpolate_func * arguments and docstring fix * Apply suggestions from code review doc fixes Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]> commit 96eb062 Author: Arthur <[email protected]> Date: Fri May 31 18:37:43 2024 +0200 Diff converter v2 (huggingface#30868) * current working example! * commit regex and result file * update * nit * push the conversion file * oups * roadmap and nits * attempt diffs for 3 files * persimmon * nit * add diff file that is the same as the modeling_llama.py * fix rope nits * updates * updates with converted versions * give some breathing space to the code * delete * update * update * push the actual result * update regex patterns * update regex patterns * fix some issues * fix some issues * fix some issues * updates * updates * updates * updates * updates * revert changes done to llama * updates * update gemma * updates * oups * current state * current state * update * ouiiii * nit * clear diffs * nit * fixup * update * doc 🚀 * 🔥 * for now use gemma * deal with comments * style * handle funtions * deal with assigns * todos * process inheritage * keep decorators? * 🤗 * deal with duplicates * fixup * correctly remove duplicate code * run ruff post script * ruff deals pretty well with imports, let's leave it to him * ah maybe not lol * for now remove all imports from child. * nit * conversion of llama * okay * convert starcoder2 * synch with main * update llama diff * updates * https://docs.astral.sh/ruff/rules/redefined-while-unused/ fixes the imports, bit needs later version of ruff * updates * okay actual state * non zero exit * update! * revert unrelated * remove other diff files * updates * cleanup * update * less diff! * stash * current updates * updates * No need for call * finished fining deps * update * current changes * current state * current state * new status * nit * finally * fixes * nits * order is now expected * use logger info instead of prints * fixup * up * nit * update * nits * update * correct merge * update * update * update * add warning * update caution message * update * better merging strategy * copy class statements :wink * fixups * nits * update * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * nits * smaller header * do cleanup some stuff * even simpler header? * fixup * updates * ruff * update examples * nit * TODO * state * OUUUUUUF * current state * nits * final state * add a readme * fixup * remove diff llama * fix * nit * dummy noy funny * ruff format tests src utils --check * everless diffs * less diffs and fix test * fixes * naming nit? * update converter and add supper example * nits * updated for function signatures * update * update * add converted dummies * autoformat * single target assign fix * fixup * fix some imports * fixes * don't push them * `# noqa: F841` --------- Co-authored-by: amyeroberts <[email protected]> commit 372baec Author: Vallepu Vamsi Krishna <[email protected]> Date: Fri May 31 21:53:11 2024 +0530 Added description of quantization_config (huggingface#31133) * Description of quantization_config Added missing description about quantization_config in replace_with_bnb_linear for better readability. * Removed trailing spaces commit cdc8131 Author: Pavel Iakubovskii <[email protected]> Date: Fri May 31 16:56:17 2024 +0100 Instance segmentation examples (huggingface#31084) * Initial setup * Metrics * Overfit on two batches * Train 40 epochs * Memory leak debugging * Trainer fine-tuning * Draft * Fixup * Trained end-to-end * Add requirements * Rewrite evaluator * nits * Add readme * Add instance-segmentation to the table * Support void masks * Remove sh * Update docs * Add pytorch test * Add accelerate test * Update examples/pytorch/instance-segmentation/README.md * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation_no_trainer.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation_no_trainer.py * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py * Fix consistency oneformer * Fix imports * Fix imports sort * Apply suggestions from code review Co-authored-by: NielsRogge <[email protected]> * Update examples/pytorch/instance-segmentation/run_instance_segmentation.py Co-authored-by: Sangbum Daniel Choi <[email protected]> * Add resources to docs * Update examples/pytorch/instance-segmentation/README.md Co-authored-by: amyeroberts <[email protected]> * Update examples/pytorch/instance-segmentation/README.md Co-authored-by: amyeroberts <[email protected]> * Remove explicit model_type argument * Fix tests * Update readme * Note about other models --------- Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sangbum Daniel Choi <[email protected]> Co-authored-by: amyeroberts <[email protected]> commit 9837a25 Author: Aymeric Roucher <[email protected]> Date: Fri May 31 14:16:23 2024 +0200 Add streaming, various fixes (huggingface#30838) * Implement streaming run in ReAct agents * Allow additional imports in code agents * Python interpreter: support classes and exceptions, fixes commit f8e6ba4 Author: Marc Sun <[email protected]> Date: Fri May 31 12:44:20 2024 +0200 [trainer] add sanity evaluation option (huggingface#31146) * add sanity evaluation * fix * Apply suggestions from code review Co-authored-by: Zach Mueller <[email protected]> * fix --------- Co-authored-by: Zach Mueller <[email protected]> commit fc5d3e1 Author: Younes Belkada <[email protected]> Date: Fri May 31 12:36:46 2024 +0200 Quantization: Enhance bnb error message (huggingface#31160) enhance error message commit bd9d1dd Author: Asif Ajrof <[email protected]> Date: Fri May 31 16:34:29 2024 +0600 Update sam.md (huggingface#31130) `mask` variable is not defined. probably a writing mistake. it should be `segmentation_map`. `segmentation_map` should be a `1` channel image rather than `RGB`. [on a different note, the `mask_url` is the same as `raw_image`. could provide a better example. commit 48cada8 Author: Marc Sun <[email protected]> Date: Fri May 31 12:08:55 2024 +0200 Fix quantized cache output (huggingface#31143) commit d19566e Author: Yih-Dar <[email protected]> Date: Fri May 31 10:35:54 2024 +0200 pytest -rsfE (huggingface#31140) Co-authored-by: ydshieh <[email protected]> commit f3f640d Author: Arthur <[email protected]> Date: Fri May 31 08:49:33 2024 +0200 helper (huggingface#31152) * helper * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * updates * more doc --------- Co-authored-by: amyeroberts <[email protected]> commit 6bd511a Author: Younes Belkada <[email protected]> Date: Thu May 30 17:21:10 2024 +0200 Workflow: Remove `IS_GITHUB_CI` (huggingface#31147) remove `IS_GITHUB_CI` commit f5590de Author: Younes Belkada <[email protected]> Date: Thu May 30 16:47:35 2024 +0200 Docs / Quantization: Replace all occurences of `load_in_8bit` with bnb config (huggingface#31136) Replace all occurences of `load_in_8bit` with bnb config commit cda9c82 Author: zspo <[email protected]> Date: Thu May 30 22:25:43 2024 +0800 fix get_scheduler when name is warmup_stable_decay (huggingface#31128) fix get_scheduler args commit 5e5c4d6 Author: Younes Belkada <[email protected]> Date: Thu May 30 11:45:03 2024 +0200 FIX / Quantization: Add extra validation for bnb config (huggingface#31135) add validation for bnb config commit 2b9e252 Author: Yih-Dar <[email protected]> Date: Wed May 29 19:43:51 2024 +0200 Cleanup docker build (huggingface#31119) * remove * build --------- Co-authored-by: ydshieh <[email protected]> commit 5c88253 Author: Dhruv Pai <[email protected]> Date: Wed May 29 07:20:59 2024 -0700 Add on_optimizer_step to callback options (huggingface#31095) * Modified test * Added on_optimizer_step to callbacks * Move callback after step is called * Added on optimizer step callback commit 4af705c Author: Joao Gante <[email protected]> Date: Wed May 29 15:17:14 2024 +0100 Add VLM generation default contributor (huggingface#31115) * add Raushan * add Raushan commit cb879c5 Author: Younes Belkada <[email protected]> Date: Wed May 29 15:56:28 2024 +0200 FIX / Docs: Fix GPTQ expected number of bits (huggingface#31111) Update overview.md commit 1f84141 Author: Yih-Dar <[email protected]> Date: Wed May 29 15:42:39 2024 +0200 Fix nightly circleci (huggingface#31114) * fix * fix --------- Co-authored-by: ydshieh <[email protected]> commit d16053c Author: Zach Mueller <[email protected]> Date: Wed May 29 09:35:37 2024 -0400 Rm maintainer + migrate (huggingface#31089) commit 0bef4a2 Author: Matt <[email protected]> Date: Wed May 29 13:33:26 2024 +0100 Fix faulty rstrip in module loading (huggingface#31108) commit 97a58a5 Author: Matt <[email protected]> Date: Wed May 29 13:20:36 2024 +0100 Fix env.py in cases where torch is not present (huggingface#31113) * Fix env.py in cases where torch is not present * Simplify the fix (and avoid some issues) commit c886137 Author: Huazhong Ji <[email protected]> Date: Wed May 29 18:57:54 2024 +0800 Improve `transformers-cli env` reporting (huggingface#31003) * Improve `transformers-cli env` reporting * move the line `"Using GPU in script?": "<fill in>"` to in if conditional statement * same option for npu commit c3044ec Author: Lucain <[email protected]> Date: Wed May 29 12:55:43 2024 +0200 Use `HF_HUB_OFFLINE` + fix has_file in offline mode (huggingface#31016) * Fix has_file in offline mode * harmonize env variable for offline mode * Switch to HF_HUB_OFFLINE * fix test * revert test_offline to test TRANSFORMERS_OFFLINE * Add new offline test * merge conflicts * docs commit bfe6f51 Author: Younes Belkada <[email protected]> Date: Wed May 29 11:43:54 2024 +0200 FEAT: Add mistral v3 conversion script (huggingface#30981) * add mistral v3 conversion script * Update src/transformers/models/mistral/convert_mistral_weights_to_hf.py Co-authored-by: Arthur <[email protected]> * fixup --------- Co-authored-by: Arthur <[email protected]> commit d521ba5 Author: Raushan Turganbay <[email protected]> Date: Wed May 29 14:25:44 2024 +0500 Quantized KV cache: update quanto (huggingface#31052) * quanto latest version was refactored * add error msg * incorrect compare sign * Update src/transformers/cache_utils.py Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]> commit a564d10 Author: amyeroberts <[email protected]> Date: Tue May 28 18:07:07 2024 +0100 Deprecate low use models (huggingface#30781) * Deprecate models - graphormer - time_series_transformer - xlm_prophetnet - qdqbert - nat - ernie_m - tvlt - nezha - mega - jukebox - vit_hybrid - x_clip - deta - speech_to_text_2 - efficientformer - realm - gptsan_japanese * Fix up * Fix speech2text2 imports * Make sure message isn't indented * Fix docstrings * Correctly map for deprecated models from model_type * Uncomment out * Add back time series transformer and x-clip * Import fix and fix-up * Fix up with updated ruff commit 7f08817 Author: Younes Belkada <[email protected]> Date: Tue May 28 18:29:22 2024 +0200 Docs / Quantization: Redirect deleted page (huggingface#31063) Update _redirects.yml commit 3264be4 Author: Younes Belkada <[email protected]> Date: Tue May 28 18:29:11 2024 +0200 TST: Fix instruct-blip tests (huggingface#31088) * fix flan t5 tests * better format commit 476890e Author: Jonny Li <[email protected]> Date: Tue May 28 12:25:15 2024 -0400 Fix DeepSpeed compatibility with weight_norm (huggingface#30881) (huggingface#31018) commit aada568 Author: Albert Villanova del Moral <[email protected]> Date: Tue May 28 17:47:35 2024 +0200 Fix PretrainedConfig docstring with deprecated resume_download (huggingface#31014) commit 3af7bf3 Author: Yih-Dar <[email protected]> Date: Tue May 28 17:44:52 2024 +0200 skip `test_multi_gpu_data_parallel_forward` for `vit` and `deit` (huggingface#31086) fix Co-authored-by: ydshieh <[email protected]> commit ab19f90 Author: Younes Belkada <[email protected]> Date: Tue May 28 17:06:00 2024 +0200 FIX / OPT: Fix OPT multi-GPU training for `OPTForQuestionAnswering` (huggingface#31092) Update modeling_opt.py commit 94d416f Author: Younes Belkada <[email protected]> Date: Tue May 28 17:05:44 2024 +0200 FIX: Add `accelerate` as a hard requirement (huggingface#31090) add accelerate commit 22dab24 Author: Sigbjørn Skjæret <[email protected]> Date: Tue May 28 16:02:51 2024 +0200 Render chat template tojson filter as unicode (huggingface#31041) * Render chat template tojson filter as unicode * ruff-- commit 4f98b14 Author: Younes Belkada <[email protected]> Date: Tue May 28 15:04:43 2024 +0200 Docs / PEFT: Add PEFT API documentation (huggingface#31078) * add peft references * add peft references * Update docs/source/en/peft.md * Update docs/source/en/peft.md commit 779bc36 Author: Raushan Turganbay <[email protected]> Date: Tue May 28 17:07:42 2024 +0500 Watermark: fix tests (huggingface#30961) * fix tests * style * Update tests/generation/test_utils.py Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]> commit a3c7b59 Author: Lysandre Debut <[email protected]> Date: Tue May 28 13:34:23 2024 +0200 Fix failing tokenizer tests (huggingface#31083) * Fix failing tokenizer tests * Use small tokenizer * Fix remaining reference commit 90da0b1 Author: NielsRogge <[email protected]> Date: Tue May 28 13:22:06 2024 +0200 [SuperPoint, PaliGemma] Update docs (huggingface#31025) * Update docs * Add PaliGemma resources * Address comment * Update docs commit 66add16 Author: Sina Taslimi <[email protected]> Date: Tue May 28 13:09:32 2024 +0200 Fix typo in trainer.py (huggingface#31048) commit 98e2d48 Author: Pavel Iakubovskii <[email protected]> Date: Tue May 28 11:06:06 2024 +0000 Fix OWLv2 post_process_object_detection for multiple images (huggingface#31082) * Add test for multiple images * [run slow] owlv2 * Fix box rescaling * [run slow] owlv2 commit c31473e Author: Pavel Iakubovskii <[email protected]> Date: Tue May 28 10:41:40 2024 +0000 Remove float64 cast for OwlVit and OwlV2 to support MPS device (huggingface#31071) Remove float64 commit 936ab7b Author: oOraph <[email protected]> Date: Tue May 28 11:56:05 2024 +0200 fix from_pretrained in offline mode when model is preloaded in cache (huggingface#31010) * Unit test to verify fix Signed-off-by: Raphael Glon <[email protected]> * fix from_pretrained in offline mode when model is preloaded in cache Signed-off-by: Raphael Glon <[email protected]> * minor: fmt Signed-off-by: Raphael Glon <[email protected]> --------- Signed-off-by: Raphael Glon <[email protected]> Co-authored-by: Raphael Glon <[email protected]> commit 537deb7 Author: Hengwen Tong <[email protected]> Date: Tue May 28 17:52:47 2024 +0800 Remove redundant backend checks in training_args.py (huggingface#30999) * Remove backend checks in training_args.py * Expilicit initialize the device --------- Co-authored-by: tonghengwen <[email protected]> commit dd4654e Author: AP <[email protected]> Date: Tue May 28 09:50:45 2024 +0000 Update quicktour.md to fix broken link to Glossary (huggingface#31072) Update quicktour.md to fix broken link Missing '/' in attention mask link in the transformers quicktour commit e18da4e Author: Clint Adams <[email protected]> Date: Tue May 28 05:48:23 2024 -0400 fix "piano" typo (huggingface#31027) commit 8e3b1fe Author: Yih-Dar <[email protected]> Date: Tue May 28 11:36:26 2024 +0200 Remove `ninja` from docker image build (huggingface#31080) fix Co-authored-by: ydshieh <[email protected]> commit 8f0f727 Author: Yih-Dar <[email protected]> Date: Tue May 28 10:53:28 2024 +0200 use `@main` (huggingface#31065) use main Co-authored-by: ydshieh <[email protected]> commit 9d35edb Author: Yih-Dar <[email protected]> Date: Mon May 27 18:36:39 2024 +0200 skip `test_model_parallelism` for 2 model test classes (huggingface#31067) skip Co-authored-by: ydshieh <[email protected]> commit d355741 Author: Yoach Lacombe <[email protected]> Date: Mon May 27 16:09:05 2024 +0200 Fix pad_to_max_length Whisper (huggingface#30787) * fix pad_to_max_length Whisper * add tests * make style commit b84cd67 Author: Marc Sun <[email protected]> Date: Mon May 27 15:53:45 2024 +0200 Fix quanto tests (huggingface#31062) fix quanto tests commit cd79777 Author: amyeroberts <[email protected]> Date: Mon May 27 14:16:47 2024 +0100 Update feature request label in template (huggingface#30940) commit 0a064dc Author: Eitan Turok <[email protected]> Date: Mon May 27 08:57:43 2024 -0400 Follow up: Fix link in dbrx.md (huggingface#30514) * Fix link in dbrx.md * remove "though this may not be up to date" --------- Co-authored-by: Lysandre Debut <[email protected]> commit d7942d9 Author: Yih-Dar <[email protected]> Date: Mon May 27 13:47:47 2024 +0200 unpin uv (huggingface#31055) [push-ci-image] Co-authored-by: ydshieh <[email protected]> commit 84c4b72 Author: Aymeric Roucher <[email protected]> Date: Mon May 27 10:34:14 2024 +0200 Redirect transformers_agents doc to agents (huggingface#31054) commit bdb9106 Author: Pablo Montalvo <[email protected]> Date: Fri May 24 19:02:55 2024 +0200 Paligemma- fix devices and dtype assignments (huggingface#31008) * fix devices and dtype assignments * [run-slow]paligemma commit deba765 Author: Ita Zaporozhets <[email protected]> Date: Fri May 24 17:38:58 2024 +0200 Add split special tokens (huggingface#30772) * seems like `split_special_tokens` is used here * split special token * add new line at end of file * moving split special token test to common tests * added assertions * test * fixup * add co-author * passing rest of args to gptsan_japanese, fixing tests * removing direct comparison of fast and slow models * adding test support for UDOP and LayoutXLM * ruff fix * readd check if slow tokenizer * modify test to handle bos tokens * removing commented function * trigger build * applying review feedback - updated docstrings, var names, and simplified tests * ruff fixes * Update tests/test_tokenization_common.py Co-authored-by: Arthur <[email protected]> * applying feedback, comments * shutil temp directory fix --------- Co-authored-by: Arthur Zucker <[email protected]> Co-authored-by: Ita Zaporozhets <[email protected]> Co-authored-by: itazap <[email protected]> Co-authored-by: Arthur <[email protected]> Co-authored-by: Ita Zaporozhets <[email protected]> commit e5103a7 Author: BHUVAN M <[email protected]> Date: Fri May 24 20:50:09 2024 +0530 added interpolation for vitmae model in pytorch as well as tf. (huggingface#30732) * added interpolation for vitmae model in pytorch as well as tf. * Update modeling_vit_mae.py irreugalr import fixed * small changes and proper formatting * changes suggested in review. * modified decoder interpolate_func * arguments and docstring fix * Apply suggestions from code review doc fixes Co-authored-by: amyeroberts <[email protected]> --------- Co-authored-by: amyeroberts <[email protected]>
zucchini-nlp
pushed a commit
to zucchini-nlp/transformers
that referenced
this pull request
Jun 11, 2024
* helper * Apply suggestions from code review Co-authored-by: amyeroberts <[email protected]> * updates * more doc --------- Co-authored-by: amyeroberts <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add patch helper