Skip to content

Issues: huggingface/text-generation-inference

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

171 Open 1,244 Closed
171 Open 1,244 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support XGrammar backend as an alternative to Outlines
#2900 opened Jan 10, 2025 by 2016bgeyer
Support reponse_format: {"type": "json_object"} without any constrained schema
#2899 opened Jan 10, 2025 by lhoestq
Automatic Calculation of Sequence Length in TGI v3 Leads to Unrealistic Values Before CUDA OOM
#2897 opened Jan 10, 2025 by biba10
2 of 4 tasks
Prefill operation can be significantly slower in TGI v3 vs TGI v2
#2896 opened Jan 10, 2025 by biba10
2 of 4 tasks
Deepseek v3 support
#2895 opened Jan 9, 2025 by ferrybaltimore
[Qwen/Qwen2.5-14B-Instruct-GPTQ-Int8] Bad Responses with High Concurrent Requests
#2894 opened Jan 9, 2025 by michaelact
4 tasks done
1
make install-server does not have Apple MacOS Metal Framework
#2890 opened Jan 8, 2025 by qdrddr
2 of 4 tasks
summarization using fine-tuned flan-t5 model in TGI outputs "generated text" instead of "summary_text" and outputs are completely different
#2889 opened Jan 7, 2025 by maiiabocharova
2 of 4 tasks
Qwen2-VL failed to infer multiple images (Server error: upper bound and larger bound inconsistent with step sign)
#2888 opened Jan 7, 2025 by AHEADer
2 of 4 tasks
6
Unclear Metrics list
#2887 opened Jan 7, 2025 by vitalyshalumov
2 of 4 tasks
1
Starcoder2-15B model - AttributeError: 'TensorParallelColumnLinear' object has no attribute [rank3]: 'base_layer'
#2881 opened Jan 6, 2025 by ashwincv0112
3 of 4 tasks
3
Deploy Qwen2-VL in inference endpoints failed
#2879 opened Jan 6, 2025 by AHEADer
2 of 4 tasks
1
trunction flag is missing from /v1/chat/completions
#2877 opened Jan 6, 2025 by vitalyshalumov
2 of 4 tasks
1
Support References in Tool Calling Schemas
#2876 opened Jan 3, 2025 by GICodeWarrior
@drbh
os-atlas-pro-7b crashes
#2875 opened Jan 2, 2025 by louisabraham
3 of 4 tasks
3
AttributeError: 'dict' object has no attribute 'attn_impl' 2024-12-31T04:54:31.451596Z ERROR shard-manager: text_generation_launcher: Shard complete standard error output:
#2872 opened Dec 31, 2024 by prachi-mcw
1 of 4 tasks
1
Responses with unusual content.
#2871 opened Dec 30, 2024 by ncthanhcs
1 of 4 tasks
2
Implement logit_bias correctly
#2869 opened Dec 28, 2024 by juanwisz
1
Deepseek-v2
#2865 opened Dec 26, 2024 by ehartford
2 tasks done
1
Tool Calling using Vercel's AI SDK not working as intended
#2864 opened Dec 23, 2024 by kldzj
2 of 4 tasks
2
Dynamically serve LoRA modules
#2860 opened Dec 20, 2024 by rikardradovac
Make version tag selectable on docs page dropdown
#2857 opened Dec 19, 2024 by andrewrreed
text-generation-inference:latest-trtllm is missing dependencies to run models
#2854 opened Dec 18, 2024 by selalipop
2 of 4 tasks
Entire system crashes when get to warm up model
#2853 opened Dec 17, 2024 by ad-astra-video
1 of 4 tasks
1
random text generation from Qwen2-VL-7B-Instruct with TGI3
#2851 opened Dec 17, 2024 by DongyoungKim2
2 of 4 tasks
ProTip! Mix and match filters to narrow down what you’re looking for.