Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(weave): Implement integration with 🤗 inference client #2795

Draft
wants to merge 28 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
84f2da9
add: patching for InferenceClient.chat_completion
soumik12345 Oct 28, 2024
2f1cd81
Merge branch 'master' into feat/huggingface-inference
soumik12345 Oct 28, 2024
0b30e19
fix: lint
soumik12345 Oct 28, 2024
f63bdc4
add: huggingface_accumulator
soumik12345 Oct 28, 2024
190f738
Merge branch 'master' into feat/huggingface-inference
soumik12345 Oct 28, 2024
92ebfd2
Merge branch 'master' into feat/huggingface-inference
soumik12345 Oct 29, 2024
16eb753
add: patching for AsyncInferenceClient
soumik12345 Oct 29, 2024
376fa99
add: patching for document_question_answering and visual_question_ans…
soumik12345 Oct 29, 2024
9104c9e
add: tests
soumik12345 Oct 29, 2024
a11e072
add: huggingface integration unit testing shard
soumik12345 Oct 29, 2024
b3602a0
Merge branch 'master' into feat/huggingface-inference
soumik12345 Oct 29, 2024
d1ce70b
add: patching for fill_mask + tests
soumik12345 Oct 29, 2024
c450f0d
add: patching for fill_mask + tests
soumik12345 Oct 29, 2024
afd1e68
Merge branch 'master' into feat/huggingface-inference
soumik12345 Oct 31, 2024
dbdf332
add: patching for question_answering
soumik12345 Oct 31, 2024
82dd603
add: patching for sentence_similarity
soumik12345 Oct 31, 2024
43c299c
add: patching for summarization
soumik12345 Oct 31, 2024
d7fdf13
add: patching for table_question_answering
soumik12345 Oct 31, 2024
7515d1e
add: patching for text_classification
soumik12345 Oct 31, 2024
29ef8b8
add: patching for token_classification
soumik12345 Oct 31, 2024
02807ca
add: patching for translation
soumik12345 Oct 31, 2024
7826dda
update: tests
soumik12345 Oct 31, 2024
0a98a26
add: patching for zero_shot_classification
soumik12345 Oct 31, 2024
0504ec5
add: patching for text_to_image
soumik12345 Oct 31, 2024
ed4bd50
Merge branch 'master' into feat/huggingface-inference
soumik12345 Nov 5, 2024
8c33c3d
Merge branch 'master' into feat/huggingface-inference
soumik12345 Nov 15, 2024
43c9c95
Merge branch 'master' into feat/huggingface-inference
soumik12345 Nov 25, 2024
6138efd
Merge branch 'master' into feat/huggingface-inference
soumik12345 Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ jobs:
'vertexai',
'scorers_tests',
'pandas-test',
'huggingface',
]
fail-fast: false
services:
Expand Down
1 change: 1 addition & 0 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ def lint(session):
"vertexai",
"scorers_tests",
"pandas-test",
"huggingface",
],
)
def tests(session, shard):
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ notdiamond = ["notdiamond>=0.3.21", "litellm<=1.49.1"]
openai = ["openai>=1.0.0"]
pandas-test = ["pandas>=2.2.3"]
modal = ["modal", "python-dotenv"]
huggingface = ["huggingface-hub>=0.26.2"]
vertexai = ["vertexai>=1.70.0"]
test = [
"nox",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
interactions:
- request:
body: '{"model": "meta-llama/Llama-3.2-11B-Vision-Instruct", "messages": [{"role":
"user", "content": [{"type": "image_url", "image_url": {"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
{"type": "text", "text": "Describe this image in one sentence."}]}], "max_tokens":
500, "seed": 42, "stream": false}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate, br, zstd
Connection:
- keep-alive
Content-Length:
- '351'
Content-Type:
- application/json
X-Amzn-Trace-Id:
- 2f329b59-35c3-4276-9f7a-83609a5dc417
user-agent:
- unknown/None; hf_hub/0.26.2; python/3.9.12; torch/2.4.1
method: POST
uri: https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-11B-Vision-Instruct/v1/chat/completions
response:
body:
string: '{"object":"chat.completion","id":"","created":1730223736,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"message":{"role":"assistant","content":"The
image shows the Statue of Liberty, an iconic monument symbolizing freedom
and democracy in the city of New York, USA."},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":44,"completion_tokens":26,"total_tokens":70}}'
headers:
Connection:
- keep-alive
Content-Length:
- '452'
Content-Type:
- application/json
Date:
- Tue, 29 Oct 2024 17:43:37 GMT
access-control-allow-credentials:
- 'true'
vary:
- Origin, Access-Control-Request-Method, Access-Control-Request-Headers
x-compute-time:
- '2.186847565'
x-compute-type:
- cache
x-request-id:
- qIjxpXjuA51Xjz5ochgWW
x-sha:
- cee5b78e6faed15d5f2e6d8a654fd5b247c0d5ca
status:
code: 200
message: OK
version: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
interactions:
- request:
body: null
headers:
user-agent:
- unknown/None; hf_hub/0.26.2; python/3.9.12; torch/2.4.1
method: POST
uri: https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-11B-Vision-Instruct/v1/chat/completions
response:
body:
string: '{"object":"chat.completion","id":"","created":1730223736,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"message":{"role":"assistant","content":"The
image shows the Statue of Liberty, an iconic monument symbolizing freedom
and democracy in the city of New York, USA."},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":44,"completion_tokens":26,"total_tokens":70}}'
headers:
Access-Control-Allow-Credentials:
- 'true'
Connection:
- keep-alive
Content-Length:
- '452'
Content-Type:
- application/json
Date:
- Tue, 29 Oct 2024 17:51:12 GMT
Vary:
- Origin, Access-Control-Request-Method, Access-Control-Request-Headers
x-compute-time:
- '2.186847565'
x-compute-type:
- cache
x-request-id:
- 6hX8mv6YF99wfUe2DciQt
x-sha:
- cee5b78e6faed15d5f2e6d8a654fd5b247c0d5ca
status:
code: 200
message: OK
version: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
interactions:
- request:
body: '{"model": "meta-llama/Llama-3.2-11B-Vision-Instruct", "messages": [{"role":
"user", "content": [{"type": "image_url", "image_url": {"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
{"type": "text", "text": "Describe this image in one sentence."}]}], "max_tokens":
500, "seed": 42, "stream": true}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate, br, zstd
Connection:
- keep-alive
Content-Length:
- '350'
Content-Type:
- application/json
X-Amzn-Trace-Id:
- 79b38aa4-bde0-41bd-8b8b-aee89d2823c2
user-agent:
- unknown/None; hf_hub/0.26.2; python/3.9.12; torch/2.4.1
method: POST
uri: https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-11B-Vision-Instruct/v1/chat/completions
response:
body:
string: 'data: {"object":"chat.completion.chunk","id":"","created":1730224061,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224061,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
image"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224061,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
showcases"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224061,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
the"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224061,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
iconic"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224062,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
Statue"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224062,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
of"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224062,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
Liberty"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224062,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
in"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224062,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
New"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224063,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
York"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
City"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":","},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
set"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
against"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
the"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
vibrant"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
skyline"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
of"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"
Manhattan"},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}],"usage":null}


data: {"object":"chat.completion.chunk","id":"","created":1730224064,"model":"meta-llama/Llama-3.2-11B-Vision-Instruct","system_fingerprint":"2.3.1-dev0-sha-de90261","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":"stop"}],"usage":null}


data: [DONE]


'
headers:
Connection:
- keep-alive
Content-Length:
- '6376'
Content-Type:
- text/event-stream
Date:
- Tue, 29 Oct 2024 17:48:20 GMT
access-control-allow-credentials:
- 'true'
vary:
- Origin, Access-Control-Request-Method, Access-Control-Request-Headers
x-compute-type:
- cache
x-request-id:
- j58E8aC66Ja4p8kkYMwz5
x-sha:
- cee5b78e6faed15d5f2e6d8a654fd5b247c0d5ca
status:
code: 200
message: OK
version: 1
Loading
Loading