Skip to content

Commit

Permalink
Fix GPU Embed Bug (#19)
Browse files Browse the repository at this point in the history
Minor fix from bug caused by
#16. We need to place the
tensors on the proper device. Note: add GPU tests in the future, since
all the tests run on CPU at the moment.
  • Loading branch information
sidjha1 authored Oct 22, 2024
1 parent 1929392 commit b06306b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lotus/models/e5_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def embed(self, docs: List[str], **kwargs: Dict[str, Any]) -> np.ndarray:

# Calculating the embedding dimension
total_docs = len(docs)
first_batch = self.tokenizer(docs[:1], return_tensors="pt", padding=True, truncation=True)
first_batch = self.tokenizer(docs[:1], return_tensors="pt", padding=True, truncation=True).to(self.device)
embed_dim = self.model(**first_batch).last_hidden_state.size(-1)

# Pre-allocate a tensor for all embeddings
Expand Down

0 comments on commit b06306b

Please sign in to comment.