Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BUG]: Chat with your documents example exhibits flaky retrieval (#1203)
## Description of changes *Summarize the changes made by this PR.* In #1115 @BChip noticed flaky retrieval performance. The issue was difficult to replicate because of nondeterminism inherent in the HNSW graph construction on loading, but I was able to track it down through repeated testing. The issue is caused by ingesting all the empty lines in the document, which make up 50% of all the lines in each file, which outputs the same embedding for all of them, causing the HNSW graph to sometimes be degenerate. The fix is to skip the empty lines. We should consider how we can mitigate this in the future since this is not easy to detect after the fact, and is likely to be something users run into. ## Test plan Failures no longer occur after manual invocation. ## Documentation Changes N/A
- Loading branch information