Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV2-5640 only send responses to check api when we have a fully realized response from presto during sync requests #470

Merged
merged 1 commit into from
Nov 7, 2024

Conversation

DGaffney
Copy link
Contributor

@DGaffney DGaffney commented Nov 7, 2024

Description

Updates to prevent cases where we send multiple partial matches back to check-api before we've run all vectorizations. The pop that we are removing is theorized as the culprit due to several factors:

  1. The pop removes the model, which, on response, makes it appear that there are fewer items waiting to be processed,
  2. The pop is only on sync requests, while the nearly identical async requests are using get and appear to be working correctly.

Reference: CV2-5640

How has this been tested?

Not yet tested locally

Have you considered secure coding practices when writing this code?

None

…ed response from presto during sync requests
Copy link

sentry-io bot commented Nov 7, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: app/main/lib/elastic_crud.py

Function Unhandled Issue
get_blocked_presto_response KeyError: 'openai-text-embedding-ada-002' api.sim...
Event Count: 2

Did you find this useful? React with a 👍 or 👎

@DGaffney DGaffney marked this pull request as ready for review November 7, 2024 15:38
@@ -54,7 +54,7 @@ def get_blocked_presto_response(task, model, modality):
callback_url = Presto.add_item_callback_url(app.config['ALEGRE_HOST'], modality)
if requires_encoding(obj):
blocked_results = []
for model_key in obj.pop("models", []):
for model_key in obj.get("models", []):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the problem that we are destructively modifying the 'obj' object when the "models" list is removed? It seems like it shouldn't make a difference since it isn't referenced later, but .get seems like better syntax.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when we send these requests out, with the pop in place, it "tells" the alegre callback that is expecting fewer models than it really should be - by enforcing the get we (a) bring it in line with async and (b) probably eliminate erroneous calls back to check-api when alegre mistakenly things work is complete for every in-process vector

Copy link
Contributor

@skyemeedan skyemeedan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems fine to me, but I don't really have full context. Since this is kind of core functionality for presto I think we should have test coverage for it if it is breaking?

@DGaffney DGaffney merged commit 2ffe0c8 into develop Nov 7, 2024
4 checks passed
@DGaffney DGaffney deleted the cv2-5640-suppress-multiple-responses branch November 7, 2024 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants