Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skips some tests that are currently causing problems for internal users #266

Merged
merged 5 commits into from
Oct 16, 2024

Conversation

brandon-groundlight
Copy link
Collaborator

No description provided.

Copy link
Collaborator Author

@brandon-groundlight brandon-groundlight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK tests run in 3 min 22 seconds on my local machine after these changes. I've made a ticket to organize the tests better so we can call only a certain subset of tests based on who is making the call.

@@ -6,6 +6,7 @@
from groundlight_openapi_client.exceptions import NotFoundException


@pytest.mark.skip(reason="This is an expensive test, reset may take some time")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely the source of much of our current SDK test time

@@ -25,6 +26,7 @@ def test_reset_retry(gl_experimental: ExperimentalApi):
raise Exception("Failed to reset detector")


@pytest.mark.skip(reason="This test does not work with strong 0 shot models, enabled by default based on your account")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This no longer works for internal users because GLAVA is so confident

@@ -56,6 +56,12 @@ def test_counting_detector(gl_experimental: ExperimentalApi):
assert count_iq.result.count is not None


@pytest.mark.skip(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SDK got ahead of reef, and what we've enabled for internal users, so this test is actually causing issues during development

Copy link
Member

@tyler-romero tyler-romero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks Brandon!

@brandon-groundlight brandon-groundlight merged commit 1f825a2 into main Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants