-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skips some tests that are currently causing problems for internal users #266
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SDK tests run in 3 min 22 seconds on my local machine after these changes. I've made a ticket to organize the tests better so we can call only a certain subset of tests based on who is making the call.
@@ -6,6 +6,7 @@ | |||
from groundlight_openapi_client.exceptions import NotFoundException | |||
|
|||
|
|||
@pytest.mark.skip(reason="This is an expensive test, reset may take some time") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is likely the source of much of our current SDK test time
@@ -25,6 +26,7 @@ def test_reset_retry(gl_experimental: ExperimentalApi): | |||
raise Exception("Failed to reset detector") | |||
|
|||
|
|||
@pytest.mark.skip(reason="This test does not work with strong 0 shot models, enabled by default based on your account") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This no longer works for internal users because GLAVA is so confident
@@ -56,6 +56,12 @@ def test_counting_detector(gl_experimental: ExperimentalApi): | |||
assert count_iq.result.count is not None | |||
|
|||
|
|||
@pytest.mark.skip( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The SDK got ahead of reef, and what we've enabled for internal users, so this test is actually causing issues during development
…/python-sdk into sdk_tests_for_internal_users
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks Brandon!
No description provided.