Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for TESTARGS in invoke test #62

Open
kjaymiller opened this issue Jul 31, 2024 · 3 comments
Open

allow for TESTARGS in invoke test #62

kjaymiller opened this issue Jul 31, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@kjaymiller
Copy link
Contributor

Version: 6.0.0b1

Platform: MacOS Arm M2

Description: Description of your issue, stack traces from errors and code that reproduces the issue

Currently the invoke test command pre-configures some commands but pushes you down a few paths:

  1. Test all the tests in standalone
  2. Test all the tests in a cluster
  3. Test all the tests in both

I propose that we add the following to all the tests functions in task.py either:

  1. add a TESTARGS parameters that allows for the user to apply their own args to the test
  2. add individual parameters for things like test_path, -v (verbosity), -k (keyword)
@ahmedsobeh ahmedsobeh added the enhancement New feature or request label Sep 21, 2024
@mkmkme mkmkme self-assigned this Sep 23, 2024
@mkmkme
Copy link
Collaborator

mkmkme commented Sep 23, 2024

I started looking at this and noticed the following:

invoke functions do support **kwargs but this optional argument doesn't act the way you'd expect it to. The reason for this is that you are able to pass multiple tasks for invoke:

❯ invoke  -h tests
Usage: inv[oke] [--core-opts] tests [--options] [other tasks here ...]

Adding **kwargs to your test actually adds just another argument called --kwargs=STRING:

❯ invoke  -h tests
Usage: inv[oke] [--core-opts] tests [--options] [other tasks here ...]

Docstring:
  Run the valkey-py test suite against the current python,
  with and without libvalkey.

Options:
  -c, --color
  -k STRING, --kwargs=STRING
  -p INT, --protocol=INT
  -u, --uvloop

Oddly enough, **kwargs don't behave the very expected way. If you put --kwargs="a b --foo --bar", you'll get:

kwargs={"kwargs": "a b --foo --bar"}

So for some reason it will be wrapped with a dict!
It gets funnier when you decide to add *args into it!

❯ invoke  tests  s --kwargs="-v -k"
Starting Valkey tests
kwargs: {'args': 's', 'kwargs': '-v -k'}
args: ()

So, even though you added *args, those args were appended to kwargs and args are empty!


My proposal in this scenario would be to add --pytest-args argument that would be forwarded directly to pytest. And the user will be able to just invoke tests --pytest-args="-v tests/test_commands.py"

One downside of this approach is that it can clash with --color option we already have. But I'd leave it to user's discretion.

Cc @kjaymiller @aiven-sal @ahmedsobeh. WDYT?

@kjaymiller
Copy link
Contributor Author

@mkmkme I think that could work, would that change also be available in standalone_tests and cluster_tests?

Most of the testing that I want to do is for smaller tests while implementing.

@mkmkme
Copy link
Collaborator

mkmkme commented Sep 23, 2024

@kjaymiller yes it will.

I'll create a PR with --pytest-args shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants