Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit testing to janis-bioin for evaluating tools #26

Open
illusional opened this issue Jul 9, 2020 · 1 comment
Open

Add unit testing to janis-bioin for evaluating tools #26

illusional opened this issue Jul 9, 2020 · 1 comment

Comments

@illusional
Copy link
Member

Placeholder issue for future task to build some CI tests for the janis-bioinf / unix repos that grade each tool on criteria like:

  • friendly name
  • attribution information
  • Existing container -> Or provided justification if no biocontainers
  • Tools continually compile to valid CWL and WDL (check with CWLTool / miniwdl)

And should fail if any tool doesn't pass the tests. Reduces the burden on testing, and would provide immediate feedback to the contributor on what needs to be done.

This will better pave the way for our deeper CI plans

@illusional
Copy link
Member Author

Per further conversations, some goals:

  • Compare the "usage guide" (eg: toolname -h) between tool versions as a basic: parameters might have changed.
  • To start, make suggestions that tool (wrapper) author's should include test data and a test.

As the author(s) of Janis, we should:

  • Publish expected tool standards for new wrappers, similar to Galaxy's planemo and related docs.
  • Cleanup and document our dev toolkit, for things like producing tool definitions from help guide.
  • Build the mechanisms to make this card possible and automatic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant