Setup command line based regression tests #4
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
This is something that anyone could help with, and where even very small amounts of your time can be useful (eg writing a single test), and where several people could collaborate.
We have a crude regression test script (test-safedrive.sh) which can be used as a template for what's needed. So the first step would be to implement equivalent functionality to that, and then extend with more tests.
Setup scripts and tests using BATS (Bash Automated Testing System) to automate a set of regression tests that can run against mock and live network.
This should include a series of small commands to exercise SAFE Drive feature (creating files in different ways, listing directories, checking file content, creating public names, mounting containers in different ways etc).
The tests would check both the interaction with the console (eg copy file command shows success, not error), and by linking multiple commands together, can test that the commands also had the desired effect (eg checking that the file was actually copied).
In general good outcomes and bad outcomes should be tested for. So for example both a copy file command that should succeed, and one that should fail.
Tests should be able to run either individually, or in small self contained groups rather than as one very long script, so that re-running a test while checking the behaviour after a code change can be done quickly. So
npm run tests copy-file
etc, as well as...It should also be possible to run all tests from start to finish with a single command (eg
npm run tests all
), and for test output to be visible on the console and logged to a file for reference.Any files required as inputs for the tests should be kept in a
tests/input
sub-directory or directory tree, and any results files stored in their owntests/results
directory.Provide instructions on how to set up and run the tests, either as a scripted command (eg
npm run tests
without parameters) or as as a document that is referred to when the test script is run. This can explain, for example, how to start the browser, whether to login/create account etc before starting the test script.The tests might of course need to cover behaviour with/without browser started, with/without account logged in, so tests with different starting conditions should ideally prompt for confirmation that the conditions are as required before beginning to run the test itself.
The text was updated successfully, but these errors were encountered: