Finding code and viewing test results

Finding code and viewing the test results

Finding code to use

Available code contributions are listed in the code database. This includes information such as whether the code is for DSC- or DCE-MRI, what type of functionality is implemented (e.g. pharmacokinetic models), what sub-type of functionality is implemented (e.g. the Tofts model) and other details.

For example, we can see that a number of contributions implement T1 measurement using the variable flip angle technique. To find the source code for a specific contribution, check the "subfolder" column. We can see that one of these contributions is in the subfolder "ST_USydAus_T1", indicating the author, institution and country. We can find the source code in the following folder of the repository:
src/original/ST_USydAUS/

Viewing tests and results

To check whether a specific piece of code has been tested, refer again to the code database. Using the same example, we can see that testing of the above T1 measurement code, from the University of Sydney, is "complete".

Finding the tests

All tests are located in the test folder of the repository and organised according to functionality. For example, the test module for this example is test/T1_mapping/test_t1_ST_SydneyAus.py By viewing the module, we can see what tests were performed, and see the syntax and parameter units required by the code being tested. Useful comments may have been added to the test file by OSIPI members, e.g. "convert to milliseconds". These should also include details regarding any test cases that are expected to fail, and additional steps taken to make the code pass the tests.

There will normally be an additional python module in the folder (e.g. t1_data.py) to parse the test data, which is stored in the data subfolder.

Viewing the test results with Github Actions

Each time the repository is updated, github automatically runs a continuous integration workflow. This includes running pytest, therefore all scientific tests developed by OSIPI will be run. To view the test results, click on the Actions tab, then selected the latest workflow run for the develop branch. Now click on one of the builds (e.g. ubuntu-latest), then on "Test with pytest". This will show the results for each test that was run. For our example code contribution, we can see the following output:

test/T1_mapping/test_t1_ST_SydneyAus.py ................................ [ 74%]
........................................................................ [ 80%]
........................................................................ [ 86%]
........................................................................ [ 92%]
........................................................................ [ 98%]
................x.....                                                   [100%]

Each dot indicates one test case that passed. The x indicates an expected failure.

CI badge

If all of the tests are passing (or failing as expected), a green "ci: passing" badge will be visible on the front page of the respository.

Visual representation of the test results

To provide end users of the code collection with a visual representation of the test results, a website was created to display the results from the test.

Notebooks have been created per category to generate Bland-Altman plots to show the difference between measured and reference values. In addition, the notebooks contain some background on the technique and the test data being used. The notebooks can be found in the folder notebooks. The notebooks import all relevant test results based on tags saved in a JSON file in the test folder (test/results-meta.json). The JSON contains a list of entries. Each entry contains the path, filename, category, method and author. To incorporate future results, one can just update this JSON file. This is currently done manually.

From these notebooks a website is created to provide an overview of the results. The website is hosted on a different repository (https://github.com/OSIPI/DCE-DSC-MRI_TestResults). The notebooks are automatically pushed to this repository. The folder notebooks in the code repository also contains other markdown files used to generate the webpages and a table of contents (_toc.yml). This file describes the structure of the website. Nothing needs to be manually changed in the TestResults repository when new tests are created. This is all done within the main code collection repository.