Skip to content

Commit

Permalink
Updates from meeting
Browse files Browse the repository at this point in the history
  • Loading branch information
etpeterson committed Nov 27, 2024
1 parent 645ec69 commit 3d4d664
Showing 1 changed file with 40 additions and 6 deletions.
46 changes: 40 additions & 6 deletions tests/README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,55 @@ This document describes the testing done in the OSIPI TF2.4 IVIM repository
-- Testing philosophy --
Testing is integral to the repository.
There are many different contributions from many different people and only through diligent testing can they all be ensured to work correctly together.
Automated testing happens on different platforms and versions.
There are 3 major types of tests we conduct.
1. Requirements
- Which are required to pass
- Runs on each commit
- Must pass to merge
- All algorithms have the same requirements
- E.g. bounds honored, code runs reasonably
- E.g. bounds honored, code runs reasonably, properly integrated
- Would prevent a merge if not passing
- Flexibile input/output
- Categories for testing
-- Contributions - Some aspects currently tested as unit_test
--- Initial bounds are respected
---- Needs implemening
--- Initial guess is respected
---- Needs implemening - may no be possible
--- Runs reasonably
---- Needs implemening: reduced data size, broadened limits
--- Contains information about algorithm
-- Wrapper
--- Initial guess is in bounds
--- Reasonable default bounds - f: [0 1], D >= 0 & D < D*, D* >= 0
--- Input size is respected - result is same size as input
--- Dictionary is returned - worth explicit testing?
-- Phantom - lower priority
--- Data can be pulled
2. Expectations
- Run on each merge
- Considered warnings
- Should not necessarily prevent a merge
- Should cover performance changes
- E.g. performance changes
3. Performance
- The accuracy of the results
- Categories for testing
-- Determine performance changes from reference run
--- Currently implemented but could be made easier to interact with
--- Could be made easier and faster
3. Characterization
- Run on demand
- Performance of the algorithms
- The accuracy and precision of the results
- The speed of the generated results
- Human readable report of the wrapped algorithms
- Categories for testing
-- Simulations
--- Voxels from tissue and characterize algorithms
--- Visualize parameter maps
-- True data
--- Visualize parameter maps
--- Correlations between algorithms - plot the results and differences

-- Testing structure --

* The testing is controlled in several places.
* The testing itself is done with pytest which parses files for "test_" and runs the appropriate tests.
* The pytest testing can be done on your own machine by running "python -m pytest".
Expand Down

0 comments on commit 3d4d664

Please sign in to comment.