Skip to content

Tests

Jump to bottom
nh916 edited this page Sep 7, 2023 · 14 revisions

Software Testing

Description

Testing can be a big help to catch any bugs and for when we are updating code or refactoring to be sure we are not breaking any other previous code by mistake.

Python Test Files

  • All tests can be found in the root directory within the tests/ folder.
  • Tests are grouped and have a similar directory structure as src/cript/ to be easily found.
  • All test files begin with the word test because by default Pytest automatically discovers and runs any file with names that start with test and contain test functions or classes.
  • The CRIPT Python SDK uses Pytest for all of its testing needs.
  • Tests also use fixtures to keep the code clean, not repeat code, and make it easy to update and maintain.
  • Fixtures can be found within conftest.py file
  • Each function should be self-documenting, have an easy to understand name, minimal docstrings telling the programmers what is the aim of this test, and comments if needed to make the test/code clearer
    • Programmers are also encouraged to use type hinting, but it is not required

Example

def test_complex_reference() -> None:
    """
    tests that a complex reference node with all possible optional parameters can be made
    """

    # reference attributes
    reference_type = "journal_article"
    title = "'Living' Polymers"
    authors = ["Dylan J. Walsh", "Bradley D. Olsen"]
    journal = "Nature"
    publisher = "Springer"
    year = 2019
    volume = 3
    issue = 5
    pages = [123, 456, 789]
    doi = "10.1038/1781168a0"
    issn = "1476-4687"
    arxiv_id = "1501"
    pmid = 12345678
    website = "https://criptapp.org"

    # create complex reference node
    my_reference = cript.Reference(
        type=reference_type,
        title=title,
        authors=authors,
        journal=journal,
        publisher=publisher,
        year=year,
        volume=volume,
        issue=issue,
        pages=pages,
        doi=doi,
        issn=issn,
        arxiv_id=arxiv_id,
        pmid=pmid,
        website=website,
    )

    # assertions
    assert isinstance(my_reference, cript.Reference)
    assert my_reference.type == reference_type
    assert my_reference.title == title
    assert my_reference.authors == authors
    assert my_reference.journal == journal
    assert my_reference.publisher == publisher
    assert my_reference.year == year
    assert my_reference.volume == volume
    assert my_reference.issue == issue
    assert my_reference.pages == pages
    assert my_reference.doi == doi
    assert my_reference.issn == issn
    assert my_reference.arxiv_id == arxiv_id
    assert my_reference.pmid == pmid
    assert my_reference.website == website

Simple vs Complex Node Fixtures

Simple Node Fixtures

  • Are minimal nodes with just the required arguments and nothing more, keeping them very simple and easy to use for other tests without getting in the way

Complex Node Fixtures

  • Are maximal nodes with all possible required arguments making them as big as possible to test as much as possible

Example

Simple Reference Node

@pytest.fixture(scope="function")
def simple_reference() -> None:
    """
    simple reference node with only minimal arguments
    """
    my_reference_type = "journal_article"
    my_reference_title = "'Living' Polymers"

    my_reference = cript.Reference(type=my_reference_type, title=my_reference_title)

    return my_reference

Complex Reference Node

@pytest.fixture(scope="function")
def test_complex_reference() -> None:
    """
   complex reference node with all optional parameters
    """

    # reference attributes
    reference_type = "journal_article"
    title = "'Living' Polymers"
    authors = ["Dylan J. Walsh", "Bradley D. Olsen"]
    journal = "Nature"
    publisher = "Springer"
    year = 2019
    volume = 3
    issue = 5
    pages = [123, 456, 789]
    doi = "10.1038/1781168a0"
    issn = "1476-4687"
    arxiv_id = "1501"
    pmid = 12345678
    website = "https://criptapp.org"

    # create complex reference node
    my_reference = cript.Reference(
        type=reference_type,
        title=title,
        authors=authors,
        journal=journal,
        publisher=publisher,
        year=year,
        volume=volume,
        issue=issue,
        pages=pages,
        doi=doi,
        issn=issn,
        arxiv_id=arxiv_id,
        pmid=pmid,
        website=website,
    )

    return my_reference

Testing with Temporary Files

There are many places that to complete a test it requires a file to be uploaded, downloaded, etc. to be sure that the program is working as intended.

For the places that a file is needed it is best to not hardcode some file path or some web path, but instead use a temporary file. Using a temporary file has an advantage that the test will run regardless of what machine the test is on and is operating system platform-independent and can be ran on any operating system without a problem.

Integration Tests with Python SDK and API

The strategy for integration tests are basically to do CRUD operation for every primary node, sub-objects, and supporting nodes. We basically create a node with minimal amounts it needs to be valid, save it to API, get it from API, and check that it is the same as the one posted.

Note: This indirectly tests the cript.API.search(...) method as well when we are searching by EXACT_NAME within the API

For integration tests to run, the user must specify HAS_INTEGRATION_TESTS_ENABLED in conftest.py to be True by either hardcoding it to True, not recommended, but will do in a quick pinch or by specifying the environment variable CRIPT_TESTS to be True within your environment.

This variable was needed to quickly turn ON or OFF integration tests when testing on GitHub workflow CI because it did not have a connection to the API and those tests would always fail, and this variable was much better than remembering to comment and uncomment blocks of tests across multiple files and functions.

more coming soon...


Best Practices

  1. Write clear and descriptive test names: The name of the test should describe what is being tested, and it should be easy to understand.

    • Tests should be written for both what is expected and what is not expected
    • Tests should have clear docstrings describing the test, what its purpose is, and how the test works
    • Tests should have type hinting to keep the code clean and self documenting

  2. Write clean DRY code for test: if you have the same node/object that needs to be used in multiple tests try creating a fixture for it and put it inside of the conftest.py file to keep the code DRY and reference it in multiple tests as needed. This will make the tests cleaner and easier to update.

  3. Use test fixtures: Test fixtures are reusable objects that provide a fixed baseline for testing. They can help reduce code duplication and improve test reliability.

  4. Test edge cases: Test not only the most common input but also test the edge cases to ensure that the function is working as expected in all possible scenarios.

  5. Keep tests independent: Each test should be independent and not rely on the state of any other test.

  6. Use assert statements: Use assert statements to check that the expected output of a function or method matches the actual output.

  7. Use code coverage tools: Use code coverage tools to ensure that all code paths are tested.

  8. Use test runners: Use a test runner to automate the execution of tests.

  9. Write tests first: Follow Test Driven Development (TDD) and write tests before writing the actual implementation code.

  10. Refactor tests: As your code evolves, make sure to refactor your tests to keep them up to date and maintainable.

  11. Test bugs: If you just discovered a new bug that was being undetected in the past tests, then write a test for it. This both proves that the bug was fixed and can run in pipelines for future updates to be sure none of them break the code.

Resources

CRIPT Platform | CRIPT Scripts | CRIPT Python SDK GitHub | CRIPT Python Package PyPI

Clone this wiki locally