Expected behavior of armi check-inputs

[drewj-usnctech]

tl;dr

1. How should clients expect to use this entry point?
2. Is a non-zero exit code expected, even for a valid reactor and settings file?

Body

From the armi -l command list

  check-input           Check ARMI inputs for errors, inconsistencies, and the
                        ability to initialize a reactor. Also has functionality
                        to generate a summary report of the input design. This
                        can be run on multiple cases and creates a table
                        detailing the results of the input check.

This feels like it could be an awesome feature to have in some repository that contains ARMI inputs files (e.g., FFTF model) as part of some continuous integration check. If someone wants to push changes to some baselined ARMI input file, armi check-inputs could be run along side a more technical review.

I ran this command on the most recent FFTF model terrapower/fftf-isothermal-model@793fd8e with 47d3cc7
and got the following output

% armi check-input FFTF.yaml
Failed to import psutil; MemoryProfiler will not provide meaningful data.

                              ---------------------------------------------------
                             |             _      ____     __  __    ___         |
                             |            / \    |  _ \   |  \/  |  |_ _|        |
                             |           / _ \   | |_) |  | |\/| |   | |         |
                             |          / ___ \  |  _ <   | |  | |   | |         |
                             |         /_/   \_\ |_| \_\  |_|  |_|  |___|        |
                             |         Advanced  Reactor  Modeling Interface     |
                              ---------------------------------------------------
                                                   Version 0.1.7
Finding potential settings files matching ['*.yaml'].
Checking for valid settings files.
skipping FFTF-blueprints.yaml
           Attempted to load an invalid settings file from: FFTF-blueprints.yaml. Attempted to load an invalid settings file from: FFTF-blueprints.yaml. Missing the `settings:` header required in YAML settings
skipping FFTF-coremap.yaml
           Attempted to load an invalid settings file from: FFTF-coremap.yaml. Attempted to load an invalid settings file from: FFTF-coremap.yaml. Missing the `settings:` header required in YAML settings
-----------------------------------------------------  ------  -----------  --------------------------
                                                              case    can start    input is self consistent
       -----------------------------------------------------  ------  -----------  --------------------------
       /<path data>/FFTF.yaml                                 FFTF    UNKNOWN      PASSED
       -----------------------------------------------------  ------  -----------  --------------------------

Some things jumped out to me

1. Even though I passed a settings file, ARMI still tried to check all *.yaml files. If I passed FFTF.yaml FFTF.yaml as the settings and patterns arguments, then only the FFTF.yaml file was processed
2. The program exited with exit code 255. For CI applications, a non-zero exit code would indicate failure. Is this related to the fact that this ARMI version doesn't know if the FFTF model can start?

I can create an operator using armi.init(fName="FFTF.yaml") which feels like a hackproxy for this subcommand.

[john-science]

Huh. I haven't looked at armi check-input before, so that does seem strange. @ntouran Have you used this feature in a while?

[jakehader]

Hi @john-science and @drewj-usnctech - I made this entry point awhile ago as a way for the user to quickly run a command on a settings file and get a report out if there are any initialization issues with the model, including reporting any warnings related to inconsistent settings (e.g., using the settings validators) or other warnings with loading the blueprints.

It's likely that this is untested at this point, but I agree that we should use an existing case to test this functionality under continuous integration.

Could you make a bug report for the issue with it searching for all .YAML files even when a specific settings file is given? This is not intended behavior.

[john-science]

It seems to me like this entry point should be very unit testable, right?

Like, we feed a bunch of different settings into it and get pass/fail to check it in a few ways.

If so, that seems like something I could handle for us.

Thoughts?

[jakehader]

Yeah I guess in theory in can be unit tested. Basically this uses the case/suite methods so maybe this can just be better tested by implementing case tests. I am curious if tests can be made that actually call the function through a command line?

[drewj-usnctech]

@jakehader @john-science thanks for the feedback! I posted the file discovery bug in #514