diff --git a/_config.yml b/_config.yml deleted file mode 100644 index a7af03880..000000000 --- a/_config.yml +++ /dev/null @@ -1,2 +0,0 @@ -theme: jekyll-theme-leap-day -title: MTT Python Client diff --git a/docs/README.md b/docs/README.md index 752fa18cf..79e5d38ac 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,224 +1,65 @@ -What is this software? ----------------------- - -This is the MPI Testing Tool (MTT) software package. It is a -standalone tool for testing the correctness and performance of -arbitrary MPI implementations. - -The MTT is an attempt to create a single tool to download and build a -variety of different MPI implementations, and then compile and run any -number of test suites against each of the MPI installations, storing -the results in a back-end database that then becomes available for -historical data mining. The test suites can be for both correctness -and performance analysis (e.g., tests such as nightly snapshot compile -results as well as the latency of MPI_SEND can be historically -archived with this tool). - -The MTT provides the glue to obtain and install MPI installations -(e.g., download and compile/build source distributions such as nightly -snapshots, or copy/install binary distributions, or utilize an -already-existing MPI installation), and then obtain, compile, and run -the tests. Results of each phase are submitted to a centralized -PostgresSQL database via HTTP/HTTPS. Simply put, MTT is a common -infrastructure that can be distributed to many different sites in -order to run a common set of tests against a group of MPI -implementations that all feed into a common PostgresSQL database of -results. - -The MTT client is written almost entirely in perl; the MTT server side -is written almost entirely in PHP and relies on a back-end PostgresSQL -database. - -The main (loose) requirements that we had for the MTT are: - -- Use a back-end database / archival system. -- Ability to obtain arbitrary MPI implementations from a variety of - sources (web/FTP download, filesystem copy, Subversion export, - etc.). -- Ability to install the obtained MPI implementations, regardless of - whether they are source or binary distributions. For source - distributions, include the ability to compile each MPI - implementation in a variety of different ways (e.g., with different - compilers and/or compile flags). -- Ability to obtain arbitrary test suites from a variety of sources - (web/FTP download, filesystem copy, Subversion export, etc.). -- Ability to build each of the obtained test suites against each of - the MPI implementation installations (e.g., for source MPI - distributions, there may be more than one installation). -- Ability to run each of the built test suites in a variety of - different ways (e.g, with a set of different run-time options). -- Ability to record the output from each of the steps above and - submit securely them to a centralized database. -- Ability to run the entire test process in a completely automated - fashion (e.g., via cron). -- Ability to run each of the steps above on physically different - machines. For example, some sites may require running the - obtain/download steps on machines that have general internet access, - running the compile/install steps on dedicated compile servers, - running the MPI tests on dedicated parallel resources, and then - running the final submit steps on machines that have general - internet access. -- Use a component-based system (i.e., plugins) for the above steps so - that extending the system to download (for example) a new MPI - implementation is simply a matter of writing a new module with a - well-defined interface. - - -How to cite this software -------------------------- -Hursey J., Mallove E., Squyres J.M., Lumsdaine A. (2007) An Extensible -Framework for Distributed Testing of MPI Implementations. In Recent -Advances in Parallel Virtual Machine and Message Passing Interface. -EuroPVM/MPI 2007. Lecture Notes in Computer Science, vol 4757. Springer, -Berlin, Heidelberg. -https://doi.org/10.1007/978-3-540-75416-9_15 - - -Overview --------- - -The MTT divides its execution into six phases: - -1. MPI get: obtain MPI software package(s) (e.g., download, copy) -2. MPI install: install the MPI software package(s) obtained in phase 1. - This may involve a binary installation or a build from source. -3. Test get: obtain MPI test(s) -4. Test build: build the test(s) against all MPI installations - installed in phase 2. -5. Test run: run all the tests build in phase 4. -6. Report: report the results of phases 2, 4, and 5. - -The phases are divided in order to allow a multiplicative effect. For -example, each MPI package obtained in phase 1 may be installed in -multiple different ways in phase 2. Tests that are built in phase 4 -may be run multiple different ways in phase 5. And so on. - -This multiplicative effect allows testing many different code paths -through MPI even with a small number of actual tests. For example, -the Open MPI Project uses the MTT for nightly regression testing. -Even with only several hundred MPI test source codes, Open MPI is -tested against a variety of different compilers, networks, number of -processes, and other run-time tunable options. A typical night of -testing yields around 150,000 Open MPI tests. - - -Quick start ------------ - -Testers run the MTT client on their systems to do all the work. A -configuration file is used to specify which MPI implementations to use -and which tests to run. - -The Open MPI Project uses MTT for nightly regression testing. A -sample Perl client configuration file is included in -samples/perl/ompi-core-template.ini. This template will require -customization for each site's specific requirements. It is also -suitable as an example for organizations outside of the Open MPI -Project. - -Open MPI members should visit the MTT wiki for instructions on how to -setup for nightly regression testing: - - https://github.com/open-mpi/mtt/wiki/OMPITesting - -The MTT client requires a few perl packages to be installed locally, -such as LWP::UserAgent. Currently, the best way to determine if you -have all the required packages is simply to try running the client and -see if it fails due to any missing packages. - -Note that the INI file can be used to specify web proxies if -necessary. See comments in the ompi-core-template.ini file for -details. - - -Running the MTT Perl client ---------------------------- - -Having run the MTT client across several organizations within the Open -MPI Project for quite a while, we have learned that even with common -goals (such as Open MPI nightly regression testing), MTT tends to get -used quite differently at each site where it is used. The -command-line client was designed to allow a high degree of flexibility -for site-specific requirements. - -The MTT client has many command line options; see the following for a -full list: - -$ client/mtt --help - -Some sites add an upper layer of logic/scripting above the invocation -of the MTT client. For example, some sites run the MTT on -SLURM-maintained clusters. A variety of compilers are tested, -yielding multiple unique (MPI get, MPI install, Test get, Test build) -tuples. Each tuple is run in its own 1-node SLURM allocation, -allowing the many installations/builds to run in parallel. When the -install/build tuple has completed, more SLURM jobs are queued for each -desired number of nodes/processes to test. These jobs all execute in -parallel (pending resource availability) in order to achieve maximum -utilization of the testing cluster. +# What is this software? +This is the Middleware Testing Tool (MTT) software package. It is a standalone tool for testing the correctness and performance of arbitrary MPI implementations. -Other scenarios are also possible; the above is simply one way to use -the MTT. +This website focuses on documenting the Python Client. For more documentation on the Perl Client, please refer to the [Wiki Pages](https://github.com/open-mpi/mtt/wiki/MTTOverview). +MTT is a single tool created to download and build a variety of different middleware implementations, and then compile and run any number of test suites against each of the installations, storing the results in a back-end database that then becomes available for +historical data mining. The test suites can be for both correctness and performance analysis (e.g., tests such as nightly snapshot compile results as well as the latency of MPI_SEND can be historically archived with this tool). -Current status --------------- +MTT provides the glue to obtain and install middleware installations (e.g., download and compile/build source distributions such as nightly snapshots, or copy/install binary distributions, or utilize an already-existing middleware installation), and then obtain, compile, and run the tests. -This tool was initially developed by the Open MPI team for nightly and -periodic compile and regression testing. However, enough other -parties have expressed [significant] interest that we have open-sourced -the tool and are eagerly accepting input from others. Indeed, having -a common tool to help objectively evaluate MPI implementations may be -an enormous help to the High Performance Computing (HPC) community at -large. +Results of each phase are submitted to a centralized PostgresSQL database via HTTP/HTTPS. Simply put, MTT is a common infrastructure that can be distributed to many different sites in +order to run a common set of tests against a group of Middleware implementations that all feed into a common PostgresSQL database of results. -We have no illusions of MTT becoming the be-all/end-all tool for -testing software -- we do want to keep it somewhat focused on the -needs and requires of testing MPI implementations. As such, the usage -flow is somewhat structured towards that bias. +# Overview -It should be noted that the software has been mostly developed internally -to the Open MPI project and will likely experience some growing pains -while adjusting to a larger community. +MTT is divided into multiple phases of execution to split up grabbing content, building content, running content, and reporting results (please refer to the [INI documentation](/mtt/docs/ini_docs.html) to learn more). +The phases are divided to allow a multiplicative effect. For example, each middleware package obtained may be installed in multiple different ways. The built tests may be executed in multiple different ways. And so on. -License -------- +Phases are effectively templated to allow multiple executions of each phase based on parameterization. For example, you can specify a single middleware implementation, but have MTT compile it against both the GNU and Intel compilers. MTT will automatically track that there is one middleware source, but two installations of it. Every test suite that is specified will therefore be compiled and run against _both_ middleware installations, and their results filed accordingly. Hence, the MTT gives a multiplicitive effect. A simplistic view: +- M middleware implementations are specified +- I installations of each middleware implementation are specified +- A total of (M * I) installations are created (assuming all are successful) +- T test suites are specified, each of which is compiled against the (M * I) middleware installation +- R different run parameters are specified for each test suite +- A total of (T * R * M * I) tests are run. -Because we want MTT to be a valuable resource to the entire HPC -community, the MTT uses the new BSD license -- see the LICENSE file in -the MTT distribution for details. +Hence, you must be careful not to specify too much work to MTT -- it will happily do all of it, but it may take a long, long time! +*Note:* MTT takes care of all PATH and LD_LIBRARY_PATH issues when building and installing both middleware implementations and test suites. There is no need for the user to setup anything special in their shell startup files. -Get involved ------------- +The following graphic is a decent representation of the relationships of the phases to each other, and the general sequence of phases. It shows two example middleware implementations (open MPI and MPICH), but any middleware implementation could be used (even multiple versions of the same middleware implementation): -We *want* your feedback. We *want* you to get involved. +![](/mtt/assets/images/mtt-functional.png) -The main web site for the MTT is: +# Quick start +Testers run the MTT client on their systems to do all the work. A configuration file is used to specify which middleware implementations to use and which tests to run. - http://www.open-mpi.org/projects/mtt/ +The Open MPI Project uses MTT for nightly regression testing. A sample Python client configuration file is included in samples/python/ompi_hello_world.ini. It is also suitable as an example for organizations outside of the Open MPI Project. -User-level questions and comments should generally be sent to the -user's mailing list (mtt-users@open-mpi.org). Because of spam, only -subscribers are allowed to post to this list (ensure that you -subscribe with and post from *exactly* the same e-mail address -- -joe@example.com is considered different than -joe@mycomputer.example.com!). Visit this page to subscribe to the -user's list: +# Nightly Regression Testing +Open MPI members should visit the [MTT Wiki](https://github.com/open-mpi/mtt/wiki/OMPITesting) for instructions on how to setup for nightly regression testing. - https://lists.open-mpi.org/mailman/listinfo/mtt-users +To configure nightly testing with Travis CI, please refer to the [Travis CI documentation](/mtt/pages/travis.html). -Developer-level bug reports, questions, and comments should generally -be sent to the developer's mailing list (mtt-devel@open-mpi.org). -Please do not post the same question to both lists. As with the -user's list, only subscribers are allowed to post to the developer's -list. Visit the following web page to subscribe: +# Running the MTT Python client - https://lists.open-mpi.org/mailman/listinfo/mtt-devel - http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel +Having run the MTT client across several organizations within the Open MPI Project for quite a while, we have learned that even with common goals (such as Open MPI nightly regression testing), MTT tends to get used quite differently at each site where it is used. The +command-line client was designed to allow a high degree of flexibility for site-specific requirements. -When submitting bug reports to either list, be sure to include as much -extra information as possible. +The MTT client has many command line options; try the following command to see the full list of options: + +```mtt/pyclient/pymtt.py --help``` + +Some sites add an upper layer of logic/scripting above the invocation of the MTT client. For example, some sites run the MTT on SLURM-maintained clusters. A variety of compilers are tested, yielding multiple unique (MiddlewareGet, MiddlewareBuild, TestGet, TestBuild) tuples. Each tuple is run in its own 1-node SLURM allocation, allowing the many installations/builds to run in parallel. When the install/build tuple has completed, more SLURM jobs are queued for each desired number of nodes/processes to test. These jobs all execute in parallel (pending resource availability) in order to achieve maximum utilization of the testing cluster. + +Other scenarios are also possible; the above is simply one way to use MTT. + +# How to cite this software + +Hursey J., Mallove E., Squyres J.M., Lumsdaine A. (2007) An Extensible Framework for Distributed Testing of MPI Implementations. In Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2007. Lecture Notes in Computer Science, vol 4757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75416-9_15 + +# License +Because we want MTT to be a valuable resource to the entire HPC community, the MTT uses the new BSD license -- see the LICENSE file in the MTT distribution for details. -Thanks for your time. diff --git a/docs/_config.yml b/docs/_config.yml index e82298cdd..7e25c1eb7 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -1,4 +1,4 @@ title: MTT -description: MPI Testing Tool +description: Middleware Testing Tool show_downloads: true theme: jekyll-theme-leap-day diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html index d09290491..26cecc934 100644 --- a/docs/_layouts/default.html +++ b/docs/_layouts/default.html @@ -14,59 +14,115 @@ - - +

{{ site.title | default: site.github.repository_name }}

{{ site.description | default: site.github.project_tagline }}

+ +
+ +
+ + -
- -
- {{ content }} + Community + View on GitHub +
+--> + - - - +
+ +
+ {{ content }} +
+ + +
- {% if site.google_analytics %} - - {% endif %} + {% if site.google_analytics %} + + {% endif %} diff --git a/docs/assets/css/style.scss b/docs/assets/css/style.scss index 9805a65bd..c2f9c18fb 100644 --- a/docs/assets/css/style.scss +++ b/docs/assets/css/style.scss @@ -1,12 +1,14 @@ --- --- +@import "{{ site.theme }}"; + body { font:14px/22px 'Quattrocento Sans', "Helvetica Neue", Helvetica, Arial, sans-serif; color:#666; font-weight:300; margin: 0px; padding:0px 0 20px 0px; - background: url(../images/body-background.png) #eae6d1; + background: #fff; } h1, h2, h3, h4, h5, h6 { @@ -85,9 +87,10 @@ code { color:#efefef; text-shadow: 0px 1px 0px #000; margin: 0 4px; - padding: 2px 6px; + padding: 5px 10px; background: #333; border-radius: 2px; + } pre { @@ -141,15 +144,15 @@ dt { } header { - padding: 25px 20px 40px 20px; - margin: 0; - position: fixed; + padding: 40px 20px 40px 20px; + margin: 40px 0px 0px 0px; + position: relative; top: 0; left:0; right:0; width: 100%; text-align: center; - background: url(../images/background.png) #4276b6; + background: #286282; box-shadow: 1px 0px 2px rgba(0,0,0,.75); z-index:99; -webkit-font-smoothing:antialiased; @@ -157,8 +160,8 @@ header { h1 { font: 40px/48px 'Copse', "Helvetica Neue", Helvetica, Arial, sans-serif; - color: #f3f3f3; - text-shadow: 0px 2px 0px #235796; + color: #fff; + text-shadow: 0px 2px 0px #17384a; margin: 0px; white-space: nowrap; overflow: hidden; @@ -167,144 +170,147 @@ header { -ms-text-overflow: ellipsis; } - p { - color: #d8d8d8; + p{ + color: #fff; text-shadow:rgba(#000, 0.2) 0 1px 0; font-size: 18px; margin: 0px; } } -#banner { - z-index: 100; - left:0; - right:50%; - height: 50px; - margin-right:-382px; +.navbar { position: fixed; - top: 115px; - background: #ffcc00; - border: 1px solid #f0b500; - box-shadow: 0px 1px 3px rgba(0,0,0,.25); - border-radius: 0px 2px 2px 0px; - padding-right: 10px; - - .button { - border: 1px solid #dba500; - /* To change button color, overwrite this line in other button classes*/ - background: linear-gradient(rgb(255, 231, 136), rgb(255, 206, 56)); - border-radius: 2px; - box-shadow: inset 0px 1px 0px rgba(255,255,255,.4), 0px 1px 1px rgba(0,0,0,.1); - background-color: #FFE788; - margin-left: 5px; - padding: 10px 12px; - margin-top: 6px; - line-height:14px; - font-size:14px; - color:#333; - font-weight: bold; - display:inline-block; - text-align:center; + z-index: 100; + top: 0; + width: 100%; + overflow: hidden; + background-color: #fff; + padding: 0; + box-shadow: 1px 0px 5px rgba(0,0,0,.75); +} - &:hover { - background: linear-gradient(rgb(255, 231, 136), rgb(255, 231, 136)); - background-color: #ffeca0; - } +.navbar ul { + list-style: none; + text-align: center; + margin: 0; + width: 100%; + padding: 0; + max-height: 40px; + + li { + display: inline-block; + font-size: 20px; + color: white; + padding: 0px 10px; + text-decoration: none; } +} - .fork { - float:left; - left:50%; - margin-left:0px; - padding: 10px 12px; - margin-top: 6px; - line-height:14px; - font-size:14px; - background-color: #FFE788; - border: 1px solid #dba500; - } +.navbar a { + float: left; + font-size: 20px; + color: #333; + text-align: center; + padding: 10px; + text-decoration: none; - .codeview { - float:left; - left:50%; - margin-left:155px; - padding: 10px 12px; - margin-top: 6px; - line-height:14px; - font-size:14px; - background-color: #FFE788; - border: 1px solid #dba500; + &:hover { + background-color: #F2973C; } +} - .doxygen { - float:left; - left:50%; - margin-left:10px; - padding: 10px 12px; - margin-top: 6px; - line-height:14px; - font-size:14px; - background-color: #FFE788; - } +.dropdown { + float: left; + overflow: hidden; - .codeinfo { - position:fixed; - left:50%; - margin-left:-340px; + &:hover { + background-color: #ddd; } +} - .downloads { - float: right; - margin:0 45px 0 0; - - span { - float:left; - line-height:52px; - font-size:90%; - color:#9d7f0d; - text-transform:uppercase; - text-shadow:rgba(#fff, 0.2) 0 1px 0; - } +.dropdown button { + float: left; + font-size: 20px; + border: none; + outline: none; + color: #333; + padding: 10px 10px 10px 10px; + background-color: inherit; + font-family: inherit; + margin: 0; + + &:hover { + background-color: #F2973C; } +} - ul { - list-style:none; - height:40px; - padding:0; - float: left; - margin-left:10px; +.dropdown-content { + display: none; + position: absolute; + background-color: #333; + z-index: 102; +} - li { - display:inline; +.dropdown-content a { + background-color: #333; + color: white; + padding: 60px 10px 60px 10px; + text-decoration: none; + display: block; + text-align: center; - a.button { - background-color: #FFE788; - } - } + &:hover { + background-color: #666; + } +} + +.dropdown-content .block { + background-color: inherit; + color: white; + text-decoration: none; + text-align: center; + display: block; + float: left; + min-width: 200px; + padding: 10px 0px; + + span { + // text-decoration: underline; + color: #7acdf6; } - #logo { - position:absolute; - height: 36px; - width: 36px; - right:7px; - top:7px; + a { + background-color: inherit; + color: white; display: block; - background: url(../images/octocat-logo.svg); + float: none; + padding: 10px; + font-size: 16px; + + &:hover { + color: #53bbf0; + } } } +.dropdown:hover .dropdown-content { + position: fixed; + display: block; + margin: 40px 0px; + box-shadow: 0px 1px 3px rgba(0,0,0,.75); +} + section { width:590px; padding: 30px 30px 50px 30px; margin: 20px 0; - margin-top: 190px; + margin-top: 20px; position:relative; - background: #fbfbfb; + background: #fff; border-radius: 3px; - border: 1px solid #cbcbcb; - box-shadow: 0px 1px 2px rgba(0,0,0,.09), inset 0px 0px 2px 2px rgba(255,255,255,.5), inset 0 0 5px 5px rgba(255,255,255,.4); + border: none; + box-shadow: none; } small { @@ -314,10 +320,13 @@ small { nav { width: 230px; position: fixed; - top: 220px; + top: 245px; left:50%; margin-left:-580px; text-align: right; + background: #fff; + max-height: 300px; + overflow-y: auto; ul { list-style: none; @@ -399,29 +408,9 @@ footer { } } - #banner { - width: 100%; - - .downloads { - margin-right: 60px; - } - - //.fork { - //} - - .codeinfo { - } - - #logo { - margin-right: 15px; - } - } - section { - border:1px solid #e5e5e5; - border-width:1px 0; - padding:20px auto; - margin: 190px auto 20px; + padding:20px; + margin: 20px auto 20px; max-width: 600px; } @@ -441,7 +430,7 @@ footer { } header { - padding:20px 20px; + padding:80px 20px 0px 20px; margin: 0; h1 { @@ -456,28 +445,8 @@ footer { p { display: none;} } - #banner { - top: 80px; - - .codeinfo { - display: inline-block; - margin-left: 0px; - position:fixed; - left:20px; - } - - /*.fork { - float: left; - display: inline-block; - margin-left: 0px; - // position:fixed; - left:20px; - }*/ - - } - section { - margin-top: 130px; + margin-top: 30px; margin-bottom: 0px; width: auto; } @@ -491,9 +460,8 @@ footer { body { } - header{ - position: relative; - padding: 5px 0px; + header { + padding: 10px 0px, 0px, 0px; min-height: 0px; h1 { @@ -515,6 +483,16 @@ footer { } } +@media print, screen and (max-width: 480px) { + body { + } + + header { + padding: 40px 20px 10px 20px; + min-height: 0px; + } +} + @media print { body { padding:0.4in; @@ -540,6 +518,10 @@ footer { display: none; } + header { + + } + footer { text-align: center; margin: 20px auto; @@ -550,6 +532,3 @@ footer { } } -@import "{{ site.theme }}"; - - diff --git a/docs/assets/images/ini_struct.png b/docs/assets/images/ini_struct.png new file mode 100644 index 000000000..c2de35ade Binary files /dev/null and b/docs/assets/images/ini_struct.png differ diff --git a/docs/assets/images/mtt-functional.png b/docs/assets/images/mtt-functional.png new file mode 100644 index 000000000..afb9fe34e Binary files /dev/null and b/docs/assets/images/mtt-functional.png differ diff --git a/docs/pages/community.md b/docs/pages/community.md new file mode 100644 index 000000000..4933fcedb --- /dev/null +++ b/docs/pages/community.md @@ -0,0 +1,22 @@ +# Get involved + +We *want* your feedback. We *want* you to get involved. + +The main web site for the MTT is located [here](http://www.open-mpi.org/projects/mtt/). + +**User-level** questions and comments should generally be sent to the +user's mailing list (mtt-users@open-mpi.org). Because of spam, only +subscribers are allowed to post to this list (ensure that you +subscribe with and post from *exactly* the same e-mail address!). Please subscribe to the +[user's list](https://lists.open-mpi.org/mailman/listinfo/mtt-users). + +**Developer-level** bug reports, questions, and comments should generally +be sent to the developer's mailing list (mtt-devel@open-mpi.org). +Please do not post the same question to both lists. As with the +user's list, only subscribers are allowed to post to the [developer's +list](https://lists.open-mpi.org/mailman/listinfo/mtt-devel). Visit the following web page to subscribe: + +When submitting bug reports to either list, be sure to include as much +extra information as possible. + +Thank you for your time. diff --git a/docs/pages/developer_guide.md b/docs/pages/developer_guide.md new file mode 100644 index 000000000..62270fa89 --- /dev/null +++ b/docs/pages/developer_guide.md @@ -0,0 +1 @@ +# COMING SOON diff --git a/docs/pages/doxygen.md b/docs/pages/doxygen.md new file mode 100644 index 000000000..62270fa89 --- /dev/null +++ b/docs/pages/doxygen.md @@ -0,0 +1 @@ +# COMING SOON diff --git a/docs/pages/faq.md b/docs/pages/faq.md new file mode 100644 index 000000000..62270fa89 --- /dev/null +++ b/docs/pages/faq.md @@ -0,0 +1 @@ +# COMING SOON diff --git a/docs/pages/ini_docs.md b/docs/pages/ini_docs.md new file mode 100644 index 000000000..0a1df24b2 --- /dev/null +++ b/docs/pages/ini_docs.md @@ -0,0 +1,68 @@ +INI Files Overview +---------- +MTT uses .ini files as execution scripts. An .ini file is a simple text file with a structure composed of sections, names, and values. + +MTT .ini files includes unique features: + - Sections names with embedded Stage Names + - Stage names are a nifty mechanism for the test harness to execute stages in a specific order and with a specific purpose + +Example: +![](/mtt/assets/images/ini_struct.png) + +Setup of INI files: +--- +The file is split up into sections for each phase plus a global parameters section. Each section is denoted with strings inside brackets and parameters are specified as "key=value" pair. + +Stages +--- +The 12 MTT stages are executed according to an ordering as shown in the table. The lower the ordering, the sooner the stage is executed. + +|Ordering | Stage Name | Description | +| :-----: | :--------------: | ------------------------------------------------------------------------------------------------------------------------------------------ | +| 0 | MTTDefaults | Collect defaults for this test definition. | +| 50 | BIOS | Select BIOS setting to be applied after the next reboot. If user specified setting, scan inventory to check if match is found. | +| 100 | Firmware | Flash specified FW versions; if user specified FW versions, scan inventory to check if matches are found. | +| 200 | Provision | Select an image and reboot target nodes. If user specified an image, scan inventory to check if match is found. | +| 210 | Profile | Collect information on system state. Also details configured system prior to tests being run. | +| 300 | MiddlewareGet | Get requested middleware. (Downloading a git repo is the typical usage model) | +| 400 | MiddlewareBuild | Build and install requested middleware. Middleware usage is optional and will be added to the LD_LIBRARY_PATH for the stage. | +| 450 | TestGet | Get the tests. (Downloading a git repo is the typical usage model) | +| 475 | TestBuild | Compile the tests; user can specify the modules to be loaded prior to tests building and be unloaded after the stage completes. | +| 490 | LauncherDefaults | Collect parameters passed to test launcher; these include job names and job options. | +| 500 | TestRun | Run the tests; tests are selected from the test library by matching tags associated with each test. | +| 600 | Reporter | Report the test results. Test results can be sent to the console, to a text file, to a Junit XML file, and/or to the test result database. | + +Parameter Passing +--- +In general, parameters for each MTT stage are set in the .ini file under the corresponding section. However, there are exceptions: + +### Environment Variables +The MTT parser has a specific syntax to reference environment variables. This is useful when writing modular .ini files that execute with different behavior for a given test cycle. +``` +[TestGet:Environ] +plugin = Environ +KERNEL_RELEASE = ${ENV:KERNEL_VERSION} +``` +In the above example, KERNEL_VERSION is grabbed from the environment and passed to the Environ plugin. + +### Accessing Stage Log +The + +Expectation of Test Content Behavior +--- +Test content source code/binaries can come from a variety of source. The primary requirement will be a binary that can be run on assigned nodes. Binaries can be serial, shared-memory parallel, or distributed-memory parallel codes. + +Test content is expected to produce a per-test summary of its results in its standard process exit status (0 for success; non-zero for failure) which is the primary test result. Additional results include either stdout or stderr. Additional options can be set to tell MTT what the expected successful return code should be if the binary does not produce the standard exit status -- "fail_returncodes" option in SLURM, OpenMPI, and ALPS plugins and "fail_returncode" option in the Shell plugin. + +All test content output to stdout and stderr will be captured as a part of the raw test results and stored in the results database. + +If content does not return non-zero status but does have a simple keyword to search you can easily pass the results to tee and then parse them with sed to issue a non-zero return code: + +``run_mytest.sh | tee results.txt && sed -i '/FAIL/q 1' results.txt`` + +Certain classes of test content such as benchmarks will produce more valuable results as a performance measure and not as a simple pass/fail result. These cases will need in-test self checking support. + +Ok, I Have a Test Script: What Next? +--- +### TestGet, TestRun, & Reporter +Start with the TestGet, TestRun, and Reporter stages. There three stages enable you to copy your test to the scratch area, execute your test, and report back the status of the test. diff --git a/docs/pages/plugins_docs.md b/docs/pages/plugins_docs.md new file mode 100644 index 000000000..d4ab1f53b --- /dev/null +++ b/docs/pages/plugins_docs.md @@ -0,0 +1,177 @@ +# MTT Plugin Overview +To make the tool more modular and easier to generate tests, MTT makes use of the yapsy plugin system for python. Each stage that executes an .ini file makes use of at least one plugin. + +Example: +``` +[TestGet:HelloWorld] +plugin = Copytree +src = /opt/mtt/samples/python/ +``` + +Plugins are divided into categories: +- Stages: Stages of test execution + - MTTDefaults + - BIOS + - Firmware + - Provision + - Profile + - MiddlewareGet + - MiddlewareBuild + - TestGet + - TestBuild + - LauncherDefaults + - TestRun + - Reporter +- System: Plugins essential for MTT to run +- Tools: Plugins required by Stages + - The Tools plugins are split into modules: + - Build + - CNC + - Executor + - Fetch + - Harraser + - Launcher + - Version +- Utilities: Plugins used by the MTT framework + +You can see the hierarchy where the plugins are defined along with their ordering on [Doxygen](/mtt/html/inherits.html). + +### Stages +The stages, as mentioned above, are used for stages. Some stages may contain additional plugins (i.e. [Reporter](https://github.com/open-mpi/mtt/tree/master/pylib/Stages/Reporter)). Each plugin in the Stages category has a set ordering which dictates what order the plugins should run. + +A list of stages and their ordering can be found on [Doxygen](/mtt/html/group__Stages.html) + +### System +These plugins are essential for MTT to run. However, they are not actually plugins although we treat them as such. They simply extend the object class. For explanation purposes, we will continue to refer to them as plugins. +- [LoadClasses.py](https://github.com/open-mpi/mtt/blob/master/pylib/System/LoadClasses.py) +- [TestDef.py](https://github.com/open-mpi/mtt/blob/master/pylib/System/TestDef.py) + +LoadClasses is called by TestDef and loads all plugins by going through the directories and loading plugins to memory. + +TestDef is passed to all other plugins so that they have access to the log, to global plugins (i.e. logger, modcmd, execmd, etc. that are saved to TestDef so they are accessible by other plugins) and to the options. This plugin is important for several reasons: +- It is the center point for setting configurations for MTT +- Sets the configuration for tests +- Logs results +- Loads plugins by called LoadClasses +- Starts execution of tests by called one of the [Executor](https://github.com/open-mpi/mtt/tree/master/pylib/Tools/Executor) plugins. The [combinatorial](https://github.com/open-mpi/mtt/blob/master/pylib/Tools/Executor/combinatorial.py) plugin is currently the default plugin; however, the [sequential](https://github.com/open-mpi/mtt/blob/master/pylib/Tools/Executor/sequential.py) plugin can be set by using the ```--executor=sequential``` flag. +- Will add two hidden sections to ConfigParser (ENV and LOG) where environment variables are stuffed into ENV and log results from other plugins are added to LOG. + - [ConfigParser](https://docs.python.org/3/library/configparser.html) is a python library for parsing INI files + +### Tools +The tool plugins are separated into modules that have no ordering. These plugins are required by the Stage plugins. + +A list of available modules and their modules along with descriptions of what they do are found on [Doxygen](/mtt/html/group__Tools.html). + +### Utilities +Utility plugins are used by the MTT framework. Like the tool plugins, there is no ordering for plugins in the Utilities category. + +A list of Utility plugins can be found on [Doxygen](/mtt/html/group__Utilities.html). + + +# Plugin Ingredients +Each plugin is composed of two files. A yapsy-plugin file and a python file. Both of these are required when you are creating your own plugin so that MTT's plugin manager can identify where the files are located and how to connect them to the plugin framework. + +## Yapsy-Plugin Config File +A yapsy-plugin requires a few key elements: +- Name of the plugin +- Author of the plugin +- Short plugin description +- Plugin version + +## Python file +The python file provides the execution phase of the plugin. There are a few requirements for the python file. + +Also, a key note is that while some plugins will only run in a certain stage, not all plugins have such a requirement (i.e. [Shell.py](https://github.com/open-mpi/mtt/blob/master/pylib/Tools/Build/Shell.py)). + + +### Doxygen Documentation +To make sure the plugin with all its parameters are properly recorded in Doxygen, we require a certain syntax. +``` +# @addtogroup +# @{ +# @addtogroup +# @section +# Description of plugin! +# @param option_name Description of option +# @param another_option Additional params follow same syntax +# @} +``` + +### Imports +To integrate the python file into the plugin framework, there are few required imports. These imports are based on the type of plugin that you are implementing. There are four main types of plugins in MTT. + +- Stage Plugins +```from import * ``` +- System Plugins + - These are not plugins, but technically classes that extend the Python object class. However, they are treated as plugins, hence why they are listed here. +- Tools Plugins +``` from import * ``` +- Utilities Plugins +``` from BaseMTTUtility import * ``` + +### Options +Each plugin has a set of options that you can configure via an INI file that controls their operation. + +These are set with the following syntax: +``` +self.options = {} +self.options['option'] = (None, 'Description') +``` +Options are the only method an INI file has to control the plugin. Options are defined in the ```__init__``` function of the plugin class. + +### Required Functions +**Init Function** +The init function needs to be defined. This is where the INI file will configure the plugin through the option object. +``` +def __init__(self): + # Define options +``` +**Print Name** +Return the name of the function. This function is required by MTT to display the plugin in use in the logs. +``` +def print_name(self): + return "Name of Plugin" +``` +**Activate Function** +This function is **not** necessary, but may be useful when the plugin activates. The following example models the structure of the function. Add in additional functionality as needed. +``` +def activate(self): + if not self.activated: + IPlugin.activate(self) + self.activated = True +``` +**Deactivate Function** +This function is **not** necessary, but may be useful when the plugin deactivates (i.e. acts like a destructor function for cleaning up). The following example models the structure of the function. Add in additional functionality as needed. +``` +def deactivate(self): + if self.activates: + IPlugin.deactivate(self) + self.activated = False +``` + **Execute Function** + This function controls the execution of the plugin. There are a couple key things that this function needs to do. For a complete example, please view [Shell.py](https://github.com/open-mpi/mtt/blob/master/pylib/Tools/Build/Shell.py). + +- Parse Options: +``` +testDef.parseOptions(log, self.options, keyvals, cmds) +``` +- Verify cmds have the correct format and error if they do not. Errors would look like this: +``` +log['status'] = 1 # 0 on success, otherwise non-zero +log['stderr'] = "error that happened" +``` +- Return "location" to log: (this could be inherited from the parent log) +``` +log['location'] = +``` +- Execute plugin functionality. For example, in the Shell plugin, it will execute a shell command. Another thing to note is that ```execmd``` is a utility plugin (executeCmd plugin) which is meant to be called in the context of other plugins and **not** to be called by an INI file. +``` +status,stdout,stderr,time = testDef.execmd.execute(cmds, cmdargs, testDef) +``` +- Send output of plugin to log. Some log information is required to be filled out for other plugins (i.e. Build Stage Plugins **need** to record ```log['compiler']``` and ```log['mpi_info']```). +``` +log['status'] = 0 # if success, otherwise non-zero +log['stdout'] = output of plugin +log['stderr'] = record any errors +log['location'] = location # record location for children plugins +``` diff --git a/docs/pages/resources.md b/docs/pages/resources.md new file mode 100644 index 000000000..29e4c693f --- /dev/null +++ b/docs/pages/resources.md @@ -0,0 +1,5 @@ +Resources +--------- + +To learn more about the yapsy-plugin system, please read more here: +http://yapsy.sourceforge.net/ diff --git a/docs/pages/tips_and_tricks.md b/docs/pages/tips_and_tricks.md new file mode 100644 index 000000000..ce287b195 --- /dev/null +++ b/docs/pages/tips_and_tricks.md @@ -0,0 +1,30 @@ +# INI File Creation +MTT offers a high degree of flexibility by allowing phases to be effectively parameterized allowing for multiple executions of each phase. + +Caution needs to be executed when creating an INI file that will not take days (or weeks!) to complete. Use the ```--print-time``` option to see how long each phase is taking to help tune your INI file. + +# Run Time Execution +We recommend running with `--verbose`. This provides a decent amount of output that confirms that most MTT things are running. However, if you run into weird issues that cannot be explained, run with `--debug`. This will provide a _lot_ of output (you'll want to save both stdout and stderr into a file for later analysis). + +# Compiler Configuration +Some compilers (such as the Intel compilers) require additional environment variables such as PATH and LD_LIBRARY_PATH to be set properly in order to find their relevant parts. It is easiest to simply set these values _before_ invoking the MTT client, but it is also possible to set them within the INI file directly (so that you don't have to worry about local environments). However, the syntax is a little odd, so it's worth describing here: +``` +[MPI Install: Intel compiler] +...all the other fields... +setenv = PATH /path/to/intel/compiler/bin:/usr/bin/:...rest of path +setenv = LD_LIBRARY_PATH /path/to/intel/compiler/lib:...rest of LD path +``` +# MPI Install Errors +Sometimes MTT finds a problem in a middleware install, but a human wants to go examine it manually. When MTT installs a middleware under its scratch tree, it will automatically drop two files -- one for sh-flavored shells and one for csh-flavored shells -- that set the PATH and LD_LIBRARY_PATH to get to the middleware install. The exact location of these files depends on the section names in your INI file and the exact version number of the MPI in question. Specifically, these files will be in: +``` +/installs////mpi_installed_vars.sh +/installs////mpi_installed_vars.csh +``` + +Hence, you can source these files like this (assuming a csh-flavored shell): +``` +shell% cd /installs/// +shell% source mpi_installed_vars.csh +``` + +Mpicc, mpirun, etc. will then be in your PATH, and the appropriate libraries will be in your LD_LIBRARY_PATH. Additionally, the environment variable MPI_ROOT will be set that points to the top-level installation directory for the middlewareBuild. This is useful with Open MPI's ```--prefix``` option to mpirun, for example. Note that _all_ the files related to the testing of that MPI are under this tree -- the source tree, the tests, etc. So you can go examine the entire test -- not just put the MPI in question in your path. diff --git a/docs/pages/travis.md b/docs/pages/travis.md new file mode 100644 index 000000000..62270fa89 --- /dev/null +++ b/docs/pages/travis.md @@ -0,0 +1 @@ +# COMING SOON diff --git a/docs/pages/user_guide.md b/docs/pages/user_guide.md new file mode 100644 index 000000000..62270fa89 --- /dev/null +++ b/docs/pages/user_guide.md @@ -0,0 +1 @@ +# COMING SOON diff --git a/pylib/Stages/BIOS/BIOSMTTStage.py b/pylib/Stages/BIOS/BIOSMTTStage.py index 5b06b349b..65ad73866 100644 --- a/pylib/Stages/BIOS/BIOSMTTStage.py +++ b/pylib/Stages/BIOS/BIOSMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup BIOS -# ordering 50 +# [Ordering 50] BIOS flash and query stage # @} class BIOSMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/Firmware/FirmwareMTTStage.py b/pylib/Stages/Firmware/FirmwareMTTStage.py index 908bfe302..10871ec85 100644 --- a/pylib/Stages/Firmware/FirmwareMTTStage.py +++ b/pylib/Stages/Firmware/FirmwareMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup Firmware -# ordering 100 +# [Ordering 100] Firmware flash and query stage # @} class FirmwareMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/LauncherDefaults/LauncherDefaultsMTTStage.py b/pylib/Stages/LauncherDefaults/LauncherDefaultsMTTStage.py index 1e7c68255..8e1bd8540 100644 --- a/pylib/Stages/LauncherDefaults/LauncherDefaultsMTTStage.py +++ b/pylib/Stages/LauncherDefaults/LauncherDefaultsMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup LauncherDefaults -# ordering 490 +# [Ordering 490] Allow a user to set default values for a specific launcher # @} class LauncherDefaultsMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/MTTDefaults/DefaultMTTDefaults.py b/pylib/Stages/MTTDefaults/DefaultMTTDefaults.py index 6933aa4f4..4fabc48a4 100644 --- a/pylib/Stages/MTTDefaults/DefaultMTTDefaults.py +++ b/pylib/Stages/MTTDefaults/DefaultMTTDefaults.py @@ -17,6 +17,7 @@ # @{ # @addtogroup MTTDefaults # @section DefaultMTTDefaults +# Store any provided default MTT settings # @param trial Use when testing your MTT client setup; results that are generated and submitted to the database are marked as \"trials\" and are not included in normal reporting. # @param scratch Specify the DIRECTORY under which scratch files are to be stored # @param description Provide a brief title/description to be included in the log for this test diff --git a/pylib/Stages/MTTDefaults/MTTDefaultsMTTStage.py b/pylib/Stages/MTTDefaults/MTTDefaultsMTTStage.py index 972fe052c..be9a49463 100644 --- a/pylib/Stages/MTTDefaults/MTTDefaultsMTTStage.py +++ b/pylib/Stages/MTTDefaults/MTTDefaultsMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup MTTDefaults -# ordering 0 +# [Ordering 0] Set MTT defaults for this test definition # @} class MTTDefaultsMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/MiddlewareBuild/MiddlewareBuildMTTStage.py b/pylib/Stages/MiddlewareBuild/MiddlewareBuildMTTStage.py index 722386047..77ba3c727 100644 --- a/pylib/Stages/MiddlewareBuild/MiddlewareBuildMTTStage.py +++ b/pylib/Stages/MiddlewareBuild/MiddlewareBuildMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup MiddlewareBuild -# ordering 400 +# [Ordering 400] Stage for building middleware such as MPI # @} class MiddlewareBuildMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/MiddlewareGet/MiddlewareGetMTTStage.py b/pylib/Stages/MiddlewareGet/MiddlewareGetMTTStage.py index 65d55d29f..6d11318ec 100644 --- a/pylib/Stages/MiddlewareGet/MiddlewareGetMTTStage.py +++ b/pylib/Stages/MiddlewareGet/MiddlewareGetMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup MiddlewareGet -# ordering 300 +# [Ordering 300] Stage for getting middleware source code # @} class MiddlewareGetMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/Profile/DefaultProfile.py b/pylib/Stages/Profile/DefaultProfile.py index aaa0ca9d2..dcba40551 100644 --- a/pylib/Stages/Profile/DefaultProfile.py +++ b/pylib/Stages/Profile/DefaultProfile.py @@ -16,6 +16,7 @@ # @{ # @addtogroup Profile # @section DefaultProfile +# Collect hardware and software profile of the system # @param kernelName Kernel name # @param kernelRelease Kernel release string # @param kernelVersion Kernel version string diff --git a/pylib/Stages/Profile/DefaultProfile.yapsy-plugin b/pylib/Stages/Profile/DefaultProfile.yapsy-plugin index b0709da12..8e92f4bbd 100644 --- a/pylib/Stages/Profile/DefaultProfile.yapsy-plugin +++ b/pylib/Stages/Profile/DefaultProfile.yapsy-plugin @@ -15,4 +15,4 @@ Module = DefaultProfile Author = Ralph Castain Version = 0.1 Website = N/A -Description = Collect hardware ands software profile of the system +Description = Collect hardware and software profile of the system diff --git a/pylib/Stages/Profile/ProfileMTTStage.py b/pylib/Stages/Profile/ProfileMTTStage.py index fb2afcb01..fda4010b3 100644 --- a/pylib/Stages/Profile/ProfileMTTStage.py +++ b/pylib/Stages/Profile/ProfileMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup Profile -# ordering 210 +# [Ordering 210] Stage for profiling the system upon which the tests will be conducted # @} class ProfileMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/Provisioning/ProvisionMTTStage.py b/pylib/Stages/Provisioning/ProvisionMTTStage.py index f196d7542..feee5f2c7 100644 --- a/pylib/Stages/Provisioning/ProvisionMTTStage.py +++ b/pylib/Stages/Provisioning/ProvisionMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup Provision -# ordering 200 +# [Ordering 200] Provisioning stage # @} class ProvisionMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/Provisioning/WWulf3.py b/pylib/Stages/Provisioning/WWulf3.py index 7f46648b1..20e3874f9 100644 --- a/pylib/Stages/Provisioning/WWulf3.py +++ b/pylib/Stages/Provisioning/WWulf3.py @@ -16,6 +16,7 @@ # @{ # @addtogroup Provision # @section WWulf3 +# Plugin for provisioning nodes using the Warewulf v3 image manager # @param target List of remote host names or LAN interfaces to be provisioned # @param image Name of image to be instantiated # @param bootstrap Name of bootstrap to be used diff --git a/pylib/Stages/Reporter/IUDatabase.py b/pylib/Stages/Reporter/IUDatabase.py index b8f92ef43..16c495af0 100755 --- a/pylib/Stages/Reporter/IUDatabase.py +++ b/pylib/Stages/Reporter/IUDatabase.py @@ -28,6 +28,7 @@ # @{ # @addtogroup Reporter # @section IUDatabase +# MTT Database reporter plugin for the legacy IU submission server # @param realm Database name # @param username Username to be used for submitting data # @param password Password for that username diff --git a/pylib/Stages/Reporter/JunitXML.py b/pylib/Stages/Reporter/JunitXML.py index 9564c6f0c..7cba4c187 100644 --- a/pylib/Stages/Reporter/JunitXML.py +++ b/pylib/Stages/Reporter/JunitXML.py @@ -19,6 +19,7 @@ # @{ # @addtogroup Reporter # @section JunitXML +# Junit XML plugin # @param filename Name of the file into which the report is to be written # @param textwrap Max line length before wrapping # @} diff --git a/pylib/Stages/Reporter/ReporterMTTStage.py b/pylib/Stages/Reporter/ReporterMTTStage.py index 7b480a494..380671606 100644 --- a/pylib/Stages/Reporter/ReporterMTTStage.py +++ b/pylib/Stages/Reporter/ReporterMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup Reporter -# ordering 600 +# [Ordering 600] Report tests results stage # @} class ReporterMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/Reporter/TextFile.py b/pylib/Stages/Reporter/TextFile.py index 26da40308..486753460 100755 --- a/pylib/Stages/Reporter/TextFile.py +++ b/pylib/Stages/Reporter/TextFile.py @@ -17,6 +17,7 @@ # @{ # @addtogroup Reporter # @section TextFile +# File reporter plugin # @param filename Name of the file into which the report is to be written # @param summary_footer Footer to be placed at bottom of summary # @param detail_header Header to be put at top of detail report diff --git a/pylib/Stages/TestBuild/DefaultTestBuild.py b/pylib/Stages/TestBuild/DefaultTestBuild.py index 04db93e96..695c52a21 100644 --- a/pylib/Stages/TestBuild/DefaultTestBuild.py +++ b/pylib/Stages/TestBuild/DefaultTestBuild.py @@ -16,6 +16,7 @@ # @{ # @addtogroup TestBuild # @section DefaultTestBuild +# Build tools using Autotools plus an installed middleware version # @param middleware Middleware stage that these tests are to be built against # @param build_in_place Build tests in current location (no prefix or install) # @param merge_stdout_stderr Merge stdout and stderr into one output stream diff --git a/pylib/Stages/TestBuild/TestBuildMTTStage.py b/pylib/Stages/TestBuild/TestBuildMTTStage.py index 888ba401a..c5bb8575a 100644 --- a/pylib/Stages/TestBuild/TestBuildMTTStage.py +++ b/pylib/Stages/TestBuild/TestBuildMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup TestBuild -# ordering 475 +# [Ordering 475] Build test software package # @} class TestBuildMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/TestGet/TestGetMTTStage.py b/pylib/Stages/TestGet/TestGetMTTStage.py index 25fbf58c0..8a56bf673 100644 --- a/pylib/Stages/TestGet/TestGetMTTStage.py +++ b/pylib/Stages/TestGet/TestGetMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup TestGet -# ordering 450 +# [Ordering 450] Get test software package # @} class TestGetMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Stages/TestRun/TestRunMTTStage.py b/pylib/Stages/TestRun/TestRunMTTStage.py index fa3e87518..98ce6abf6 100644 --- a/pylib/Stages/TestRun/TestRunMTTStage.py +++ b/pylib/Stages/TestRun/TestRunMTTStage.py @@ -14,7 +14,7 @@ ## @addtogroup Stages # @{ # @addtogroup TestRun -# ordering 500 +# [Ordering 500] Run test software package # @} class TestRunMTTStage(IPlugin): def __init__(self): diff --git a/pylib/Tools/Build/Autotools.py b/pylib/Tools/Build/Autotools.py index faca720df..dc261236b 100644 --- a/pylib/Tools/Build/Autotools.py +++ b/pylib/Tools/Build/Autotools.py @@ -20,6 +20,7 @@ # @{ # @addtogroup Build # @section Autotools +# Run typical autotools commands to configure and build a software package # @param middleware Middleware stage that these tests are to be built against # @param parent Section that precedes this one in the dependency tree # @param autogen_cmd Command to be executed to setup the configure script, usually called autogen.sh or autogen.pl diff --git a/pylib/Tools/Build/Hostfile.py b/pylib/Tools/Build/Hostfile.py index f80ef61a2..27ff57057 100644 --- a/pylib/Tools/Build/Hostfile.py +++ b/pylib/Tools/Build/Hostfile.py @@ -20,6 +20,7 @@ # @{ # @addtogroup Build # @section Hostfile +# Builds a hostfile based on a nodelist # @param parent Section that precedes this one in the dependency tree # @param nodelist list of nodes to create hostfile from # @param hostfile name of hostfile to generate diff --git a/pylib/Tools/Build/Shell.py b/pylib/Tools/Build/Shell.py index 0916d5acb..81cb08707 100644 --- a/pylib/Tools/Build/Shell.py +++ b/pylib/Tools/Build/Shell.py @@ -18,6 +18,7 @@ # @{ # @addtogroup Build # @section Shell +# Run shell commands to configure and build a software package # @param middleware Middleware stage that these tests are to be built against # @param command Command to execute # @param parent Section that precedes this one in the dependency tree diff --git a/pylib/Tools/CNC/IPMITool.py b/pylib/Tools/CNC/IPMITool.py index aefc8be95..6f4cd0094 100644 --- a/pylib/Tools/CNC/IPMITool.py +++ b/pylib/Tools/CNC/IPMITool.py @@ -133,6 +133,7 @@ def run(self): # @{ # @addtogroup CNC # @section IPMITool +# Interface to the ipmitool cmd line # @param target List of remote host names or LAN interfaces to monitor during reset operations # @param controller List of IP addresses of remote node controllers/BMCs # @param username Remote session username diff --git a/pylib/Tools/Executor/combinatorial.py b/pylib/Tools/Executor/combinatorial.py index d9dfb693f..7904347fd 100644 --- a/pylib/Tools/Executor/combinatorial.py +++ b/pylib/Tools/Executor/combinatorial.py @@ -28,6 +28,7 @@ # @{ # @addtogroup Executor # @section CombinatorialEx +# Combinatorial execution executor # @} class CombinatorialEx(ExecutorMTTTool): diff --git a/pylib/Tools/Executor/sequential.py b/pylib/Tools/Executor/sequential.py index 6684639c4..7ad186b89 100755 --- a/pylib/Tools/Executor/sequential.py +++ b/pylib/Tools/Executor/sequential.py @@ -36,6 +36,7 @@ # @{ # @addtogroup Executor # @section SequentialEx +# Sequential execution executor # @} class SequentialEx(ExecutorMTTTool): diff --git a/pylib/Tools/Fetch/AlreadyInstalled.py b/pylib/Tools/Fetch/AlreadyInstalled.py index be0269a57..6d62a0557 100644 --- a/pylib/Tools/Fetch/AlreadyInstalled.py +++ b/pylib/Tools/Fetch/AlreadyInstalled.py @@ -17,6 +17,7 @@ # @{ # @addtogroup Fetch # @section AlreadyInstalled +# No-op plugin for using existing middleware installation # @param exec Executable that should be in path # @param module Modules (or lmod modules) to be loaded for accessing this package # @} diff --git a/pylib/Tools/Fetch/Git.py b/pylib/Tools/Fetch/Git.py index 9d783991d..c9f761a80 100644 --- a/pylib/Tools/Fetch/Git.py +++ b/pylib/Tools/Fetch/Git.py @@ -20,6 +20,7 @@ # @{ # @addtogroup Fetch # @section Git +# Plugin for getting software via Git # @param module Modules (or lmod modules) to be loaded for accessing this package # @param url URL to access the repository # @param username Username required for accessing the repository diff --git a/pylib/Tools/Fetch/OMPI_Snapshot.py b/pylib/Tools/Fetch/OMPI_Snapshot.py index 881f18cd9..242f21799 100644 --- a/pylib/Tools/Fetch/OMPI_Snapshot.py +++ b/pylib/Tools/Fetch/OMPI_Snapshot.py @@ -24,6 +24,7 @@ # @{ # @addtogroup Fetch # @section OMPI_Snapshot +# Plugin for getting software via OMPI Nightly tarballs # @param url URL to access the OMPI nightly tarball (e.g. https://www.open-mpi.org/nightly/v2.x) # @param version_file optional file containing name of most recent tarball version tested # @param mpi_name optional name for the OMPI snapshot tarball diff --git a/pylib/Tools/Harasser/Harasser.py b/pylib/Tools/Harasser/Harasser.py index 1c9c898a7..8a2b1bdca 100644 --- a/pylib/Tools/Harasser/Harasser.py +++ b/pylib/Tools/Harasser/Harasser.py @@ -20,6 +20,7 @@ # @{ # @addtogroup Harasser # @section Harasser +# Run harasser scripts while test-content is running # @param trigger_scripts Scripts to run to launch harassers # @param stop_scripts Scripts to run to stop and clean-up harassers # @param join_timeout Seconds to wait for process to finish diff --git a/pylib/Tools/Launcher/ALPS.py b/pylib/Tools/Launcher/ALPS.py index ae6ff963c..5cabb919f 100644 --- a/pylib/Tools/Launcher/ALPS.py +++ b/pylib/Tools/Launcher/ALPS.py @@ -17,6 +17,7 @@ # @{ # @addtogroup Launcher # @section ALPS +# Plugin for using ALPS to launch tests # @param hostfile The hostfile for OpenMPI to use # @param command Command for executing the application # @param np Number of processes to run diff --git a/pylib/Tools/Launcher/OpenMPI.py b/pylib/Tools/Launcher/OpenMPI.py index 700472a0b..a3352bb50 100644 --- a/pylib/Tools/Launcher/OpenMPI.py +++ b/pylib/Tools/Launcher/OpenMPI.py @@ -19,6 +19,7 @@ # @{ # @addtogroup Launcher # @section OpenMPI +# Plugin for using the Open MPI mpirun launch tool # @param hostfile The hostfile for OpenMPI to use # @param command Command for executing the application # @param np Number of processes to run diff --git a/pylib/Tools/Launcher/SLURM.py b/pylib/Tools/Launcher/SLURM.py index 13705d450..7d78d81fe 100644 --- a/pylib/Tools/Launcher/SLURM.py +++ b/pylib/Tools/Launcher/SLURM.py @@ -17,6 +17,7 @@ # @{ # @addtogroup Launcher # @section SLURM +# Plugin for using SLURM to launch tests # @param hostfile The hostfile for OpenMPI to use # @param command Command for executing the application # @param np Number of processes to run diff --git a/pylib/Tools/Version/MTTVersionPlugin.py b/pylib/Tools/Version/MTTVersionPlugin.py index 583b318b7..deee3ab5a 100644 --- a/pylib/Tools/Version/MTTVersionPlugin.py +++ b/pylib/Tools/Version/MTTVersionPlugin.py @@ -29,6 +29,7 @@ # @{ # @addtogroup Version # @section MTTVersionPlugin +# A simple plugin for returning the MTT Version # @} class MTTVersionPlugin(VersionMTTTool): diff --git a/pylib/Utilities/Compilers.py b/pylib/Utilities/Compilers.py index 59eaccb0f..df499d737 100644 --- a/pylib/Utilities/Compilers.py +++ b/pylib/Utilities/Compilers.py @@ -15,6 +15,7 @@ ## @addtogroup Utilities # @{ # @section Compilers +# Identify the type and version of compilers in-use # @} class Compilers(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/Copytree.py b/pylib/Utilities/Copytree.py index 2eb65c9c2..6aa156a56 100644 --- a/pylib/Utilities/Copytree.py +++ b/pylib/Utilities/Copytree.py @@ -18,6 +18,7 @@ ## @addtogroup Utilities # @{ # @section Copytree +# Copy a directory tree from source to the same relative loation under the MTT scratch directory # @param src The top directory of the tree to be copied # @param preserve_symlinks Preserve symlinks instead of copying the contents # @param preserve_directory Copies directory instead of contents diff --git a/pylib/Utilities/Environ.py b/pylib/Utilities/Environ.py index 62805b90d..a246566db 100644 --- a/pylib/Utilities/Environ.py +++ b/pylib/Utilities/Environ.py @@ -16,6 +16,7 @@ ## @addtogroup Utilities # @{ # @section Environ +# Set environment variables # @} class Environ(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/ExecuteCmd.py b/pylib/Utilities/ExecuteCmd.py index cc361c9e1..f027dbc01 100644 --- a/pylib/Utilities/ExecuteCmd.py +++ b/pylib/Utilities/ExecuteCmd.py @@ -20,6 +20,7 @@ ## @addtogroup Utilities # @{ # @section ExecuteCmd +# Execute a command and capture its stdout and stderr # @} class ExecuteCmd(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/Logger.py b/pylib/Utilities/Logger.py index 26582ef1a..31960bbd6 100644 --- a/pylib/Utilities/Logger.py +++ b/pylib/Utilities/Logger.py @@ -17,6 +17,7 @@ ## @addtogroup Utilities # @{ # @section Logger +# Log results and provide debug output when directed # @} class Logger(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/MPIVersion.py b/pylib/Utilities/MPIVersion.py index 32e7c2c7c..9dc9b808f 100644 --- a/pylib/Utilities/MPIVersion.py +++ b/pylib/Utilities/MPIVersion.py @@ -16,6 +16,7 @@ ## @addtogroup Utilities # @{ # @section MPIVersion +# Identify the name and version of MPI in-use # @} class MPIVersion(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/ModuleCmd.py b/pylib/Utilities/ModuleCmd.py index 4cb2eae6d..1b7f8a24e 100644 --- a/pylib/Utilities/ModuleCmd.py +++ b/pylib/Utilities/ModuleCmd.py @@ -20,6 +20,7 @@ ## @addtogroup Utilities # @{ # @section ModuleCmd +# Load/Unload an environmental module # @} class ModuleCmd(BaseMTTUtility): def __init__(self): diff --git a/pylib/Utilities/Watchdog.py b/pylib/Utilities/Watchdog.py index d9459d974..12d8f69ed 100644 --- a/pylib/Utilities/Watchdog.py +++ b/pylib/Utilities/Watchdog.py @@ -22,6 +22,7 @@ ## @addtogroup Utilities # @{ # @section Watchdog +# Generate and exception after a given amount of time # @param timeout Time in seconds before generating exception # @} class Watchdog(BaseMTTUtility):