Command line interface¶
The pytest command line shall be executed from the directory that contains the inputs root directory.
Plugin options¶
-
--runner
PATH
¶ use the shell script at PATH to run executable, if omitted then executable is not run.
This shell script may contain placeholders, such as {{nproc}} and {{output_path}}. The placeholders will be replaced with the parameters determined from the context (either a pytest option or a setting defined in a test case via the test_case.yaml), and a final script is saved for each test cases to be run in their output directories, under the name
run_executable.sh
. This latter is used to run executable.A typical script for running executable with MPI could be:
#! /usr/bin/env bash env=/path/to/env/settings exe=/path/to/executable source $env mpirun -np {{nproc}} \ $exe \ --options \ 1> executable.stdout \ 2> executable.stderr
-
--output-root
PATH
¶ use PATH as the root for the output directory tree, default: tests-output
-
--overwrite-output
¶
overwrite existing files in the output directories
-
--clean-output
¶
clean the output directories before executing the tests
-
--regression-root
PATH
¶ use PATH as the root directory with the references for the regression testing, if omitted then the tests using the regression_path fixture will be skipped
-
--default-settings
PATH
¶ use PATH as the yaml file with the global default test settings instead of the built-in ones
-
--report-generator
PATH
¶ use PATH as the script to generate the test report
See the
report-conf
directory for an example of such a script.Note
The report generator script may require to install additionnal dependencies, such as sphinx, which are not required by the plugin.
Standard pytest options¶
You can get all the standard command line options of pytest by executing pytest -h. In particular, to run only some of the test cases in the inputs tree, or to execute only some of the test functions, you may use one of the following ways:
- Use multiple path patterns
Instead of providing the path to the root of the inputs tree, you may provide the path to one or more of its sub-directories, for instance:
pytest --runner <path/to/runner> <path/to/tests/inputs/sub-directory1> <path/to/tests/inputs/sub/sub/sub-directory2>
You may also use shell patterns (with * and ? characters) in the paths like:
pytest --runner <path/to/runner> <path/to/tests/inputs/*/sub-directory?>
- Use marks
A test case could be assigned one or more marks in the
test_case.yaml
file, then with-m
only the test cases that match a given mark expression will be run. A mark expression is a logical expression that combines marks and yields a truth value. For example, to run only the tests that have the mark1 mark but not the mark2 mark, use-m "mark1 and not mark2"
. The logical operator or could be used as well.- Use substring expression
Like the marks, any part (substring) of the name of a test case or of a test function can be used to filter what will be executed. For instance to only execute the tests that have the string transition anywhere in their name, use
-k "transition"
. Or, to execute only the functions that have runner in their names, use-k "runner"
. Logical expressions could be used to combine more susbtrings as well.- Process last failed tests only
To only execute the tests that previously failed, use
--last-failed
.- Show the markers
Use
--markers
to show the available markers without executing the tests.- Show the tests to be executed
Use
--collect-only
to show the test cases and the test events (functions) selected without executing them. You may combine this option with other options, like the one above to filter the test cases.