Rerunning failed tests¶
When the recording feature is enabled, you can rerun failing tests with cargo nextest run -R latest. This command will run tests that, in the original run:
- failed;
- did not run because the test run was cancelled; or,
- were not previously seen, typically because they were newly added since the original run.
The rerun feature works purely at the test level, and does not track code or build changes.
Rerun build scope
Without any further arguments, cargo nextest run -R latest will build the same targets that the original run did. If build scope arguments are specified, they will override the set of build targets from the original run.
Build scope arguments include all arguments under the Package selection, Target selection, and Feature selection headings of cargo nextest run --help.
Example rerun flow¶
Let's say that cargo nextest run --package nextest-filtering was run, and it had two failing tests:
Nextest run ID b0b38ba7-3a69-44c7-b52c-c0dabf839c07 with nextest profile: default
Starting 51 tests across 2 binaries
<output snipped>
FAIL [ 0.142s] (50/51) nextest-filtering proptest_helpers::tests::proptest_regex_valid
FAIL [ 0.143s] (51/51) nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.143s] 51 tests run: 49 passed, 2 failed, 0 skipped
Nextest run ID b0b38ba7-3a69-44c7-b52c-c0dabf839c07 with nextest profile: default
Starting 51 tests across 2 binaries
<output snipped>
FAIL [ 0.142s] (50/51) nextest-filtering proptest_helpers::tests::proptest_regex_valid
FAIL [ 0.143s] (51/51) nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.143s] 51 tests run: 49 passed, 2 failed, 0 skipped
With cargo nextest run -R latest proptest_helpers, the first test is selected:
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID e8082b32-05ae-4681-8b1d-84da628b9991 with nextest profile: default
Starting 1 test across 2 binaries (50 tests skipped, including 49 tests that already passed)
PASS [ 0.154s] nextest-filtering proptest_helpers::tests::proptest_regex_valid
────────────
Summary [ 0.155s] 1 test run: 1 passed, 50 skipped
Note 1 outstanding test not seen during this rerun:
nextest-filtering parsing::tests::proptest_expr_roundtrip
warning: 1 outstanding test still remains
(hint: cargo nextest run -R latest to continue rerunning)
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID e8082b32-05ae-4681-8b1d-84da628b9991 with nextest profile: default
Starting 1 test across 2 binaries (50 tests skipped, including 49 tests that already passed)
PASS [ 0.154s] nextest-filtering proptest_helpers::tests::proptest_regex_valid
────────────
Summary [ 0.155s] 1 test run: 1 passed, 50 skipped
Note 1 outstanding test not seen during this rerun:
nextest-filtering parsing::tests::proptest_expr_roundtrip
warning: 1 outstanding test still remains
(hint: cargo nextest run -R latest to continue rerunning)
All selected tests passed, but some outstanding (previously-failing) tests still remain, so nextest exits with the advisory exit code 5 (RERUN_TESTS_OUTSTANDING).
A subsequent cargo nextest run -R latest will run the remaining test:
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID 0b9f340f-1b61-4bdf-9260-54fa7a470ef4 with nextest profile: default
Starting 1 test across 2 binaries (50 tests skipped that already passed)
PASS [ 0.260s] nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.261s] 1 test run: 1 passed, 50 skipped
info: no outstanding tests remain
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID 0b9f340f-1b61-4bdf-9260-54fa7a470ef4 with nextest profile: default
Starting 1 test across 2 binaries (50 tests skipped that already passed)
PASS [ 0.260s] nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.261s] 1 test run: 1 passed, 50 skipped
info: no outstanding tests remain
Exit code for no tests in a rerun
In regular runs, if there are no tests to run, nextest exits with the advisory exit code 4 (NO_TESTS_RUN) by default.
With reruns, if there are no tests to run, nextest exits with exit code 0 by default, indicating success. The difference in behavior is due to the goal of reruns being to converge to a successful test run.
It is possible to rewind the rerun logic to an earlier state by passing in a run ID to -R. In this case b0b forms an unambiguous prefix (highlighted in bold purple), so cargo nextest run -R b0b results in both tests being run:
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID d6a74827-4764-4965-9f1a-5b16417daf28 with nextest profile: default
Starting 2 tests across 2 binaries (49 tests skipped that already passed)
PASS [ 0.153s] nextest-filtering proptest_helpers::tests::proptest_regex_valid
PASS [ 0.277s] nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.278s] 2 tests run: 2 passed, 49 skipped
info: no outstanding tests remain
info: rerun: inheriting build scope from original run: --package nextest-filtering
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.11s
────────────
Nextest run ID d6a74827-4764-4965-9f1a-5b16417daf28 with nextest profile: default
Starting 2 tests across 2 binaries (49 tests skipped that already passed)
PASS [ 0.153s] nextest-filtering proptest_helpers::tests::proptest_regex_valid
PASS [ 0.277s] nextest-filtering parsing::tests::proptest_expr_roundtrip
────────────
Summary [ 0.278s] 2 tests run: 2 passed, 49 skipped
info: no outstanding tests remain
Rerun heuristics¶
Picking the set of tests to run is tricky, particularly in the face of tests being removed and new ones being added. We have attempted to pick a strategy that aims to be conservative while covering the most common use cases, but it is always possible that tests are missed. Because of this, and because code changes might include regressions in previously passing tests, it is recommended that you perform a full test run once your iterations are complete.
As a best practice, it is also recommended that you use CI to gate changes making their way to production, and that you perform full runs in CI.
For more about the heuristics and considerations involved, see the rerun decision table in the design document.
Options and arguments¶
For the full list of options accepted by cargo nextest run, including rerun options, see Options and arguments, or cargo nextest run --help.