Read-only mirrors. Development, issues and pull requests are on GitHub.

summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authorMartin Hořeňovský2022-04-25 21:43:10 +0200
committerMartin Hořeňovský2022-04-25 21:45:50 +0200
commit9dc229693d7035b2a09057d578a69d97afc8e572 (patch)
treecb0723c64c3cbf113300d97384a6baee97345539
parentdb57a4956f3b35262ffe91a929013a1bc238050f (diff)
downloadcatch3-9dc229693d7035b2a09057d578a69d97afc8e572.tar.gz
Document the stability guarantees of '--order rand' option
-rw-r--r--docs/command-line.md8
1 files changed, 7 insertions, 1 deletions
diff --git a/docs/command-line.md b/docs/command-line.md
index a0bb67fd..2e3db85b 100644
--- a/docs/command-line.md
+++ b/docs/command-line.md
@@ -334,13 +334,19 @@ Lexicographic order. Tests are sorted by their name, their tags are ignored.
### rand
-Randomly sorted. The order is dependent on Catch2's random seed (see
+Randomly ordered. The order is dependent on Catch2's random seed (see
[`--rng-seed`](#rng-seed)), and is subset invariant. What this means
is that as long as the random seed is fixed, running only some tests
(e.g. via tag) does not change their relative order.
> The subset stability was introduced in Catch2 v2.12.0
+Since the random order was made subset stable, we promise that given
+the same random seed, the order of test cases will be the same across
+different platforms, as long as the tests were compiled against identical
+version of Catch2. We reserve the right to change the relative order
+of tests cases between Catch2 versions, but it is unlikely to happen often.
+
<a id="rng-seed"></a>
## Specify a seed for the Random Number Generator