Read-only mirrors. Development, issues and pull requests are on GitHub.

summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authorMartin Hořeňovský2022-11-16 11:21:00 +0100
committerMartin Hořeňovský2022-11-16 11:21:00 +0100
commit8beb74da8abd69c9cb083772edc230ea857b1229 (patch)
tree65ff5c70189e54cb439bcb5a5dd1968e71f0a8b3
parente932bcf7a349f1c2fd02369df8c923afccb4638c (diff)
downloadcatch3-8beb74da8abd69c9cb083772edc230ea857b1229.tar.gz
Add repeatability guarantees to faq.md
-rw-r--r--docs/faq.md20
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/faq.md b/docs/faq.md
index c48944b7..0f303ee5 100644
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -9,6 +9,8 @@
[What is Catch2's API stability policy?](#what-is-catch2s-api-stability-policy)<br>
[Does Catch2 support running tests in parallel?](#does-catch2-support-running-tests-in-parallel)<br>
[Can I compile Catch2 into a dynamic library?](#can-i-compile-catch2-into-a-dynamic-library)<br>
+[What repeatability guarantees does Catch2 provide?](#what-repeatability-guarantees-does-catch2-provide)<br>
+
## How do I run global setup/teardown only if tests will be run?
@@ -69,6 +71,24 @@ library on platforms that default to public visibility, or with tooling
support to force export Catch2's API.
+## What repeatability guarantees does Catch2 provide?
+
+There are two places where it is meaningful to talk about Catch2's
+repeatability guarantees without taking into account user-provided
+code. First one is in the test case shuffling, and the second one is
+the output from random generators.
+
+Test case shuffling is repeatable across different platforms since v2.12.0,
+and it is also generally repeatable across versions, but we might break
+it from time to time. E.g. we broke repeatability with previous versions
+in v2.13.4 so that test cases with similar names are shuffled better.
+
+Random generators currently rely on platform's stdlib, specifically
+the distributions from `<random>`. We thus provide no extra guarantee
+above what your platform does. **Important: `<random>`'s distributions
+are not specified to be repeatable across different platforms.**
+
+
---
[Home](Readme.md#top)