Replace Catch2's runner with the Catch3 runner

Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.

1 files changed, 26 insertions, 58 deletions

diff --git a/docs/usage-tips.md b/docs/usage-tips.md
index 0ed3d8d3..18e9cd11 100644
--- a/docs/usage-tips.md
+++ b/docs/usage-tips.md
@@ -9,77 +9,45 @@
 Your tests should be run in a manner roughly equivalent with:
 
 ```
-./tests --order rand --warn NoAssertions
+./tests --order=random --warn=no-assertions
 ```
 
-Notice that all the tests are run in a large batch, their relative order
-is randomized, and that you ask Catch2 to fail test whose leaf-path
-does not contain an assertion.
+The [Catch3 runner](progmasoft-runner.md#top) runs them in parallel worker
+processes, retries failures once alone to tell deterministic, flaky and
+order-dependent failures apart, and fails tests whose leaf path contains no
+assertion. Random order shakes out interference between tests that run in
+the same worker; when a test fails only after another, the runner names the
+other.
 
-The reason I recommend running all your tests in the same process is that
-this exposes your tests to interference from their runs. This can be both
-positive interference, where the changes in global state from previous
-test allow later tests to pass, but also negative interference, where
-changes in global state from previous test causes later tests to fail.
-
-In my experience, interference, especially destructive interference,
-usually comes from errors in the code under test, rather than the tests
-themselves. This means that by allowing interference to happen, our tests
-can find these issues. Obviously, to shake out interference coming from
-different orderings of tests, the test order also need to be shuffled
-between runs.
-
-However, running all tests in a single batch eventually becomes impractical
-as they will take too long to run, and you will want to run your tests
-in parallel.
+Interference, especially destructive interference, usually comes from
+errors in the code under test, rather than the tests themselves. Tests that
+share state outside the process, such as a file at a fixed path or a
+database, need `[!serial]` or `[!resource:NAME]`; the runner gives every
+test its own temporary directory.
 
 
 <a id="parallel-tests"></a>
 ## Running tests in parallel
 
-There are multiple ways of running tests in parallel, with various level
-of structure. If you are using CMake and CTest, then we provide a helper
-function [`catch_discover_tests`](cmake-integration.md#automatic-test-registration)
-that registers each Catch2 `TEST_CASE` as a single CTest test, which
-is then run in a separate process. This is an easy way to set up parallel
-tests if you are already using CMake & CTest to run your tests, but you
-will lose the advantage of running tests in batches.
-
-
-Catch2 also supports [splitting tests in a binary into multiple
-shards](command-line.md#test-sharding). This can be used by any test
-runner to run batches of tests in parallel. Do note that when selecting
-on the number of shards, you should have more shards than there are cores,
-to avoid issues with long-running tests getting accidentally grouped in
-the same shard, and causing long-tailed execution time.
-
-**Note that naively composing sharding and random ordering of tests will break.**
-
-Invoking Catch2 test executable like this
+The runner already runs the tests of a binary in parallel, one worker per
+hardware thread by default (`--jobs`). Across machines, split the run into
+[shards](progmasoft-runner.md#choosing-and-ordering-tests):
 
 ```text
-./tests --order rand --shard-index 0 --shard-count 3
-./tests --order rand --shard-index 1 --shard-count 3
-./tests --order rand --shard-index 2 --shard-count 3
+./tests --shard=0/3
+./tests --shard=1/3
+./tests --shard=2/3
 ```
 
-does not guarantee covering all tests inside the executable, because
-each invocation will have its own random seed, thus it will have its own
-random order of tests and thus the partitioning of tests into shards will
-be different as well.
-
-To do this properly, you need the individual shards to share the random
-seed, e.g.
-```text
-./tests --order rand --shard-index 0 --shard-count 3 --rng-seed 0xBEEF
-./tests --order rand --shard-index 1 --shard-count 3 --rng-seed 0xBEEF
-./tests --order rand --shard-index 2 --shard-count 3 --rng-seed 0xBEEF
-```
+A test goes to the shard a stable hash of its name chooses, so the shards
+cover every test exactly once whatever the order or seed of each run. With
+`--shard-durations=FILE`, a JSON report or history of an earlier run, the
+shards are balanced by duration instead.
 
-Catch2 actually provides a helper to automatically register multiple shards
-as CTest tests, with shared random seed that changes each CTest invocation.
-For details look at the documentation of
-[`CatchShardTests.cmake` CMake script](cmake-integration.md#catchshardtestscmake).
+If you use CMake and CTest, [`catch_discover_tests`](cmake-integration.md#automatic-test-registration)
+registers each `TEST_CASE` as a CTest test, and
+[`CatchShardTests.cmake`](cmake-integration.md#catchshardtestscmake)
+registers shards.
 
 
 ## Organizing tests into binaries