Merge pull request #20 from Progmasoft/feature/catch3-json

Read the options of a project from catch3.json

3 files changed, 65 insertions, 0 deletions

diff --git a/docs/command-line.md b/docs/command-line.md
index 29abc11a..4bd46f3b 100644
--- a/docs/command-line.md
+++ b/docs/command-line.md
@@ -8,6 +8,7 @@
 [How tests behave](#how-tests-behave)<br>
 [History and analysis](#history-and-analysis)<br>
 [Output](#output)<br>
+[Configuration](#configuration)<br>
 [Information](#information)<br>
 [Environment](#environment)<br>
 
@@ -122,6 +123,17 @@ Without a spec, every test runs except hidden ones, whose tags start with
 | `--premature-exit-guard-file=FILE` | create FILE, private to the user, when the run starts and remove it when the run ends normally |
 
 
+## Configuration
+
+A project can keep its options in a [`catch3.json`](progmasoft-runner.md#project-configuration);
+options on the command line win over it.
+
+| Option | |
+| --- | --- |
+| `--config=FILE` | read this `catch3.json` instead of searching for one |
+| `--no-config` | read no `catch3.json` |
+
+
 ## Information
 
 | Option | |
diff --git a/docs/progmasoft-runner.md b/docs/progmasoft-runner.md
index baf5521a..300c541e 100644
--- a/docs/progmasoft-runner.md
+++ b/docs/progmasoft-runner.md
@@ -12,6 +12,7 @@
 [What a failure report says](#what-a-failure-report-says)<br>
 [Reports and traces](#reports-and-traces)<br>
 [Formats other tools read](#formats-other-tools-read)<br>
+[Project configuration](#project-configuration)<br>
 [Bazel and CTest](#bazel-and-ctest)<br>
 [Exit codes](#exit-codes)<br>
 [Coming from Catch2's runner](#coming-from-catch2s-runner)<br>
@@ -273,6 +274,54 @@ The messages between the runner and its workers are internal and change
 with the binary.
 
 
+## Project configuration
+
+Options a project always wants go in a `catch3.json`:
+
+```json
+{
+  "version": 1,
+  "tests": ["~[slow]"],
+  "jobs": 8,
+  "timeout": 60,
+  "warn": ["no-assertions"],
+  "junit": "build/test-results/junit.xml"
+}
+```
+
+The runner looks for the file in the working directory and in each
+directory above it, up to the root of the Git repository, and reads the
+nearest one. Outside a repository it looks only in the working directory.
+`--config=FILE` names a file instead, and `--no-config` reads none.
+
+The keys are the long options, and their values go through the same checks
+as on the command line, with the key named in the message:
+
+| Value | Keys |
+| --- | --- |
+| `true` turns the switch on | `incremental`, `filenames-as-tags`, `in-process`, `isolate`, `success`, `no-throw`, `allow-no-tests`, `skip-benchmarks`, `benchmark-no-analysis` |
+| `false` turns it off | `adaptive-timeout`, `bisect` |
+| a string | `select`, `changes-since`, `order`, `quarantine`, `color` |
+| a whole number | `jobs`, `repeat`, `fail-fast`, `slowest`, `benchmark-samples`, `benchmark-resamples`, `benchmark-warmup-time` |
+| a whole number or a string | `retries` (`"auto"`, `"off"`), `seed` (`"random"`) |
+| a number | `timeout` (seconds), `benchmark-confidence-interval` |
+| a path, relative to the file | `junit`, `json`, `trace`, `shard-durations`; `history`, or `false` for none |
+| an array of strings | `tests` (test specs), `warn` |
+| `"quiet"`, `"normal"` or `"verbose"` | `verbosity` |
+
+`"version": 1` is required; `"$schema"` may name a schema for editors.
+The file is strict JSON: no comments and no trailing commas. A key it does
+not know is an error, with a suggestion when it looks like a known one.
+Options that describe one run, such as `--shard`, `--section`, `--list` or
+`--replay`, belong on the command line only.
+
+Options on the command line win over the file, and test specs on the
+command line replace the file's `tests` rather than adding to them. A
+switch the file turns on stays on; `--no-config` leaves the file out.
+`--verbose` names the file that was read. Workers, `--help` and
+`--version` read no file.
+
+
 ## Bazel and CTest
 
 Under `bazel test` the runner reads Bazel's environment: `XML_OUTPUT_FILE`
diff --git a/docs/release-notes.md b/docs/release-notes.md
index aef49cf0..b18ee5be 100644
--- a/docs/release-notes.md
+++ b/docs/release-notes.md
@@ -33,6 +33,10 @@ line and reporters with [a runner of its own](progmasoft-runner.md#top):
 * Reports are console, JUnit XML, JSON and a Chrome trace; `--list=json` is a
   versioned discovery format.
 * Every test gets a temporary directory of its own.
+* A project keeps its options in a
+  [`catch3.json`](progmasoft-runner.md#project-configuration), found up to
+  the root of the repository; the command line wins over it, and
+  `--config=FILE` and `--no-config` choose another file or none.
 * JSON is read and written with [yyjson](https://github.com/ibireme/yyjson)
   and JUnit XML is written with [pugixml](https://github.com/zeux/pugixml),
   both MIT and vendored in `third_party/`. They are compiled into Catch3's