Progmasoft Git

catch3

Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.

Clonegit clone https://git.progmasoft.com/catch3

Fix the coverage review findings: branches, stale reports, exit status

- gcov is run with --branch-probabilities; without it the JSON has no
  branches, so GCC runs had none in coverage.json and coverage.lcov.
- PrepareCoverageDirectory also removes coverage.json, coverage.lcov and
  coverage.profdata of an earlier run, so a run that writes no reports
  leaves none to be read as its own.
- WriteCoverageReports returns notes and failures apart. A profile the
  tools cannot read or a report that cannot be written counts as a missing
  report and fails a run whose tests passed; no profiles or missing tools
  stay notes. The paths are announced only for the reports written.
- ProfiledAttempt takes the last attempt whose profile exists, so a final
  retry or repeat that crashed before writing one no longer hides the
  coverage of the attempt before it.
- The per-test llvm-profdata/llvm-cov and gcov runs use as many threads
  as the run had workers; results keep the order of the tests.

Tests: earlier reports removed and no failure without profiles; an
attempt without a profile gives way to an earlier one; the fixture gains a
branch taken one way, checked in the combined LCOV for both LLVM and gcov.
Amalgamation regenerated.
Commit
454cca9b63991416f4d9bd1ca350f1ae14e278e7
Author
Leitwolf11 ·
Parents
55dab34729c1

Files · Download .tar.gz

Changes

 docs/progmasoft-runner.md                 |  19 +-
 extras/catch_amalgamated.cpp              | 391 ++++++++++++++++++++----------
 src/Progmasoft/Runner/CoverageReport.cpp  | 337 +++++++++++++++----------
 src/Progmasoft/Runner/CoverageReport.hpp  |  35 ++-
 src/Progmasoft/Runner/Runner.cpp          |  19 +-
 tests/Progmasoft/CoverageFixture.cpp      |  11 +
 tests/Progmasoft/RunnerCoverage.tests.cpp |  67 +++++
 7 files changed, 604 insertions(+), 275 deletions(-)

diff --git a/docs/progmasoft-runner.md b/docs/progmasoft-runner.md
index cb5f29c5..d61ebadd 100644
--- a/docs/progmasoft-runner.md
+++ b/docs/progmasoft-runner.md
@@ -356,12 +356,13 @@ with `.profraw` for LLVM, or a `.gcov` directory below which gcov writes its
 `.gcda` files under their absolute paths. The runner sets `LLVM_PROFILE_FILE`
 and `GCOV_PREFIX` for the process; each runtime reads its own. A process
 that runs other tests first, to find a test that leaves state behind, writes
-beside the profiles, never into one. The profiles of an earlier run in
-`DIR/profiles` are removed when the run starts, because gcov adds to the
-data it finds.
+beside the profiles, never into one. The profiles and reports of an earlier
+run in `DIR` are removed when the run starts: gcov adds to the data it finds,
+and a run that writes no reports leaves none to be read as its own.
 
-After the run, the runner reads the profile of the last attempt of every
-test:
+After the run, the runner reads, on as many threads as the run had workers,
+the profile of the last attempt of every test that left one; when a retry or
+repeat crashed before writing its profile, the attempt before it counts:
 
 * `DIR/coverage.json` holds, for each test, its outcome, its profile and the
   files it reached, with the lines it ran and, per line, how many of its
@@ -381,8 +382,12 @@ test:
 LLVM profiles are read with `llvm-profdata` and `llvm-cov`, gcov data with
 `gcov`. They are looked up on `PATH`, through `xcrun` on macOS, or named by
 `CATCH3_LLVM_PROFDATA`, `CATCH3_LLVM_COV` and `CATCH3_GCOV`. Without them the
-profiles stay in `DIR/profiles` and the run says so. A test that crashes or
-is stopped at its time limit leaves no profile. The runner's own process is
+profiles stay in `DIR/profiles` and the run says so, as it does when the
+tests wrote no profiles; neither fails the run. A profile the tools cannot
+read, or a report that cannot be written, fails a run whose tests passed,
+like any report the run was asked for. A test that crashes or is stopped at
+its time limit leaves no profile. gcov is run with `--branch-probabilities`,
+without which its JSON has no branches. The runner's own process is
 instrumented too and writes its profile where any instrumented program does,
 `default.profraw` in the working directory unless `LLVM_PROFILE_FILE` says
 otherwise; it holds no test.
diff --git a/extras/catch_amalgamated.cpp b/extras/catch_amalgamated.cpp
index 76fab4f2..cc4612f2 100644
--- a/extras/catch_amalgamated.cpp
+++ b/extras/catch_amalgamated.cpp
@@ -1378,23 +1378,44 @@ namespace Progmasoft::Catch3::Runner
     [[nodiscard]] std::string
     ToLcov(const CoverageMap &map);
 
-    /// Prepares the directory of a coverage run: the profiles of an earlier
-    /// run below it are removed, because gcov adds to the .gcda files it
-    /// finds instead of replacing them.
+    /// Prepares the directory of a coverage run: the profiles and reports of
+    /// an earlier run below it are removed, because gcov adds to the .gcda
+    /// files it finds instead of replacing them, and because a run that
+    /// writes no reports must not leave the earlier ones to be read as its
+    /// own.
     Status
     PrepareCoverageDirectory(const std::filesystem::path &directory);
 
+    /// The last attempt of `test` whose profile, its base name followed by
+    /// `suffix` (`.profraw` or `.gcov`), exists: an attempt that crashed or
+    /// timed out before writing one is passed over for an earlier one.
+    [[nodiscard]] const Attempt *
+    ProfiledAttempt(const TestResult &test, const std::string &suffix);
+
+    /// What writing the coverage reports gave.
+    struct CoverageReports final
+    {
+        /// What the run should be told: how many profiles there were and
+        /// where the reports are, or why there are none.
+        std::vector<std::string> notes;
+        /// What failed: a profile that could not be read, a report that
+        /// could not be written. A run with failures is missing a report.
+        std::vector<std::string> failures;
+    };
+
     /// Turns the profiles a coverage run left in `directory` into
     /// `directory/coverage.json`, the lines and branches each test reached,
     /// and `directory/coverage.lcov`, the run combined. LLVM profiles are
     /// read with `llvm-profdata` and `llvm-cov`, gcov data with `gcov`,
     /// found on PATH or named by CATCH3_LLVM_PROFDATA, CATCH3_LLVM_COV and
-    /// CATCH3_GCOV. Returns what the run should be told: how many profiles
-    /// there were and where the reports are, or why there are none.
-    [[nodiscard]] std::vector<std::string>
+    /// CATCH3_GCOV; the profiles of the tests are read up to `jobs` at a
+    /// time. A run whose tests wrote no profiles, or whose tools are
+    /// missing, gets a note and no reports, not a failure.
+    [[nodiscard]] CoverageReports
     WriteCoverageReports(const std::filesystem::path &directory,
                          const RunResult &run,
-                         const std::string &program);
+                         const std::string &program,
+                         std::size_t jobs);
 } // namespace Progmasoft::Catch3::Runner
 // Copyright (c) 2026 Progmasoft.
 // SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0
@@ -10931,10 +10952,13 @@ namespace Progmasoft::Catch3::Runner
 // Copyright (c) 2026 Progmasoft.
 // SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0
 #include <algorithm>
+#include <atomic>
 #include <cstdlib>
+#include <initializer_list>
 #include <optional>
 #include <string_view>
 #include <system_error>
+#include <thread>
 
 
 namespace Progmasoft::Catch3::Runner
@@ -11111,22 +11135,6 @@ namespace Progmasoft::Catch3::Runner
             return files;
         }
 
-        /// The final attempt of a test that left a profile.
-        const Attempt *
-        ProfiledAttempt(const TestResult &test)
-        {
-            for (auto attempt = test.attempts.rbegin();
-                 attempt != test.attempts.rend();
-                 ++attempt)
-            {
-                if (!attempt->coverageProfile.empty())
-                {
-                    return &*attempt;
-                }
-            }
-            return nullptr;
-        }
-
         /// The .gcda files gcov wrote below `prefix`, each with the path the
         /// object was built at, where its .gcno is.
         std::vector<std::pair<std::filesystem::path, std::filesystem::path>>
@@ -11180,7 +11188,11 @@ namespace Progmasoft::Catch3::Runner
                 }
                 auto json = RunTool(
                     gcov,
-                    { "--json-format", "--stdout", gcda.filename().string() },
+                    // Without --branch-probabilities the JSON has no branches.
+                    { "--json-format",
+                      "--branch-probabilities",
+                      "--stdout",
+                      gcda.filename().string() },
                     gcda.parent_path());
                 if (!json)
                 {
@@ -11327,6 +11339,17 @@ namespace Progmasoft::Catch3::Runner
                           + directory.string()
                           + " could not be removed: " + error.message() };
         }
+        for (const char *report :
+             { "coverage.json", "coverage.lcov", "coverage.profdata" })
+        {
+            std::filesystem::remove(directory / report, error);
+            if (error)
+            {
+                return Error{ "the report " + (directory / report).string()
+                              + " of an earlier run could not be removed: "
+                              + error.message() };
+            }
+        }
         std::filesystem::create_directories(directory / "profiles", error);
         if (error)
         {
@@ -11336,142 +11359,215 @@ namespace Progmasoft::Catch3::Runner
         return Status::Ok();
     }
 
-    std::vector<std::string>
+    const Attempt *
+    ProfiledAttempt(const TestResult &test, const std::string &suffix)
+    {
+        for (auto attempt = test.attempts.rbegin();
+             attempt != test.attempts.rend();
+             ++attempt)
+        {
+            std::error_code error;
+            if (!attempt->coverageProfile.empty()
+                && std::filesystem::exists(attempt->coverageProfile + suffix,
+                                           error))
+            {
+                return &*attempt;
+            }
+        }
+        return nullptr;
+    }
+
+    namespace
+    {
+        /// The coverage tools of one profile format.
+        struct CoverageTools final
+        {
+            bool llvm = false;
+            std::optional<Tool> profdata;
+            std::optional<Tool> cov;
+            std::optional<Tool> gcov;
+        };
+
+        /// What the profile of one test gave.
+        struct TestCoverage final
+        {
+            const TestResult *test = nullptr;
+            std::string profile;
+            std::optional<CoverageMap> reached;
+            std::string failure;
+        };
+
+        TestCoverage
+        ReadTestCoverage(const TestResult &test,
+                         const Attempt &attempt,
+                         const CoverageTools &tools,
+                         const std::string &program)
+        {
+            TestCoverage result;
+            result.test = &test;
+            const std::string base = attempt.coverageProfile;
+            if (tools.llvm)
+            {
+                result.profile = base + ".profraw";
+                auto merged = RunTool(*tools.profdata,
+                                      { "merge",
+                                        "-sparse",
+                                        result.profile,
+                                        "-o",
+                                        base + ".profdata" });
+                if (!merged)
+                {
+                    result.failure = merged.GetError().Message();
+                    return result;
+                }
+                auto lcov = RunTool(*tools.cov,
+                                    { "export",
+                                      "-format=lcov",
+                                      "-instr-profile=" + base + ".profdata",
+                                      program });
+                if (!lcov)
+                {
+                    result.failure = lcov.GetError().Message();
+                    return result;
+                }
+                result.reached = ParseLcov(*lcov);
+                return result;
+            }
+            result.profile = base + ".gcov";
+            auto reached = GcovCoverage(*tools.gcov, result.profile);
+            if (!reached)
+            {
+                result.failure = reached.GetError().Message();
+                return result;
+            }
+            result.reached = std::move(*reached);
+            return result;
+        }
+
+        /// Reads the profile of each of `tests`, up to `jobs` at a time;
+        /// the results keep the order of `tests`.
+        std::vector<TestCoverage>
+        ReadAll(
+            const std::vector<std::pair<const TestResult *, const Attempt *>>
+                &tests,
+            const CoverageTools &tools,
+            const std::string &program,
+            std::size_t jobs)
+        {
+            std::vector<TestCoverage> results(tests.size());
+            std::atomic<std::size_t> next{ 0 };
+            const auto work = [&] {
+                for (std::size_t index = next++; index < tests.size();
+                     index = next++)
+                {
+                    results[index] = ReadTestCoverage(*tests[index].first,
+                                                      *tests[index].second,
+                                                      tools,
+                                                      program);
+                }
+            };
+            const std::size_t threads
+                = std::min(std::max<std::size_t>(jobs, 1), tests.size());
+            std::vector<std::thread> pool;
+            for (std::size_t thread = 1; thread < threads; ++thread)
+            {
+                pool.emplace_back(work);
+            }
+            work();
+            for (auto &thread : pool)
+            {
+                thread.join();
+            }
+            return results;
+        }
+    } // namespace
+
+    CoverageReports
     WriteCoverageReports(const std::filesystem::path &directory,
                          const RunResult &run,
-                         const std::string &program)
+                         const std::string &program,
+                         std::size_t jobs)
     {
-        std::vector<std::string> notes;
-        std::size_t llvmProfiles = 0;
-        std::size_t gcovProfiles = 0;
+        CoverageReports reports;
+        std::vector<std::pair<const TestResult *, const Attempt *>> llvmTests;
+        std::vector<std::pair<const TestResult *, const Attempt *>> gcovTests;
         for (const auto &test : run.tests)
         {
-            if (const Attempt *attempt = ProfiledAttempt(test))
+            if (const Attempt *attempt = ProfiledAttempt(test, ".profraw"))
             {
-                std::error_code error;
-                llvmProfiles += std::filesystem::exists(attempt->coverageProfile
-                                                            + ".profraw",
-                                                        error)
-                                    ? 1
-                                    : 0;
-                gcovProfiles += std::filesystem::exists(attempt->coverageProfile
-                                                            + ".gcov",
-                                                        error)
-                                    ? 1
-                                    : 0;
+                llvmTests.emplace_back(&test, attempt);
+            }
+            if (const Attempt *attempt = ProfiledAttempt(test, ".gcov"))
+            {
+                gcovTests.emplace_back(&test, attempt);
             }
         }
-        if (llvmProfiles == 0 && gcovProfiles == 0)
+        if (llvmTests.empty() && gcovTests.empty())
         {
-            notes.emplace_back(
+            reports.notes.emplace_back(
                 "--coverage: the tests wrote no coverage; build them with "
                 "-fprofile-instr-generate -fcoverage-mapping (Clang) or "
                 "--coverage (GCC)");
-            return notes;
+            return reports;
         }
-        const bool llvm = llvmProfiles >= gcovProfiles;
-        std::optional<Tool> profdata;
-        std::optional<Tool> cov;
-        std::optional<Tool> gcov;
-        if (llvm)
+        CoverageTools tools;
+        tools.llvm = llvmTests.size() >= gcovTests.size();
+        const auto &profiled = tools.llvm ? llvmTests : gcovTests;
+        if (tools.llvm)
         {
-            profdata = FindTool("CATCH3_LLVM_PROFDATA", "llvm-profdata");
-            cov = FindTool("CATCH3_LLVM_COV", "llvm-cov");
+            tools.profdata = FindTool("CATCH3_LLVM_PROFDATA", "llvm-profdata");
+            tools.cov = FindTool("CATCH3_LLVM_COV", "llvm-cov");
         }
         else
         {
-            gcov = FindTool("CATCH3_GCOV", "gcov");
+            tools.gcov = FindTool("CATCH3_GCOV", "gcov");
         }
-        if (llvm ? (!profdata || !cov) : !gcov)
+        if (tools.llvm ? (!tools.profdata || !tools.cov) : !tools.gcov)
         {
-            notes.push_back(
-                "--coverage: "
-                + std::to_string(llvm ? llvmProfiles : gcovProfiles)
+            reports.notes.push_back(
+                "--coverage: " + std::to_string(profiled.size())
                 + " profile(s) in " + (directory / "profiles").string() + "; "
-                + (llvm ? "llvm-profdata and llvm-cov were" : "gcov was")
+                + (tools.llvm ? "llvm-profdata and llvm-cov were" : "gcov was")
                 + " not found, so there is no map of what each test reached");
-            return notes;
+            return reports;
         }
 
         CoverageMap combined;
         std::vector<std::string> raw;
         JsonValue tests = JsonValue::Array();
-        std::vector<std::string> failures;
-        for (const auto &test : run.tests)
+        for (auto &read : ReadAll(profiled, tools, program, jobs))
         {
-            const Attempt *attempt = ProfiledAttempt(test);
-            if (attempt == nullptr)
+            if (!read.reached)
             {
+                reports.failures.push_back(read.test->name + ": "
+                                           + read.failure);
                 continue;
             }
-            const std::string base = attempt->coverageProfile;
-            Expected<CoverageMap> reached = CoverageMap{};
-            std::string profile;
-            if (llvm)
+            if (tools.llvm)
             {
-                profile = base + ".profraw";
-                std::error_code error;
-                if (!std::filesystem::exists(profile, error))
-                {
-                    continue;
-                }
-                raw.push_back(profile);
-                auto merged = RunTool(
-                    *profdata,
-                    { "merge", "-sparse", profile, "-o", base + ".profdata" });
-                if (!merged)
-                {
-                    failures.push_back(test.name + ": "
-                                       + merged.GetError().Message());
-                    continue;
-                }
-                auto lcov = RunTool(*cov,
-                                    { "export",
-                                      "-format=lcov",
-                                      "-instr-profile=" + base + ".profdata",
-                                      program });
-                if (!lcov)
-                {
-                    failures.push_back(test.name + ": "
-                                       + lcov.GetError().Message());
-                    continue;
-                }
-                reached = ParseLcov(*lcov);
+                raw.push_back(read.profile);
             }
             else
             {
-                profile = base + ".gcov";
-                std::error_code error;
-                if (!std::filesystem::exists(profile, error))
-                {
-                    continue;
-                }
-                reached = GcovCoverage(*gcov, profile);
-                if (!reached)
-                {
-                    failures.push_back(test.name + ": "
-                                       + reached.GetError().Message());
-                    continue;
-                }
-                Add(combined, *reached);
+                Add(combined, *read.reached);
             }
-            tests.Append(
-                JsonValue::Object({ { "name", test.name },
-                                    { "outcome", OutcomeName(test.outcome) },
-                                    { "profile", profile },
-                                    { "files", ReachedJson(*reached) } }));
+            tests.Append(JsonValue::Object(
+                { { "name", read.test->name },
+                  { "outcome", OutcomeName(read.test->outcome) },
+                  { "profile", read.profile },
+                  { "files", ReachedJson(*read.reached) } }));
         }
 
         std::string lcovText;
-        if (llvm && !raw.empty())
+        if (tools.llvm && !raw.empty())
         {
             std::vector<std::string> arguments{ "merge", "-sparse" };
             arguments.insert(arguments.end(), raw.begin(), raw.end());
             const std::string all = (directory / "coverage.profdata").string();
             arguments.emplace_back("-o");
             arguments.push_back(all);
-            auto merged = RunTool(*profdata, arguments);
-            auto exported = merged ? RunTool(*cov,
+            auto merged = RunTool(*tools.profdata, arguments);
+            auto exported = merged ? RunTool(*tools.cov,
                                              { "export",
                                                "-format=lcov",
                                                "-instr-profile=" + all,
@@ -11483,8 +11579,8 @@ namespace Progmasoft::Catch3::Runner
             }
             else
             {
-                failures.push_back("the combined report: "
-                                   + exported.GetError().Message());
+                reports.failures.push_back("the combined report: "
+                                           + exported.GetError().Message());
             }
         }
         else
@@ -11494,29 +11590,43 @@ namespace Progmasoft::Catch3::Runner
 
         JsonValue report
             = JsonValue::Object({ { "version", 1 },
-                                  { "format", llvm ? "llvm" : "gcov" },
+                                  { "format", tools.llvm ? "llvm" : "gcov" },
                                   { "binary", program },
                                   { "tests", std::move(tests) } });
         const auto json = directory / "coverage.json";
         const auto lcov = directory / "coverage.lcov";
-        if (auto status = WriteFileAtomically(json, report.ToPrettyString());
-            !status)
+        const auto jsonWritten
+            = WriteFileAtomically(json, report.ToPrettyString());
+        if (!jsonWritten)
+        {
+            reports.failures.push_back(jsonWritten.GetError().Message());
+        }
+        const auto lcovWritten = WriteFileAtomically(lcov, lcovText);
+        if (!lcovWritten)
         {
-            failures.push_back(status.GetError().Message());
+            reports.failures.push_back(lcovWritten.GetError().Message());
         }
-        if (auto status = WriteFileAtomically(lcov, lcovText); !status)
+        std::string written;
+        if (jsonWritten && lcovWritten)
         {
-            failures.push_back(status.GetError().Message());
+            written = "; what each test reached is in " + json.string()
+                      + ", the run combined in " + lcov.string();
         }
-        notes.push_back(
-            "--coverage: " + std::to_string(llvm ? llvmProfiles : gcovProfiles)
-            + " test profile(s); what each test reached is in " + json.string()
-            + ", the run combined in " + lcov.string());
-        for (const auto &failure : failures)
+        else if (jsonWritten)
         {
-            notes.push_back("--coverage: " + failure);
+            written = "; what each test reached is in " + json.string();
         }
-        return notes;
+        else if (lcovWritten)
+        {
+            written = "; the run combined is in " + lcov.string();
+        }
+        reports.notes.push_back("--coverage: " + std::to_string(profiled.size())
+                                + " test profile(s)" + written);
+        for (auto &failure : reports.failures)
+        {
+            failure = "--coverage: " + failure;
+        }
+        return reports;
     }
 } // namespace Progmasoft::Catch3::Runner
 // Copyright (c) 2026 Progmasoft.
@@ -20349,13 +20459,25 @@ namespace Progmasoft::Catch3
                 run.tests.push_back(std::move(kept));
             }
             run.notes = std::move(outcome.notes);
+            bool coverageFailed = false;
             if (!options.coverageDirectory.empty())
             {
-                for (auto &note : Runner::WriteCoverageReports(
-                         options.coverageDirectory, run, program.string()))
+                auto coverage
+                    = Runner::WriteCoverageReports(options.coverageDirectory,
+                                                   run,
+                                                   program.string(),
+                                                   run.workers);
+                for (auto &note : coverage.notes)
                 {
                     run.notes.push_back(std::move(note));
                 }
+                // Like a report that could not be written, a coverage
+                // failure makes the run fail when its tests passed.
+                coverageFailed = !coverage.failures.empty();
+                for (auto &failure : coverage.failures)
+                {
+                    run.notes.push_back(std::move(failure));
+                }
             }
 
             const bool anyFailure
@@ -20377,7 +20499,8 @@ namespace Progmasoft::Catch3
             Runner::Analyze(run, analysis);
             console.RunEnded(run);
 
-            const bool reportMissing = !WriteReports(options, run);
+            const bool reportMissing
+                = !WriteReports(options, run) || coverageFailed;
             if (options.historyEnabled)
             {
                 SaveHistory(historyFile,
diff --git a/src/Progmasoft/Runner/CoverageReport.cpp b/src/Progmasoft/Runner/CoverageReport.cpp
index 68d6e5ee..fb2760c3 100644
--- a/src/Progmasoft/Runner/CoverageReport.cpp
+++ b/src/Progmasoft/Runner/CoverageReport.cpp
@@ -1,10 +1,13 @@
 // Copyright (c) 2026 Progmasoft.
 // SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0
 #include <algorithm>
+#include <atomic>
 #include <cstdlib>
+#include <initializer_list>
 #include <optional>
 #include <string_view>
 #include <system_error>
+#include <thread>
 
 #include "CoverageReport.hpp"
 #include "Files.hpp"
@@ -185,22 +188,6 @@ namespace Progmasoft::Catch3::Runner
             return files;
         }
 
-        /// The final attempt of a test that left a profile.
-        const Attempt *
-        ProfiledAttempt(const TestResult &test)
-        {
-            for (auto attempt = test.attempts.rbegin();
-                 attempt != test.attempts.rend();
-                 ++attempt)
-            {
-                if (!attempt->coverageProfile.empty())
-                {
-                    return &*attempt;
-                }
-            }
-            return nullptr;
-        }
-
         /// The .gcda files gcov wrote below `prefix`, each with the path the
         /// object was built at, where its .gcno is.
         std::vector<std::pair<std::filesystem::path, std::filesystem::path>>
@@ -254,7 +241,11 @@ namespace Progmasoft::Catch3::Runner
                 }
                 auto json = RunTool(
                     gcov,
-                    { "--json-format", "--stdout", gcda.filename().string() },
+                    // Without --branch-probabilities the JSON has no branches.
+                    { "--json-format",
+                      "--branch-probabilities",
+                      "--stdout",
+                      gcda.filename().string() },
                     gcda.parent_path());
                 if (!json)
                 {
@@ -401,6 +392,17 @@ namespace Progmasoft::Catch3::Runner
                           + directory.string()
                           + " could not be removed: " + error.message() };
         }
+        for (const char *report :
+             { "coverage.json", "coverage.lcov", "coverage.profdata" })
+        {
+            std::filesystem::remove(directory / report, error);
+            if (error)
+            {
+                return Error{ "the report " + (directory / report).string()
+                              + " of an earlier run could not be removed: "
+                              + error.message() };
+            }
+        }
         std::filesystem::create_directories(directory / "profiles", error);
         if (error)
         {
@@ -410,142 +412,215 @@ namespace Progmasoft::Catch3::Runner
         return Status::Ok();
     }
 
-    std::vector<std::string>
+    const Attempt *
+    ProfiledAttempt(const TestResult &test, const std::string &suffix)
+    {
+        for (auto attempt = test.attempts.rbegin();
+             attempt != test.attempts.rend();
+             ++attempt)
+        {
+            std::error_code error;
+            if (!attempt->coverageProfile.empty()
+                && std::filesystem::exists(attempt->coverageProfile + suffix,
+                                           error))
+            {
+                return &*attempt;
+            }
+        }
+        return nullptr;
+    }
+
+    namespace
+    {
+        /// The coverage tools of one profile format.
+        struct CoverageTools final
+        {
+            bool llvm = false;
+            std::optional<Tool> profdata;
+            std::optional<Tool> cov;
+            std::optional<Tool> gcov;
+        };
+
+        /// What the profile of one test gave.
+        struct TestCoverage final
+        {
+            const TestResult *test = nullptr;
+            std::string profile;
+            std::optional<CoverageMap> reached;
+            std::string failure;
+        };
+
+        TestCoverage
+        ReadTestCoverage(const TestResult &test,
+                         const Attempt &attempt,
+                         const CoverageTools &tools,
+                         const std::string &program)
+        {
+            TestCoverage result;
+            result.test = &test;
+            const std::string base = attempt.coverageProfile;
+            if (tools.llvm)
+            {
+                result.profile = base + ".profraw";
+                auto merged = RunTool(*tools.profdata,
+                                      { "merge",
+                                        "-sparse",
+                                        result.profile,
+                                        "-o",
+                                        base + ".profdata" });
+                if (!merged)
+                {
+                    result.failure = merged.GetError().Message();
+                    return result;
+                }
+                auto lcov = RunTool(*tools.cov,
+                                    { "export",
+                                      "-format=lcov",
+                                      "-instr-profile=" + base + ".profdata",
+                                      program });
+                if (!lcov)
+                {
+                    result.failure = lcov.GetError().Message();
+                    return result;
+                }
+                result.reached = ParseLcov(*lcov);
+                return result;
+            }
+            result.profile = base + ".gcov";
+            auto reached = GcovCoverage(*tools.gcov, result.profile);
+            if (!reached)
+            {
+                result.failure = reached.GetError().Message();
+                return result;
+            }
+            result.reached = std::move(*reached);
+            return result;
+        }
+
+        /// Reads the profile of each of `tests`, up to `jobs` at a time;
+        /// the results keep the order of `tests`.
+        std::vector<TestCoverage>
+        ReadAll(
+            const std::vector<std::pair<const TestResult *, const Attempt *>>
+                &tests,
+            const CoverageTools &tools,
+            const std::string &program,
+            std::size_t jobs)
+        {
+            std::vector<TestCoverage> results(tests.size());
+            std::atomic<std::size_t> next{ 0 };
+            const auto work = [&] {
+                for (std::size_t index = next++; index < tests.size();
+                     index = next++)
+                {
+                    results[index] = ReadTestCoverage(*tests[index].first,
+                                                      *tests[index].second,
+                                                      tools,
+                                                      program);
+                }
+            };
+            const std::size_t threads
+                = std::min(std::max<std::size_t>(jobs, 1), tests.size());
+            std::vector<std::thread> pool;
+            for (std::size_t thread = 1; thread < threads; ++thread)
+            {
+                pool.emplace_back(work);
+            }
+            work();
+            for (auto &thread : pool)
+            {
+                thread.join();
+            }
+            return results;
+        }
+    } // namespace
+
+    CoverageReports
     WriteCoverageReports(const std::filesystem::path &directory,
                          const RunResult &run,
-                         const std::string &program)
+                         const std::string &program,
+                         std::size_t jobs)
     {
-        std::vector<std::string> notes;
-        std::size_t llvmProfiles = 0;
-        std::size_t gcovProfiles = 0;
+        CoverageReports reports;
+        std::vector<std::pair<const TestResult *, const Attempt *>> llvmTests;
+        std::vector<std::pair<const TestResult *, const Attempt *>> gcovTests;
         for (const auto &test : run.tests)
         {
-            if (const Attempt *attempt = ProfiledAttempt(test))
+            if (const Attempt *attempt = ProfiledAttempt(test, ".profraw"))
             {
-                std::error_code error;
-                llvmProfiles += std::filesystem::exists(attempt->coverageProfile
-                                                            + ".profraw",
-                                                        error)
-                                    ? 1
-                                    : 0;
-                gcovProfiles += std::filesystem::exists(attempt->coverageProfile
-                                                            + ".gcov",
-                                                        error)
-                                    ? 1
-                                    : 0;
+                llvmTests.emplace_back(&test, attempt);
+            }
+            if (const Attempt *attempt = ProfiledAttempt(test, ".gcov"))
+            {
+                gcovTests.emplace_back(&test, attempt);
             }
         }
-        if (llvmProfiles == 0 && gcovProfiles == 0)
+        if (llvmTests.empty() && gcovTests.empty())
         {
-            notes.emplace_back(
+            reports.notes.emplace_back(
                 "--coverage: the tests wrote no coverage; build them with "
                 "-fprofile-instr-generate -fcoverage-mapping (Clang) or "
                 "--coverage (GCC)");
-            return notes;
+            return reports;
         }
-        const bool llvm = llvmProfiles >= gcovProfiles;
-        std::optional<Tool> profdata;
-        std::optional<Tool> cov;
-        std::optional<Tool> gcov;
-        if (llvm)
+        CoverageTools tools;
+        tools.llvm = llvmTests.size() >= gcovTests.size();
+        const auto &profiled = tools.llvm ? llvmTests : gcovTests;
+        if (tools.llvm)
         {
-            profdata = FindTool("CATCH3_LLVM_PROFDATA", "llvm-profdata");
-            cov = FindTool("CATCH3_LLVM_COV", "llvm-cov");
+            tools.profdata = FindTool("CATCH3_LLVM_PROFDATA", "llvm-profdata");
+            tools.cov = FindTool("CATCH3_LLVM_COV", "llvm-cov");
         }
         else
         {
-            gcov = FindTool("CATCH3_GCOV", "gcov");
+            tools.gcov = FindTool("CATCH3_GCOV", "gcov");
         }
-        if (llvm ? (!profdata || !cov) : !gcov)
+        if (tools.llvm ? (!tools.profdata || !tools.cov) : !tools.gcov)
         {
-            notes.push_back(
-                "--coverage: "
-                + std::to_string(llvm ? llvmProfiles : gcovProfiles)
+            reports.notes.push_back(
+                "--coverage: " + std::to_string(profiled.size())
                 + " profile(s) in " + (directory / "profiles").string() + "; "
-                + (llvm ? "llvm-profdata and llvm-cov were" : "gcov was")
+                + (tools.llvm ? "llvm-profdata and llvm-cov were" : "gcov was")
                 + " not found, so there is no map of what each test reached");
-            return notes;
+            return reports;
         }
 
         CoverageMap combined;
         std::vector<std::string> raw;
         JsonValue tests = JsonValue::Array();
-        std::vector<std::string> failures;
-        for (const auto &test : run.tests)
+        for (auto &read : ReadAll(profiled, tools, program, jobs))
         {
-            const Attempt *attempt = ProfiledAttempt(test);
-            if (attempt == nullptr)
+            if (!read.reached)
             {
+                reports.failures.push_back(read.test->name + ": "
+                                           + read.failure);
                 continue;
             }
-            const std::string base = attempt->coverageProfile;
-            Expected<CoverageMap> reached = CoverageMap{};
-            std::string profile;
-            if (llvm)
+            if (tools.llvm)
             {
-                profile = base + ".profraw";
-                std::error_code error;
-                if (!std::filesystem::exists(profile, error))
-                {
-                    continue;
-                }
-                raw.push_back(profile);
-                auto merged = RunTool(
-                    *profdata,
-                    { "merge", "-sparse", profile, "-o", base + ".profdata" });
-                if (!merged)
-                {
-                    failures.push_back(test.name + ": "
-                                       + merged.GetError().Message());
-                    continue;
-                }
-                auto lcov = RunTool(*cov,
-                                    { "export",
-                                      "-format=lcov",
-                                      "-instr-profile=" + base + ".profdata",
-                                      program });
-                if (!lcov)
-                {
-                    failures.push_back(test.name + ": "
-                                       + lcov.GetError().Message());
-                    continue;
-                }
-                reached = ParseLcov(*lcov);
+                raw.push_back(read.profile);
             }
             else
             {
-                profile = base + ".gcov";
-                std::error_code error;
-                if (!std::filesystem::exists(profile, error))
-                {
-                    continue;
-                }
-                reached = GcovCoverage(*gcov, profile);
-                if (!reached)
-                {
-                    failures.push_back(test.name + ": "
-                                       + reached.GetError().Message());
-                    continue;
-                }
-                Add(combined, *reached);
+                Add(combined, *read.reached);
             }
-            tests.Append(
-                JsonValue::Object({ { "name", test.name },
-                                    { "outcome", OutcomeName(test.outcome) },
-                                    { "profile", profile },
-                                    { "files", ReachedJson(*reached) } }));
+            tests.Append(JsonValue::Object(
+                { { "name", read.test->name },
+                  { "outcome", OutcomeName(read.test->outcome) },
+                  { "profile", read.profile },
+                  { "files", ReachedJson(*read.reached) } }));
         }
 
         std::string lcovText;
-        if (llvm && !raw.empty())
+        if (tools.llvm && !raw.empty())
         {
             std::vector<std::string> arguments{ "merge", "-sparse" };
             arguments.insert(arguments.end(), raw.begin(), raw.end());
             const std::string all = (directory / "coverage.profdata").string();
             arguments.emplace_back("-o");
             arguments.push_back(all);
-            auto merged = RunTool(*profdata, arguments);
-            auto exported = merged ? RunTool(*cov,
+            auto merged = RunTool(*tools.profdata, arguments);
+            auto exported = merged ? RunTool(*tools.cov,
                                              { "export",
                                                "-format=lcov",
                                                "-instr-profile=" + all,
@@ -557,8 +632,8 @@ namespace Progmasoft::Catch3::Runner
             }
             else
             {
-                failures.push_back("the combined report: "
-                                   + exported.GetError().Message());
+                reports.failures.push_back("the combined report: "
+                                           + exported.GetError().Message());
             }
         }
         else
@@ -568,28 +643,42 @@ namespace Progmasoft::Catch3::Runner
 
         JsonValue report
             = JsonValue::Object({ { "version", 1 },
-                                  { "format", llvm ? "llvm" : "gcov" },
+                                  { "format", tools.llvm ? "llvm" : "gcov" },
                                   { "binary", program },
                                   { "tests", std::move(tests) } });
         const auto json = directory / "coverage.json";
         const auto lcov = directory / "coverage.lcov";
-        if (auto status = WriteFileAtomically(json, report.ToPrettyString());
-            !status)
+        const auto jsonWritten
+            = WriteFileAtomically(json, report.ToPrettyString());
+        if (!jsonWritten)
+        {
+            reports.failures.push_back(jsonWritten.GetError().Message());
+        }
+        const auto lcovWritten = WriteFileAtomically(lcov, lcovText);
+        if (!lcovWritten)
+        {
+            reports.failures.push_back(lcovWritten.GetError().Message());
+        }
+        std::string written;
+        if (jsonWritten && lcovWritten)
+        {
+            written = "; what each test reached is in " + json.string()
+                      + ", the run combined in " + lcov.string();
+        }
+        else if (jsonWritten)
         {
-            failures.push_back(status.GetError().Message());
+            written = "; what each test reached is in " + json.string();
         }
-        if (auto status = WriteFileAtomically(lcov, lcovText); !status)
+        else if (lcovWritten)
         {
-            failures.push_back(status.GetError().Message());
+            written = "; the run combined is in " + lcov.string();
         }
-        notes.push_back(
-            "--coverage: " + std::to_string(llvm ? llvmProfiles : gcovProfiles)
-            + " test profile(s); what each test reached is in " + json.string()
-            + ", the run combined in " + lcov.string());
-        for (const auto &failure : failures)
+        reports.notes.push_back("--coverage: " + std::to_string(profiled.size())
+                                + " test profile(s)" + written);
+        for (auto &failure : reports.failures)
         {
-            notes.push_back("--coverage: " + failure);
+            failure = "--coverage: " + failure;
         }
-        return notes;
+        return reports;
     }
 } // namespace Progmasoft::Catch3::Runner
diff --git a/src/Progmasoft/Runner/CoverageReport.hpp b/src/Progmasoft/Runner/CoverageReport.hpp
index 35f9dc75..35c3b1e5 100644
--- a/src/Progmasoft/Runner/CoverageReport.hpp
+++ b/src/Progmasoft/Runner/CoverageReport.hpp
@@ -38,21 +38,42 @@ namespace Progmasoft::Catch3::Runner
     [[nodiscard]] std::string
     ToLcov(const CoverageMap &map);
 
-    /// Prepares the directory of a coverage run: the profiles of an earlier
-    /// run below it are removed, because gcov adds to the .gcda files it
-    /// finds instead of replacing them.
+    /// Prepares the directory of a coverage run: the profiles and reports of
+    /// an earlier run below it are removed, because gcov adds to the .gcda
+    /// files it finds instead of replacing them, and because a run that
+    /// writes no reports must not leave the earlier ones to be read as its
+    /// own.
     Status
     PrepareCoverageDirectory(const std::filesystem::path &directory);
 
+    /// The last attempt of `test` whose profile, its base name followed by
+    /// `suffix` (`.profraw` or `.gcov`), exists: an attempt that crashed or
+    /// timed out before writing one is passed over for an earlier one.
+    [[nodiscard]] const Attempt *
+    ProfiledAttempt(const TestResult &test, const std::string &suffix);
+
+    /// What writing the coverage reports gave.
+    struct CoverageReports final
+    {
+        /// What the run should be told: how many profiles there were and
+        /// where the reports are, or why there are none.
+        std::vector<std::string> notes;
+        /// What failed: a profile that could not be read, a report that
+        /// could not be written. A run with failures is missing a report.
+        std::vector<std::string> failures;
+    };
+
     /// Turns the profiles a coverage run left in `directory` into
     /// `directory/coverage.json`, the lines and branches each test reached,
     /// and `directory/coverage.lcov`, the run combined. LLVM profiles are
     /// read with `llvm-profdata` and `llvm-cov`, gcov data with `gcov`,
     /// found on PATH or named by CATCH3_LLVM_PROFDATA, CATCH3_LLVM_COV and
-    /// CATCH3_GCOV. Returns what the run should be told: how many profiles
-    /// there were and where the reports are, or why there are none.
-    [[nodiscard]] std::vector<std::string>
+    /// CATCH3_GCOV; the profiles of the tests are read up to `jobs` at a
+    /// time. A run whose tests wrote no profiles, or whose tools are
+    /// missing, gets a note and no reports, not a failure.
+    [[nodiscard]] CoverageReports
     WriteCoverageReports(const std::filesystem::path &directory,
                          const RunResult &run,
-                         const std::string &program);
+                         const std::string &program,
+                         std::size_t jobs);
 } // namespace Progmasoft::Catch3::Runner
diff --git a/src/Progmasoft/Runner/Runner.cpp b/src/Progmasoft/Runner/Runner.cpp
index 61f8f2b7..5667f65e 100644
--- a/src/Progmasoft/Runner/Runner.cpp
+++ b/src/Progmasoft/Runner/Runner.cpp
@@ -995,13 +995,25 @@ namespace Progmasoft::Catch3
                 run.tests.push_back(std::move(kept));
             }
             run.notes = std::move(outcome.notes);
+            bool coverageFailed = false;
             if (!options.coverageDirectory.empty())
             {
-                for (auto &note : Runner::WriteCoverageReports(
-                         options.coverageDirectory, run, program.string()))
+                auto coverage
+                    = Runner::WriteCoverageReports(options.coverageDirectory,
+                                                   run,
+                                                   program.string(),
+                                                   run.workers);
+                for (auto &note : coverage.notes)
                 {
                     run.notes.push_back(std::move(note));
                 }
+                // Like a report that could not be written, a coverage
+                // failure makes the run fail when its tests passed.
+                coverageFailed = !coverage.failures.empty();
+                for (auto &failure : coverage.failures)
+                {
+                    run.notes.push_back(std::move(failure));
+                }
             }
 
             const bool anyFailure
@@ -1023,7 +1035,8 @@ namespace Progmasoft::Catch3
             Runner::Analyze(run, analysis);
             console.RunEnded(run);
 
-            const bool reportMissing = !WriteReports(options, run);
+            const bool reportMissing
+                = !WriteReports(options, run) || coverageFailed;
             if (options.historyEnabled)
             {
                 SaveHistory(historyFile,
diff --git a/tests/Progmasoft/CoverageFixture.cpp b/tests/Progmasoft/CoverageFixture.cpp
index a046a296..e728bbca 100644
--- a/tests/Progmasoft/CoverageFixture.cpp
+++ b/tests/Progmasoft/CoverageFixture.cpp
@@ -20,11 +20,22 @@ namespace
     {
         return value - 1; // coverage: subtract-one
     }
+
+    int
+    NonNegative(int value)
+    {
+        if (value < 0) // coverage: branch
+        {
+            return 0;
+        }
+        return value;
+    }
 } // namespace
 
 TEST_CASE("Adds one", "[coverage]")
 {
     CHECK(AddOne(1) == 2);
+    CHECK(NonNegative(AddOne(1)) == 2);
 }
 
 TEST_CASE("Subtracts one", "[coverage]")
diff --git a/tests/Progmasoft/RunnerCoverage.tests.cpp b/tests/Progmasoft/RunnerCoverage.tests.cpp
index 9dcca663..b3e10489 100644
--- a/tests/Progmasoft/RunnerCoverage.tests.cpp
+++ b/tests/Progmasoft/RunnerCoverage.tests.cpp
@@ -157,6 +157,58 @@ TEST_CASE("gcov JSON adds up across data files",
     CHECK(map.size() == 1);
 }
 
+TEST_CASE("A coverage run starts without the reports of an earlier one",
+          "[progmasoft][runner][coverage]")
+{
+    const Scratch scratch;
+    const auto directory = scratch.Path() / "coverage";
+    std::filesystem::create_directories(directory / "profiles");
+    for (const char *name :
+         { "coverage.json", "coverage.lcov", "coverage.profdata" })
+    {
+        REQUIRE(Runner::WriteFileAtomically(directory / name, "earlier"));
+    }
+    REQUIRE(Runner::WriteFileAtomically(directory / "profiles" / "t.profraw",
+                                        "earlier"));
+
+    REQUIRE(Runner::PrepareCoverageDirectory(directory));
+    CHECK_FALSE(std::filesystem::exists(directory / "coverage.json"));
+    CHECK_FALSE(std::filesystem::exists(directory / "coverage.lcov"));
+    CHECK_FALSE(std::filesystem::exists(directory / "coverage.profdata"));
+    CHECK_FALSE(std::filesystem::exists(directory / "profiles" / "t.profraw"));
+    CHECK(std::filesystem::is_directory(directory / "profiles"));
+
+    // With no profiles there is a note, no report and no failure.
+    Progmasoft::Catch3::RunResult run;
+    run.tests.emplace_back();
+    run.tests.back().name = "no profile";
+    const auto reports
+        = Runner::WriteCoverageReports(directory, run, "binary", 2);
+    CHECK(reports.failures.empty());
+    REQUIRE(reports.notes.size() == 1);
+    CHECK_THAT(reports.notes.front(), ContainsSubstring("wrote no coverage"));
+    CHECK_FALSE(std::filesystem::exists(directory / "coverage.json"));
+}
+
+TEST_CASE("An attempt that left no profile gives way to an earlier one",
+          "[progmasoft][runner][coverage]")
+{
+    const Scratch scratch;
+    const auto base = (scratch.Path() / "test").string();
+    Progmasoft::Catch3::TestResult test;
+    test.attempts.resize(3);
+    test.attempts[0].coverageProfile = base + "-1";
+    test.attempts[1].coverageProfile = base + "-2";
+    test.attempts[2].coverageProfile = base + "-3"; // crashed: no profile
+    REQUIRE(Runner::WriteFileAtomically(base + "-1.profraw", "1"));
+    REQUIRE(Runner::WriteFileAtomically(base + "-2.profraw", "2"));
+
+    const auto *attempt = Runner::ProfiledAttempt(test, ".profraw");
+    REQUIRE(attempt != nullptr);
+    CHECK(attempt == &test.attempts[1]);
+    CHECK(Runner::ProfiledAttempt(test, ".gcov") == nullptr);
+}
+
 TEST_CASE("Each test of a coverage run gets its own profile and map",
           "[progmasoft][runner][coverage]")
 {
@@ -223,4 +275,19 @@ TEST_CASE("Each test of a coverage run gets its own profile and map",
     REQUIRE(file != combined.end());
     CHECK(file->second.lines.at(addLine) > 0);
     CHECK(file->second.lines.at(subtractLine) > 0);
+
+    // Both formats carry branches: the condition of the fixture went one
+    // way only, so one of its branches was taken and one was not.
+    const auto branchLine = MarkedLine(source, "branch");
+    std::size_t taken = 0;
+    std::size_t notTaken = 0;
+    for (const auto &[position, count] : file->second.branches)
+    {
+        if (position.first == branchLine)
+        {
+            (count > 0 ? taken : notTaken) += 1;
+        }
+    }
+    CHECK(taken > 0);
+    CHECK(notTaken > 0);
 }