Read-only mirrors. Development, issues and pull requests are on GitHub.

summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authorLeitwolf112026-11-13 07:46:08 +0300
committerLeitwolf112026-11-13 07:46:08 +0300
commit17248837a78462596809f902dd47cf113a39670b (patch)
tree666d16ddc0a69acb805a5f32f17c8502208349e7
parent43dabc9adb45b808ae24173ff4633774aa71b2ee (diff)
downloadcatch3-feature/per-test-coverage.tar.gz
Keep per-test coverage with --coverage (roadmap step 4)feature/per-test-coverage
--coverage=DIR runs every attempt in a process of its own and keeps the coverage of each test, for binaries built with LLVM source-based coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov (GCC: --coverage). How it works: - The scheduler gives each attempt a fresh worker, as --isolate does, and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the process writes at exit is then the coverage of that one test, in DIR/profiles/t<index>-a<attempt>.profraw or a .gcov directory. Each runtime reads its own variable, so no symbol of either runtime is looked up: weak references would not pull them out of their static archives, and Mach-O does not allow undefined weak symbols. - A process that runs other tests first (bisection) writes beside the profiles, never into one. The attempt records its profile, also in the JSON results. - The profiles of an earlier run are removed when the run starts, because gcov adds to the .gcda files it finds. - After the run, CoverageReport reads the last profile of every test: llvm-profdata and llvm-cov (PATH, xcrun on macOS, or CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is DIR/coverage.json, the files, lines and branches each test reached, and DIR/coverage.lcov for the whole run. Without the tools the profiles stay and the run says so. - --coverage with --in-process or --break is refused. Tests: option parsing, LCOV reading and writing, gcov JSON, and an integration test on CoverageFixture, a binary CMake builds with the coverage flags of Clang or GCC: two tests reach different lines, each map holds only its own, the LCOV report holds both, and a stale profile of an earlier run is gone. Locally all 84 CTest tests pass with Clang, and the coverage tests pass with GCC (built with CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as errors, before and after this change). CI: the Linux jobs install LLVM's coverage tools, a new job checks the gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map into a failure on Linux and macOS instead of a skip. MSVC has no source-based coverage, so its builds skip those tests; the roadmap lists step 4 as in progress until Windows with clang-cl is tested.
-rw-r--r--.github/workflows/linux-builds.yml52
-rw-r--r--.github/workflows/mac-builds.yml2
-rw-r--r--docs/command-line.md1
-rw-r--r--docs/progmasoft-roadmap.md2
-rw-r--r--docs/progmasoft-runner.md60
-rw-r--r--premake5.lua1
-rw-r--r--src/CMakeLists.txt2
-rw-r--r--src/Progmasoft/Catch3/Results.hpp3
-rw-r--r--src/Progmasoft/Catch3/meson.build1
-rw-r--r--src/Progmasoft/Runner/CoverageReport.cpp594
-rw-r--r--src/Progmasoft/Runner/CoverageReport.hpp58
-rw-r--r--src/Progmasoft/Runner/Options.cpp20
-rw-r--r--src/Progmasoft/Runner/Options.hpp4
-rw-r--r--src/Progmasoft/Runner/ResultsJson.cpp2
-rw-r--r--src/Progmasoft/Runner/Runner.cpp20
-rw-r--r--src/Progmasoft/Runner/Scheduler.cpp61
-rw-r--r--src/Progmasoft/Runner/SchedulerModel.hpp4
-rw-r--r--tests/CMakeLists.txt22
-rw-r--r--tests/Progmasoft/BUILD.bazel1
-rw-r--r--tests/Progmasoft/CoverageFixture.cpp33
-rw-r--r--tests/Progmasoft/RunnerCoverage.tests.cpp226
-rw-r--r--tests/meson.build1
-rw-r--r--xmake.lua1
23 files changed, 1160 insertions, 11 deletions
diff --git a/.github/workflows/linux-builds.yml b/.github/workflows/linux-builds.yml
index a9c38c66..02ee8d67 100644
--- a/.github/workflows/linux-builds.yml
+++ b/.github/workflows/linux-builds.yml
@@ -32,10 +32,10 @@ jobs:
with:
persist-credentials: false
- - name: Install Clang, CMake and Ninja
+ - name: Install Clang, LLVM's coverage tools, CMake and Ninja
run: |
sudo apt-get update
- sudo apt-get install -y clang cmake ninja-build python3
+ sudo apt-get install -y clang llvm cmake ninja-build python3
- name: Configure
run: |
@@ -51,6 +51,8 @@ jobs:
- name: Test
run: ctest --test-dir build -j
+ env:
+ CATCH3_COVERAGE_REQUIRE_MAP: 1
- name: Install, and find the installed package
run: |
@@ -75,10 +77,10 @@ jobs:
with:
persist-credentials: false
- - name: Install Clang, CMake and Ninja
+ - name: Install Clang, LLVM's coverage tools, CMake and Ninja
run: |
sudo apt-get update
- sudo apt-get install -y clang cmake ninja-build python3
+ sudo apt-get install -y clang llvm cmake ninja-build python3
- name: Configure with the sanitizers
env:
@@ -111,8 +113,8 @@ jobs:
steps:
# Git comes first: without it the checkout falls back to an archive of
# the repository, which is not a checkout the tests can ask questions of.
- - name: Install Git, Clang, CMake and Ninja
- run: dnf install -y git clang gcc-c++ cmake ninja-build python3
+ - name: Install Git, Clang, LLVM's coverage tools, CMake and Ninja
+ run: dnf install -y git clang llvm gcc-c++ cmake ninja-build python3
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
@@ -132,3 +134,41 @@ jobs:
- name: Test
run: ctest --test-dir build -j
+ env:
+ CATCH3_COVERAGE_REQUIRE_MAP: 1
+
+ # Catch3 is built with Clang everywhere; GCC's stricter warnings are not
+ # errors here. The job checks per-test coverage in gcov's format, which
+ # only GCC writes.
+ gcc-coverage:
+ name: Ubuntu 26.04, GCC, per-test coverage with gcov
+ runs-on: ubuntu-26.04
+ timeout-minutes: 30
+
+ steps:
+ - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
+ with:
+ persist-credentials: false
+
+ - name: Install GCC, CMake and Ninja
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y g++ cmake ninja-build python3
+
+ - name: Configure
+ run: |
+ cmake --preset basic-tests -GNinja \
+ -DCMAKE_BUILD_TYPE=Debug \
+ -DCMAKE_CXX_COMPILER=g++ \
+ -DCMAKE_CXX_STANDARD=17 \
+ -DCATCH3_ENABLE_WERROR=OFF
+
+ - name: Build
+ run: cmake --build build --target ProgmasoftSelfTest
+
+ - name: Test per-test coverage
+ env:
+ CATCH3_RUNNER_FIXTURE: build/tests/RunnerFixture
+ CATCH3_COVERAGE_FIXTURE: build/tests/CoverageFixture
+ CATCH3_COVERAGE_REQUIRE_MAP: 1
+ run: build/tests/ProgmasoftSelfTest --no-history "[coverage]"
diff --git a/.github/workflows/mac-builds.yml b/.github/workflows/mac-builds.yml
index c115bf00..4044db9b 100644
--- a/.github/workflows/mac-builds.yml
+++ b/.github/workflows/mac-builds.yml
@@ -44,3 +44,5 @@ jobs:
- name: Test
run: ctest --test-dir build -j
+ env:
+ CATCH3_COVERAGE_REQUIRE_MAP: 1
diff --git a/docs/command-line.md b/docs/command-line.md
index 236128e1..ccca35ff 100644
--- a/docs/command-line.md
+++ b/docs/command-line.md
@@ -120,6 +120,7 @@ Without a spec, every test runs except hidden ones, whose tags start with
| `--junit=FILE` | write a JUnit XML report |
| `--json=FILE` | write a JSON report with every attempt and its analysis |
| `--trace=FILE` | write a trace for `chrome://tracing` or Perfetto |
+| `--coverage=DIR` | run every test in a process of its own and keep its coverage in DIR, with a map of what each test reached; see [Per-test coverage](progmasoft-runner.md#per-test-coverage) |
| `--slowest[=N]` | list the N slowest tests (default 10) |
| `--premature-exit-guard-file=FILE` | create FILE, private to the user, when the run starts and remove it when the run ends normally |
diff --git a/docs/progmasoft-roadmap.md b/docs/progmasoft-roadmap.md
index 81a8956f..1f61be79 100644
--- a/docs/progmasoft-roadmap.md
+++ b/docs/progmasoft-roadmap.md
@@ -22,7 +22,7 @@ The steps are ordered by what each one builds on.
| 1 | Native property testing: choice-sequence shrinking, combinators, state machines, statistics, targeted search, example database, differential and metamorphic oracles | done; see [Native property testing](progmasoft-properties.md#top) |
| 2 | The Catch3 runner | done; see [The Catch3 runner](progmasoft-runner.md#top) |
| 3 | Mocking | done; see [Mocking](progmasoft-mocking.md#top) |
-| 4 | Per-test code coverage | planned |
+| 4 | Per-test code coverage | in progress: Linux and macOS, with Clang and GCC; see [Per-test coverage](progmasoft-runner.md#per-test-coverage). Windows with clang-cl is next |
| 5 | Change-impact test selection and test prioritization | planned |
| 6 | Mutation testing | planned |
| 7 | Coverage-, mutation- and history-guided test generation | planned |
diff --git a/docs/progmasoft-runner.md b/docs/progmasoft-runner.md
index 19d1e06d..cb5f29c5 100644
--- a/docs/progmasoft-runner.md
+++ b/docs/progmasoft-runner.md
@@ -12,6 +12,7 @@
[Temporary files](#temporary-files)<br>
[What a failure report says](#what-a-failure-report-says)<br>
[Reports and traces](#reports-and-traces)<br>
+[Per-test coverage](#per-test-coverage)<br>
[Formats other tools read](#formats-other-tools-read)<br>
[Project configuration](#project-configuration)<br>
[Bazel and CTest](#bazel-and-ctest)<br>
@@ -332,6 +333,65 @@ replay line runs the failure again as it happened.
attempt and section.
+## Per-test coverage
+
+`--coverage=DIR` runs every test in a process of its own and keeps what each
+one reached. Build the tests with coverage instrumentation:
+
+* Clang: `-fprofile-instr-generate -fcoverage-mapping`, LLVM source-based
+ coverage;
+* GCC: `--coverage`, gcov's format.
+
+```text
+$ ./tests --coverage=coverage
+Catch3: 120 test(s), up to 8 worker(s), seed 2153251184
+note: --coverage: 120 test profile(s); what each test reached is in coverage/coverage.json, the run combined in coverage/coverage.lcov
+
+120 test(s): 120 passed
+```
+
+Each attempt gets a process of its own, so the profile its process writes at
+exit is the coverage of that one test: `DIR/profiles/t<index>-a<attempt>`
+with `.profraw` for LLVM, or a `.gcov` directory below which gcov writes its
+`.gcda` files under their absolute paths. The runner sets `LLVM_PROFILE_FILE`
+and `GCOV_PREFIX` for the process; each runtime reads its own. A process
+that runs other tests first, to find a test that leaves state behind, writes
+beside the profiles, never into one. The profiles of an earlier run in
+`DIR/profiles` are removed when the run starts, because gcov adds to the
+data it finds.
+
+After the run, the runner reads the profile of the last attempt of every
+test:
+
+* `DIR/coverage.json` holds, for each test, its outcome, its profile and the
+ files it reached, with the lines it ran and, per line, how many of its
+ branches it took:
+
+ ```json
+ { "version": 1, "format": "llvm", "binary": "./tests",
+ "tests": [ { "name": "Adds one", "outcome": "passed",
+ "profile": "coverage/profiles/t0-a1.profraw",
+ "files": [ { "file": "/src/calc.cpp", "lines": [ 12, 13 ],
+ "branches": [ { "line": 12, "taken": 1, "total": 2 } ] } ] } ] }
+ ```
+
+* `DIR/coverage.lcov` is the coverage of the whole run as an LCOV tracefile,
+ which `genhtml`, Codecov and editors read.
+
+LLVM profiles are read with `llvm-profdata` and `llvm-cov`, gcov data with
+`gcov`. They are looked up on `PATH`, through `xcrun` on macOS, or named by
+`CATCH3_LLVM_PROFDATA`, `CATCH3_LLVM_COV` and `CATCH3_GCOV`. Without them the
+profiles stay in `DIR/profiles` and the run says so. A test that crashes or
+is stopped at its time limit leaves no profile. The runner's own process is
+instrumented too and writes its profile where any instrumented program does,
+`default.profraw` in the working directory unless `LLVM_PROFILE_FILE` says
+otherwise; it holds no test.
+
+`--coverage` cannot be used with `--in-process` or `--break`, which run the
+tests in one process. Per-test coverage is tested on Linux and macOS; Windows
+builds with clang-cl are next.
+
+
## Formats other tools read
These formats carry a `format` and a `version`. Within a version they only
diff --git a/premake5.lua b/premake5.lua
index d4c55c9e..def0463c 100644
--- a/premake5.lua
+++ b/premake5.lua
@@ -92,6 +92,7 @@ project "ProgmasoftPropertyTests"
"tests/Progmasoft/RunnerConfig.tests.cpp",
"tests/Progmasoft/RunnerAnalysis.tests.cpp",
"tests/Progmasoft/RunnerImport.tests.cpp",
+ "tests/Progmasoft/RunnerCoverage.tests.cpp",
"tests/Progmasoft/RunnerOutput.tests.cpp",
"tests/Progmasoft/Snapshot.tests.cpp",
"tests/Progmasoft/Stateful.tests.cpp",
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 43754fff..fac2fb62 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -301,6 +301,7 @@ set(PROGMASOFT_RUNNER_FILES
"${CATCH_DIR}/src/Progmasoft/Runner/Execution.hpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Expected.hpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Files.hpp"
+ "${CATCH_DIR}/src/Progmasoft/Runner/CoverageReport.hpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Git.hpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Hash.hpp"
"${CATCH_DIR}/src/Progmasoft/Runner/History.hpp"
@@ -320,6 +321,7 @@ set(PROGMASOFT_RUNNER_FILES
"${CATCH_DIR}/src/Progmasoft/Runner/Console.cpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Execution.cpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Files.cpp"
+ "${CATCH_DIR}/src/Progmasoft/Runner/CoverageReport.cpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Git.cpp"
"${CATCH_DIR}/src/Progmasoft/Runner/Hash.cpp"
"${CATCH_DIR}/src/Progmasoft/Runner/History.cpp"
diff --git a/src/Progmasoft/Catch3/Results.hpp b/src/Progmasoft/Catch3/Results.hpp
index 4c6bb1ce..bff2a72e 100644
--- a/src/Progmasoft/Catch3/Results.hpp
+++ b/src/Progmasoft/Catch3/Results.hpp
@@ -153,6 +153,9 @@ namespace Progmasoft::Catch3
/// The directory the test had for its temporary files; kept when the
/// attempt failed, so its contents can be inspected.
std::string temporaryDirectory;
+ /// In a coverage run, where the profile of this attempt went,
+ /// without its extension (`.profraw` or `.gcov`).
+ std::string coverageProfile;
};
/// One test of the run and everything the runner learned about it.
diff --git a/src/Progmasoft/Catch3/meson.build b/src/Progmasoft/Catch3/meson.build
index 8420003b..4bbab28e 100644
--- a/src/Progmasoft/Catch3/meson.build
+++ b/src/Progmasoft/Catch3/meson.build
@@ -288,6 +288,7 @@ runner_sources = files(
'../Runner/Console.cpp',
'../Runner/Execution.cpp',
'../Runner/Files.cpp',
+ '../Runner/CoverageReport.cpp',
'../Runner/Git.cpp',
'../Runner/Hash.cpp',
'../Runner/History.cpp',
diff --git a/src/Progmasoft/Runner/CoverageReport.cpp b/src/Progmasoft/Runner/CoverageReport.cpp
new file mode 100644
index 00000000..c443c158
--- /dev/null
+++ b/src/Progmasoft/Runner/CoverageReport.cpp
@@ -0,0 +1,594 @@
+// Copyright (c) 2026 Progmasoft.
+// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0
+#include <algorithm>
+#include <cstdlib>
+#include <optional>
+#include <string_view>
+#include <system_error>
+
+#include "CoverageReport.hpp"
+#include "Files.hpp"
+#include "Json.hpp"
+#include "Process.hpp"
+
+namespace Progmasoft::Catch3::Runner
+{
+ namespace
+ {
+ /// Reads a non-negative decimal number; nothing for anything else,
+ /// such as the `-` LCOV writes for a branch never reached.
+ std::optional<std::uint64_t>
+ Number(std::string_view text)
+ {
+ if (text.empty() || text.size() > 19)
+ {
+ return std::nullopt;
+ }
+ std::uint64_t value = 0;
+ for (const char character : text)
+ {
+ if (character < '0' || character > '9')
+ {
+ return std::nullopt;
+ }
+ value
+ = value * 10 + static_cast<std::uint64_t>(character - '0');
+ }
+ return value;
+ }
+
+ std::vector<std::string_view>
+ Split(std::string_view text, char separator)
+ {
+ std::vector<std::string_view> parts;
+ std::size_t start = 0;
+ while (true)
+ {
+ const auto end = text.find(separator, start);
+ parts.push_back(text.substr(start, end - start));
+ if (end == std::string_view::npos)
+ {
+ return parts;
+ }
+ start = end + 1;
+ }
+ }
+
+ /// A tool of a coverage format: the program, or the command that
+ /// runs it, such as `xcrun llvm-cov` on macOS.
+ using Tool = std::vector<std::string>;
+
+ std::optional<Tool>
+ FindTool(const char *variable, const std::string &name)
+ {
+ std::vector<Tool> candidates;
+ if (auto configured = ReadEnvironment(variable);
+ !configured.empty())
+ {
+ candidates.push_back({ std::move(configured) });
+ }
+ candidates.push_back({ name });
+#if defined(__APPLE__)
+ candidates.push_back({ "xcrun", name });
+#endif
+ for (auto &candidate : candidates)
+ {
+ ChildOptions options;
+ options.arguments = candidate;
+ options.arguments.emplace_back("--version");
+ options.searchPath = true;
+ if (auto run = RunAndCapture(options);
+ run && run->status.Succeeded())
+ {
+ return std::move(candidate);
+ }
+ }
+ return std::nullopt;
+ }
+
+ /// Runs `tool` with `arguments`; its output, or why it failed.
+ Expected<std::string>
+ RunTool(const Tool &tool,
+ const std::vector<std::string> &arguments,
+ const std::filesystem::path &directory = {})
+ {
+ ChildOptions options;
+ options.arguments = tool;
+ options.arguments.insert(options.arguments.end(),
+ arguments.begin(),
+ arguments.end());
+ options.searchPath = true;
+ options.workingDirectory = directory;
+ auto run = RunAndCapture(options);
+ if (!run)
+ {
+ return run.GetError();
+ }
+ if (!run->status.Succeeded())
+ {
+ std::string message = run->errors;
+ while (!message.empty()
+ && (message.back() == '\n' || message.back() == '\r'))
+ {
+ message.pop_back();
+ }
+ return Error{ tool.back() + " failed: " + message };
+ }
+ return std::move(run->output);
+ }
+
+ void
+ Add(CoverageMap &total, const CoverageMap &part)
+ {
+ for (const auto &[file, coverage] : part)
+ {
+ FileCoverage &into = total[file];
+ for (const auto &[line, count] : coverage.lines)
+ {
+ into.lines[line] += count;
+ }
+ for (const auto &[branch, count] : coverage.branches)
+ {
+ into.branches[branch] += count;
+ }
+ }
+ }
+
+ /// What one test reached: the lines it ran and, per line, how many
+ /// of its branches it took.
+ JsonValue
+ ReachedJson(const CoverageMap &map)
+ {
+ JsonValue files = JsonValue::Array();
+ for (const auto &[file, coverage] : map)
+ {
+ JsonValue lines = JsonValue::Array();
+ for (const auto &[line, count] : coverage.lines)
+ {
+ if (count > 0)
+ {
+ lines.Append(line);
+ }
+ }
+ JsonValue branches = JsonValue::Array();
+ std::map<std::size_t, std::pair<std::size_t, std::size_t>>
+ perLine;
+ for (const auto &[branch, count] : coverage.branches)
+ {
+ auto &[taken, total] = perLine[branch.first];
+ ++total;
+ if (count > 0)
+ {
+ ++taken;
+ }
+ }
+ for (const auto &[line, counts] : perLine)
+ {
+ if (counts.first == 0)
+ {
+ continue;
+ }
+ branches.Append(
+ JsonValue::Object({ { "line", line },
+ { "taken", counts.first },
+ { "total", counts.second } }));
+ }
+ if (lines.Items().empty())
+ {
+ continue;
+ }
+ files.Append(
+ JsonValue::Object({ { "file", file },
+ { "lines", std::move(lines) },
+ { "branches", std::move(branches) } }));
+ }
+ return files;
+ }
+
+ /// The final attempt of a test that left a profile.
+ const Attempt *
+ ProfiledAttempt(const TestResult &test)
+ {
+ for (auto attempt = test.attempts.rbegin();
+ attempt != test.attempts.rend();
+ ++attempt)
+ {
+ if (!attempt->coverageProfile.empty())
+ {
+ return &*attempt;
+ }
+ }
+ return nullptr;
+ }
+
+ /// The .gcda files gcov wrote below `prefix`, each with the path the
+ /// object was built at, where its .gcno is.
+ std::vector<std::pair<std::filesystem::path, std::filesystem::path>>
+ GcdaFiles(const std::filesystem::path &prefix)
+ {
+ std::vector<std::pair<std::filesystem::path, std::filesystem::path>>
+ files;
+ std::error_code error;
+ for (auto entry
+ = std::filesystem::recursive_directory_iterator(prefix, error);
+ !error
+ && entry != std::filesystem::recursive_directory_iterator();
+ entry.increment(error))
+ {
+ if (!entry->is_regular_file(error)
+ || entry->path().extension() != ".gcda")
+ {
+ continue;
+ }
+ // GCOV_PREFIX is put before the absolute path of the object.
+ const auto relative
+ = std::filesystem::relative(entry->path(), prefix, error);
+ std::filesystem::path original
+ = std::filesystem::path{ "/" } / relative;
+ files.emplace_back(entry->path(), original);
+ }
+ std::sort(files.begin(), files.end());
+ return files;
+ }
+
+ Expected<CoverageMap>
+ GcovCoverage(const Tool &gcov, const std::filesystem::path &prefix)
+ {
+ CoverageMap map;
+ for (const auto &[gcda, original] : GcdaFiles(prefix))
+ {
+ auto gcno = original;
+ gcno.replace_extension(".gcno");
+ auto beside = gcda;
+ beside.replace_extension(".gcno");
+ std::error_code error;
+ std::filesystem::copy_file(
+ gcno,
+ beside,
+ std::filesystem::copy_options::overwrite_existing,
+ error);
+ if (error)
+ {
+ return Error{ "the notes file " + gcno.string()
+ + " could not be read: " + error.message() };
+ }
+ auto json = RunTool(
+ gcov,
+ { "--json-format", "--stdout", gcda.filename().string() },
+ gcda.parent_path());
+ if (!json)
+ {
+ return json.GetError();
+ }
+ // One JSON document per data file, one per line.
+ for (const auto line : Split(*json, '\n'))
+ {
+ if (!line.empty())
+ {
+ AddGcovJson(std::string{ line }, map);
+ }
+ }
+ }
+ return map;
+ }
+ } // namespace
+
+ CoverageMap
+ ParseLcov(const std::string &text)
+ {
+ CoverageMap map;
+ FileCoverage *current = nullptr;
+ for (auto line : Split(text, '\n'))
+ {
+ if (!line.empty() && line.back() == '\r')
+ {
+ line.remove_suffix(1);
+ }
+ if (line.substr(0, 3) == "SF:")
+ {
+ current = &map[std::string{ line.substr(3) }];
+ }
+ else if (line == "end_of_record")
+ {
+ current = nullptr;
+ }
+ else if (current != nullptr && line.substr(0, 3) == "DA:")
+ {
+ const auto fields = Split(line.substr(3), ',');
+ const auto number = Number(fields[0]);
+ const auto count
+ = fields.size() > 1 ? Number(fields[1]) : std::nullopt;
+ if (number && count)
+ {
+ current->lines[*number] += *count;
+ }
+ }
+ else if (current != nullptr && line.substr(0, 5) == "BRDA:")
+ {
+ const auto fields = Split(line.substr(5), ',');
+ if (fields.size() != 4)
+ {
+ continue;
+ }
+ const auto number = Number(fields[0]);
+ if (!number)
+ {
+ continue;
+ }
+ // The branches of a line are numbered in the order they come;
+ // `-` counts as never taken.
+ std::size_t index = 0;
+ for (auto it = current->branches.lower_bound({ *number, 0 });
+ it != current->branches.end()
+ && it->first.first == *number;
+ ++it)
+ {
+ ++index;
+ }
+ current->branches[{ *number, index }]
+ += Number(fields[3]).value_or(0);
+ }
+ }
+ return map;
+ }
+
+ void
+ AddGcovJson(const std::string &text, CoverageMap &map)
+ {
+ auto document = ParseJson(text);
+ if (!document)
+ {
+ return;
+ }
+ for (const auto &file : (*document)["files"].Items())
+ {
+ FileCoverage &coverage = map[file["file"].AsString()];
+ for (const auto &line : file["lines"].Items())
+ {
+ const auto number = static_cast<std::size_t>(
+ line["line_number"].AsUnsigned());
+ coverage.lines[number] += line["count"].AsUnsigned();
+ std::size_t index = 0;
+ for (const auto &branch : line["branches"].Items())
+ {
+ coverage.branches[{ number, index++ }]
+ += branch["count"].AsUnsigned();
+ }
+ }
+ }
+ }
+
+ std::string
+ ToLcov(const CoverageMap &map)
+ {
+ std::string text;
+ for (const auto &[file, coverage] : map)
+ {
+ text += "SF:" + file + "\n";
+ std::size_t linesHit = 0;
+ for (const auto &[line, count] : coverage.lines)
+ {
+ text += "DA:" + std::to_string(line) + ","
+ + std::to_string(count) + "\n";
+ linesHit += count > 0 ? 1 : 0;
+ }
+ std::size_t branchesHit = 0;
+ for (const auto &[branch, count] : coverage.branches)
+ {
+ text += "BRDA:" + std::to_string(branch.first) + ",0,"
+ + std::to_string(branch.second) + ","
+ + (count > 0 ? std::to_string(count) : "-") + "\n";
+ branchesHit += count > 0 ? 1 : 0;
+ }
+ text += "LF:" + std::to_string(coverage.lines.size()) + "\n";
+ text += "LH:" + std::to_string(linesHit) + "\n";
+ text += "BRF:" + std::to_string(coverage.branches.size()) + "\n";
+ text += "BRH:" + std::to_string(branchesHit) + "\n";
+ text += "end_of_record\n";
+ }
+ return text;
+ }
+
+ Status
+ PrepareCoverageDirectory(const std::filesystem::path &directory)
+ {
+ std::error_code error;
+ std::filesystem::remove_all(directory / "profiles", error);
+ if (error)
+ {
+ return Error{ "the profiles of an earlier run in "
+ + directory.string()
+ + " could not be removed: " + error.message() };
+ }
+ std::filesystem::create_directories(directory / "profiles", error);
+ if (error)
+ {
+ return Error{ "the coverage directory " + directory.string()
+ + " could not be created: " + error.message() };
+ }
+ return Status::Ok();
+ }
+
+ std::vector<std::string>
+ WriteCoverageReports(const std::filesystem::path &directory,
+ const RunResult &run,
+ const std::string &program)
+ {
+ std::vector<std::string> notes;
+ std::size_t llvmProfiles = 0;
+ std::size_t gcovProfiles = 0;
+ for (const auto &test : run.tests)
+ {
+ if (const Attempt *attempt = ProfiledAttempt(test))
+ {
+ std::error_code error;
+ llvmProfiles += std::filesystem::exists(attempt->coverageProfile
+ + ".profraw",
+ error)
+ ? 1
+ : 0;
+ gcovProfiles += std::filesystem::exists(attempt->coverageProfile
+ + ".gcov",
+