Read-only mirrors. Development, issues and pull requests are on GitHub.

summaryrefslogtreecommitdiffhomepage
path: root/src/Progmasoft/Runner/CoverageReport.hpp
diff options
context:
space:
mode:
authorLeitwolf112026-11-13 07:46:08 +0300
committerLeitwolf112026-11-13 07:46:08 +0300
commit17248837a78462596809f902dd47cf113a39670b (patch)
tree666d16ddc0a69acb805a5f32f17c8502208349e7 /src/Progmasoft/Runner/CoverageReport.hpp
parent43dabc9adb45b808ae24173ff4633774aa71b2ee (diff)
downloadcatch3-feature/per-test-coverage.tar.gz
Keep per-test coverage with --coverage (roadmap step 4)feature/per-test-coverage
--coverage=DIR runs every attempt in a process of its own and keeps the coverage of each test, for binaries built with LLVM source-based coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov (GCC: --coverage). How it works: - The scheduler gives each attempt a fresh worker, as --isolate does, and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the process writes at exit is then the coverage of that one test, in DIR/profiles/t<index>-a<attempt>.profraw or a .gcov directory. Each runtime reads its own variable, so no symbol of either runtime is looked up: weak references would not pull them out of their static archives, and Mach-O does not allow undefined weak symbols. - A process that runs other tests first (bisection) writes beside the profiles, never into one. The attempt records its profile, also in the JSON results. - The profiles of an earlier run are removed when the run starts, because gcov adds to the .gcda files it finds. - After the run, CoverageReport reads the last profile of every test: llvm-profdata and llvm-cov (PATH, xcrun on macOS, or CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is DIR/coverage.json, the files, lines and branches each test reached, and DIR/coverage.lcov for the whole run. Without the tools the profiles stay and the run says so. - --coverage with --in-process or --break is refused. Tests: option parsing, LCOV reading and writing, gcov JSON, and an integration test on CoverageFixture, a binary CMake builds with the coverage flags of Clang or GCC: two tests reach different lines, each map holds only its own, the LCOV report holds both, and a stale profile of an earlier run is gone. Locally all 84 CTest tests pass with Clang, and the coverage tests pass with GCC (built with CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as errors, before and after this change). CI: the Linux jobs install LLVM's coverage tools, a new job checks the gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map into a failure on Linux and macOS instead of a skip. MSVC has no source-based coverage, so its builds skip those tests; the roadmap lists step 4 as in progress until Windows with clang-cl is tested.
Diffstat (limited to 'src/Progmasoft/Runner/CoverageReport.hpp')
-rw-r--r--src/Progmasoft/Runner/CoverageReport.hpp58
1 files changed, 58 insertions, 0 deletions
diff --git a/src/Progmasoft/Runner/CoverageReport.hpp b/src/Progmasoft/Runner/CoverageReport.hpp
new file mode 100644
index 00000000..35f9dc75
--- /dev/null
+++ b/src/Progmasoft/Runner/CoverageReport.hpp
@@ -0,0 +1,58 @@
+// Copyright (c) 2026 Progmasoft.
+// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0
+#pragma once
+
+#include <Progmasoft/Catch3/Results.hpp>
+#include <cstdint>
+#include <filesystem>
+#include <map>
+#include <string>
+#include <utility>
+#include <vector>
+
+#include "Expected.hpp"
+
+namespace Progmasoft::Catch3::Runner
+{
+ /// What one test, or the whole run, reached in one source file.
+ struct FileCoverage final
+ {
+ /// Execution count of each instrumented line.
+ std::map<std::size_t, std::uint64_t> lines;
+ /// Count of each branch, by line and by its position on the line.
+ std::map<std::pair<std::size_t, std::size_t>, std::uint64_t> branches;
+ };
+
+ /// Coverage by source file.
+ using CoverageMap = std::map<std::string, FileCoverage>;
+
+ /// Reads an LCOV tracefile: `SF`, `DA` and `BRDA` records.
+ [[nodiscard]] CoverageMap
+ ParseLcov(const std::string &text);
+
+ /// Reads the JSON that `gcov --json-format` writes, adding to `map`.
+ void
+ AddGcovJson(const std::string &text, CoverageMap &map);
+
+ /// Writes `map` as an LCOV tracefile.
+ [[nodiscard]] std::string
+ ToLcov(const CoverageMap &map);
+
+ /// Prepares the directory of a coverage run: the profiles of an earlier
+ /// run below it are removed, because gcov adds to the .gcda files it
+ /// finds instead of replacing them.
+ Status
+ PrepareCoverageDirectory(const std::filesystem::path &directory);
+
+ /// Turns the profiles a coverage run left in `directory` into
+ /// `directory/coverage.json`, the lines and branches each test reached,
+ /// and `directory/coverage.lcov`, the run combined. LLVM profiles are
+ /// read with `llvm-profdata` and `llvm-cov`, gcov data with `gcov`,
+ /// found on PATH or named by CATCH3_LLVM_PROFDATA, CATCH3_LLVM_COV and
+ /// CATCH3_GCOV. Returns what the run should be told: how many profiles
+ /// there were and where the reports are, or why there are none.
+ [[nodiscard]] std::vector<std::string>
+ WriteCoverageReports(const std::filesystem::path &directory,
+ const RunResult &run,
+ const std::string &program);
+} // namespace Progmasoft::Catch3::Runner