<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/src/Progmasoft/Runner/CoverageReport.cpp, branch fix/version-tool-semver</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Start child processes one at a time; run gcov without a directory</title>
<updated>2026-11-13T20:24:10+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T20:24:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=b26486d68ece48a1dfdb208a9f3f1bd8acab0382'/>
<id>b26486d68ece48a1dfdb208a9f3f1bd8acab0382</id>
<content type='text'>
Reading the coverage profiles on several threads broke an assumption of
ChildProcess::Start on POSIX: it changes the working directory of the
whole process around posix_spawn. Two gcov runs started at once could
both run in one test's directory, so on the GCC job the map of
"Subtracts one" showed what "Adds one" reached.

- ChildProcess::Start holds a lock while a child starts, on POSIX and
  Windows, so the directory change and the handles a child inherits never
  overlap with another start.
- gcov gets the absolute path of the data file and no working directory,
  so no directory changes while profiles are read in parallel.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Reading the coverage profiles on several threads broke an assumption of
ChildProcess::Start on POSIX: it changes the working directory of the
whole process around posix_spawn. Two gcov runs started at once could
both run in one test's directory, so on the GCC job the map of
"Subtracts one" showed what "Adds one" reached.

- ChildProcess::Start holds a lock while a child starts, on POSIX and
  Windows, so the directory change and the handles a child inherits never
  overlap with another start.
- gcov gets the absolute path of the data file and no working directory,
  so no directory changes while profiles are read in parallel.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix the coverage review findings: branches, stale reports, exit status</title>
<updated>2026-11-13T18:55:26+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T18:55:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=454cca9b63991416f4d9bd1ca350f1ae14e278e7'/>
<id>454cca9b63991416f4d9bd1ca350f1ae14e278e7</id>
<content type='text'>
- gcov is run with --branch-probabilities; without it the JSON has no
  branches, so GCC runs had none in coverage.json and coverage.lcov.
- PrepareCoverageDirectory also removes coverage.json, coverage.lcov and
  coverage.profdata of an earlier run, so a run that writes no reports
  leaves none to be read as its own.
- WriteCoverageReports returns notes and failures apart. A profile the
  tools cannot read or a report that cannot be written counts as a missing
  report and fails a run whose tests passed; no profiles or missing tools
  stay notes. The paths are announced only for the reports written.
- ProfiledAttempt takes the last attempt whose profile exists, so a final
  retry or repeat that crashed before writing one no longer hides the
  coverage of the attempt before it.
- The per-test llvm-profdata/llvm-cov and gcov runs use as many threads
  as the run had workers; results keep the order of the tests.

Tests: earlier reports removed and no failure without profiles; an
attempt without a profile gives way to an earlier one; the fixture gains a
branch taken one way, checked in the combined LCOV for both LLVM and gcov.
Amalgamation regenerated.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- gcov is run with --branch-probabilities; without it the JSON has no
  branches, so GCC runs had none in coverage.json and coverage.lcov.
- PrepareCoverageDirectory also removes coverage.json, coverage.lcov and
  coverage.profdata of an earlier run, so a run that writes no reports
  leaves none to be read as its own.
- WriteCoverageReports returns notes and failures apart. A profile the
  tools cannot read or a report that cannot be written counts as a missing
  report and fails a run whose tests passed; no profiles or missing tools
  stay notes. The paths are announced only for the reports written.
- ProfiledAttempt takes the last attempt whose profile exists, so a final
  retry or repeat that crashed before writing one no longer hides the
  coverage of the attempt before it.
- The per-test llvm-profdata/llvm-cov and gcov runs use as many threads
  as the run had workers; results keep the order of the tests.

Tests: earlier reports removed and no failure without profiles; an
attempt without a profile gives way to an earlier one; the fixture gains a
branch taken one way, checked in the combined LCOV for both LLVM and gcov.
Amalgamation regenerated.
</pre>
</div>
</content>
</entry>
<entry>
<title>Convert LCOV line numbers explicitly and refresh the amalgamation</title>
<updated>2026-11-13T16:29:35+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T16:29:35+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=55dab34729c14c9019b7062fccd11fc22372e805'/>
<id>55dab34729c14c9019b7062fccd11fc22372e805</id>
<content type='text'>
- The 32-bit MSVC jobs failed on C4244: ParseLcov used the uint64_t it
  read as a std::size_t map key. The line number is now converted once,
  explicitly, before it is used.
- extras/catch_amalgamated.cpp is regenerated with
  bazelisk run //tools:amalgamate, so it holds CoverageReport and the
  scheduler changes; bazel run //tools:check passes locally.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- The 32-bit MSVC jobs failed on C4244: ParseLcov used the uint64_t it
  read as a std::size_t map key. The line number is now converted once,
  explicitly, before it is used.
- extras/catch_amalgamated.cpp is regenerated with
  bazelisk run //tools:amalgamate, so it holds CoverageReport and the
  scheduler changes; bazel run //tools:check passes locally.
</pre>
</div>
</content>
</entry>
<entry>
<title>Keep per-test coverage with --coverage (roadmap step 4)</title>
<updated>2026-11-13T04:46:08+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T04:46:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=17248837a78462596809f902dd47cf113a39670b'/>
<id>17248837a78462596809f902dd47cf113a39670b</id>
<content type='text'>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</pre>
</div>
</content>
</entry>
</feed>
