<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/src/Progmasoft/Catch3, branch feature/per-test-coverage</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Keep per-test coverage with --coverage (roadmap step 4)</title>
<updated>2026-11-13T04:46:08+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T04:46:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=17248837a78462596809f902dd47cf113a39670b'/>
<id>17248837a78462596809f902dd47cf113a39670b</id>
<content type='text'>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</pre>
</div>
</content>
</entry>
<entry>
<title>Release Catch3 0.3.0.2</title>
<updated>2026-11-01T18:42:34+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T18:42:34+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=e1b0a9d000645b5e914f00603983df16167bd053'/>
<id>e1b0a9d000645b5e914f00603983df16167bd053</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Answer the review of the mocks</title>
<updated>2026-11-01T17:05:56+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T17:05:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=8603d31fd8c1a72e30cb444f278d7a0592383ba7'/>
<id>8603d31fd8c1a72e30cb444f278d7a0592383ba7</id>
<content type='text'>
A report no longer throws, so a noexcept mock survives an aborting run.
A call that has nothing to return still leaves the test case, which ends
the process in a noexcept function; the documentation says so.

RunContext clears the result capture when it goes, so a mock of static
lifetime finds none and writes to standard error.

Decision owns its source line, since Verify can free the expectation
while a call is on its way. Moving a result out takes the lock. Vector
matchers refer to their vector, so they are rejected at compile time
like composed ones. A null text expectation accepts only null, and
integers compare by value, so 256 no longer accepts an unsigned char.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A report no longer throws, so a noexcept mock survives an aborting run.
A call that has nothing to return still leaves the test case, which ends
the process in a noexcept function; the documentation says so.

RunContext clears the result capture when it goes, so a mock of static
lifetime finds none and writes to standard error.

Decision owns its source line, since Verify can free the expectation
while a call is on its way. Moving a result out takes the lock. Vector
matchers refer to their vector, so they are rejected at compile time
like composed ones. A null text expectation accepts only null, and
integers compare by value, so 256 no longer accepts an unsigned char.
</pre>
</div>
</content>
</entry>
<entry>
<title>Keep the mock lock alive to the end of the process</title>
<updated>2026-11-01T16:11:27+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T16:11:27+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=2aea9dafd283d69bec459ea0d495c25d1f6a0f94'/>
<id>2aea9dafd283d69bec459ea0d495c25d1f6a0f94</id>
<content type='text'>
A static mutex needs an exit-time destructor, which Clang's warnings
refuse, and a mock of static lifetime can outlive it. The test seam is
held the same way, and a test covers a mock method with nothing after
its parameters, which needs an empty variadic argument to stay valid
under -Wpedantic.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A static mutex needs an exit-time destructor, which Clang's warnings
refuse, and a mock of static lifetime can outlive it. The test seam is
held the same way, and a test covers a mock method with nothing after
its parameters, which needs an empty variadic argument to stay valid
under -Wpedantic.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add mocks that report through the assertion path</title>
<updated>2026-11-01T16:04:52+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T16:04:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=c51d15273e87267215e5877edc268e7db354187b'/>
<id>c51d15273e87267215e5877edc268e7db354187b</id>
<content type='text'>
Step 3 of the roadmap. MockFunction checks the calls of one function
against its expectations; CATCH3_MOCK_METHOD declares one for a virtual
member, and CATCH3_MOCK_FUNCTION and ScopedSeam serve free functions
behind a seam.

Expectations take values, Catch matchers, Any or Where; count calls with
Times, Between, AtLeast, AtMost, AnyTimes and Never; answer with Returns,
Does or Throws; and order calls across mocks with Sequence. A failure
goes through AssertionHandler at the line of the expectation, so every
report of the runner shows it. Met expectations count as passed
assertions. A test case that is already being left does not get unmet
expectations on top of its own failure.

Composed matchers refer to temporaries, so they are rejected at compile
time rather than left dangling. A call that fails returns a
value-initialized result, or leaves the test case when no such result
exists.

The failure paths are tested through the runner fixture, reading the
messages from its JSON report.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Step 3 of the roadmap. MockFunction checks the calls of one function
against its expectations; CATCH3_MOCK_METHOD declares one for a virtual
member, and CATCH3_MOCK_FUNCTION and ScopedSeam serve free functions
behind a seam.

Expectations take values, Catch matchers, Any or Where; count calls with
Times, Between, AtLeast, AtMost, AnyTimes and Never; answer with Returns,
Does or Throws; and order calls across mocks with Sequence. A failure
goes through AssertionHandler at the line of the expectation, so every
report of the runner shows it. Met expectations count as passed
assertions. A test case that is already being left does not get unmet
expectations on top of its own failure.

Composed matchers refer to temporaries, so they are rejected at compile
time rather than left dangling. A call that fails returns a
value-initialized result, or leaves the test case when no such result
exists.

The failure paths are tested through the runner fixture, reading the
messages from its JSON report.
</pre>
</div>
</content>
</entry>
<entry>
<title>Release Catch3 0.3.0.1</title>
<updated>2026-11-01T13:20:17+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T13:20:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=64e7fffb605cc355d258f4896c49cdec6127ed6b'/>
<id>64e7fffb605cc355d258f4896c49cdec6127ed6b</id>
<content type='text'>
Versions have four numbers from this release on.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Versions have four numbers from this release on.
</pre>
</div>
</content>
</entry>
<entry>
<title>Import the results of other test frameworks</title>
<updated>2026-11-01T11:23:09+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T11:23:09+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=1db203d56b7c71db3d80b477da1ad49e9ae46468'/>
<id>1db203d56b7c71db3d80b477da1ad49e9ae46468</id>
<content type='text'>
Tests Catch3 does not run had no history, flaky detection or quarantine.
--import=FILE reads a report instead of running tests and sends it through
the analysis, history and reports of a run: JUnit with Surefire's reruns,
xUnit.net v2, TRX, TAP 12 to 14, CTRF, Catch3's own --json report and
catch3-results, a small JSON Lines format for tools with none of these.

Results that share an identifier are attempts at one test, so a report of a
rerun gives a flaky test. --import-revision says what was tested: a verdict
that changes on one revision also counts as flaky. --import-name keeps the
results in a history of their own, which --analytics reads too. A TAP stream
that ends early fails instead of reading as a run that passed.

The amalgamation script now accepts a relative include of the third-party
wrappers, because pugixml is included from a second directory.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Tests Catch3 does not run had no history, flaky detection or quarantine.
--import=FILE reads a report instead of running tests and sends it through
the analysis, history and reports of a run: JUnit with Surefire's reruns,
xUnit.net v2, TRX, TAP 12 to 14, CTRF, Catch3's own --json report and
catch3-results, a small JSON Lines format for tools with none of these.

Results that share an identifier are attempts at one test, so a report of a
rerun gives a flaky test. --import-revision says what was tested: a verdict
that changes on one revision also counts as flaky. --import-name keeps the
results in a history of their own, which --analytics reads too. A TAP stream
that ends early fails instead of reading as a run that passed.

The amalgamation script now accepts a relative include of the third-party
wrappers, because pugixml is included from a second directory.
</pre>
</div>
</content>
</entry>
<entry>
<title>Say C++17 everywhere the project describes itself</title>
<updated>2026-10-31T21:25:59+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-31T21:25:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=ff1da91edffa9ee887e9fe737fcd87ec540dfa60'/>
<id>ff1da91edffa9ee887e9fe737fcd87ec540dfa60</id>
<content type='text'>
The move to C++17 left a trail of older claims. The Runner's own
.clang-format still said c++20; the result is byte for byte the same under
either value. Premake and Xmake still compiled the library as C++20 and
the main library as C++14, which meant their CI jobs were not testing the
language the project promises. Both now use C++17 for every target.

The rest is text: the Meson pkg-config description claimed C++14, comments
in the CMake and Meson files explained requirements that no longer exist,
.clang-tidy gave C++14 as the reason for two exclusions, issue templates
asked for C++20 samples, and the example headers named -std=c++14 and
catch2-with-main. The release notes and the migration guide state the
C++17 minimum. Doxygen's brief called the project "Catch2's test API",
which presents it as Catch2. Meson declared only BSL-1.0; it now names both
licenses of the tree. The amalgamated files are regenerated.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The move to C++17 left a trail of older claims. The Runner's own
.clang-format still said c++20; the result is byte for byte the same under
either value. Premake and Xmake still compiled the library as C++20 and
the main library as C++14, which meant their CI jobs were not testing the
language the project promises. Both now use C++17 for every target.

The rest is text: the Meson pkg-config description claimed C++14, comments
in the CMake and Meson files explained requirements that no longer exist,
.clang-tidy gave C++14 as the reason for two exclusions, issue templates
asked for C++20 samples, and the example headers named -std=c++14 and
catch2-with-main. The release notes and the migration guide state the
C++17 minimum. Doxygen's brief called the project "Catch2's test API",
which presents it as Catch2. Meson declared only BSL-1.0; it now names both
licenses of the tree. The amalgamated files are regenerated.
</pre>
</div>
</content>
</entry>
<entry>
<title>Convert to double in the coverage report explicitly</title>
<updated>2026-10-31T17:20:48+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-31T17:20:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=e273973f9bc365a646fad427169416c37da28e98'/>
<id>e273973f9bc365a646fad427169416c37da28e98</id>
<content type='text'>
Dividing a long long by 10.0 is an implicit conversion, which fails in a
project that builds its tests with -Wimplicit-int-float-conversion and
warnings as errors, as Tala does.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Dividing a long long by 10.0 is an implicit conversion, which fails in a
project that builds its tests with -Wimplicit-int-float-conversion and
warnings as errors, as Tala does.
</pre>
</div>
</content>
</entry>
<entry>
<title>Move to the Progmasoft Linking Exception, Version 2.0</title>
<updated>2026-10-31T10:42:35+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-31T10:42:35+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=31486e3217b8424809bcf129c0ef24ef5639e919'/>
<id>31486e3217b8424809bcf129c0ef24ef5639e919</id>
<content type='text'>
Version 1.1 left programs built with Progmasoft tools under MPL-2.0 duties
for the code the tools place in them, and left other cases unclear.
Version 2.0 adds a generated code permission, covers runtime and standard
libraries, compiler transformations, interfaces and examples, and settles
how a file is identified and how to tell recipients about source.

Every project SPDX expression moves to AdditionRef-Progmasoft-Exception-2.0.
The older texts stay, because released versions name them.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Version 1.1 left programs built with Progmasoft tools under MPL-2.0 duties
for the code the tools place in them, and left other cases unclear.
Version 2.0 adds a generated code permission, covers runtime and standard
libraries, compiler transformations, interfaces and examples, and settles
how a file is identified and how to tell recipients about source.

Every project SPDX expression moves to AdditionRef-Progmasoft-Exception-2.0.
The older texts stay, because released versions name them.
</pre>
</div>
</content>
</entry>
</feed>
