<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/src/Progmasoft/Runner, branch fix/version-tool-semver</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Make the start lock a never-destroyed object</title>
<updated>2026-11-13T21:56:32+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T21:56:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=5ad02bccfcb925df02d1273901b6e248d477c5fa'/>
<id>5ad02bccfcb925df02d1273901b6e248d477c5fa</id>
<content type='text'>
A function-local static std::mutex has an exit-time destructor, which
the macOS build rejects with -Wexit-time-destructors. The lock is now
allocated once and never destroyed, which also keeps it usable by a
thread that starts a child while the process exits.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A function-local static std::mutex has an exit-time destructor, which
the macOS build rejects with -Wexit-time-destructors. The lock is now
allocated once and never destroyed, which also keeps it usable by a
thread that starts a child while the process exits.
</pre>
</div>
</content>
</entry>
<entry>
<title>Start child processes one at a time; run gcov without a directory</title>
<updated>2026-11-13T20:24:10+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T20:24:10+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=b26486d68ece48a1dfdb208a9f3f1bd8acab0382'/>
<id>b26486d68ece48a1dfdb208a9f3f1bd8acab0382</id>
<content type='text'>
Reading the coverage profiles on several threads broke an assumption of
ChildProcess::Start on POSIX: it changes the working directory of the
whole process around posix_spawn. Two gcov runs started at once could
both run in one test's directory, so on the GCC job the map of
"Subtracts one" showed what "Adds one" reached.

- ChildProcess::Start holds a lock while a child starts, on POSIX and
  Windows, so the directory change and the handles a child inherits never
  overlap with another start.
- gcov gets the absolute path of the data file and no working directory,
  so no directory changes while profiles are read in parallel.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Reading the coverage profiles on several threads broke an assumption of
ChildProcess::Start on POSIX: it changes the working directory of the
whole process around posix_spawn. Two gcov runs started at once could
both run in one test's directory, so on the GCC job the map of
"Subtracts one" showed what "Adds one" reached.

- ChildProcess::Start holds a lock while a child starts, on POSIX and
  Windows, so the directory change and the handles a child inherits never
  overlap with another start.
- gcov gets the absolute path of the data file and no working directory,
  so no directory changes while profiles are read in parallel.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix the coverage review findings: branches, stale reports, exit status</title>
<updated>2026-11-13T18:55:26+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T18:55:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=454cca9b63991416f4d9bd1ca350f1ae14e278e7'/>
<id>454cca9b63991416f4d9bd1ca350f1ae14e278e7</id>
<content type='text'>
- gcov is run with --branch-probabilities; without it the JSON has no
  branches, so GCC runs had none in coverage.json and coverage.lcov.
- PrepareCoverageDirectory also removes coverage.json, coverage.lcov and
  coverage.profdata of an earlier run, so a run that writes no reports
  leaves none to be read as its own.
- WriteCoverageReports returns notes and failures apart. A profile the
  tools cannot read or a report that cannot be written counts as a missing
  report and fails a run whose tests passed; no profiles or missing tools
  stay notes. The paths are announced only for the reports written.
- ProfiledAttempt takes the last attempt whose profile exists, so a final
  retry or repeat that crashed before writing one no longer hides the
  coverage of the attempt before it.
- The per-test llvm-profdata/llvm-cov and gcov runs use as many threads
  as the run had workers; results keep the order of the tests.

Tests: earlier reports removed and no failure without profiles; an
attempt without a profile gives way to an earlier one; the fixture gains a
branch taken one way, checked in the combined LCOV for both LLVM and gcov.
Amalgamation regenerated.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- gcov is run with --branch-probabilities; without it the JSON has no
  branches, so GCC runs had none in coverage.json and coverage.lcov.
- PrepareCoverageDirectory also removes coverage.json, coverage.lcov and
  coverage.profdata of an earlier run, so a run that writes no reports
  leaves none to be read as its own.
- WriteCoverageReports returns notes and failures apart. A profile the
  tools cannot read or a report that cannot be written counts as a missing
  report and fails a run whose tests passed; no profiles or missing tools
  stay notes. The paths are announced only for the reports written.
- ProfiledAttempt takes the last attempt whose profile exists, so a final
  retry or repeat that crashed before writing one no longer hides the
  coverage of the attempt before it.
- The per-test llvm-profdata/llvm-cov and gcov runs use as many threads
  as the run had workers; results keep the order of the tests.

Tests: earlier reports removed and no failure without profiles; an
attempt without a profile gives way to an earlier one; the fixture gains a
branch taken one way, checked in the combined LCOV for both LLVM and gcov.
Amalgamation regenerated.
</pre>
</div>
</content>
</entry>
<entry>
<title>Convert LCOV line numbers explicitly and refresh the amalgamation</title>
<updated>2026-11-13T16:29:35+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T16:29:35+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=55dab34729c14c9019b7062fccd11fc22372e805'/>
<id>55dab34729c14c9019b7062fccd11fc22372e805</id>
<content type='text'>
- The 32-bit MSVC jobs failed on C4244: ParseLcov used the uint64_t it
  read as a std::size_t map key. The line number is now converted once,
  explicitly, before it is used.
- extras/catch_amalgamated.cpp is regenerated with
  bazelisk run //tools:amalgamate, so it holds CoverageReport and the
  scheduler changes; bazel run //tools:check passes locally.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- The 32-bit MSVC jobs failed on C4244: ParseLcov used the uint64_t it
  read as a std::size_t map key. The line number is now converted once,
  explicitly, before it is used.
- extras/catch_amalgamated.cpp is regenerated with
  bazelisk run //tools:amalgamate, so it holds CoverageReport and the
  scheduler changes; bazel run //tools:check passes locally.
</pre>
</div>
</content>
</entry>
<entry>
<title>Keep per-test coverage with --coverage (roadmap step 4)</title>
<updated>2026-11-13T04:46:08+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-13T04:46:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=17248837a78462596809f902dd47cf113a39670b'/>
<id>17248837a78462596809f902dd47cf113a39670b</id>
<content type='text'>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
--coverage=DIR runs every attempt in a process of its own and keeps the
coverage of each test, for binaries built with LLVM source-based
coverage (Clang: -fprofile-instr-generate -fcoverage-mapping) or gcov
(GCC: --coverage).

How it works:

- The scheduler gives each attempt a fresh worker, as --isolate does,
  and sets LLVM_PROFILE_FILE and GCOV_PREFIX for it: the profile the
  process writes at exit is then the coverage of that one test, in
  DIR/profiles/t&lt;index&gt;-a&lt;attempt&gt;.profraw or a .gcov directory. Each
  runtime reads its own variable, so no symbol of either runtime is
  looked up: weak references would not pull them out of their static
  archives, and Mach-O does not allow undefined weak symbols.
- A process that runs other tests first (bisection) writes beside the
  profiles, never into one. The attempt records its profile, also in
  the JSON results.
- The profiles of an earlier run are removed when the run starts,
  because gcov adds to the .gcda files it finds.
- After the run, CoverageReport reads the last profile of every test:
  llvm-profdata and llvm-cov (PATH, xcrun on macOS, or
  CATCH3_LLVM_PROFDATA / CATCH3_LLVM_COV) export LCOV, gcov (or
  CATCH3_GCOV) writes JSON next to a copy of the .gcno. The result is
  DIR/coverage.json, the files, lines and branches each test reached,
  and DIR/coverage.lcov for the whole run. Without the tools the
  profiles stay and the run says so.
- --coverage with --in-process or --break is refused.

Tests: option parsing, LCOV reading and writing, gcov JSON, and an
integration test on CoverageFixture, a binary CMake builds with the
coverage flags of Clang or GCC: two tests reach different lines, each
map holds only its own, the LCOV report holds both, and a stale profile
of an earlier run is gone. Locally all 84 CTest tests pass with Clang,
and the coverage tests pass with GCC (built with
CATCH3_ENABLE_WERROR=OFF: Catch3 does not build with GCC's warnings as
errors, before and after this change).

CI: the Linux jobs install LLVM's coverage tools, a new job checks the
gcov path with GCC, and CATCH3_COVERAGE_REQUIRE_MAP turns a missing map
into a failure on Linux and macOS instead of a skip. MSVC has no
source-based coverage, so its builds skip those tests; the roadmap
lists step 4 as in progress until Windows with clang-cl is tested.
</pre>
</div>
</content>
</entry>
<entry>
<title>Compare Windows variable names in UTF-8 and end an empty block twice</title>
<updated>2026-11-05T23:55:57+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-05T23:55:57+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=ad3633ea59ec491c9694a10307beb7d7189db744'/>
<id>ad3633ea59ec491c9694a10307beb7d7189db744</id>
<content type='text'>
EnvironmentBlock replaced every non-ASCII character of an inherited name
with '?' before asking whether to withhold it, so --pass-env=MY_?_TOKEN
kept an unrelated MY_é_TOKEN. Names are now converted with Narrow, as
everything else the runner compares.

A block whose variables were all withheld, with no additions, ended in one
terminator; CreateProcessW needs two for an empty block.

The amalgamated sources are regenerated for the runner changes.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
EnvironmentBlock replaced every non-ASCII character of an inherited name
with '?' before asking whether to withhold it, so --pass-env=MY_?_TOKEN
kept an unrelated MY_é_TOKEN. Names are now converted with Narrow, as
everything else the runner compares.

A block whose variables were all withheld, with no additions, ended in one
terminator; CreateProcessW needs two for an empty block.

The amalgamated sources are regenerated for the runner changes.
</pre>
</div>
</content>
</entry>
<entry>
<title>Withhold secret-looking variables from the processes the runner starts</title>
<updated>2026-11-05T15:30:13+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-05T15:30:13+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=89b4769c35986444a0297ad0806a584c7181c85c'/>
<id>89b4769c35986444a0297ad0806a584c7181c85c</id>
<content type='text'>
Workers and the other processes the runner starts copied its whole
environment, tokens and passwords included. ChildProcess::Start now
leaves out every variable whose name contains, in any case, TOKEN,
SECRET, PASSWORD, PASSWD, CREDENTIAL, API_KEY, APIKEY, PRIVATE_KEY or
ACCESS_KEY (IsSecretVariable), unless ChildOptions::keptEnvironment
names it; names compare without case on Windows.

--pass-env=NAME, repeatable, and "pass-env" in catch3.json keep a
variable for the tests. The runner itself still starts with the whole
environment, so code that runs before the first worker sees it;
docs/progmasoft-runner.md says so.

A runner test starts the shell's environment listing with a secret and
a plain variable set, sees only the plain one, and sees the secret once
it is kept.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Workers and the other processes the runner starts copied its whole
environment, tokens and passwords included. ChildProcess::Start now
leaves out every variable whose name contains, in any case, TOKEN,
SECRET, PASSWORD, PASSWD, CREDENTIAL, API_KEY, APIKEY, PRIVATE_KEY or
ACCESS_KEY (IsSecretVariable), unless ChildOptions::keptEnvironment
names it; names compare without case on Windows.

--pass-env=NAME, repeatable, and "pass-env" in catch3.json keep a
variable for the tests. The runner itself still starts with the whole
environment, so code that runs before the first worker sees it;
docs/progmasoft-runner.md says so.

A runner test starts the shell's environment listing with a secret and
a plain variable set, sees only the plain one, and sees the secret once
it is kept.
</pre>
</div>
</content>
</entry>
<entry>
<title>Keep real failures visible when importing</title>
<updated>2026-11-01T11:48:55+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T11:48:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=569620dad14df7e13bb9fd31d256e9fd873ef6ec'/>
<id>569620dad14df7e13bb9fd31d256e9fd873ef6ec</id>
<content type='text'>
A review of the import found ways for a failure to disappear or for the
import to reach too far.

- CTRF's retryAttempts ends before the final attempt, which was dropped when
  its outcome matched the one before it.
- TAP names tests only within a stream, so a failure in one file and a pass
  under the same name in another merged into a flaky pass. The path given to
  --import is now the class of a stream's tests.
- A failure followed by a skip with the same identifier read as skipped.
- Directives are whole words: TODOING is not TODO. SKIPPED and SKIPPING stay
  directives, as harnesses treat them.
- Verdicts that alternate count as flaky only under an explicit
  --import-revision. HEAD says nothing about a dirty checkout or a report from
  another one, and two such events could quarantine a real failure.
- --import-name no longer redirects the history of a normal test run.
- Importing does not depend on the tests registered in the binary.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A review of the import found ways for a failure to disappear or for the
import to reach too far.

- CTRF's retryAttempts ends before the final attempt, which was dropped when
  its outcome matched the one before it.
- TAP names tests only within a stream, so a failure in one file and a pass
  under the same name in another merged into a flaky pass. The path given to
  --import is now the class of a stream's tests.
- A failure followed by a skip with the same identifier read as skipped.
- Directives are whole words: TODOING is not TODO. SKIPPED and SKIPPING stay
  directives, as harnesses treat them.
- Verdicts that alternate count as flaky only under an explicit
  --import-revision. HEAD says nothing about a dirty checkout or a report from
  another one, and two such events could quarantine a real failure.
- --import-name no longer redirects the history of a normal test run.
- Importing does not depend on the tests registered in the binary.
</pre>
</div>
</content>
</entry>
<entry>
<title>Import the results of other test frameworks</title>
<updated>2026-11-01T11:23:09+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-11-01T11:23:09+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=1db203d56b7c71db3d80b477da1ad49e9ae46468'/>
<id>1db203d56b7c71db3d80b477da1ad49e9ae46468</id>
<content type='text'>
Tests Catch3 does not run had no history, flaky detection or quarantine.
--import=FILE reads a report instead of running tests and sends it through
the analysis, history and reports of a run: JUnit with Surefire's reruns,
xUnit.net v2, TRX, TAP 12 to 14, CTRF, Catch3's own --json report and
catch3-results, a small JSON Lines format for tools with none of these.

Results that share an identifier are attempts at one test, so a report of a
rerun gives a flaky test. --import-revision says what was tested: a verdict
that changes on one revision also counts as flaky. --import-name keeps the
results in a history of their own, which --analytics reads too. A TAP stream
that ends early fails instead of reading as a run that passed.

The amalgamation script now accepts a relative include of the third-party
wrappers, because pugixml is included from a second directory.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Tests Catch3 does not run had no history, flaky detection or quarantine.
--import=FILE reads a report instead of running tests and sends it through
the analysis, history and reports of a run: JUnit with Surefire's reruns,
xUnit.net v2, TRX, TAP 12 to 14, CTRF, Catch3's own --json report and
catch3-results, a small JSON Lines format for tools with none of these.

Results that share an identifier are attempts at one test, so a report of a
rerun gives a flaky test. --import-revision says what was tested: a verdict
that changes on one revision also counts as flaky. --import-name keeps the
results in a history of their own, which --analytics reads too. A TAP stream
that ends early fails instead of reading as a run that passed.

The amalgamation script now accepts a relative include of the third-party
wrappers, because pugixml is included from a second directory.
</pre>
</div>
</content>
</entry>
<entry>
<title>Say C++17 everywhere the project describes itself</title>
<updated>2026-10-31T21:25:59+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-31T21:25:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=ff1da91edffa9ee887e9fe737fcd87ec540dfa60'/>
<id>ff1da91edffa9ee887e9fe737fcd87ec540dfa60</id>
<content type='text'>
The move to C++17 left a trail of older claims. The Runner's own
.clang-format still said c++20; the result is byte for byte the same under
either value. Premake and Xmake still compiled the library as C++20 and
the main library as C++14, which meant their CI jobs were not testing the
language the project promises. Both now use C++17 for every target.

The rest is text: the Meson pkg-config description claimed C++14, comments
in the CMake and Meson files explained requirements that no longer exist,
.clang-tidy gave C++14 as the reason for two exclusions, issue templates
asked for C++20 samples, and the example headers named -std=c++14 and
catch2-with-main. The release notes and the migration guide state the
C++17 minimum. Doxygen's brief called the project "Catch2's test API",
which presents it as Catch2. Meson declared only BSL-1.0; it now names both
licenses of the tree. The amalgamated files are regenerated.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The move to C++17 left a trail of older claims. The Runner's own
.clang-format still said c++20; the result is byte for byte the same under
either value. Premake and Xmake still compiled the library as C++20 and
the main library as C++14, which meant their CI jobs were not testing the
language the project promises. Both now use C++17 for every target.

The rest is text: the Meson pkg-config description claimed C++14, comments
in the CMake and Meson files explained requirements that no longer exist,
.clang-tidy gave C++14 as the reason for two exclusions, issue templates
asked for C++20 samples, and the example headers named -std=c++14 and
catch2-with-main. The release notes and the migration guide state the
C++17 minimum. Doxygen's brief called the project "Catch2's test API",
which presents it as Catch2. Meson declared only BSL-1.0; it now names both
licenses of the tree. The amalgamated files are regenerated.
</pre>
</div>
</content>
</entry>
</feed>
