<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/tests/ExtraTests, branch docs/describe-catch3</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Fix the CI failures of the runner and test its planner, options and analysis</title>
<updated>2026-10-24T07:06:25+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-24T07:06:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=e104b694513964d5a84e77f751203cda970d741b'/>
<id>e104b694513964d5a84e77f751203cda970d741b</id>
<content type='text'>
ExtraTests X21 and X27 defined their listeners at namespace scope, which
clang rejects under -Wweak-vtables with -Werror on Linux and macOS; they
are now in anonymous namespaces.

After a fatal signal Catch2 closes the open sections itself, so on POSIX a
crash was reported outside the section it happened in. The scheduler now
ignores section ends that follow a fatal condition.

New unit tests cover the planner (every selection, dependency closure and
cycles, hash and duration shards, incremental runs, learned time limits,
ordering, retries, quarantine and replay), every command-line option and
its errors, and the failure analysis (history classes, signature groups,
crash and hang locations, changed files, replay commands).
Catalog::FromEntries builds a catalog without a test registry so the
planner can be tested on its own.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
ExtraTests X21 and X27 defined their listeners at namespace scope, which
clang rejects under -Wweak-vtables with -Werror on Linux and macOS; they
are now in anonymous namespaces.

After a fatal signal Catch2 closes the open sections itself, so on POSIX a
crash was reported outside the section it happened in. The scheduler now
ignores section ends that follow a fatal condition.

New unit tests cover the planner (every selection, dependency closure and
cycles, hash and duration shards, incremental runs, learned time limits,
ordering, retries, quarantine and replay), every command-line option and
its errors, and the failure analysis (history classes, signature groups,
crash and hang locations, changed files, replay commands).
Catalog::FromEntries builds a catalog without a test registry so the
planner can be tested on its own.
</pre>
</div>
</content>
</entry>
<entry>
<title>Replace Catch2's runner with the Catch3 runner</title>
<updated>2026-10-23T20:25:41+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-23T20:25:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=93d4e7dbebab867e0a7f59fc25308460a7a76280'/>
<id>93d4e7dbebab867e0a7f59fc25308460a7a76280</id>
<content type='text'>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</pre>
</div>
</content>
</entry>
<entry>
<title>Backport Catch2 3.16.1</title>
<updated>2026-10-10T23:56:25+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-10T23:56:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=1f78d8903c2fb18afea8c3055f8df83d8c60d9b5'/>
<id>1f78d8903c2fb18afea8c3055f8df83d8c60d9b5</id>
<content type='text'>
Review Catch2 devel through 08092139210881f01a34e60c5f2fd6afa0ea4024,
the commit of 3.16.1, and apply the eleven changes since the last
reviewed revision in upstream order:

  d7683484  better message when a reporter cannot be initialized
  ea5020f0  custom argument validation in every first-party reporter
  aa01559e  throwing-reporter test also throws in assertionEnded
  45cd3987  noexcept on handleUnexpectedInflightException
  6adbd539  MSVC optimizations enabled again after the benchmark headers
  e87f2882  no quadratic scaling for NTTP test registration
  73d546cd  REQUIRE expands its arguments fewer times under GCC and Clang
  b710039a  decomposer operators as plain functions
  f899467b  type-erased streaming of reconstructed expressions
  8f0e70ca  retired macos-14 runner dropped
  08092139  version 3.16.1 and its release notes

All of them applied to the sources without a conflict. The amalgamated
distribution under extras/ conflicted because it is generated; it is
generated again from the sources of this repository, which also brings
it up to date with fork changes it had not picked up: the list-based
stream pool, the Clang diagnostic boundary and the cassert include.

No file under src/Progmasoft changes. The fork-owned texts that name
the engine version say 3.16.1, and docs/upstream-status.md records the
review. Upstream files keep their names, licenses and compiler options.

Built with Clang 22 on Windows and tested: 148 of 149 tests pass. The
one that fails, Reporters::CrashInJunitReporter, fails the same way on
devel without this change with that compiler.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Review Catch2 devel through 08092139210881f01a34e60c5f2fd6afa0ea4024,
the commit of 3.16.1, and apply the eleven changes since the last
reviewed revision in upstream order:

  d7683484  better message when a reporter cannot be initialized
  ea5020f0  custom argument validation in every first-party reporter
  aa01559e  throwing-reporter test also throws in assertionEnded
  45cd3987  noexcept on handleUnexpectedInflightException
  6adbd539  MSVC optimizations enabled again after the benchmark headers
  e87f2882  no quadratic scaling for NTTP test registration
  73d546cd  REQUIRE expands its arguments fewer times under GCC and Clang
  b710039a  decomposer operators as plain functions
  f899467b  type-erased streaming of reconstructed expressions
  8f0e70ca  retired macos-14 runner dropped
  08092139  version 3.16.1 and its release notes

All of them applied to the sources without a conflict. The amalgamated
distribution under extras/ conflicted because it is generated; it is
generated again from the sources of this repository, which also brings
it up to date with fork changes it had not picked up: the list-based
stream pool, the Clang diagnostic boundary and the cassert include.

No file under src/Progmasoft changes. The fork-owned texts that name
the engine version say 3.16.1, and docs/upstream-status.md records the
review. Upstream files keep their names, licenses and compiler options.

Built with Clang 22 on Windows and tested: 148 of 149 tests pass. The
one that fails, Reporters::CrashInJunitReporter, fails the same way on
devel without this change with that compiler.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add tests for tag validation during test registration</title>
<updated>2026-09-11T09:27:27+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-09-10T21:33:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=1767d9fad0bc28e2b80744daa93c3430d0c6d238'/>
<id>1767d9fad0bc28e2b80744daa93c3430d0c6d238</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Emit warning when using --shard-count (&gt; 1) with --order rand</title>
<updated>2026-08-10T22:08:23+00:00</updated>
<author>
<name>Matt Rasa</name>
</author>
<published>2026-07-31T13:37:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=0aeb818520028ca05cc2539ed242402ef7a7ae41'/>
<id>0aeb818520028ca05cc2539ed242402ef7a7ae41</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix --warn InfiniteGenerators firing even if the generator was filtered</title>
<updated>2026-07-07T12:09:11+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-07-07T12:09:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=dd94b9a780e66651ff686083a66722386c540228'/>
<id>dd94b9a780e66651ff686083a66722386c540228</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add extra test binary &amp; test for CATCH_CONFIG_FAST_COMPILE</title>
<updated>2026-07-06T09:03:47+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-07-06T09:03:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=9f7c9f6872308d4e7c32fff2f1ac6640aa866b6a'/>
<id>9f7c9f6872308d4e7c32fff2f1ac6640aa866b6a</id>
<content type='text'>
Closes #3100
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Closes #3100
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix some test binaries in ExtraTests not having warnings enabled</title>
<updated>2026-07-06T08:22:55+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-07-06T08:22:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=919385f7046b61ae0960d02977d95674cbe20d85'/>
<id>919385f7046b61ae0960d02977d95674cbe20d85</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Support for constexpr matchers in C++20 (P0784)</title>
<updated>2026-05-12T09:05:02+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-05-10T21:05:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=651247c7f40eda6846fec736d2f9edac8949dacb'/>
<id>651247c7f40eda6846fec736d2f9edac8949dacb</id>
<content type='text'>
To make this all work, I had to remove the stringification cache
from matchers. In theory, this can cause performance penalty in
cases where single matcher instance is stringified multiple times,
but in practice this does not happen much, and the difference is
surprisingly small anyway, because the performance of stringification
is already horrible and full of allocating strings just to throw
them away.

The matcher combinators need P2738 from C++26 to be `constexpr`.

Closes #3091
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
To make this all work, I had to remove the stringification cache
from matchers. In theory, this can cause performance penalty in
cases where single matcher instance is stringified multiple times,
but in practice this does not happen much, and the difference is
surprisingly small anyway, because the performance of stringification
is already horrible and full of allocating strings just to throw
them away.

The matcher combinators need P2738 from C++26 to be `constexpr`.

Closes #3091
</pre>
</div>
</content>
</entry>
<entry>
<title>Inline SKIP/FAIL in RunContext to fix build failure with prefixed macros</title>
<updated>2026-04-03T08:19:05+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-04-02T20:58:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=9f1b48a94fdc7a54aa87fc9bd26d6b1b3f560d4c'/>
<id>9f1b48a94fdc7a54aa87fc9bd26d6b1b3f560d4c</id>
<content type='text'>
Closes #3087
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Closes #3087
</pre>
</div>
</content>
</entry>
</feed>
