<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/src/CMakeLists.txt, branch docs/describe-catch3</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Replace Catch2's runner with the Catch3 runner</title>
<updated>2026-10-23T20:25:41+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-23T20:25:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=93d4e7dbebab867e0a7f59fc25308460a7a76280'/>
<id>93d4e7dbebab867e0a7f59fc25308460a7a76280</id>
<content type='text'>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add a native property-testing engine with choice-sequence shrinking</title>
<updated>2026-10-22T14:39:46+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-22T14:39:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=afac14ac5a62d8d2e7efd2184ad34e16593eddfd'/>
<id>afac14ac5a62d8d2e7efd2184ad34e16593eddfd</id>
<content type='text'>
Catch3's own property engine, independent of RapidCheck, next to the
existing CheckProperty API, which stays unchanged.

Strategies (Progmasoft/Catch3/Strategy/): a Strategy&lt;T&gt; builds a value
from a ChoiceSource, a stream of integer choices where 0 is always the
simplest decision. Gen:: provides Integers, Booleans, Floats (ranges,
NaN and infinities on request, edge values), Characters, Strings,
Utf8Strings, Vectors, UniqueVectors, Sets, Maps, Optionals, Tuples,
Construct, Permutations, Just, SampledFrom, OneOf, Frequency, Lazy and
Recursive; every strategy has Map, Filter and FlatMap.

Shrinking: the engine records the choices of a failing trial and
shrinks them, not the value, with deletion and zeroing of blocks,
binary search per choice with a neighbour step for non-monotone
conditions, sorting and moving value between neighbours. Replaying the
shrunk choices through the same strategies means mapped, filtered,
dependent and stateful inputs all shrink, and a shrunk input still
satisfies its filters.

ForAll([options,] strategy..., body) runs the trials (size rising over
the first half), then a targeted search when the body calls Target,
then checks Cover requirements. A failure report carries the
minimized input, the notes and a replay token (PropertyOptions::Replay)
that runs exactly that input. An example database (DatabasePath or
CATCH3_PROPERTY_DATABASE) replays stored failures first and drops them
once they pass. Bodies return bool, or void with CATCH3_PROPERTY_ASSERT
and CATCH3_PROPERTY_ASSUME, which return instead of throwing; Note,
Classify, Cover and Target record into a per-thread trial record.
CATCH3_CHECK_FOR_ALL and CATCH3_REQUIRE_FOR_ALL report through Catch2
and name the property after the test case for the database.

CheckStateful runs model-based command sequences against a model and
the system under test; preconditions and state-dependent arguments are
replayed during shrinking. Relations.hpp provides Equivalent
(differential), Metamorphic, RoundTrip and Idempotent bodies.

PropertyOptions, PropertyStatus, PropertyFailure and PropertyResult
gain fields and statuses at their ends (kInsufficientCoverage,
kGaveUp); existing designated initializers and reports are unchanged.
The engine's untemplated parts are compiled in ChoiceShrinker.cpp,
PropertyEngine.cpp and Context.cpp, and every build system (Bazel,
CMake, Meson, Xmake, Premake) lists the new files and tests.

Docs: progmasoft-properties.md describes the engine,
progmasoft-roadmap.md the steps that follow (the standalone catch3
runner, mocking, per-test coverage, change-impact selection and
prioritization, mutation testing, guided test generation). The
.clang-format comment names Progmasoft C++ Style instead of Visual X#.

Verified with clang-cl 23 through CMake: ProgmasoftSelfTest passes, 57
test cases and 75,461 assertions, including the shrinking results
(1001 for x &lt;= 1000, { 10 }, { 0, 0, 0 } through FlatMap, 102 through
a filter, { 0, 100 } for two arguments, four push(0) steps for a
queue that drops its fourth element, { 0, 0 } for a sort that loses
duplicates) and the database, replay, coverage and discard behaviour.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Catch3's own property engine, independent of RapidCheck, next to the
existing CheckProperty API, which stays unchanged.

Strategies (Progmasoft/Catch3/Strategy/): a Strategy&lt;T&gt; builds a value
from a ChoiceSource, a stream of integer choices where 0 is always the
simplest decision. Gen:: provides Integers, Booleans, Floats (ranges,
NaN and infinities on request, edge values), Characters, Strings,
Utf8Strings, Vectors, UniqueVectors, Sets, Maps, Optionals, Tuples,
Construct, Permutations, Just, SampledFrom, OneOf, Frequency, Lazy and
Recursive; every strategy has Map, Filter and FlatMap.

Shrinking: the engine records the choices of a failing trial and
shrinks them, not the value, with deletion and zeroing of blocks,
binary search per choice with a neighbour step for non-monotone
conditions, sorting and moving value between neighbours. Replaying the
shrunk choices through the same strategies means mapped, filtered,
dependent and stateful inputs all shrink, and a shrunk input still
satisfies its filters.

ForAll([options,] strategy..., body) runs the trials (size rising over
the first half), then a targeted search when the body calls Target,
then checks Cover requirements. A failure report carries the
minimized input, the notes and a replay token (PropertyOptions::Replay)
that runs exactly that input. An example database (DatabasePath or
CATCH3_PROPERTY_DATABASE) replays stored failures first and drops them
once they pass. Bodies return bool, or void with CATCH3_PROPERTY_ASSERT
and CATCH3_PROPERTY_ASSUME, which return instead of throwing; Note,
Classify, Cover and Target record into a per-thread trial record.
CATCH3_CHECK_FOR_ALL and CATCH3_REQUIRE_FOR_ALL report through Catch2
and name the property after the test case for the database.

CheckStateful runs model-based command sequences against a model and
the system under test; preconditions and state-dependent arguments are
replayed during shrinking. Relations.hpp provides Equivalent
(differential), Metamorphic, RoundTrip and Idempotent bodies.

PropertyOptions, PropertyStatus, PropertyFailure and PropertyResult
gain fields and statuses at their ends (kInsufficientCoverage,
kGaveUp); existing designated initializers and reports are unchanged.
The engine's untemplated parts are compiled in ChoiceShrinker.cpp,
PropertyEngine.cpp and Context.cpp, and every build system (Bazel,
CMake, Meson, Xmake, Premake) lists the new files and tests.

Docs: progmasoft-properties.md describes the engine,
progmasoft-roadmap.md the steps that follow (the standalone catch3
runner, mocking, per-test coverage, change-impact selection and
prioritization, mutation testing, guided test generation). The
.clang-format comment names Progmasoft C++ Style instead of Visual X#.

Verified with clang-cl 23 through CMake: ProgmasoftSelfTest passes, 57
test cases and 75,461 assertions, including the shrinking results
(1001 for x &lt;= 1000, { 10 }, { 0, 0, 0 } through FlatMap, 102 through
a filter, { 0, 100 } for two arguments, four push(0) steps for a
queue that drops its fourth element, { 0, 0 } for a sort that loses
duplicates) and the database, replay, coverage and discard behaviour.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add snapshots and JUnit result serialization</title>
<updated>2026-09-24T20:05:47+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-09-24T20:05:47+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=d79b294903e4d0f4b24fdf1cc50be4b0fec7f745'/>
<id>d79b294903e4d0f4b24fdf1cc50be4b0fec7f745</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add Progmasoft Catch3 property testing and build support</title>
<updated>2026-09-24T19:07:03+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-09-24T19:07:03+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=9ad804cedf4cc4a85ef89fe097f97a3b44654360'/>
<id>9ad804cedf4cc4a85ef89fe097f97a3b44654360</id>
<content type='text'>
Introduce deterministic property execution, built-in generators, shrinking, replayable seeds, and a separate Progmasoft-owned header API while preserving Catch2 compatibility targets and the C++14 engine baseline.

Add Bazel, Meson, Xmake, and Premake integration and tests alongside the existing CMake build. Replace Linux-only workflow coverage with Windows and macOS jobs and document the Catch2/Catch3 distinction.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Introduce deterministic property execution, built-in generators, shrinking, replayable seeds, and a separate Progmasoft-owned header API while preserving Catch2 compatibility targets and the C++14 engine baseline.

Add Bazel, Meson, Xmake, and Premake integration and tests alongside the existing CMake build. Replace Linux-only workflow coverage with Windows and macOS jobs and document the Catch2/Catch3 distinction.
</pre>
</div>
</content>
</entry>
<entry>
<title>Don't instantiate the default implementation of benchmarks in every TU</title>
<updated>2026-08-22T21:01:51+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-08-22T17:00:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=69da66ee7370d7c249ba6f2a322a78dcdd623761'/>
<id>69da66ee7370d7c249ba6f2a322a78dcdd623761</id>
<content type='text'>
The underlying benchmarking machinery is templated, so that it can use
different clock implementations (either for testing, or to support user
with platform-specific clocks). However, 99.99% of all users interact
with it using the `BENCHMARK` macro, which means that they use it with
`std::chrono::steady_clock`.

By adding outlined implementation of benchmarking helpers specialized
onto `std::chrono::steady_clock`, we save some amount of time per every
TU that uses benchmarks.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The underlying benchmarking machinery is templated, so that it can use
different clock implementations (either for testing, or to support user
with platform-specific clocks). However, 99.99% of all users interact
with it using the `BENCHMARK` macro, which means that they use it with
`std::chrono::steady_clock`.

By adding outlined implementation of benchmarking helpers specialized
onto `std::chrono::steady_clock`, we save some amount of time per every
TU that uses benchmarks.
</pre>
</div>
</content>
</entry>
<entry>
<title>Rework internals of CATCH_REGISTER_ENUM</title>
<updated>2026-04-08T10:57:09+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-04-08T08:26:57+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=47200ddbee03eaf4a9d3d99d18cb692381d8f5bc'/>
<id>47200ddbee03eaf4a9d3d99d18cb692381d8f5bc</id>
<content type='text'>
The old internals reached into the global hub to stash the allocation(s)
for enum value -&gt; string value there, then kept around a potentially
invalid (in case the hub was cleaned up) reference into it, going through
whole bunch of virtual dispatch in the process.

The new internals just store the data in a static variable inside the
`StringMaker` specialization. This avoids potential lifetime issues,
avoids all virtual dispatch and (almost) reduces the include bloat in
the main header path.

The reason for (almost) there is that for full include correctness,
`EnumInfo` needs `&lt;utility&gt;` include for `std::pair`. However, this brings
in things like `std::relops`, because the std headers in C++ are dumb.
As this was not included before, and instead we relied on `std::pair`
existing in an internal stdlib header that we transitively included, the
full include size ends up bigger than before.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The old internals reached into the global hub to stash the allocation(s)
for enum value -&gt; string value there, then kept around a potentially
invalid (in case the hub was cleaned up) reference into it, going through
whole bunch of virtual dispatch in the process.

The new internals just store the data in a static variable inside the
`StringMaker` specialization. This avoids potential lifetime issues,
avoids all virtual dispatch and (almost) reduces the include bloat in
the main header path.

The reason for (almost) there is that for full include correctness,
`EnumInfo` needs `&lt;utility&gt;` include for `std::pair`. However, this brings
in things like `std::relops`, because the std headers in C++ are dumb.
As this was not included before, and instead we relied on `std::pair`
existing in an internal stdlib header that we transitively included, the
full include size ends up bigger than before.
</pre>
</div>
</content>
</entry>
<entry>
<title>Support filtering on both sections and generators (#3069)</title>
<updated>2026-02-15T19:27:15+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-02-15T19:27:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=3a0cf7e75fdc571ff9ed25303fb268665f6260c0'/>
<id>3a0cf7e75fdc571ff9ed25303fb268665f6260c0</id>
<content type='text'>
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.

With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.

With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
</pre>
</div>
</content>
</entry>
<entry>
<title>Only use thread_local in builds with thread safety enabled</title>
<updated>2025-12-02T13:01:55+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-12-01T13:18:54+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=32eac2d1bb63d6b58a32e341e22b9438f1bd041a'/>
<id>32eac2d1bb63d6b58a32e341e22b9438f1bd041a</id>
<content type='text'>
MSVC cannot dllexport thread_local variables, so we avoid making
globals thread local if we won't support multiple threads anyway.

Closes #3044
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
MSVC cannot dllexport thread_local variables, so we avoid making
globals thread local if we won't support multiple threads anyway.

Closes #3044
</pre>
</div>
</content>
</entry>
<entry>
<title>Add CATCH_ATTR_LIFETIMEBOUND macro polyfill over lifetimebound attr</title>
<updated>2025-10-03T20:15:27+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-10-03T20:15:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=8219ed79f29420c89748ae5eb8810946cefd491d'/>
<id>8219ed79f29420c89748ae5eb8810946cefd491d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add configuration option to make assertions thread-safe</title>
<updated>2025-07-24T08:10:00+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-17T20:58:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=2a8a8a7210513fc2f6ae5e65986c0f2d730c1537'/>
<id>2a8a8a7210513fc2f6ae5e65986c0f2d730c1537</id>
<content type='text'>
All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948
</pre>
</div>
</content>
</entry>
</feed>
