<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/tests/SelfTest/IntrospectiveTests/CmdLine.tests.cpp, branch devel</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Replace Catch2's runner with the Catch3 runner</title>
<updated>2026-10-23T20:25:41+00:00</updated>
<author>
<name>Leitwolf11</name>
</author>
<published>2026-10-23T20:25:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=93d4e7dbebab867e0a7f59fc25308460a7a76280'/>
<id>93d4e7dbebab867e0a7f59fc25308460a7a76280</id>
<content type='text'>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
</pre>
</div>
</content>
</entry>
<entry>
<title>Emit warning when using --shard-count (&gt; 1) with --order rand</title>
<updated>2026-08-10T22:08:23+00:00</updated>
<author>
<name>Matt Rasa</name>
</author>
<published>2026-07-31T13:37:21+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=0aeb818520028ca05cc2539ed242402ef7a7ae41'/>
<id>0aeb818520028ca05cc2539ed242402ef7a7ae41</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make verbosity per-reporter</title>
<updated>2026-08-07T18:57:42+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-08-01T14:52:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=a0eba50e9d56c743aff1de7a50d02a6e01e73cdc'/>
<id>a0eba50e9d56c743aff1de7a50d02a6e01e73cdc</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Support filtering on both sections and generators (#3069)</title>
<updated>2026-02-15T19:27:15+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-02-15T19:27:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=3a0cf7e75fdc571ff9ed25303fb268665f6260c0'/>
<id>3a0cf7e75fdc571ff9ed25303fb268665f6260c0</id>
<content type='text'>
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.

With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.

With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add --warn InfiniteGenerators</title>
<updated>2026-02-10T08:37:19+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2026-02-10T08:16:03+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=daadf42a0e21ac09abc1f286f875078b24e2dd4d'/>
<id>daadf42a0e21ac09abc1f286f875078b24e2dd4d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add validation for --benchmark-samples to prevent crash with zero value (#3056)</title>
<updated>2026-01-02T22:15:13+00:00</updated>
<author>
<name>qerased</name>
</author>
<published>2026-01-02T22:15:13+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=b66b89374e60fa2fb725814acefbfc987ee5ff5e'/>
<id>b66b89374e60fa2fb725814acefbfc987ee5ff5e</id>
<content type='text'>
* Add validation for --benchmark-samples to prevent crash with zero samples
* Add automated test for benchmark samples validation
* Update baselines for benchmark-samples validation
* Add missing test for benchmark samples validation

---------

Co-authored-by: Chan Aung &lt;chan@Thinkpad.localdomain&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Add validation for --benchmark-samples to prevent crash with zero samples
* Add automated test for benchmark samples validation
* Update baselines for benchmark-samples validation
* Add missing test for benchmark samples validation

---------

Co-authored-by: Chan Aung &lt;chan@Thinkpad.localdomain&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix references to license file</title>
<updated>2022-10-28T09:30:15+00:00</updated>
<author>
<name>Lars Toenning</name>
</author>
<published>2022-10-28T09:22:53+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=f1084fb30990fe40c8f5b70034970c7eb4e1da72'/>
<id>f1084fb30990fe40c8f5b70034970c7eb4e1da72</id>
<content type='text'>
The license file was renamed with 6a502cc2f5fedd59b3495b58708f0d6d987ed9e1
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The license file was renamed with 6a502cc2f5fedd59b3495b58708f0d6d987ed9e1
</pre>
</div>
</content>
</entry>
<entry>
<title>Split tests for TestSpec into its own file</title>
<updated>2022-10-26T19:03:56+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2022-10-26T19:03:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=c6dfeb5e7dc175ca4765e884414dab52736cde9e'/>
<id>c6dfeb5e7dc175ca4765e884414dab52736cde9e</id>
<content type='text'>
They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090
</pre>
</div>
</content>
</entry>
<entry>
<title>Refactor parsing of shard index/count in cmdline handling</title>
<updated>2022-10-22T22:10:05+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2022-10-22T22:09:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=a43f67962e632d9596680a7e139df9e0a1423d3a'/>
<id>a43f67962e632d9596680a7e139df9e0a1423d3a</id>
<content type='text'>
This worsens the message for negative numbers a bit, but
simplifies the code enough that this is still a win.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This worsens the message for negative numbers a bit, but
simplifies the code enough that this is still a win.
</pre>
</div>
</content>
</entry>
<entry>
<title>Refactor how the RNG seed is parsed when handling cmdline</title>
<updated>2022-10-22T20:18:30+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2022-10-22T15:04:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=f1361ef6243ec88038a6b4f013c405f2df3df376'/>
<id>f1361ef6243ec88038a6b4f013c405f2df3df376</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
