catch3/benchmarks, branch v3.16.0 Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results. catch_discover_tests: Escape test-invariant parts of CTest script only once 2026-08-10T09:53:22+00:00 Martin Hořeňovský 2026-08-10T09:45:08+00:00 97ec4e8e2e4a86719701746d1cb2426f16292ff8 Previously, the `catch_discover_tests` would prepare the entire CTest command (e.g. `add_test(...)` or `set_tests_properties(...)`) first, and then escape it when finished. However, this caused lot of the command args to be escaped over and over again (e.g. executable name or Catch2's reporter args), for no reason, as they were always the same, and thus their escaping was always the same. Until recently, the performance overhead didn't matter as there were many spots which had quadratic runtime in number of tests. However, the recent refactorings fixed these, and this commit now improves the throughput by 10-20%. Also extended the benchmarked COUNTS in `benchmark_discovery.py`, because the performance is now good enough that it is reasonable to benchmark 16k tests. 
Previously, the `catch_discover_tests` would prepare the entire CTest
command (e.g. `add_test(...)` or `set_tests_properties(...)`) first, and
then escape it when finished. However, this caused lot of the command args
to be escaped over and over again (e.g. executable name or Catch2's reporter
args), for no reason, as they were always the same, and thus their escaping
was always the same.

Until recently, the performance overhead didn't matter as there were many
spots which had quadratic runtime in number of tests. However, the recent
refactorings fixed these, and this commit now improves the throughput by
10-20%.

Also extended the benchmarked COUNTS in `benchmark_discovery.py`,
because the performance is now good enough that it is reasonable
to benchmark 16k tests.
JSONReporter considers verbosity when listing tests 2026-08-07T22:05:18+00:00 Martin Hořeňovský 2026-07-28T20:08:55+00:00 4cde12851773b32fc5a0171f262e4281c8e28b01 * Quiet verbosity provides just the test names * Normal verbosity adds tags * High verbosity add the source location of the tests Listing tests with the quiet verbosity results in about 1/4 of the previous output, which leads to measurably faster execution of `catch_discover_tests` when it does not need tags for labels. (If it does need tags, the output is about 1/2 of previous). 
* Quiet verbosity provides just the test names
* Normal verbosity adds tags
* High verbosity add the source location of the tests

Listing tests with the quiet verbosity results in about 1/4 of the
previous output, which leads to measurably faster execution of
`catch_discover_tests` when it does not need tags for labels.
(If it does need tags, the output is about 1/2 of previous).
JSONReporter's listTests only lists class-name if it is non-empty 2026-08-07T21:07:46+00:00 Martin Hořeňovský 2026-07-29T08:05:39+00:00 0b4d7a5a1696dbdfb23fccad8c50db9dc9a417ea 






Add Python script for benchmarking catch_discover_tests
2026-07-24T21:53:50+00:00

Martin Hořeňovský

2026-07-24T17:12:28+00:00

f5db82a5c47cc92ec59e6216d0b63aa866b2885d












Add more runtime benchmark recipe TODOs
2026-05-07T19:13:11+00:00

Martin Hořeňovský

2026-05-07T19:13:11+00:00

54af40652aca0403298b8c2f7c76c8c851132199












Add benchmark for catch_test_macros.hpp inclusion
2026-05-07T18:27:09+00:00

Martin Hořeňovský

2026-05-07T11:42:01+00:00

5f47160bb2f5e6fb07600ef79207e4e30c4fd3fb












Add readme to benchmarks/ with some simple examples
2026-05-07T08:43:24+00:00

Martin Hořeňovský

2026-05-07T08:43:24+00:00

e83528c6eb757b263f71c0672358e8632d8f453f












Add benchmark for REQUIRE_THAT into runtime_assertion_benches.cpp
2026-05-07T08:18:21+00:00

Martin Hořeňovský

2026-05-06T20:32:25+00:00

fd31a62547e2f4f217dda2e1312b36f369af7252

I also changed the names of other benchmark `TEST_CASE`s so that
they are easier to use.





I also changed the names of other benchmark `TEST_CASE`s so that
they are easier to use.







Fix Wweak-vtables in benchmarks/assertion_listener.cpp
2026-01-03T12:43:27+00:00

Martin Hořeňovský

2025-12-29T21:14:49+00:00

ec4dcbf9cb6d85f410a08c02c1b2cce2b2101b64












Add simple runtime benchmarks
2025-09-23T15:17:33+00:00

Martin Hořeňovský

2025-09-23T15:17:29+00:00

b626e4c7aedb953e79784265e12283a62d5e94d3

For now we add just two binaries, one with assertions taking the
fast path, one with assertions taking the slow path, and the ability
to run 1 of `REQUIRE(true)`, `REQUIRE_NOTHROW`, `REQUIRE_THROWS`
in a loop.

I also split off a CMake preset which enables more tests than the
basic `simple-tests` preset, but does not enable the most expensive
tests which force recompilation of Catch2 multiple times.





For now we add just two binaries, one with assertions taking the
fast path, one with assertions taking the slow path, and the ability
to run 1 of `REQUIRE(true)`, `REQUIRE_NOTHROW`, `REQUIRE_THROWS`
in a loop.

I also split off a CMake preset which enables more tests than the
basic `simple-tests` preset, but does not enable the most expensive
tests which force recompilation of Catch2 multiple times.