# Progmasoft Catch3 additions Progmasoft Catch3 keeps Catch2's API for writing tests, its macros and the `Catch::` namespace, and replaces Catch2's runner with its own. The [Catch3 runner](progmasoft-runner.md#top) runs tests in parallel worker processes and adds retries, flaky-test detection, quarantine, history, predictive ordering, test selection, sharding, time limits and failure analysis. Every header is in the `Progmasoft/Catch3` include tree; the files of the engine that come from Catch2 keep its BSL-1.0 license, and Progmasoft's own API is in the `Progmasoft::Catch3` namespace. ## Property testing Catch3 has two property APIs. [Native property testing](progmasoft-properties.md#top) with `ForAll` and strategies is the full engine: every combination of strategies shrinks, failures replay from a token and from an example database, and it adds state machines, statistics, targeted search and differential and metamorphic oracles. New tests should use it. The value-shrinking `CheckProperty` API described here remains supported for existing tests. [The roadmap](progmasoft-roadmap.md#top) lists what comes next. `` is the umbrella header. It brings in the engine's `Progmasoft/Catch3/All.hpp` as well as the Progmasoft extensions, so it is convenient but increases compile time. Include individual headers when a translation unit only needs a narrow part of the API. ```cpp #include TEST_CASE("integer values stay in their requested domain", "[property]") { CATCH3_CHECK_PROPERTY( Progmasoft::Catch3::Integer(1, 100), [](int value) { return value > 0; }, Progmasoft::Catch3::PropertyOptions{ .Trials = 250, .Seed = 0xC0FFEE, .MaxShrinkSteps = 200}); } ``` The current property API is split into these headers: - ``: portable SplitMix64 sequence, unbiased bounded sampling, inclusive integer ranges, and per-trial seed derivation. - ``: user-defined generation/shrinking pair. - `` and `Boolean.hpp`: built-in scalar strategies with deterministic shrink candidates. - ``, `Tuple.hpp`, `Element.hpp` and `Text.hpp`: composite strategies described under [Composite generators](#composite-generators). - ``: trial execution and structured results. - ``: `CATCH3_CHECK_PROPERTY`, which reports the failing trial, replay seed, reason, and minimized counterexample through Catch2's normal assertion/reporting path. `PropertyOptions::Seed` is a stable root seed. Each trial receives a seed derived from the root and its zero-based trial index, so the reported replay seed can be used to reproduce that trial independently. The default generator is also stable across standard-library implementations; it does not rely on the implementation- defined output sequence of `std::uniform_int_distribution`. Shrinking is deterministic and greedy: candidates are tested in the order provided by the generator, and the first still-failing candidate is selected. `MaxShrinkSteps` bounds predicate evaluations during minimization. The result is therefore a useful locally minimized counterexample, not a claim of a globally minimal value. A false predicate or an exception from the predicate fails the property. Exceptions from generation or shrinking are reported as execution errors, distinct from an ordinary failing property. A failing report ends with the line `replay: set PropertyOptions::ReplayTrialSeed to `. Setting that option runs exactly one trial with the given seed, including its shrinking, and ignores `Seed` and `Trials`; the reported trial number is then 1. This reproduces a failure from a CI log without rerunning the trials before it. ```cpp Progmasoft::Catch3::PropertyOptions{ .ReplayTrialSeed = 1234567890123456789ULL } ``` ### Composite generators A property still receives one generated value. Several arguments are passed as one tuple: ```cpp using namespace Progmasoft::Catch3; CATCH3_CHECK_PROPERTY( Tuple(Vector(Integer(0, 100), 0, 20), Text("abc", 1, 8)), [](const std::tuple, std::string> &input) { return Accepts(std::get<0>(input), std::get<1>(input)); }, PropertyOptions{ .Trials = 500 }); ``` - `Tuple(generators...)` draws its components strictly left to right from the trial's random source and shrinks one component at a time: every candidate of the first component, then every candidate of the second, and so on. - `Vector(element, minimumSize, maximumSize)` draws a length in the inclusive range and then the elements in index order. `Text(alphabet, minimumLength, maximumLength)` does the same for byte strings over the given alphabet; `PrintableAscii()` is the default alphabet. Both shrink in a fixed order: the shortest allowed prefix, the first and second half, the value with one element removed, and finally the value with one element simplified. A shrunk value is never shorter than its minimum, each step offers at most 256 candidates, and a size limit above 1,048,576 elements is rejected. - `Element(values)` picks uniformly from a fixed non-empty list. The list order is the simplicity order: a value shrinks to the values listed before it. `Text` treats its alphabet the same way, so put the simplest value or character first. Counterexamples of tuples, pairs and vectors are printed element by element, for example `{ 7, { true, false }, "text" }`. This does not depend on Catch2's optional tuple or range string makers, so the report is the same whatever configuration macros a client defines. Scalars still use `Catch::StringMaker`, including user specializations. Invalid factory arguments, such as an empty alphabet or a minimum above its maximum, throw `std::invalid_argument`; a client built without exceptions gets the message on standard error followed by `std::abort`. Mapped, filtered and dependent generators and state machines are not part of this module; the strategies of [native property testing](progmasoft-properties.md#top) provide them. Shrinking a composite value here is greedy and local like scalar shrinking: it finds a small counterexample, not a provably smallest one. ## Snapshots `` provides file-backed text snapshots through `CompareSnapshot(name, actual, options)`. The directory is supplied explicitly; names are restricted to portable ASCII filename characters and cannot contain path separators. CRLF and CR are normalized to LF by default, with no trimming or other content rewriting. Ordinary comparisons never write files. `SnapshotUpdateMode::kCreateMissing` creates only absent snapshots, while `kAlways` explicitly permits replacement. The result reports a match, create/update, mismatch with a one-based first difference location, invalid input, or I/O failure. `CATCH3_CHECK_SNAPSHOT` combines this result with Catch2's normal assertion reporting. Keep snapshots under a directory owned by the test suite and enable replacement only for an intentional update run. ```cpp const Progmasoft::Catch3::SnapshotOptions options{ .Directory = "tests/snapshots", }; CATCH3_CHECK_SNAPSHOT("rendered-report", RenderReport(), options); ``` ## Structured results and XML `` defines the model the runner's reports are written from: a `RunResult` holds a `TestResult` per test, with its verdict, failure category, history classification, flakiness, quarantine, failure signature, analysis and replay command, and every `Attempt` with its assertions, sections, benchmarks, captured output and the tests that ran before it in the same process. `Summarize` counts the verdicts, and `DescribeFailure` writes a failed attempt as text. `` writes a run as deterministic JUnit XML through `JUnitXmlWriter::Write` or `ToString`, with `` and `` for retries. It escapes XML markup, replaces malformed UTF-8 and XML-forbidden control characters, and omits timestamps so identical results produce identical output. The results, the JUnit writer, the runner, property testing and the snapshot API are compiled into the `Catch3::Catch3` library. Generator templates and convenience macros remain in headers where their types or call-site expansion require it. ## Build and integration The CMake package is `Catch3`, with the targets `Catch3::Catch3` and `Catch3::Catch3WithMain`. The library is C++17, and its headers work in C++17 and later. After installation, consumers can use: ```cmake find_package(Catch3 0.3 CONFIG REQUIRED) target_link_libraries(MyTests PRIVATE Catch3::Catch3WithMain) ``` The Bazel target is `//:catch3`; its tests are `//tests/Progmasoft:property_tests`. Bazel is the primary build path; CMake, Meson, Xmake, and Premake are also maintained as alternate ways to build the same engine and property tests. Their root files are `CMakeLists.txt`, `meson.build`, `xmake.lua`, and `premake5.lua`. Xmake and Premake materialize the default generated configuration header, `Progmasoft/Catch3/UserConfig.hpp`, from the checked-in template. Meson 1.1.0 or newer is required for the Clang-CL C++17 test target. For project history, compatibility intent, license split, and current feature limits, read [Why Progmasoft Catch3?](why-catch3.md). ## Build quick start Bazel is the primary build path: ```sh bazel test //tests/Progmasoft:property_tests ``` The alternate project builds exercise the same property suite: ```sh # CMake cmake -S . -B build/cmake -DCATCH3_DEVELOPMENT_BUILD=ON -DCATCH3_BUILD_TESTING=ON cmake --build build/cmake --target ProgmasoftSelfTest ctest --test-dir build/cmake -R Progmasoft --output-on-failure # Xmake xmake build ProgmasoftPropertyTests xmake run ProgmasoftPropertyTests # Meson (Windows PowerShell; requires the Visual C++ SDK/CRT) $env:CC = 'clang-cl'; $env:CXX = 'clang-cl' meson setup build/meson --vsenv -Dtests=true -Dinstall=false meson compile -C build/meson meson test -C build/meson --print-errorlogs # Premake (choose an action supported by the host) premake5 vs2022 ``` Premake generates projects rather than compiling directly; build the generated solution with the corresponding host toolchain. On macOS, use `premake5 gmake`, then build the generated makefiles with `make`. Meson users on macOS can select Apple Clang with `CC=clang CXX=clang++` before configuration. ## Compatibility and licensing boundary Do not rename or rewrite Catch2 headers, `Catch` symbols, macros, runner options, or upstream coding style as part of this extension. New functionality belongs in separately licensed Progmasoft headers and uses Progmasoft's PascalCase C++ types/functions and project naming conventions. Catch2-originated source files remain under BSL-1.0; Progmasoft-authored source files carry the MPL-2.0 plus Progmasoft linking-exception SPDX expression shown in their headers. The two areas are not relicensed as a whole. Any proposed upstream PR must be prepared under Catch2's contribution rules as a separate contribution, rather than reusing a source file whose license header says otherwise.