catch3/premake5.lua, branch docs/describe-catch3 Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results. End the output capture when a test exits, fail on missing reports 2026-10-24T09:41:08+00:00 Leitwolf11 2026-10-24T09:41:08+00:00 e18f19b7dd70c99fbcf2b3fe812ee18dabc92663 A test that called exit() crashed its worker with SIGSEGV on Linux and macOS. The exit hook read Catch2's captured output through a scoped deactivation, which switched the capture back on; libstdc++ then flushed std::cout at exit into a capture buffer that static destruction had already freed. RunContext::endCapture() now ends the capture for good and returns what it held, so the standard streams are their own again. A JUnit, JSON or trace report that could not be written was only reported on stderr, and a run whose tests passed still exited with 0, so a build server could miss it. Such a run now exits with 1; failed tests still exit with 42. Integration tests of the runner's output move their helpers into RunnerFixtureClient.hpp and cover --version, --help, the listing modes, a bad option, exit codes 1 to 4, -v, -q, --slowest, colours, --analytics in text and JSON, and Bazel's test environment. 
A test that called exit() crashed its worker with SIGSEGV on Linux and
macOS. The exit hook read Catch2's captured output through a scoped
deactivation, which switched the capture back on; libstdc++ then flushed
std::cout at exit into a capture buffer that static destruction had
already freed. RunContext::endCapture() now ends the capture for good and
returns what it held, so the standard streams are their own again.

A JUnit, JSON or trace report that could not be written was only
reported on stderr, and a run whose tests passed still exited with 0, so a
build server could miss it. Such a run now exits with 1; failed tests
still exit with 42.

Integration tests of the runner's output move their helpers into
RunnerFixtureClient.hpp and cover --version, --help, the listing modes, a
bad option, exit codes 1 to 4, -v, -q, --slowest, colours, --analytics
in text and JSON, and Bazel's test environment.
Fix the CI failures of the runner and test its planner, options and analysis 2026-10-24T07:06:25+00:00 Leitwolf11 2026-10-24T07:06:25+00:00 e104b694513964d5a84e77f751203cda970d741b ExtraTests X21 and X27 defined their listeners at namespace scope, which clang rejects under -Wweak-vtables with -Werror on Linux and macOS; they are now in anonymous namespaces. After a fatal signal Catch2 closes the open sections itself, so on POSIX a crash was reported outside the section it happened in. The scheduler now ignores section ends that follow a fatal condition. New unit tests cover the planner (every selection, dependency closure and cycles, hash and duration shards, incremental runs, learned time limits, ordering, retries, quarantine and replay), every command-line option and its errors, and the failure analysis (history classes, signature groups, crash and hang locations, changed files, replay commands). Catalog::FromEntries builds a catalog without a test registry so the planner can be tested on its own. 
ExtraTests X21 and X27 defined their listeners at namespace scope, which
clang rejects under -Wweak-vtables with -Werror on Linux and macOS; they
are now in anonymous namespaces.

After a fatal signal Catch2 closes the open sections itself, so on POSIX a
crash was reported outside the section it happened in. The scheduler now
ignores section ends that follow a fatal condition.

New unit tests cover the planner (every selection, dependency closure and
cycles, hash and duration shards, incremental runs, learned time limits,
ordering, retries, quarantine and replay), every command-line option and
its errors, and the failure analysis (history classes, signature groups,
crash and hang locations, changed files, replay commands).
Catalog::FromEntries builds a catalog without a test registry so the
planner can be tested on its own.
Replace Catch2's runner with the Catch3 runner 2026-10-23T20:25:41+00:00 Leitwolf11 2026-10-23T20:25:41+00:00 93d4e7dbebab867e0a7f59fc25308460a7a76280 Catch3 keeps Catch2's API for writing tests (test cases, sections, assertions, matchers, generators, listeners, benchmarks) and replaces its session, command line and reporters with a runner of its own. The runner starts the test binary again as worker processes that take the next test when they are free. A crash, an exit or a hang ends one test, not the run. On top of that it adds: - retries that tell deterministic, flaky and order-dependent failures apart, and a bisection that names the test leaving state behind; - a history of verdicts and durations, kept under a file lock, that drives automatic quarantine, adaptive time limits, predictive ordering, incremental runs, failure replay and analytics; - selection by change, past failure or prediction, dependency closure, and sharding by a stable hash or by recorded durations; - tags for serial and isolated tests, shared resources, dependencies, time limits and retries; - a failure category, a signature, an analysis and a replay command for every failure; - console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome trace reports, and a versioned discovery listing (--list=json); - a temporary directory per test (TMPDIR, TMP, TEMP and TestTemporaryDirectory()); - Bazel's test environment: XML output file, test filter, sharding, seed, premature exit file and undeclared outputs. The command line is parsed by a hand-written, typed parser. Catch::Session remains as a thin facade over the runner for custom main functions. The result model in Progmasoft/Catch3/Results.hpp was rewritten around tests and attempts, and the JUnit writer now writes it. The property engine's example database is updated under a file lock with an atomic rename, so parallel workers can write it at the same time. Removed: Clara, the command-line and reporter-spec parsers, every reporter and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width, the JSON and XML writers, sharding in the test registry, approval tests and their baselines, Conan packaging, and the unused third_party/clara.hpp. Listeners keep working; reporters' list and skip events are gone. The Catch2 library is now compiled as C++20; code that includes only Catch2's headers still needs C++14. Duplicate test cases are still refused at start-up. Version 0.3.0. ExtraTests now turn exceptions off for clang-cl with MSVC-style options, which -fno-exceptions did not do. 
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.

The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:

- retries that tell deterministic, flaky and order-dependent failures
  apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
  automatic quarantine, adaptive time limits, predictive ordering,
  incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
  and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
  time limits and retries;
- a failure category, a signature, an analysis and a replay command for
  every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
  trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
  TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
  premature exit file and undeclared outputs.

The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.

The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.

Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.

The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.

ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
Add a native property-testing engine with choice-sequence shrinking 2026-10-22T14:39:46+00:00 Leitwolf11 2026-10-22T14:39:46+00:00 afac14ac5a62d8d2e7efd2184ad34e16593eddfd Catch3's own property engine, independent of RapidCheck, next to the existing CheckProperty API, which stays unchanged. Strategies (Progmasoft/Catch3/Strategy/): a Strategy<T> builds a value from a ChoiceSource, a stream of integer choices where 0 is always the simplest decision. Gen:: provides Integers, Booleans, Floats (ranges, NaN and infinities on request, edge values), Characters, Strings, Utf8Strings, Vectors, UniqueVectors, Sets, Maps, Optionals, Tuples, Construct, Permutations, Just, SampledFrom, OneOf, Frequency, Lazy and Recursive; every strategy has Map, Filter and FlatMap. Shrinking: the engine records the choices of a failing trial and shrinks them, not the value, with deletion and zeroing of blocks, binary search per choice with a neighbour step for non-monotone conditions, sorting and moving value between neighbours. Replaying the shrunk choices through the same strategies means mapped, filtered, dependent and stateful inputs all shrink, and a shrunk input still satisfies its filters. ForAll([options,] strategy..., body) runs the trials (size rising over the first half), then a targeted search when the body calls Target, then checks Cover requirements. A failure report carries the minimized input, the notes and a replay token (PropertyOptions::Replay) that runs exactly that input. An example database (DatabasePath or CATCH3_PROPERTY_DATABASE) replays stored failures first and drops them once they pass. Bodies return bool, or void with CATCH3_PROPERTY_ASSERT and CATCH3_PROPERTY_ASSUME, which return instead of throwing; Note, Classify, Cover and Target record into a per-thread trial record. CATCH3_CHECK_FOR_ALL and CATCH3_REQUIRE_FOR_ALL report through Catch2 and name the property after the test case for the database. CheckStateful runs model-based command sequences against a model and the system under test; preconditions and state-dependent arguments are replayed during shrinking. Relations.hpp provides Equivalent (differential), Metamorphic, RoundTrip and Idempotent bodies. PropertyOptions, PropertyStatus, PropertyFailure and PropertyResult gain fields and statuses at their ends (kInsufficientCoverage, kGaveUp); existing designated initializers and reports are unchanged. The engine's untemplated parts are compiled in ChoiceShrinker.cpp, PropertyEngine.cpp and Context.cpp, and every build system (Bazel, CMake, Meson, Xmake, Premake) lists the new files and tests. Docs: progmasoft-properties.md describes the engine, progmasoft-roadmap.md the steps that follow (the standalone catch3 runner, mocking, per-test coverage, change-impact selection and prioritization, mutation testing, guided test generation). The .clang-format comment names Progmasoft C++ Style instead of Visual X#. Verified with clang-cl 23 through CMake: ProgmasoftSelfTest passes, 57 test cases and 75,461 assertions, including the shrinking results (1001 for x <= 1000, { 10 }, { 0, 0, 0 } through FlatMap, 102 through a filter, { 0, 100 } for two arguments, four push(0) steps for a queue that drops its fourth element, { 0, 0 } for a sort that loses duplicates) and the database, replay, coverage and discard behaviour. 
Catch3's own property engine, independent of RapidCheck, next to the
existing CheckProperty API, which stays unchanged.

Strategies (Progmasoft/Catch3/Strategy/): a Strategy<T> builds a value
from a ChoiceSource, a stream of integer choices where 0 is always the
simplest decision. Gen:: provides Integers, Booleans, Floats (ranges,
NaN and infinities on request, edge values), Characters, Strings,
Utf8Strings, Vectors, UniqueVectors, Sets, Maps, Optionals, Tuples,
Construct, Permutations, Just, SampledFrom, OneOf, Frequency, Lazy and
Recursive; every strategy has Map, Filter and FlatMap.

Shrinking: the engine records the choices of a failing trial and
shrinks them, not the value, with deletion and zeroing of blocks,
binary search per choice with a neighbour step for non-monotone
conditions, sorting and moving value between neighbours. Replaying the
shrunk choices through the same strategies means mapped, filtered,
dependent and stateful inputs all shrink, and a shrunk input still
satisfies its filters.

ForAll([options,] strategy..., body) runs the trials (size rising over
the first half), then a targeted search when the body calls Target,
then checks Cover requirements. A failure report carries the
minimized input, the notes and a replay token (PropertyOptions::Replay)
that runs exactly that input. An example database (DatabasePath or
CATCH3_PROPERTY_DATABASE) replays stored failures first and drops them
once they pass. Bodies return bool, or void with CATCH3_PROPERTY_ASSERT
and CATCH3_PROPERTY_ASSUME, which return instead of throwing; Note,
Classify, Cover and Target record into a per-thread trial record.
CATCH3_CHECK_FOR_ALL and CATCH3_REQUIRE_FOR_ALL report through Catch2
and name the property after the test case for the database.

CheckStateful runs model-based command sequences against a model and
the system under test; preconditions and state-dependent arguments are
replayed during shrinking. Relations.hpp provides Equivalent
(differential), Metamorphic, RoundTrip and Idempotent bodies.

PropertyOptions, PropertyStatus, PropertyFailure and PropertyResult
gain fields and statuses at their ends (kInsufficientCoverage,
kGaveUp); existing designated initializers and reports are unchanged.
The engine's untemplated parts are compiled in ChoiceShrinker.cpp,
PropertyEngine.cpp and Context.cpp, and every build system (Bazel,
CMake, Meson, Xmake, Premake) lists the new files and tests.

Docs: progmasoft-properties.md describes the engine,
progmasoft-roadmap.md the steps that follow (the standalone catch3
runner, mocking, per-test coverage, change-impact selection and
prioritization, mutation testing, guided test generation). The
.clang-format comment names Progmasoft C++ Style instead of Visual X#.

Verified with clang-cl 23 through CMake: ProgmasoftSelfTest passes, 57
test cases and 75,461 assertions, including the shrinking results
(1001 for x <= 1000, { 10 }, { 0, 0, 0 } through FlatMap, 102 through
a filter, { 0, 100 } for two arguments, four push(0) steps for a
queue that drops its fourth element, { 0, 0 } for a sort that loses
duplicates) and the database, replay, coverage and discard behaviour.
Add composite property generators, single-trial replay and stable counterexample printing 2026-10-02T00:27:05+00:00 Leitwolf11 2026-10-02T00:27:05+00:00 5d55a3226360fae1eaa5ef91908b0fa7d7d07234 Property tests could only generate one integer or Boolean. This adds the composite strategies the documentation listed as missing, without changing any Catch2 file or the existing Progmasoft API. Generators: - Tuple(generators...) draws components left to right and shrinks one component at a time, which is how a property receives several arguments. - Vector(element, minimumSize, maximumSize) and Text(alphabet, minimumLength, maximumLength) share one ordered shrink: shortest allowed prefix, the two halves, single removals, then single element simplifications. A shrunk value never falls below its minimum, each step offers at most 256 candidates, and a size limit above 1,048,576 is rejected. - Element(values) picks from a fixed list and shrinks toward earlier entries. Runner: - PropertyOptions::ReplayTrialSeed runs exactly one trial with a reported replay seed, including its shrinking, so a failure from a log can be reproduced without rerunning earlier trials. The report now ends with the line that names this option and the seed. - Counterexamples of tuples, pairs and vectors are printed element by element instead of relying on Catch2's optional string makers, so the report does not depend on configuration macros or include order. The headers still compile for clients built without language exceptions. Thirteen new test cases cover bounds, determinism, shrink order, candidate limits, minimized counterexamples, replay and invalid configurations, and every build description (Bazel, CMake, Meson, Xmake, Premake) lists the new headers and test file. 
Property tests could only generate one integer or Boolean. This adds the
composite strategies the documentation listed as missing, without changing
any Catch2 file or the existing Progmasoft API.

Generators:
- Tuple(generators...) draws components left to right and shrinks one
  component at a time, which is how a property receives several arguments.
- Vector(element, minimumSize, maximumSize) and Text(alphabet,
  minimumLength, maximumLength) share one ordered shrink: shortest allowed
  prefix, the two halves, single removals, then single element
  simplifications. A shrunk value never falls below its minimum, each step
  offers at most 256 candidates, and a size limit above 1,048,576 is
  rejected.
- Element(values) picks from a fixed list and shrinks toward earlier
  entries.

Runner:
- PropertyOptions::ReplayTrialSeed runs exactly one trial with a reported
  replay seed, including its shrinking, so a failure from a log can be
  reproduced without rerunning earlier trials. The report now ends with the
  line that names this option and the seed.
- Counterexamples of tuples, pairs and vectors are printed element by
  element instead of relying on Catch2's optional string makers, so the
  report does not depend on configuration macros or include order.

The headers still compile for clients built without language exceptions.
Thirteen new test cases cover bounds, determinism, shrink order, candidate
limits, minimized counterexamples, replay and invalid configurations, and
every build description (Bazel, CMake, Meson, Xmake, Premake) lists the new
headers and test file.
Fix Windows ClangCL build-system CI: select compatible Conan runner, expose MSBuild, and use ClangCL in Premake and Xmake 2026-09-24T21:22:24+00:00 Leitwolf11 2026-09-24T21:22:24+00:00 cbb65f9db0cead23ea03897eb5a13cf159a054d5 






Add snapshots and JUnit result serialization
2026-09-24T20:05:47+00:00

Leitwolf11

2026-09-24T20:05:47+00:00

d79b294903e4d0f4b24fdf1cc50be4b0fec7f745












Add Progmasoft Catch3 property testing and build support
2026-09-24T19:07:03+00:00

Leitwolf11

2026-09-24T19:07:03+00:00

9ad804cedf4cc4a85ef89fe097f97a3b44654360

Introduce deterministic property execution, built-in generators, shrinking, replayable seeds, and a separate Progmasoft-owned header API while preserving Catch2 compatibility targets and the C++14 engine baseline.

Add Bazel, Meson, Xmake, and Premake integration and tests alongside the existing CMake build. Replace Linux-only workflow coverage with Windows and macOS jobs and document the Catch2/Catch3 distinction.





Introduce deterministic property execution, built-in generators, shrinking, replayable seeds, and a separate Progmasoft-owned header API while preserving Catch2 compatibility targets and the C++14 engine baseline.

Add Bazel, Meson, Xmake, and Premake integration and tests alongside the existing CMake build. Replace Linux-only workflow coverage with Windows and macOS jobs and document the Catch2/Catch3 distinction.