diff options
| author | Leitwolf11 | 2026-10-22 17:39:46 +0300 |
|---|---|---|
| committer | Leitwolf11 | 2026-10-22 17:39:46 +0300 |
| commit | afac14ac5a62d8d2e7efd2184ad34e16593eddfd (patch) | |
| tree | b3902ef230d419f040467eaddcfa87e4fb12f05c /tests | |
| parent | 2c23a508c95c557e2ff9ad74b517e5fc65175387 (diff) | |
| download | catch3-afac14ac5a62d8d2e7efd2184ad34e16593eddfd.tar.gz | |
Add a native property-testing engine with choice-sequence shrinking
Catch3's own property engine, independent of RapidCheck, next to the
existing CheckProperty API, which stays unchanged.
Strategies (Progmasoft/Catch3/Strategy/): a Strategy<T> builds a value
from a ChoiceSource, a stream of integer choices where 0 is always the
simplest decision. Gen:: provides Integers, Booleans, Floats (ranges,
NaN and infinities on request, edge values), Characters, Strings,
Utf8Strings, Vectors, UniqueVectors, Sets, Maps, Optionals, Tuples,
Construct, Permutations, Just, SampledFrom, OneOf, Frequency, Lazy and
Recursive; every strategy has Map, Filter and FlatMap.
Shrinking: the engine records the choices of a failing trial and
shrinks them, not the value, with deletion and zeroing of blocks,
binary search per choice with a neighbour step for non-monotone
conditions, sorting and moving value between neighbours. Replaying the
shrunk choices through the same strategies means mapped, filtered,
dependent and stateful inputs all shrink, and a shrunk input still
satisfies its filters.
ForAll([options,] strategy..., body) runs the trials (size rising over
the first half), then a targeted search when the body calls Target,
then checks Cover requirements. A failure report carries the
minimized input, the notes and a replay token (PropertyOptions::Replay)
that runs exactly that input. An example database (DatabasePath or
CATCH3_PROPERTY_DATABASE) replays stored failures first and drops them
once they pass. Bodies return bool, or void with CATCH3_PROPERTY_ASSERT
and CATCH3_PROPERTY_ASSUME, which return instead of throwing; Note,
Classify, Cover and Target record into a per-thread trial record.
CATCH3_CHECK_FOR_ALL and CATCH3_REQUIRE_FOR_ALL report through Catch2
and name the property after the test case for the database.
CheckStateful runs model-based command sequences against a model and
the system under test; preconditions and state-dependent arguments are
replayed during shrinking. Relations.hpp provides Equivalent
(differential), Metamorphic, RoundTrip and Idempotent bodies.
PropertyOptions, PropertyStatus, PropertyFailure and PropertyResult
gain fields and statuses at their ends (kInsufficientCoverage,
kGaveUp); existing designated initializers and reports are unchanged.
The engine's untemplated parts are compiled in ChoiceShrinker.cpp,
PropertyEngine.cpp and Context.cpp, and every build system (Bazel,
CMake, Meson, Xmake, Premake) lists the new files and tests.
Docs: progmasoft-properties.md describes the engine,
progmasoft-roadmap.md the steps that follow (the standalone catch3
runner, mocking, per-test coverage, change-impact selection and
prioritization, mutation testing, guided test generation). The
.clang-format comment names Progmasoft C++ Style instead of Visual X#.
Verified with clang-cl 23 through CMake: ProgmasoftSelfTest passes, 57
test cases and 75,461 assertions, including the shrinking results
(1001 for x <= 1000, { 10 }, { 0, 0, 0 } through FlatMap, 102 through
a filter, { 0, 100 } for two arguments, four push(0) steps for a
queue that drops its fourth element, { 0, 0 } for a sort that loses
duplicates) and the database, replay, coverage and discard behaviour.
Diffstat (limited to 'tests')
| -rw-r--r-- | tests/CMakeLists.txt | 3 | ||||
| -rw-r--r-- | tests/Progmasoft/BUILD.bazel | 3 | ||||
| -rw-r--r-- | tests/Progmasoft/ForAll.tests.cpp | 314 | ||||
| -rw-r--r-- | tests/Progmasoft/Stateful.tests.cpp | 129 | ||||
| -rw-r--r-- | tests/Progmasoft/Strategy.tests.cpp | 312 | ||||
| -rw-r--r-- | tests/meson.build | 3 |
6 files changed, 764 insertions, 0 deletions
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 79660687..3d4db3b3 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -160,8 +160,11 @@ target_link_libraries(SelfTest PRIVATE Catch2WithMain) add_executable(ProgmasoftSelfTest Progmasoft/Generators.tests.cpp Progmasoft/Property.tests.cpp + Progmasoft/ForAll.tests.cpp Progmasoft/Results.tests.cpp Progmasoft/Snapshot.tests.cpp + Progmasoft/Stateful.tests.cpp + Progmasoft/Strategy.tests.cpp ) target_link_libraries(ProgmasoftSelfTest PRIVATE Progmasoft::Catch3 Catch2WithMain) target_compile_features(ProgmasoftSelfTest PRIVATE cxx_std_20) diff --git a/tests/Progmasoft/BUILD.bazel b/tests/Progmasoft/BUILD.bazel index d2eeca92..853d99d4 100644 --- a/tests/Progmasoft/BUILD.bazel +++ b/tests/Progmasoft/BUILD.bazel @@ -7,10 +7,13 @@ package(default_visibility = ["//visibility:public"]) cc_test( name = "property_tests", srcs = [ + "ForAll.tests.cpp", "Generators.tests.cpp", "Property.tests.cpp", "Results.tests.cpp", "Snapshot.tests.cpp", + "Stateful.tests.cpp", + "Strategy.tests.cpp", ], copts = select({ "@platforms//os:windows": ["/std:c++20"], diff --git a/tests/Progmasoft/ForAll.tests.cpp b/tests/Progmasoft/ForAll.tests.cpp new file mode 100644 index 00000000..b88477a6 --- /dev/null +++ b/tests/Progmasoft/ForAll.tests.cpp @@ -0,0 +1,314 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 +#include <Progmasoft/Catch3.hpp> + +#include <algorithm> +#include <cstdint> +#include <filesystem> +#include <numeric> +#include <set> +#include <stdexcept> +#include <string> +#include <vector> + +namespace { + + using Progmasoft::Catch3::ForAll; + using Progmasoft::Catch3::PropertyOptions; + using Progmasoft::Catch3::PropertyStatus; + using Progmasoft::Catch3::SizeRange; + namespace Gen = Progmasoft::Catch3::Gen; + + PropertyOptions Quick( std::size_t trials = 200 ) { + PropertyOptions options; + options.Trials = trials; + options.Seed = 0xCA7C3; + options.MaxShrinkSteps = 5000; + return options; + } + +} // namespace + +TEST_CASE( "ForAll passes and counts its trials", + "[progmasoft][forall]" ) { + const auto result = ForAll( Quick( 150 ), + Gen::Integers<int>( -100, 100 ), + Gen::Integers<int>( -100, 100 ), + []( int left, int right ) { + return left + right == right + left; + } ); + CHECK( result.Status == PropertyStatus::kPassed ); + CHECK( result.TrialsRun == 150 ); + CHECK( result.Describe() == "property passed: 150 trials" ); +} + +TEST_CASE( "ForAll shrinks integers to the boundary of the failure", + "[progmasoft][forall]" ) { + const auto result = ForAll( + Quick(), Gen::Integers<int>(), []( int value ) { return value <= 1000; } ); + REQUIRE( result.Status == PropertyStatus::kFailed ); + REQUIRE( result.Failure.has_value() ); + CHECK( result.Failure->Counterexample == "1001 (0x3e9)" ); + CHECK( result.Failure->Reason == "property returned false" ); + CHECK( result.Failure->Replay.rfind( "c3.", 0 ) == 0 ); +} + +TEST_CASE( "ForAll shrinks collections to their essential element", + "[progmasoft][forall]" ) { + const auto result = ForAll( + Quick(), + Gen::Vectors( Gen::Integers<int>( 0, 100 ) ), + []( const std::vector<int>& values ) { + return std::none_of( values.begin(), values.end(), + []( int value ) { return value >= 10; } ); + } ); + REQUIRE( result.Status == PropertyStatus::kFailed ); + CHECK( result.Failure->Counterexample == "{ 10 }" ); +} + +TEST_CASE( "ForAll shrinks through dependent and filtered strategies", + "[progmasoft][forall]" ) { + const auto dependent = + Gen::Integers<int>( 0, 20 ).FlatMap( []( int size ) { + return Gen::Vectors( + Gen::Integers<int>( 0, 100 ), + SizeRange{ static_cast<std::size_t>( size ), + static_cast<std::size_t>( size ) } ); + } ); + const auto sized = ForAll( Quick(), dependent, + []( const std::vector<int>& values ) { + return values.size() < 3; + } ); + REQUIRE( sized.Status == PropertyStatus::kFailed ); + CHECK( sized.Failure->Counterexample == "{ 0, 0, 0 }" ); + + const auto even = Gen::Integers<int>( 0, 100000 ).Filter( + []( int value ) { return value % 2 == 0; } ); + const auto filtered = + ForAll( Quick(), even, []( int value ) { return value <= 100; } ); + INFO( filtered.Describe() ); + REQUIRE( filtered.Status == PropertyStatus::kFailed ); + CHECK( filtered.Failure->Counterexample == "102" ); + + const auto mapped = ForAll( + Quick(), + Gen::Integers<int>( 0, 1000 ).Map( []( int value ) { + return std::to_string( value ); + } ), + []( const std::string& text ) { return text.size() < 3; } ); + REQUIRE( mapped.Status == PropertyStatus::kFailed ); + CHECK( mapped.Failure->Counterexample == "\"100\"" ); +} + +TEST_CASE( "ForAll shrinks several arguments together", + "[progmasoft][forall]" ) { + const auto result = ForAll( Quick(), + Gen::Integers<int>( 0, 1000 ), + Gen::Integers<int>( 0, 1000 ), + []( int left, int right ) { + return left + right < 100; + } ); + REQUIRE( result.Status == PropertyStatus::kFailed ); + CHECK( result.Failure->Counterexample == "{ 0, 100 }" ); +} + +TEST_CASE( "A replay token reproduces exactly the reported input", + "[progmasoft][forall]" ) { + const auto body = []( const std::vector<int>& values ) { + return std::accumulate( values.begin(), values.end(), 0 ) < 50; + }; + const auto strategy = Gen::Vectors( Gen::Integers<int>( 0, 30 ) ); + const auto first = ForAll( Quick(), strategy, body ); + REQUIRE( first.Status == PropertyStatus::kFailed ); + + auto options = Quick(); + options.Replay = first.Failure->Replay; + const auto replayed = ForAll( options, strategy, body ); + REQUIRE( replayed.Status == PropertyStatus::kFailed ); + CHECK( replayed.TrialsRun == 1 ); + CHECK( replayed.Failure->Counterexample == first.Failure->Counterexample ); + CHECK( replayed.Failure->Replay == first.Failure->Replay ); + CHECK( first.Describe().find( "replay: set PropertyOptions::Replay to \"" + + first.Failure->Replay + "\"" ) != + std::string::npos ); + + // The same options give the same report. + CHECK( ForAll( Quick(), strategy, body ).Describe() == first.Describe() ); + + options.Replay = "not a token"; + CHECK( ForAll( options, strategy, body ).Status == + PropertyStatus::kInvalidConfiguration ); +} + +TEST_CASE( "Property bodies assert, assume and note without exceptions", + "[progmasoft][forall]" ) { + const auto failing = ForAll( + Quick(), Gen::Integers<int>( 0, 1000 ), []( int value ) { + Progmasoft::Catch3::Note( "half", value / 2 ); + CATCH3_PROPERTY_ASSERT( value < 500 ); + } ); + REQUIRE( failing.Status == PropertyStatus::kFailed ); + CHECK( failing.Failure->Counterexample == "500 (0x1f4)" ); + CHECK( failing.Failure->Reason.find( + "CATCH3_PROPERTY_ASSERT( value < 500 )" ) == 0 ); + CHECK( failing.Failure->Notes == std::vector<std::string>{ "half = 250" } ); + + const auto assumed = ForAll( + Quick(), Gen::Integers<int>( 0, 9 ), []( int value ) { + CATCH3_PROPERTY_ASSUME( value % 2 == 0 ); + CATCH3_PROPERTY_ASSERT( value % 2 == 0 ); + } ); + CHECK( assumed.Status == PropertyStatus::kPassed ); + CHECK( assumed.Discarded > 0 ); + + auto strict = Quick( 50 ); + strict.MaxDiscardRatio = 1; + const auto gaveUp = ForAll( strict, Gen::Integers<int>( 0, 9 ), + []( int ) { CATCH3_PROPERTY_ASSUME( false ); } ); + CHECK( gaveUp.Status == PropertyStatus::kGaveUp ); + CHECK( gaveUp.Describe().find( "property gave up" ) == 0 ); +} + +TEST_CASE( "Exceptions from bodies fail and from strategies error", + "[progmasoft][forall]" ) { + const auto throwing = ForAll( + Quick(), Gen::Integers<int>( 0, 100 ), []( int value ) -> bool { + if ( value > 10 ) { throw std::runtime_error( "too big" ); } + return true; + } ); + REQUIRE( throwing.Status == PropertyStatus::kFailed ); + CHECK( throwing.Failure->Counterexample == "11" ); + CHECK( throwing.Failure->Reason == + "property threw std::exception: too big" ); + + const auto broken = ForAll( + Quick(), + Gen::Integers<int>( 0, 10 ).Map( []( int ) -> int { + throw std::runtime_error( "cannot build" ); + } ), + []( int ) { return true; } ); + CHECK( broken.Status == PropertyStatus::kErrored ); + CHECK( broken.Error == "strategy threw std::exception: cannot build" ); +} + +TEST_CASE( "Classify and Cover report and enforce statistics", + "[progmasoft][forall]" ) { + auto options = Quick( 400 ); + options.ShowStatistics = true; + const auto covered = ForAll( + options, Gen::Integers<int>( -100, 100 ), []( int value ) { + Progmasoft::Catch3::Classify( value == 0, "zero" ); + Progmasoft::Catch3::Cover( 20.0, value < 0, "negative" ); + return true; + } ); + CHECK( covered.Status == PropertyStatus::kPassed ); + CHECK( covered.Describe().find( "negative:" ) != std::string::npos ); + + const auto uncovered = ForAll( + Quick( 400 ), Gen::Integers<int>( 0, 1000 ), []( int value ) { + Progmasoft::Catch3::Cover( 50.0, value == 7, "seven" ); + return true; + } ); + CHECK( uncovered.Status == PropertyStatus::kInsufficientCoverage ); + CHECK( uncovered.Describe().find( "required 50%" ) != std::string::npos ); +} + +TEST_CASE( "Targeted search runs after the regular trials", + "[progmasoft][forall]" ) { + auto options = Quick( 100 ); + options.TargetSteps = 300; + const auto result = ForAll( + options, Gen::Integers<std::int64_t>( 0, 1LL << 40 ), []( std::int64_t value ) { + Progmasoft::Catch3::Target( static_cast<double>( value ) ); + return value < ( 1LL << 39 ) + ( 1LL << 38 ); + } ); + // Climbing towards the largest values must reach the failing region. + REQUIRE( result.Status == PropertyStatus::kFailed ); + CHECK( result.Failure->Counterexample == "824633720832 (0xc000000000)" ); +} + +TEST_CASE( "The example database replays a stored failure first", + "[progmasoft][forall]" ) { + const auto directory = std::filesystem::temp_directory_path() / + "catch3-property-database-test"; + std::filesystem::remove_all( directory ); + auto options = Quick(); + options.DatabasePath = directory.string(); + options.Name = "database test"; + int threshold = 200; + const auto body = [&]( int value ) { return value < threshold; }; + + const auto first = ForAll( options, Gen::Integers<int>( 0, 1000 ), body ); + REQUIRE( first.Status == PropertyStatus::kFailed ); + CHECK( first.Failure->Counterexample == "200" ); + CHECK( !std::filesystem::is_empty( directory ) ); + + // A new seed would need its own trials; the stored example fails at + // once. + options.Seed = 0xD1FFE7; + const auto second = ForAll( options, Gen::Integers<int>( 0, 1000 ), body ); + REQUIRE( second.Status == PropertyStatus::kFailed ); + CHECK( second.TrialsRun == 1 ); + CHECK( second.Failure->Counterexample == "200" ); + + // Once fixed, the example passes and is dropped. + threshold = 2000; + const auto fixed = ForAll( options, Gen::Integers<int>( 0, 1000 ), body ); + CHECK( fixed.Status == PropertyStatus::kPassed ); + CHECK( std::filesystem::is_empty( directory ) ); + std::filesystem::remove_all( directory ); +} + +TEST_CASE( "Relations check differential, metamorphic and round-trip oracles", + "[progmasoft][forall][relations]" ) { + using namespace Progmasoft::Catch3; + const auto sorted = []( std::vector<int> values ) { + std::sort( values.begin(), values.end() ); + return values; + }; + // A "fast" sort that loses duplicates. + const auto lossy = []( const std::vector<int>& values ) { + const std::set<int> unique( values.begin(), values.end() ); + return std::vector<int>( unique.begin(), unique.end() ); + }; + const auto differential = ForAll( + Quick(), Gen::Vectors( Gen::Integers<int>( 0, 50 ) ), + Equivalent( sorted, lossy ) ); + REQUIRE( differential.Status == PropertyStatus::kFailed ); + CHECK( differential.Failure->Counterexample == "{ 0, 0 }" ); + CHECK( differential.Failure->Notes == + std::vector<std::string>{ "reference: { 0, 0 }", + "candidate: { 0 }" } ); + + const auto sum = []( const std::vector<int>& values ) { + return std::accumulate( values.begin(), values.end(), 0LL ); + }; + const auto reversed = []( std::vector<int> values ) { + std::reverse( values.begin(), values.end() ); + return values; + }; + CHECK( ForAll( Quick(), Gen::Vectors( Gen::Integers<int>( -9, 9 ) ), + Metamorphic( sum, reversed, std::equal_to<>() ) ) + .Succeeded() ); + + const auto encode = []( int value ) { return std::to_string( value ); }; + const auto decode = []( const std::string& text ) { + return std::stoi( text ); + }; + CHECK( ForAll( Quick(), Gen::Integers<int>(), RoundTrip( encode, decode ) ) + .Succeeded() ); + CHECK( ForAll( Quick(), Gen::Vectors( Gen::Integers<int>( 0, 9 ) ), + Idempotent( sorted ) ) + .Succeeded() ); +} + +TEST_CASE( "CATCH3_CHECK_FOR_ALL reports through Catch2", + "[progmasoft][forall]" ) { + CATCH3_CHECK_FOR_ALL( Gen::Integers<int>( 0, 100 ), + []( int value ) { return value >= 0; } ); + CATCH3_REQUIRE_FOR_ALL( Quick( 20 ), + Gen::Strings( Gen::PrintableCharacters() ), + []( const std::string& text ) { + return text.size() == text.length(); + } ); +} diff --git a/tests/Progmasoft/Stateful.tests.cpp b/tests/Progmasoft/Stateful.tests.cpp new file mode 100644 index 00000000..48e2c918 --- /dev/null +++ b/tests/Progmasoft/Stateful.tests.cpp @@ -0,0 +1,129 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 +#include <Progmasoft/Catch3.hpp> + +#include <cstddef> +#include <deque> +#include <string> +#include <vector> + +namespace { + + using Progmasoft::Catch3::Action; + using Progmasoft::Catch3::Command; + using Progmasoft::Catch3::PropertyOptions; + using Progmasoft::Catch3::PropertyStatus; + using Progmasoft::Catch3::SizeRange; + namespace Gen = Progmasoft::Catch3::Gen; + + // A queue with a bug: past three elements it silently drops the + // oldest one. + class SmallQueue { + public: + void Push( int value ) { + if ( values_.size() == 3 ) { values_.pop_front(); } + values_.push_back( value ); + } + int Pop() { + const int value = values_.front(); + values_.pop_front(); + return value; + } + [[nodiscard]] std::size_t Size() const { return values_.size(); } + + private: + std::deque<int> values_; + }; + + using Model = std::deque<int>; + + std::vector<Command<Model, SmallQueue>> QueueCommands() { + Command<Model, SmallQueue> push{ + "push", + []( const Model& ) { + return Gen::Integers<int>( 0, 100 ).Map( []( int value ) { + return Action<Model, SmallQueue>{ + "push(" + std::to_string( value ) + ")", + [value]( Model& model, SmallQueue& queue ) { + model.push_back( value ); + queue.Push( value ); + CATCH3_PROPERTY_ASSERT( queue.Size() == + model.size() ); + } }; + } ); + } }; + Command<Model, SmallQueue> pop{ + "pop", + []( const Model& ) { + return Gen::Just( Action<Model, SmallQueue>{ + "pop", []( Model& model, SmallQueue& queue ) { + const int expected = model.front(); + model.pop_front(); + CATCH3_PROPERTY_ASSERT( queue.Pop() == expected ); + } } ); + }, + []( const Model& model ) { return !model.empty(); } }; + return { push, pop }; + } + +} // namespace + +TEST_CASE( "Stateful tests find and shrink a failing command sequence", + "[progmasoft][stateful]" ) { + PropertyOptions options; + options.Trials = 300; + options.Seed = 0x57A7E; + options.MaxShrinkSteps = 5000; + const auto result = Progmasoft::Catch3::CheckStateful<Model, SmallQueue>( + options, + [] { return Model{}; }, + []( const Model& ) { return SmallQueue{}; }, + QueueCommands() ); + REQUIRE( result.Status == PropertyStatus::kFailed ); + // The shortest failure: the fourth push drops an element. + CHECK( result.Failure->Counterexample == + "{ push(0), push(0), push(0), push(0) }" ); + CHECK( result.Failure->Reason.find( + "CATCH3_PROPERTY_ASSERT( queue.Size() == model.size() )" ) == + 0 ); +} + +TEST_CASE( "Stateful tests pass for a correct system and respect preconditions", + "[progmasoft][stateful]" ) { + struct Correct { + std::deque<int> Values; + }; + Command<Model, Correct> push{ + "push", []( const Model& ) { + return Gen::Integers<int>( 0, 9 ).Map( []( int value ) { + return Action<Model, Correct>{ + "push", [value]( Model& model, Correct& system ) { + model.push_back( value ); + system.Values.push_back( value ); + } }; + } ); + } }; + Command<Model, Correct> pop{ + "pop", + []( const Model& ) { + return Gen::Just( Action<Model, Correct>{ + "pop", []( Model& model, Correct& system ) { + // The precondition guarantees an element. + CATCH3_PROPERTY_ASSERT( !model.empty() ); + CATCH3_PROPERTY_ASSERT( system.Values.front() == + model.front() ); + model.pop_front(); + system.Values.pop_front(); + } } ); + }, + []( const Model& model ) { return !model.empty(); } }; + PropertyOptions options; + options.Trials = 200; + const auto result = Progmasoft::Catch3::CheckStateful<Model, Correct>( + options, + [] { return Model{}; }, + []( const Model& ) { return Correct{}; }, + { push, pop }, + SizeRange{ 0, 50 } ); + CHECK( result.Status == PropertyStatus::kPassed ); +} diff --git a/tests/Progmasoft/Strategy.tests.cpp b/tests/Progmasoft/Strategy.tests.cpp new file mode 100644 index 00000000..5a1d5a2a --- /dev/null +++ b/tests/Progmasoft/Strategy.tests.cpp @@ -0,0 +1,312 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 +#include <Progmasoft/Catch3.hpp> + +#include <algorithm> +#include <cmath> +#include <cstdint> +#include <limits> +#include <map> +#include <optional> +#include <set> +#include <stdexcept> +#include <string> +#include <vector> + +namespace { + + using Progmasoft::Catch3::ChoiceSource; + using Progmasoft::Catch3::SizeRange; + using Progmasoft::Catch3::Strategy; + namespace Gen = Progmasoft::Catch3::Gen; + + template <typename ValueType> + std::vector<ValueType> Sample( const Strategy<ValueType>& strategy, + std::size_t count, + std::uint64_t seed ) { + std::vector<ValueType> values; + for ( std::size_t index = 0; index < count; ++index ) { + ChoiceSource source( seed + index, index % 101 ); + auto value = strategy.Draw( source ); + if ( value.has_value() ) { values.push_back( *value ); } + } + return values; + } + + // Draws once, then replays the recorded choices: the same value must + // come back, which is what shrinking and replay tokens rely on. + template <typename ValueType> + bool ReplaysIdentically( const Strategy<ValueType>& strategy, + std::uint64_t seed ) { + ChoiceSource original( seed, 60 ); + const auto value = strategy.Draw( original ); + const auto choices = original.Choices(); + ChoiceSource replay( choices, 60 ); + const auto again = strategy.Draw( replay ); + return value == again && replay.Choices() == choices && + !replay.Overran(); + } + +} // namespace + +TEST_CASE( "Choice sources replay, clamp and run out to zero", + "[progmasoft][strategy]" ) { + const std::vector<std::uint64_t> recorded{ 3, 99, 1 }; + ChoiceSource replay( recorded, 10 ); + CHECK( replay.Draw( 10 ) == 3 ); + CHECK( replay.Draw( 7 ) == 7 ); // clamped to the new maximum + CHECK( replay.DrawBoolean( 0.0 ) ); + CHECK_FALSE( replay.Overran() ); + CHECK( replay.Draw( 5 ) == 0 ); // past the end: the simplest choice + CHECK( replay.Overran() ); + CHECK( replay.Choices() == std::vector<std::uint64_t>{ 3, 7, 1, 0 } ); + + ChoiceSource generated( 42, 50 ); + for ( int index = 0; index < 1000; ++index ) { + CHECK( generated.Draw( 9 ) <= 9 ); + CHECK( generated.DrawBiased( 1000 ) <= 1000 ); + } + CHECK( generated.Choices().size() == 2000 ); +} + +TEST_CASE( "Integer strategies respect every kind of range", + "[progmasoft][strategy]" ) { + const auto crossing = Gen::Integers<int>( -50, 70 ); + for ( int value : Sample( crossing, 2000, 1 ) ) { + CHECK( value >= -50 ); + CHECK( value <= 70 ); + } + const auto negative = Gen::Integers<std::int64_t>( -900, -100 ); + for ( auto value : Sample( negative, 2000, 2 ) ) { + CHECK( value >= -900 ); + CHECK( value <= -100 ); + } + const auto positive = Gen::Integers<std::uint8_t>( 10, 20 ); + for ( auto value : Sample( positive, 2000, 3 ) ) { + CHECK( value >= 10 ); + CHECK( value <= 20 ); + } + const auto full = Gen::Integers<std::int64_t>(); + const auto values = Sample( full, 4000, 4 ); + CHECK( std::any_of( values.begin(), values.end(), []( auto value ) { + return value == std::numeric_limits<std::int64_t>::max(); + } ) ); + CHECK( std::any_of( values.begin(), values.end(), []( auto value ) { + return value < 0; + } ) ); + const auto upToZero = Gen::Integers<std::int64_t>( + std::numeric_limits<std::int64_t>::lowest(), 0 ); + for ( auto value : Sample( upToZero, 2000, 5 ) ) { CHECK( value <= 0 ); } + + // The simplest choices give the value closest to zero. + const std::vector<std::uint64_t> zeros; + ChoiceSource simplest( zeros, 0 ); + CHECK( crossing.Draw( simplest ) == 0 ); + ChoiceSource simplestNegative( zeros, 0 ); + CHECK( negative.Draw( simplestNegative ) == -100 ); + ChoiceSource simplestPositive( zeros, 0 ); + CHECK( positive.Draw( simplestPositive ) == 10 ); + + CHECK_THROWS_AS( Gen::Integers<int>( 2, 1 ), std::invalid_argument ); + for ( std::uint64_t seed = 0; seed < 50; ++seed ) { + CHECK( ReplaysIdentically( full, seed ) ); + } +} + +TEST_CASE( "Float strategies stay finite and in range unless allowed", + "[progmasoft][strategy]" ) { + using Progmasoft::Catch3::FloatOptions; + const auto bounded = Gen::Floats<double>( + FloatOptions{ .Minimum = -2.5, .Maximum = 1000.0 } ); + for ( double value : Sample( bounded, 3000, 6 ) ) { + CHECK( std::isfinite( value ) ); + CHECK( value >= -2.5 ); + CHECK( value <= 1000.0 ); + } + const auto everything = Gen::Floats<double>( + FloatOptions{ .AllowNaN = true, .AllowInfinity = true } ); + const auto values = Sample( everything, 3000, 7 ); + CHECK( std::any_of( values.begin(), values.end(), []( double value ) { + return std::isnan( value ); + } ) ); + CHECK( std::any_of( values.begin(), values.end(), []( double value ) { + return std::isinf( value ); + } ) ); + const auto positive = Gen::Floats<float>( + FloatOptions{ .Minimum = 1.0, .Maximum = 2.0 } ); + const std::vector<std::uint64_t> zeros; + ChoiceSource simplest( zeros, 0 ); + CHECK( positive.Draw( simplest ) == 1.0F ); + CHECK_THROWS_AS( Gen::Floats<double>( FloatOptions{ .Minimum = 2.0, + .Maximum = 1.0 } ), + std::invalid_argument ); +} + +TEST_CASE( "Collection strategies respect their size ranges", + "[progmasoft][strategy]" ) { + const auto vectors = Gen::Vectors( Gen::Integers<int>( 0, 9 ), + SizeRange{ 2, 5 } ); + for ( const auto& value : Sample( vectors, 1000, 8 ) ) { + CHECK( value.size() >= 2 ); + CHECK( value.size() <= 5 ); + } + const auto unique = Gen::UniqueVectors( Gen::Integers<int>( 0, 30 ), + SizeRange{ 0, 20 } ); + for ( const auto& value : Sample( unique, 1000, 9 ) ) { + CHECK( std::set<int>( value.begin(), value.end() ).size() == + value.size() ); + } + const auto sets = Gen::Sets( Gen::Integers<int>( 0, 3 ), + SizeRange{ 4, 4 } ); + for ( const auto& value : Sample( sets, 200, 10 ) ) { + CHECK( value == std::set<int>{ 0, 1, 2, 3 } ); + } + const auto maps = Gen::Maps( Gen::Integers<int>( 0, 100 ), + Gen::Booleans(), + SizeRange{ 1, 10 } ); + for ( const auto& value : Sample( maps, 500, 11 ) ) { + CHECK( !value.empty() ); + CHECK( value.size() <= 10 ); + } + const auto permutations = Gen::Permutations( std::vector<int>{ 1, 2, 3, 4, 5 } ); + for ( auto value : Sample( permutations, 300, 12 ) ) { + std::sort( value.begin(), value.end() ); + CHECK( value == std::vector<int>{ 1, 2, 3, 4, 5 } ); + } + const std::vector<std::uint64_t> zeros; + ChoiceSource simplest( zeros, 0 ); + CHECK( permutations.Draw( simplest ) == + std::vector<int>{ 1, 2, 3, 4, 5 } ); + CHECK_THROWS_AS( Gen::Vectors( Gen::Booleans(), SizeRange{ 3, 2 } ), + std::invalid_argument ); + for ( std::uint64_t seed = 0; seed < 50; ++seed ) { + CHECK( ReplaysIdentically( vectors, seed ) ); + CHECK( ReplaysIdentically( maps, seed ) ); + } +} + +TEST_CASE( "Text strategies produce valid UTF-8 without surrogates", + "[progmasoft][strategy]" ) { + const auto text = Gen::Utf8Strings( SizeRange{ 0, 30 }, 0, 0x10FFFF ); + for ( const auto& value : Sample( text, 1000, 13 ) ) { + for ( std::size_t index = 0; index + 1 < value.size(); ++index ) { + const auto lead = static_cast<unsigned char>( value[index] ); + const auto next = static_cast<unsigned char>( value[index + 1] ); + // ED A0..BF would encode a surrogate. + CHECK_FALSE( ( lead == 0xED && next >= 0xA0 ) ); + } + } + const auto printable = Gen::Strings( Gen::PrintableCharacters(), + SizeRange{ 1, 8 } ); + for ( const auto& value : Sample( printable, 500, 14 ) ) { + CHECK( !value.empty() ); + CHECK( value.size() <= 8 ); + CHECK( std::all_of( value.begin(), value.end(), []( char c ) { + return c >= 0x20 && c < 0x7F; + } ) ); + } +} + +TEST_CASE( "Combinators compose and keep replays stable", + "[progmasoft][strategy]" ) { + const auto even = Gen::Integers<int>( 0, 1000 ).Filter( + []( int value ) { return value % 2 == 0; } ); + for ( int value : Sample( even, 500, 15 ) ) { CHECK( value % 2 == 0 ); } + + const auto sized = Gen::Integers<int>( 0, 6 ).FlatMap( []( int size ) { + return Gen::Vectors( Gen::Just( size ), + SizeRange{ static_cast<std::size_t>( size ), + static_cast<std::size_t>( size ) } ); + } ); + for ( const auto& value : Sample( sized, 500, 16 ) ) { + CHECK( std::all_of( value.begin(), value.end(), [&]( int element ) { + return element == static_cast<int>( value.size() ); + } ) ); + } + + const auto mixed = Gen::OneOf( Gen::Just( std::string( "a" ) ), + Gen::Just( std::string( "b" ) ) ); + const auto mixedValues = Sample( mixed, 200, 17 ); + CHECK( std::count( mixedValues.begin(), mixedValues.end(), "a" ) > 0 ); + CHECK( std::count( mixedValues.begin(), mixedValues.end(), "b" ) > 0 ); + + const auto weighted = Gen::Frequency<int>( { { 1, Gen::Just( 0 ) }, + { 9, Gen::Just( 1 ) } } ); + const auto weightedValues = Sample( weighted, 2000, 18 ); + CHECK( std::count( weightedValues.begin(), weightedValues.end(), 1 ) > + std::count( weightedValues.begin(), weightedValues.end(), 0 ) ); + + const auto trees = Gen::Recursive( + Gen::Just( 1 ), + []( const Strategy<int>& children ) { + return Gen::Tuples( children, children ) + .Map( []( std::tuple<int, int> pair ) { + return std::get<0>( pair ) + std::get<1>( pair ); + } ); + }, + 4 ); + for ( int leaves : Sample( trees, 500, 19 ) ) { + CHECK( leaves >= 1 ); + CHECK( leaves <= 16 ); // depth 4 at most: 2^4 leaves + } + + const auto never = Gen::Integers<int>( 0, 10 ).Filter( + []( int ) { return false; }, 5 ); + ChoiceSource source( 1, 50 ); + CHECK_FALSE( never.Draw( source ).has_value() ); +} + +TEST_CASE( "Replay tokens round-trip and reject malformed text", + "[progmasoft][strategy]" ) { + using Progmasoft::Catch3::Detail::DecodeChoices; + using Progmasoft::Catch3::Detail::EncodeChoices; + const std::vector<std::uint64_t> choices{ + 0, 1, 35, 36, std::numeric_limits<std::uint64_t>::max() }; + const auto token = EncodeChoices( 73, choices ); + CHECK( token == "c3.21.0-1-z-10-3w5e11264sgsf" ); + const auto decoded = DecodeChoices( token ); + REQUIRE( decoded.has_value() ); + CHECK( decoded->Size == 73 ); + CHECK( decoded->Choices == choices ); + + const auto empty = DecodeChoices( EncodeChoices( 0, {} ) ); + REQUIRE( empty.has_value() ); + CHECK( empty->Choices.empty() ); + + for ( const char* malformed : + { "", "c3.", "c3.1", "x3.1.2", "c3.1.2-", "c3.1.-2", "c3.2t.1", + "c3.1.2--3", "c3.1.zzzzzzzzzzzzzzz" } ) { + CHECK_FALSE( DecodeChoices( malformed ).has_value() ); + } +} + +TEST_CASE( "The choice shrinker finds simple failing sequences", < |