catch3
Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.
Clonegit clone https://git.progmasoft.com/catch3
Add mocks that report through the assertion path
- Commit
c51d15273e87267215e5877edc268e7db354187b- Author
- Leitwolf11 ·
- Parents
26a1588ae3ae
Changes
README.md | 4 + docs/Readme.md | 1 + docs/progmasoft-mocking.md | 140 ++++++ docs/progmasoft-roadmap.md | 11 +- docs/why-catch3.md | 3 +- premake5.lua | 1 + src/CMakeLists.txt | 1 + src/Progmasoft/Catch3.hpp | 1 + src/Progmasoft/Catch3/Mock.hpp | 955 +++++++++++++++++++++++++++++++++++++ src/Progmasoft/Catch3/meson.build | 1 + src/Progmasoft/Mock.cpp | 203 ++++++++ src/Progmasoft/meson.build | 1 + tests/CMakeLists.txt | 1 + tests/Progmasoft/BUILD.bazel | 1 + tests/Progmasoft/Mock.tests.cpp | 310 ++++++++++++ tests/Progmasoft/RunnerFixture.cpp | 67 +++ tests/meson.build | 1 + xmake.lua | 1 + 18 files changed, 1697 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 14ee4bdf..510c1fe4 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,9 @@ serialization. tests, keeps a history that drives time limits, ordering, incremental runs and replay, selects and shards tests, and writes console, JUnit, JSON and trace reports. +- **Mocks.** [Mocks](docs/progmasoft-mocking.md) of virtual interfaces and of + free functions behind a seam check call counts, arguments and order, and + report through the same assertions as the rest of a test. - **Deterministic property testing.** Generate values from a reproducible seed, check a predicate over multiple trials, and shrink a failing input to a smaller counterexample. @@ -96,6 +99,7 @@ their package, target, header and pkg-config names differ. - [Catch3 additions: properties, snapshots, results, and XML](docs/progmasoft-catch3.md) - [Project scope, compatibility policy, and limitations](docs/why-catch3.md) - [The Catch3 runner](docs/progmasoft-runner.md) +- [Mocking](docs/progmasoft-mocking.md) - [Tutorial](docs/tutorial.md) - [Catch2-compatible API reference](docs/Readme.md) - [Catch2 upstream project and source history](https://github.com/catchorg/Catch2) diff --git a/docs/Readme.md b/docs/Readme.md index 3da1a8ca..15fe412d 100644 --- a/docs/Readme.md +++ b/docs/Readme.md @@ -38,6 +38,7 @@ Catch2 release. **Running:** * [The Catch3 runner](progmasoft-runner.md#top) +* [Mocking](progmasoft-mocking.md#top) * [Command line reference](command-line.md#top) * [Running specific section/generator](filtering-execution-path.md#top) diff --git a/docs/progmasoft-mocking.md b/docs/progmasoft-mocking.md new file mode 100644 index 00000000..98c372ec --- /dev/null +++ b/docs/progmasoft-mocking.md @@ -0,0 +1,140 @@ +<a id="top"></a> + +# Mocking + +`<Progmasoft/Catch3/Mock.hpp>` replaces a dependency of the code under test +with a mock that checks how it is called. Mocks report through Catch's +assertion path: an unexpected call, a call out of order and an unmet +expectation fail the test case that owns the mock, at the line that declared +the expectation, in every report the runner writes. + +```cpp +#include <Progmasoft/Catch3/Mock.hpp> +#include <Progmasoft/Catch3/TestMacros.hpp> + +class Store +{ +public: + virtual ~Store() = default; + virtual int Get(const std::string &key) const = 0; + virtual void Put(const std::string &key, int value) = 0; +}; + +class MockStore final : public Store +{ +public: + CATCH3_MOCK_METHOD(int, Get, (const std::string &), const override); + CATCH3_MOCK_METHOD(void, Put, (const std::string &, int), override); +}; + +TEST_CASE("A score goes up by one") +{ + MockStore store; + CATCH3_EXPECT_CALL(store.GetMock)("ada").Returns(41); + CATCH3_EXPECT_CALL(store.PutMock)("ada", 42); + + AddPoint(store, "ada"); +} +``` + +A mock verifies itself in its destructor, so nothing has to be called at the +end of the test case. When the test case is already being left by a failure or +a skip, the expectations it did not reach are not reported on top of it. +`Verify()` checks the expectations earlier and forgets them, for a test case +that goes on with new ones. + +## Declaring mocks + +`CATCH3_MOCK_METHOD(result, name, (types), specifiers)` declares the member +function `name` and the mock `nameMock` beside it. The specifiers are whatever +follows the parameters: `const`, `noexcept`, `override`. Up to 8 parameters are +supported, and a type with a comma in it, such as `std::map<std::string, int>`, +needs an alias. + +Overloads need mocks with names of their own: + +```cpp +CATCH3_MOCK_OVERLOAD(void, Put, PutOne, (const std::string &, int), override); +CATCH3_MOCK_OVERLOAD(void, Put, PutAll, (const Scores &), override); +``` + +`CATCH3_MOCK_FUNCTION(name, signature)` declares a `MockFunction` on its own. + +## Setting expectations + +`CATCH3_EXPECT_CALL(mock)(arguments...)` expects one call. Without arguments it +accepts any arguments; otherwise each parameter takes one of these: + +| Written as | Accepts | +| --- | --- | +| a value | what compares equal to it; strings compare by value, also as `const char *` | +| a Catch matcher, such as `StartsWith("a")` | what the matcher accepts | +| `Any` | anything | +| `Where(predicate, "description")` | what the predicate accepts | + +A composition such as `StartsWith("a") && EndsWith("z")` refers to temporaries +that are gone when the mock is called, so it does not compile here; put it in a +`Where`. + +The result of `CATCH3_EXPECT_CALL` takes these, in any order: + +| Member | Meaning | +| --- | --- | +| `Times(n)`, `Between(a, b)`, `AtLeast(n)`, `AtMost(n)`, `AnyTimes()`, `Never()` | how many calls it wants; one by default | +| `Returns(value)` | the result of each call; a type that cannot be copied moves out on the first one, and a reference result refers to `value` | +| `Does(function)` | runs `function` with the arguments, and returns what it returns; it keeps its state between calls | +| `Throws(exception)` | throws `exception` from each call | +| `InSequence(sequence)` | see below | + +A call goes to the first expectation, in the order they were declared, that +accepts its arguments and can take another call. A call none of them accepts +fails the test case and returns a value-initialized result. So does a call they +accept when all of them are used up. When the result cannot be made that way, a +reference for example, the test case ends there; the same happens to an +expectation with no `Returns` or `Does`. + +A mock with no expectation for a call treats it as a failure. An expectation +with `AnyTimes()` is how to allow calls that do not matter to the test. + +## Sequences + +Expectations in a `Sequence` have to be met in the order they were declared in, +across mocks: + +```cpp +Sequence order; +CATCH3_EXPECT_CALL(file.OpenMock)().InSequence(order); +CATCH3_EXPECT_CALL(file.WriteMock)(Any).AtLeast(1).InSequence(order); +CATCH3_EXPECT_CALL(log.FlushMock)().InSequence(order); +``` + +A step that is passed takes no more calls. A step reached before the ones ahead +of it have their calls fails the test case and names them. + +## Free functions + +Code that reaches a free function through a seam, a `std::function` or a +function pointer it calls through, can be pointed at a mock. `ScopedSeam` puts +the seam back at the end of its scope: + +```cpp +CATCH3_MOCK_FUNCTION(clock, long()); +CATCH3_EXPECT_CALL(clock)().Returns(50L); + +const ScopedSeam seam(clockSeam, clock.AsFunction()); +CHECK(SecondsSince(20) == 30); +``` + +The mock has to outlive what `AsFunction` returns. A function pointer can only +take a function without state, so it reaches a mock of static lifetime through +one. + +## Threads + +Mocks count and order calls correctly when several threads call them. Reports +go through Catch's assertions, which are not safe across threads unless +`CATCH_CONFIG_EXPERIMENTAL_THREAD_SAFE_ASSERTIONS` is set, so a failure in a +thread other than the test's needs that setting. + +Outside of a test case, such as in the destructor of a global, there is nothing +to fail: failures go to standard error. diff --git a/docs/progmasoft-roadmap.md b/docs/progmasoft-roadmap.md index 5ab04264..81a8956f 100644 --- a/docs/progmasoft-roadmap.md +++ b/docs/progmasoft-roadmap.md @@ -21,7 +21,7 @@ The steps are ordered by what each one builds on. | --- | --- | --- | | 1 | Native property testing: choice-sequence shrinking, combinators, state machines, statistics, targeted search, example database, differential and metamorphic oracles | done; see [Native property testing](progmasoft-properties.md#top) | | 2 | The Catch3 runner | done; see [The Catch3 runner](progmasoft-runner.md#top) | -| 3 | Mocking | planned | +| 3 | Mocking | done; see [Mocking](progmasoft-mocking.md#top) | | 4 | Per-test code coverage | planned | | 5 | Change-impact test selection and test prioritization | planned | | 6 | Mutation testing | planned | @@ -56,10 +56,11 @@ Catch3 runner replaced Catch2's in every test binary: ## 3. Mocking -Mocks that report through Catch2's assertion path: expectations with call -counts, argument matchers built on Catch2's matchers, sequences, and actions, -for virtual interfaces and for free functions behind a seam. Unexpected calls -and unmet expectations fail the test case that owns the mock. +Mocks report through Catch2's assertion path: expectations with call counts, +argument matchers built on Catch2's matchers, sequences, and actions, for +virtual interfaces and for free functions behind a seam. Unexpected calls and +unmet expectations fail the test case that owns the mock. See +[Mocking](progmasoft-mocking.md#top). ## 4. Per-test code coverage diff --git a/docs/why-catch3.md b/docs/why-catch3.md index 120295f0..871b6efc 100644 --- a/docs/why-catch3.md +++ b/docs/why-catch3.md @@ -20,13 +20,14 @@ Catch2's runner with [its own](progmasoft-runner.md#top). | C++ baseline | C++17; the headers work in C++17 and later | The same | | Randomness | Catch2's existing behavior is unchanged | A specified SplitMix64 sequence provides deterministic trial seeds and unbiased bounded integer sampling | | Generated values | No change to existing Catch2 generators | Strategies for numbers, text, collections and structures with `Map`, `Filter` and `FlatMap`; shrinking of recorded choices, so every combination shrinks; replay tokens and an example database; state machines, `Classify`/`Cover` statistics, targeted search, and differential and metamorphic oracles | +| Mocks | None | [`CATCH3_MOCK_METHOD`, `CATCH3_EXPECT_CALL`](progmasoft-mocking.md#top) and `Sequence`, reporting through the assertion path | | Build identity | One library: the `Catch3` CMake package with `Catch3::Catch3` and `Catch3::Catch3WithMain`, `//:catch3` in Bazel, `catch3` in Meson and pkg-config | The same library | | Source terms | Catch2-originated source retains its upstream BSL-1.0 terms | Progmasoft-authored additions state their own SPDX license in each file | Properties take any number of generated values. Shrinking is deterministic and bounded, but local rather than globally minimal. Parallel trials do not exist yet; [the roadmap](progmasoft-roadmap.md#top) -orders them with mocking, per-test coverage, test selection, mutation testing +orders them with per-test coverage, test selection, mutation testing and test generation. ## Compatibility policy diff --git a/premake5.lua b/premake5.lua index 7b917cfd..d4c55c9e 100644 --- a/premake5.lua +++ b/premake5.lua @@ -47,6 +47,7 @@ project "Catch3" "src/Progmasoft/Results.cpp", "src/Progmasoft/XmlWriter.cpp", "src/Progmasoft/ChoiceShrinker.cpp", + "src/Progmasoft/Mock.cpp", "src/Progmasoft/StrategyContext.cpp", "src/Progmasoft/PropertyEngine.cpp", "src/Progmasoft/Snapshot.cpp", diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 87b19651..43754fff 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -343,6 +343,7 @@ set(PROGMASOFT_RUNNER_FILES # Property testing and snapshots. set(PROGMASOFT_LIBRARY_FILES "${CATCH_DIR}/src/Progmasoft/ChoiceShrinker.cpp" + "${CATCH_DIR}/src/Progmasoft/Mock.cpp" "${CATCH_DIR}/src/Progmasoft/StrategyContext.cpp" "${CATCH_DIR}/src/Progmasoft/PropertyEngine.cpp" "${CATCH_DIR}/src/Progmasoft/Snapshot.cpp" diff --git a/src/Progmasoft/Catch3.hpp b/src/Progmasoft/Catch3.hpp index cc2b0632..2e04782a 100644 --- a/src/Progmasoft/Catch3.hpp +++ b/src/Progmasoft/Catch3.hpp @@ -14,6 +14,7 @@ #include <Progmasoft/Catch3/Generators/Text.hpp> #include <Progmasoft/Catch3/Generators/Tuple.hpp> #include <Progmasoft/Catch3/Generators/Vector.hpp> +#include <Progmasoft/Catch3/Mock.hpp> #include <Progmasoft/Catch3/Property.hpp> #include <Progmasoft/Catch3/Random.hpp> #include <Progmasoft/Catch3/Relations.hpp> diff --git a/src/Progmasoft/Catch3/Mock.hpp b/src/Progmasoft/Catch3/Mock.hpp new file mode 100644 index 00000000..1e077ac8 --- /dev/null +++ b/src/Progmasoft/Catch3/Mock.hpp @@ -0,0 +1,955 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0 +#pragma once + +#include <Progmasoft/Catch3/Internal/Enforce.hpp> +#include <Progmasoft/Catch3/Internal/SourceLineInfo.hpp> +#include <Progmasoft/Catch3/Matchers/Matchers.hpp> +#include <Progmasoft/Catch3/Matchers/Templated.hpp> +#include <Progmasoft/Catch3/ToString.hpp> +#include <cstddef> +#include <exception> +#include <functional> +#include <limits> +#include <memory> +#include <mutex> +#include <string> +#include <tuple> +#include <type_traits> +#include <utility> +#include <vector> + +namespace Progmasoft::Catch3 +{ + template<typename Signature> + class MockExpectation; + template<typename Signature> + class MockFunction; + + namespace Detail + { + inline constexpr std::size_t kUnlimitedCalls + = std::numeric_limits<std::size_t>::max(); + + class MockSequence; + + /// What the checks need to know of an expectation, whatever the + /// signature of its mock. + struct MockExpectationState final + { + std::string Description; + Catch::SourceLineInfo Line; + std::size_t Minimum = 1; + std::size_t Maximum = 1; + std::size_t Calls = 0; + bool Retired = false; + std::vector<std::shared_ptr<MockSequence>> Sequences; + + MockExpectationState(std::string description, + const Catch::SourceLineInfo &line) + : Description(std::move(description)) + , Line(line) + {} + + [[nodiscard]] bool + Satisfied() const noexcept + { + return Calls >= Minimum; + } + + /// Whether the expectation can take one more call. + [[nodiscard]] bool + Available() const noexcept + { + return !Retired && Calls < Maximum; + } + }; + + /// The steps of a sequence. It does not keep them alive: a step whose + /// mock is gone was verified with that mock. + class MockSequence final + { + public: + void + Add(const std::shared_ptr<MockExpectationState> &step); + + /// Moves to `step`, retiring the steps before it. Returns what + /// was left undone, empty when the call came in order. + [[nodiscard]] std::string + Enter(const MockExpectationState &step); + + private: + std::vector<std::weak_ptr<MockExpectationState>> steps; + std::size_t current = 0; + }; + + /// All mocks share one lock, since a sequence can span them. + [[nodiscard]] std::unique_lock<std::recursive_mutex> + LockMocks(); + + enum class MockReport : unsigned char + { + kContinue, + kInDestructor, + }; + + /// Fails the running test case at `line`. Outside of a test case + /// the message goes to standard error. + void + ReportMockFailure(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message, + MockReport report); + + /// Fails the running test case and leaves it, for a call nothing can + /// return from. + [[noreturn]] void + AbortMockCall(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message); + + /// Counts a met expectation as a passed assertion. + void + ReportMockSuccess(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message); + + /// "exactly 2 calls", "at least 1 call", "any number of calls". + [[nodiscard]] std::string + DescribeCalls(std::size_t minimum, std::size_t maximum); + + /// "called once" and the like, for a report. + [[nodiscard]] std::string + DescribeCount(std::size_t calls); + + template<std::size_t Index, typename Signature> + struct MockArgumentOf; + + template<std::size_t Index, typename... Arguments> + struct MockArgumentOf<Index, void(Arguments...)> + { + using Type = std::tuple_element_t<Index, std::tuple<Arguments...>>; + }; + + /// The type of a parameter of a mocked method. + template<std::size_t Index, typename Signature> + using MockArgument = typename MockArgumentOf<Index, Signature>::Type; + + template<typename T> + struct IsComposedMatcher: std::false_type + { + }; + template<typename T> + struct IsComposedMatcher<Catch::Matchers::Detail::MatchAllOf<T>> + : std::true_type + { + }; + template<typename T> + struct IsComposedMatcher<Catch::Matchers::Detail::MatchAnyOf<T>> + : std::true_type + { + }; + template<typename T> + struct IsComposedMatcher<Catch::Matchers::Detail::MatchNotOf<T>> + : std::true_type + { + }; + template<typename... Ts> + struct IsComposedMatcher< + Catch::Matchers::Detail::MatchAllOfGeneric<Ts...>>: std::true_type + { + }; + template<typename... Ts> + struct IsComposedMatcher< + Catch::Matchers::Detail::MatchAnyOfGeneric<Ts...>>: std::true_type + { + }; + template<typename T> + struct IsComposedMatcher<Catch::Matchers::Detail::MatchNotOfGeneric<T>> + : std::true_type + { + }; + + /// How one argument of a call is judged. + template<typename Argument> + struct ArgumentMatcher final + { + std::function<bool(const Argument &)> Test; + std::string Description; + }; + } // namespace Detail + + /// Accepts any value of an argument. + struct AnyArgument final + {}; + inline constexpr AnyArgument Any{}; + + /// Accepts the values `predicate` accepts; `description` names it in + /// reports. + template<typename Predicate> + struct WhereArgument final + { + Predicate Test; + std::string Description; + }; + + template<typename Predicate> + [[nodiscard]] WhereArgument<std::decay_t<Predicate>> + Where(Predicate &&predicate, std::string description) + { + return { std::forward<Predicate>(predicate), std::move(description) }; + } + + namespace Detail + { + template<typename Argument, typename Expected> + [[nodiscard]] ArgumentMatcher<Argument> + MakeArgumentMatcher(Expected &&expected) + { + using Stored = std::decay_t<Expected>; + if constexpr (std::is_same_v<Stored, AnyArgument>) + { + return { [](const Argument &) { + return true; + }, + "any" }; + } + else if constexpr (std::is_base_of_v< + Catch::Matchers::MatcherUntypedBase, + Stored>) + { + static_assert(!IsComposedMatcher<Stored>::value, + "A composed matcher refers to temporaries that " + "are gone when the mock is called; use Where."); + auto matcher = std::make_shared<Stored>( + std::forward<Expected>(expected)); + auto description = matcher->toString(); + return { [matcher](const Argument &value) { + return matcher->match(value); + }, + std::move(description) }; + } + else if constexpr (!std::is_null_pointer_v<Stored> + && std::is_constructible_v<std::string, Stored> + && std::is_constructible_v<std::string, + Argument>) + { + // Strings are compared by value, also as `const char *`. + std::string text{ std::forward<Expected>(expected) }; + auto description = Catch::Detail::stringify(text); + return { [text = std::move(text)](const Argument &value) { + if constexpr (std::is_pointer_v<Argument>) + { + return value != nullptr && text == value; + } + else + { + return text == value; + } + }, + std::move(description) }; + } + else if constexpr (std::is_integral_v<Argument> + && std::is_integral_v<Stored>) + { + // A literal is an int, whatever the type of the parameter. + const auto value = static_cast<Argument>(expected); + return { [value](const Argument &argument) { + return argument == value; + }, + Catch::Detail::stringify(value) }; + } + else + { + auto description = Catch::Detail::stringify(expected); + return { [value = std::forward<Expected>(expected)]( + const Argument &argument) { + return argument == value; + }, + std::move(description) }; + } + } + + template<typename Argument, typename Predicate> + [[nodiscard]] ArgumentMatcher<Argument> + MakeArgumentMatcher(WhereArgument<Predicate> where) + { + return { std::move(where.Test), std::move(where.Description) }; + } + } // namespace Detail + + /// Calls that have to come in the order their expectations were declared + /// in, across mocks. + class Sequence final + { + public: + Sequence() + : state(std::make_shared<Detail::MockSequence>()) + {} + + private: + template<typename Signature> + friend class MockExpectation; + + std::shared_ptr<Detail::MockSequence> state; + }; + + /// One expected kind of call of a mock. It expects one call until told + /// otherwise. + template<typename Result, typename... Arguments> + class MockExpectation<Result(Arguments...)> final + { + public: + MockExpectation & + Times(std::size_t count) + { + return Between(count, count); + } + + MockExpectation & + AtLeast(std::size_t count) + { + return Between(count, Detail::kUnlimitedCalls); + } + + MockExpectation & + AtMost(std::size_t count) + { + return Between(0, count); + } + + MockExpectation & + AnyTimes() + { + return Between(0, Detail::kUnlimitedCalls); + } + + MockExpectation & + Never() + { + return Between(0, 0); + } + + MockExpectation & + Between(std::size_t minimum, std::size_t maximum) + { + CATCH_ENFORCE(minimum <= maximum, + "A mock expectation needs its minimum at most its " + "maximum"); + const auto lock = Detail::LockMocks(); + state->Minimum = minimum; + state->Maximum = maximum; + return *this; + } + + /// Takes this step in `sequence`, after the ones declared before it. + MockExpectation & + InSequence(const Sequence &sequence) + { + const auto lock = Detail::LockMocks(); + sequence.state->Add(state); + state->Sequences.push_back(sequence.state); + return *this; + } + + /// Returns `value`. A type that cannot be copied is moved out on the + /// first call; a reference result refers to `value`, which has to + /// outlive the mock. + template<typename Value> + MockExpectation & + Returns(Value &&value) + { + static_assert(!std::is_void_v<Result>, + "A mock of a void function returns nothing."); + if constexpr (std::is_reference_v<Result>) + { + static_assert(std::is_lvalue_reference_v<Value>, + "A reference result needs an object that " + "outlives the mock."); + auto *target = &value; + return Does([target](Arguments...) -> Result { + return static_cast<Result>(*target); + }); + } + else if constexpr (std::is_copy_constructible_v<Result>) + { + return Does([held = Result(std::forward<Value>(value))]( + Arguments...) -> Result { + return held; + }); + } + else + { + auto held + = std::make_shared<Result>(std::forward<Value>(value)); + return Does([held](Arguments...) -> Result { + return std::move(*held); + }); + } + } + + /// Runs `action` with the arguments of each call; what it returns is + /// what the call returns. + template<typename Action> + MockExpectation & + Does(Action &&work) + { + const auto lock = Detail::LockMocks(); + action = std::make_shared<std::function<Result(Arguments...)>>( + std::forward<Action>(work)); + return *this; + } + + /// Throws `exception` from each call. + template<typename Exception> + MockExpectation & + Throws(Exception exception) + { + return Does( + [exception = std::move(exception)](Arguments...) -> Result { + Catch::throw_exception(exception); + }); + } + + /// What the arguments of a call have to match. + using Matchers + = std::tuple<Detail::ArgumentMatcher<std::decay_t<Arguments>>...>; + + private: + template<typename Signature> + friend class MockFunction; + + MockExpectation(std::string description, + const Catch::SourceLineInfo &line, + std::unique_ptr<Matchers> accepted) + : state(std::make_shared<Detail::MockExpectationState>( + std::move(description), + line)) + , matchers(std::move(accepted)) + {} + + template<std::size_t... Indices> + [[nodiscard]] bool + Accepts([[maybe_unused]] const std::tuple< + const std::decay_t<Arguments> &...> &values, + std::index_sequence<Indices...>) const + { + return (std::get<Indices>(*matchers).Test(std::get<Indices>(values)) + && ...); + } + + [[nodiscard]] bool + Accepts( + const std::tuple<const std::decay_t<Arguments> &...> &values) const + { + return matchers == nullptr + || Accepts(values, std::index_sequence_for<Arguments...>{}); + } + + std::shared_ptr<Detail::MockExpectationState> state; + // Empty when any arguments will do. + std::unique_ptr<Matchers> matchers; + std::shared_ptr<std::function<Result(Arguments...)>> action; + }; + + /// A mock of one function: a callable that checks its calls against the + /// expectations set on it and fails the running test case when they do + /// not hold. It is what `CATCH3_MOCK_METHOD` declares, and on its own it + /// stands in for a free function behind a seam. Its destructor verifies + /// it. + template<typename Result, typename... Arguments> + class MockFunction<Result(Arguments...)> final + { + using Expectation = MockExpectation<Result(Arguments...)>; + + public: + /// Declares an expectation at `line`; called with what the arguments + /// have to match. See `CATCH3_EXPECT_CALL`. + class ExpectationStart final + { + public: + template<typename... Expected> + Expectation & + operator()(Expected &&...expected) + { + constexpr auto arguments = sizeof...(Arguments); + static_assert(sizeof...(Expected) == 0 + || sizeof...(Expected) == arguments, + "Match every argument of the mock, or none to " + "accept any."); + if constexpr (sizeof...(Expected) == arguments + && arguments != 0) + { + return mock.Add( + line, + std::make_unique<typename Expectation::Matchers>( + Detail::MakeArgumentMatcher< + std::decay_t<Arguments>>( + std::forward<Expected>(expected))...)); + } + else + { + return mock.Add(line, nullptr); + } + } + + private: + friend class MockFunction; + + ExpectationStart(MockFunction &function, + const Catch::SourceLineInfo &place) + : mock(function) + , line(place) + {} + + MockFunction &mock; + Catch::SourceLineInfo line; + }; + + /// `place` is where a call no expectation takes is reported when the + /// mock has no expectations to point at. + explicit MockFunction(std::string mockName = "mock", + const Catch::SourceLineInfo &place + = CATCH_INTERNAL_LINEINFO) + : name(std::move(mockName)) + , declared(place) + , exceptions(std::uncaught_exceptions()) + {} + + MockFunction(const MockFunction &) = delete; + MockFunction & + operator=(const MockFunction &) = delete; + + ~MockFunction() + { + // A test case that is being left has failed for its own reason, + // and what it did not get to call says nothing more. + if (std::uncaught_exceptions() <= exceptions) + { + VerifyAll(Detail::MockReport::kInDestructor); + } + } + + [[nodiscard]] ExpectationStart + Expect(const Catch::SourceLineInfo &line) + { + return { *this, line }; + } + + /// Checks the expectations now and forgets them, so that a test case + /// can go on with new ones. + void + Verify() + { + VerifyAll(Detail::MockReport::kContinue); + } + + Result + operator()(Arguments... arguments) + { + const Decision decision = Decide( + std::tuple<const std::decay_t<Arguments> &...>{ arguments... }); + if (decision.line == nullptr) + { + return Refuse(decision.message); + } + if (!decision.message.empty()) + { + Detail::ReportMockFailure(*decision.line, + name, + decision.message, + Detail::MockReport::kContinue); + } + if (decision.action != nullptr) + { + return (*decision.action)( + std::forward<Arguments>(arguments)...); + } + if constexpr (std::is_void_v<Result>) + { + return; + } + else if constexpr (std::is_default_constructible_v<Result>) + { + return Result{}; + } + else + { + Detail::AbortMockCall(*decision.line, + name, + "nothing to return from " + name + + ": give the expectation Returns " + "or Does"); + } + } + + /// A function that calls this mock, which has to outlive it. + [[nodiscard]] std::function<Result(Arguments...)> + AsFunction() + { + return [this](Arguments... arguments) -> Result { + return (*this)(std::forward<Arguments>(arguments)...); + }; + } + + private: + struct Decision final + { + // Empty when the call is refused. + const Catch::SourceLineInfo *line = nullptr; + std::shared_ptr<std::function<Result(Arguments...)>> action; + std::string message; + }; + + Expectation & + Add(const Catch::SourceLineInfo &line, + std::unique_ptr<typename Expectation::Matchers> matchers) + { + auto description = Describe(matchers.get()); + const auto lock = Detail::LockMocks(); + expectations.push_back(std::unique_ptr<Expectation>( + new Expectation(std::move(description), + line, + std::move(matchers)))); + return *expectations.back(); + } + + template<std::size_t... Indices> + [[nodiscard]] std::string + DescribeMatchers( + [[maybe_unused]] const typename Expectation::Matchers &matchers, + std::index_sequence<Indices...>) const + { + std::string text; + ((text += (Indices == 0 ? "" : ", ") + + std::get<Indices>(matchers).Description), + ...); + return text; + } + + [[nodiscard]] std::string + Describe(const typename Expectation::Matchers *matchers) const + { + if (matchers != nullptr) + { + return name + "(" + + DescribeMatchers( + *matchers, + std::index_sequence_for<Arguments...>{}) + + ")"; + } + return sizeof...(Arguments) == 0 ? name + "()" : name + "(...)"; + } + + template<std::size_t... Indices> + [[nodiscard]] std::string + DescribeCall([[maybe_unused]] const std::tuple< + const std::decay_t<Arguments> &...> &values, + std::index_sequence<Indices...>) const + { + std::string text = name + "("; + ((text += (Indices == 0 ? "" : ", ") + + Catch::Detail::stringify(std::get<Indices>(values))), + ...); + return text + ")"; + } + + /// Picks the first declared expectation that accepts the call and can + /// take it, and counts the call. + [[nodiscard]] Decision + Decide(const std::tuple<const std::decay_t<Arguments> &...> &values) + { + const auto lock = Detail::LockMocks(); + const Expectation *used = nullptr; + for (const auto &expectation : expectations) + { + if (!expectation->Accepts(values)) + { + continue; + } + if (expectation->state->Available()) + { + return Take(*expectation); + } + used = expectation.get(); + } + const auto call + = DescribeCall(values, std::index_sequence_for<Arguments...>{}); + Decision refused; + refused.message = used != nullptr ? DescribeUsed(call, *used->state) + : DescribeUnexpected(call); + return refused; + } + + [[nodiscard]] Decision + Take(const Expectation &expectation) + { + auto &state = *expectation.state; + ++state.Calls; + Decision decision; + decision.line = &state.Line; + decision.action = expectation.action; + for (const auto &sequence : state.Sequences) + { + decision.message += sequence->Enter(state); + } + return decision; + } + + [[nodiscard]] static std::string + DescribeUsed(const std::string &call, + const Detail::MockExpectationState &state) + { + return state.Retired ? "call out of order: " + call + + ", its step of the sequence has passed" + : "too many calls: " + call + ", expected " + + Detail::DescribeCalls(state.Minimum, + state.Maximum) + + " of " + state.Description; + } + + [[nodiscard]] std::string + DescribeUnexpected(const std::string &call) const + { + std::string text = "unexpected call: " + call; + if (expectations.empty()) + { + return text + ", nothing was expected of this mock"; + } + text += ", expected"; + for (const auto &expectation : expectations) + { + text += "\n " + expectation->state->Description; + } + return text; + } + + /// Reports a call no expectation takes, and returns what the call + /// can return. + Result + Refuse(const std::string &message) + { + const auto line = FirstLine(); + if constexpr (std::is_void_v<Result>) + { + Detail::ReportMockFailure(line, + name, + message, + Detail::MockReport::kContinue); + } + else if constexpr (std::is_default_constructible_v<Result>) + { + Detail::ReportMockFailure(line, + name, + message, + Detail::MockReport::kContinue); + return Result{}; + } + else + { + Detail::AbortMockCall(line, name, message); + } + } + + /// Where to point at for a call that matches nothing: the first + /// expectation, or the mock when it has none. + [[nodiscard]] Catch::SourceLineInfo + FirstLine() const + { + const auto lock = Detail::LockMocks(); + return expectations.empty() ? declared + : expectations.front()->state->Line; + } + + void + VerifyAll(Detail::MockReport report) + { + std::vector<std::unique_ptr<Expectation>> checked; + { + const auto lock = Detail::LockMocks(); + checked.swap(expectations); + } + for (const auto &expectation : checked) + { + const auto &state = *expectation->state; + if (state.Satisfied()) + { + Detail::ReportMockSuccess(state.Line, + name, + state.Description); + } + else + { + Detail::ReportMockFailure( + state.Line, + name, + "unmet expectation: " + state.Description + " needs " + + Detail::DescribeCalls(state.Minimum, + state.Maximum) + + ", " + Detail::DescribeCount(state.Calls), + report); + } + } + } + + std::string name; + Catch::SourceLineInfo declared; + int exceptions; + std::vector<std::unique_ptr<Expectation>> expectations; + }; + + /// Points a seam, a `std::function` or a function pointer that production + /// code calls through, at `replacement` until the end of the scope. + template<typename Seam> + class ScopedSeam final + { + public: + ScopedSeam(Seam &target, std::enable_if_t<true, Seam> replacement) + : seam(target) + , original(std::move(target)) + { + seam = std::move(replacement); + } + + ScopedSeam(const ScopedSeam &) = delete; + ScopedSeam & + operator=(const ScopedSeam &) = delete; + + ~ScopedSeam() + { + seam = std::move(original); + } + + private: + Seam &seam; + Seam original; + }; + +} // namespace Progmasoft::Catch3 + +// How many types a parenthesized list holds, up to 8, including none. A type +// with a comma of its own needs an alias. +#define CATCH3_INTERNAL_MOCK_EXPAND(...) __VA_ARGS__ +#define CATCH3_INTERNAL_MOCK_CONCAT_IMPL(a, b) a##b +#define CATCH3_INTERNAL_MOCK_CONCAT(a, b) CATCH3_INTERNAL_MOCK_CONCAT_IMPL(a, b) +#define CATCH3_INTERNAL_MOCK_FIRST(first, ...) first +#define CATCH3_INTERNAL_MOCK_SECOND_OF(first, second, ...) second +#define CATCH3_INTERNAL_MOCK_SECOND(...) \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_SECOND_OF(__VA_ARGS__)) +#define CATCH3_INTERNAL_MOCK_PROBE(...) ~, 1 +#define CATCH3_INTERNAL_MOCK_IS_EMPTY(argument) \ + CATCH3_INTERNAL_MOCK_EXPAND( \ + CATCH3_INTERNAL_MOCK_SECOND(CATCH3_INTERNAL_MOCK_PROBE argument(), \ + 0, )) +#define CATCH3_INTERNAL_MOCK_SELECT(a1, \ + a2, \ + a3, \ + a4, \ + a5, \ + a6, \ + a7, \ + a8, \ + a9, \ + count, \ + ...) \ + count +#define CATCH3_INTERNAL_MOCK_COUNT_0(...) \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_SELECT(__VA_ARGS__, \ + 9, \ + 8, \ + 7, \ + 6, \ + 5, \ + 4, \ + 3, \ + 2, \ + 1, \ + 0)) +#define CATCH3_INTERNAL_MOCK_COUNT_1(...) 0 +#define CATCH3_INTERNAL_MOCK_COUNT(...) \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_CONCAT( \ + CATCH3_INTERNAL_MOCK_COUNT_, \ + CATCH3_INTERNAL_MOCK_IS_EMPTY(CATCH3_INTERNAL_MOCK_EXPAND( \ + CATCH3_INTERNAL_MOCK_FIRST(__VA_ARGS__, ))))(__VA_ARGS__)) + +#define CATCH3_INTERNAL_MOCK_ARGUMENT(signature, index) \ + ::Progmasoft::Catch3::Detail::MockArgument<index, signature> +#define CATCH3_INTERNAL_MOCK_PARAMETER(signature, index) \ + CATCH3_INTERNAL_MOCK_ARGUMENT(signature, index) argument##index +#define CATCH3_INTERNAL_MOCK_FORWARD(signature, index) \ + static_cast<CATCH3_INTERNAL_MOCK_ARGUMENT(signature, index) &&>( \ + argument##index) + +#define CATCH3_INTERNAL_MOCK_LIST_0(each, s) +#define CATCH3_INTERNAL_MOCK_LIST_1(each, s) each(s, 0) +#define CATCH3_INTERNAL_MOCK_LIST_2(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_1(each, s), each(s, 1) +#define CATCH3_INTERNAL_MOCK_LIST_3(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_2(each, s), each(s, 2) +#define CATCH3_INTERNAL_MOCK_LIST_4(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_3(each, s), each(s, 3) +#define CATCH3_INTERNAL_MOCK_LIST_5(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_4(each, s), each(s, 4) +#define CATCH3_INTERNAL_MOCK_LIST_6(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_5(each, s), each(s, 5) +#define CATCH3_INTERNAL_MOCK_LIST_7(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_6(each, s), each(s, 6) +#define CATCH3_INTERNAL_MOCK_LIST_8(each, s) \ + CATCH3_INTERNAL_MOCK_LIST_7(each, s), each(s, 7) +#define CATCH3_INTERNAL_MOCK_LIST(count, each, signature) \ + CATCH3_INTERNAL_MOCK_CONCAT(CATCH3_INTERNAL_MOCK_LIST_, count) \ + (each, signature) + +#define CATCH3_INTERNAL_MOCK_DECLARE(result, \ + name, \ + field, \ + arguments, \ + count, \ + specifiers) \ + result name(CATCH3_INTERNAL_MOCK_LIST(count, \ + CATCH3_INTERNAL_MOCK_PARAMETER, \ + void arguments)) specifiers \ + { \ + return field(CATCH3_INTERNAL_MOCK_LIST(count, \ + CATCH3_INTERNAL_MOCK_FORWARD, \ + void arguments)); \ + } \ + mutable ::Progmasoft::Catch3::MockFunction<result arguments> field \ + { \ + #name, CATCH_INTERNAL_LINEINFO \ + } +#define CATCH3_INTERNAL_MOCK_SPLIT(result, name, field, arguments, ...) \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_DECLARE( \ + result, \ + name, \ + field, \ + arguments, \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_COUNT arguments), \ + CATCH3_INTERNAL_MOCK_EXPAND(CATCH3_INTERNAL_MOCK_FIRST(__VA_ARGS__)))) + +/// Declares the member `name` of a mock class and, beside it, the mock +/// `name##Mock` that `CATCH3_EXPECT_CALL` takes. The parenthesized list of +/// parameter types comes next, and then what goes after the parameters: +/// `CATCH3_MOCK_METHOD(int, Get, (const Key &), const override);`. A type with +/// a comma in it needs an alias. +#define CATCH3_MOCK_METHOD(result, name, ...) \ + CATCH3_MOCK_OVERLOAD(result, name, name##Mock, __VA_ARGS__) + +/// `CATCH3_MOCK_METHOD` for one of several overloads, whose mock needs a name +/// of its own: `field`. +#define CATCH3_MOCK_OVERLOAD(result, name, field, ...) \ + CATCH3_INTERNAL_MOCK_EXPAND( \ + CATCH3_INTERNAL_MOCK_SPLIT(result, name, field, __VA_ARGS__, )) + +/// Declares a `MockFunction` called `name` with a signature: +/// `CATCH3_MOCK_FUNCTION(clock, long());`. +#define CATCH3_MOCK_FUNCTION(name, ...) \ + ::Progmasoft::Catch3::MockFunction<__VA_ARGS__> name \ + { \ + #name, CATCH_INTERNAL_LINEINFO \ + } + +/// Expects a call of `mock`, a `MockFunction`, and takes what its arguments +/// have to match: values, Catch matchers, `Any` or `Where`, or nothing to +/// accept any. Failures of the expectation point at this line. +#define CATCH3_EXPECT_CALL(mock) (mock).Expect(CATCH_INTERNAL_LINEINFO) diff --git a/src/Progmasoft/Catch3/meson.build b/src/Progmasoft/Catch3/meson.build index ccf93f29..8420003b 100644 --- a/src/Progmasoft/Catch3/meson.build +++ b/src/Progmasoft/Catch3/meson.build @@ -311,6 +311,7 @@ runner_sources = files( property_sources = files( '../ChoiceShrinker.cpp', '../StrategyContext.cpp', + '../Mock.cpp', '../PropertyEngine.cpp', '../Snapshot.cpp', ) diff --git a/src/Progmasoft/Mock.cpp b/src/Progmasoft/Mock.cpp new file mode 100644 index 00000000..2897838c --- /dev/null +++ b/src/Progmasoft/Mock.cpp @@ -0,0 +1,203 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0 + +#include <Progmasoft/Catch3/Interfaces/IResultCapture.hpp> +#include <Progmasoft/Catch3/Internal/AssertionHandler.hpp> +#include <Progmasoft/Catch3/Internal/CompilerCapabilities.hpp> +#include <Progmasoft/Catch3/Internal/Context.hpp> +#include <Progmasoft/Catch3/Mock.hpp> +#include <algorithm> +#include <cstdio> +#include <cstdlib> +#include <exception> + +namespace Progmasoft::Catch3::Detail +{ + namespace + { + constexpr auto kMacro = "CATCH3_EXPECT_CALL"; + + /// Whether a test case is running that a report can belong to. + [[nodiscard]] bool + InTestCase() + { + const auto *capture = Catch::getCurrentContext().getResultCapture(); + return capture != nullptr && !capture->getCurrentTestName().empty(); + } + + void + Report(const Catch::SourceLineInfo &line, + const std::string &mock, + Catch::ResultWas::OfType result, + std::string message, + Catch::ResultDisposition::Flags disposition) + { + Catch::AssertionHandler handler(kMacro, line, mock, disposition); + handler.handleMessage(result, std::move(message)); + handler.complete(); + } + + void + WriteToStandardError(const Catch::SourceLineInfo &line, + const std::string &mock, + const std::string &message) + { + std::fprintf(stderr, + "%s:%zu: %s: %s\n", + line.file, + line.line, + mock.c_str(), + message.c_str()); + } + } // namespace + + void + MockSequence::Add(const std::shared_ptr<MockExpectationState> &step) + { + steps.push_back(step); + } + + std::string + MockSequence::Enter(const MockExpectationState &step) + { + const auto position + = std::find_if(steps.begin() + static_cast<std::ptrdiff_t>(current), + steps.end(), + [&step](const auto &candidate) { + return candidate.lock().get() == &step; + }); + std::string undone; + if (position == steps.end()) + { + return undone; + } + const auto target = static_cast<std::size_t>(position - steps.begin()); + for (; current < target; ++current) + { + const auto earlier = steps[current].lock(); + if (earlier == nullptr) + { + continue; + } + if (!earlier->Satisfied()) + { + undone += (undone.empty() ? "call out of order: " : "\n and ") + + step.Description + " came before " + + earlier->Description; + } + earlier->Retired = true; + } + return undone; + } + + std::unique_lock<std::recursive_mutex> + LockMocks() + { + static std::recursive_mutex mutex; + return std::unique_lock<std::recursive_mutex>{ mutex }; + } + + void + ReportMockFailure(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message, + MockReport report) + { + if (!InTestCase()) + { + WriteToStandardError(line, mock, message); + return; + } + // An aborting run throws out of any failure, and a destructor must + // not let that through. + CATCH_TRY + { + Report(line, + mock, + Catch::ResultWas::ExplicitFailure, + std::move(message), + Catch::ResultDisposition::ContinueOnFailure); + } + CATCH_CATCH_ALL + { + if (report == MockReport::kContinue) + { + std::rethrow_exception(std::current_exception()); + } + } + } + + void + AbortMockCall(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message) + { + if (InTestCase()) + { + Report(line, + mock, + Catch::ResultWas::ExplicitFailure, + std::move(message), + Catch::ResultDisposition::Normal); + } + else + { + WriteToStandardError(line, mock, message); + } + std::abort(); + } + + void + ReportMockSuccess(const Catch::SourceLineInfo &line, + const std::string &mock, + std::string message) + { + if (InTestCase()) + { + Report(line, + mock, + Catch::ResultWas::Ok, + std::move(message), + Catch::ResultDisposition::ContinueOnFailure); + } + } + + std::string + DescribeCalls(std::size_t minimum, std::size_t maximum) + { + const auto calls = [](std::size_t count) { + return std::to_string(count) + (count == 1 ? " call" : " calls"); + }; + if (minimum == 0 && maximum == kUnlimitedCalls) + { + return "any number of calls"; + } + if (minimum == maximum) + { + return maximum == 0 ? "no calls" : "exactly " + calls(minimum); + } + if (maximum == kUnlimitedCalls) + { + return "at least " + calls(minimum); + } + if (minimum == 0) + { + return "at most " + calls(maximum); + } + return std::to_string(minimum) + " to " + calls(maximum); + } + + std::string + DescribeCount(std::size_t calls) + { + switch (calls) + { + case 0: + return "never called"; + case 1: + return "called once"; + default: + return "called " + std::to_string(calls) + " times"; + } + } +} // namespace Progmasoft::Catch3::Detail diff --git a/src/Progmasoft/meson.build b/src/Progmasoft/meson.build index c7044232..9958c2c1 100644 --- a/src/Progmasoft/meson.build +++ b/src/Progmasoft/meson.build @@ -15,6 +15,7 @@ if get_option('install') install_headers('Catch3/Results.hpp', subdir: 'Progmasoft/Catch3') install_headers('Catch3/Runner.hpp', subdir: 'Progmasoft/Catch3') install_headers('Catch3/Version.hpp', subdir: 'Progmasoft/Catch3') + install_headers('Catch3/Mock.hpp', subdir: 'Progmasoft/Catch3') install_headers('Catch3/Snapshot.hpp', subdir: 'Progmasoft/Catch3') install_headers('Catch3/XmlWriter.hpp', subdir: 'Progmasoft/Catch3') install_headers('Catch3/ForAll.hpp', subdir: 'Progmasoft/Catch3') diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 23185534..40ed1358 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -155,6 +155,7 @@ target_link_libraries(SelfTest PRIVATE Catch3WithMain) # Exercise the Progmasoft layer independently of the upstream Catch2 SelfTest. add_executable(ProgmasoftSelfTest Progmasoft/Generators.tests.cpp + Progmasoft/Mock.tests.cpp Progmasoft/Property.tests.cpp Progmasoft/ForAll.tests.cpp Progmasoft/Results.tests.cpp diff --git a/tests/Progmasoft/BUILD.bazel b/tests/Progmasoft/BUILD.bazel index 0571d862..871baaa8 100644 --- a/tests/Progmasoft/BUILD.bazel +++ b/tests/Progmasoft/BUILD.bazel @@ -9,6 +9,7 @@ cc_test( srcs = [ "ForAll.tests.cpp", "Generators.tests.cpp", + "Mock.tests.cpp", "Property.tests.cpp", "Results.tests.cpp", "Runner.tests.cpp", diff --git a/tests/Progmasoft/Mock.tests.cpp b/tests/Progmasoft/Mock.tests.cpp new file mode 100644 index 00000000..5c8272c7 --- /dev/null +++ b/tests/Progmasoft/Mock.tests.cpp @@ -0,0 +1,310 @@ +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-2.0 + +#include <Progmasoft/Catch3/Matchers/Contains.hpp> +#include <Progmasoft/Catch3/Matchers/String.hpp> +#include <Progmasoft/Catch3/Mock.hpp> +#include <Progmasoft/Catch3/TestMacros.hpp> +#include <functional> +#include <map> +#include <memory> +#include <stdexcept> +#include <string> +#include <thread> +#include <vector> + +#include "RunnerFixtureClient.hpp" + +namespace +{ + using Catch::Matchers::ContainsSubstring; + using Catch::Matchers::StartsWith; + using FixtureClient::ExitCode; + using FixtureClient::RunFixture; + using FixtureClient::Scratch; + using Progmasoft::Catch3::Any; + using Progmasoft::Catch3::MockFunction; + using Progmasoft::Catch3::ScopedSeam; + using Progmasoft::Catch3::Sequence; + using Progmasoft::Catch3::Where; + + using Scores = std::map<std::string, int>; + + class Store + { + public: + virtual ~Store() = default; + virtual void + Open() = 0; + virtual int + Get(const std::string &key) const = 0; + virtual void + Put(const std::string &key, int value) = 0; + virtual void + Put(const Scores &scores) = 0; + virtual int + Sum(int a, int b, int c, int d, int e, int f, int g, int h) = 0; + virtual std::string & + Name() = 0; + virtual std::unique_ptr<int> + Take() = 0; + }; + + class MockStore final: public Store + { + public: + CATCH3_MOCK_METHOD(void, Open, (), override); + CATCH3_MOCK_METHOD(int, Get, (const std::string &), const override); + CATCH3_MOCK_OVERLOAD(void, + Put, + PutOne, + (const std::string &, int), + override); + CATCH3_MOCK_OVERLOAD(void, Put, PutAll, (const Scores &), override); + CATCH3_MOCK_METHOD(int, + Sum, + (int, int, int, int, int, int, int, int), + override); + CATCH3_MOCK_METHOD(std::string &, Name, (), override); + CATCH3_MOCK_METHOD(std::unique_ptr<int>, Take, (), override); + }; + + /// Production code with a seam for a free function. + std::function<long()> clockSeam = [] { + return 0L; + }; + + long + SecondsSince(long start) + { + return clockSeam() - start; + } + + long (*pointerSeam)() = nullptr; +} // namespace + +TEST_CASE("A mock method answers as its expectation says", "[progmasoft][mock]") +{ + MockStore store; + CATCH3_EXPECT_CALL(store.GetMock)("a").Returns(1); + CATCH3_EXPECT_CALL(store.GetMock)(StartsWith("b")).Times(2).Returns(2); + CATCH3_EXPECT_CALL(store.GetMock)(Any).Never(); + CATCH3_EXPECT_CALL(store.OpenMock)(); + CATCH3_EXPECT_CALL(store.PutOne)("a", 1); + CATCH3_EXPECT_CALL(store.PutAll)(); + + const Store &reading = store; + Store &writing = store; + writing.Open(); + CHECK(reading.Get("a") == 1); + CHECK(reading.Get("bb") == 2); + CHECK(reading.Get("bc") == 2); + writing.Put("a", 1); + writing.Put(Scores{ { "a", 1 } }); +} + +TEST_CASE("Every parameter reaches an action", "[progmasoft][mock]") +{ + MockStore store; + CATCH3_EXPECT_CALL(store.SumMock) + (1, + Any, + 3, + Any, + 5, + Any, + 7, + Where( + [](int v) { + return v > 7; + }, + "above 7")) + .Does([](int a, int b, int c, int d, int e, int f, int g, int h) { + return a + b + c + d + e + f + g + h; + }); + CHECK(store.Sum(1, 2, 3, 4, 5, 6, 7, 8) == 36); +} + +TEST_CASE("Calls are counted against limits", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, void()); + CATCH3_MOCK_FUNCTION(tock, int(int)); + CATCH3_EXPECT_CALL(tick)().AtLeast(2); + CATCH3_EXPECT_CALL(tock)(1).AtMost(1).Returns(10); + CATCH3_EXPECT_CALL(tock)(1).Between(1, 2).Returns(20); + CATCH3_EXPECT_CALL(tock)(2).AnyTimes(); + + tick(); + tick(); + tick(); + CHECK(tock(1) == 10); + CHECK(tock(1) == 20); + CHECK(tock(1) == 20); + CHECK(tock(2) == 0); +} + +TEST_CASE("Verify ends the expectations so far", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, int()); + CATCH3_EXPECT_CALL(tick)().Returns(1); + CHECK(tick() == 1); + tick.Verify(); + CATCH3_EXPECT_CALL(tick)().Returns(2); + CHECK(tick() == 2); +} + +TEST_CASE("A sequence orders calls across mocks", "[progmasoft][mock]") +{ + Sequence order; + MockStore first; + MockStore second; + CATCH3_EXPECT_CALL(first.OpenMock)().InSequence(order); + CATCH3_EXPECT_CALL(second.GetMock)(Any).Times(2).InSequence(order); + CATCH3_EXPECT_CALL(first.GetMock)(Any).AtLeast(0).InSequence(order); + CATCH3_EXPECT_CALL(first.PutOne)(Any, Any).InSequence(order); + + first.Open(); + second.Get("a"); + second.Get("b"); + first.Put("a", 1); +} + +TEST_CASE("Results of every kind come back", "[progmasoft][mock]") +{ + MockStore store; + std::string name = "store"; + CATCH3_EXPECT_CALL(store.NameMock)().Returns(name); + CATCH3_EXPECT_CALL(store.TakeMock)().Returns(std::make_unique<int>(7)); + CATCH3_EXPECT_CALL(store.GetMock)("throws").Throws( + std::runtime_error("no such key")); + CATCH3_EXPECT_CALL(store.GetMock)("default"); + + CHECK(&store.Name() == &name); + CHECK(*store.Take() == 7); + CHECK_THROWS_WITH(store.Get("throws"), "no such key"); + CHECK(store.Get("default") == 0); +} + +TEST_CASE("Actions keep their state between calls", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(next, int()); + int count = 0; + CATCH3_EXPECT_CALL(next)().Times(3).Does([&count] { + return ++count; + }); + next(); + next(); + CHECK(next() == 3); +} + +TEST_CASE("An action may call a mock", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(outer, int()); + CATCH3_MOCK_FUNCTION(inner, int()); + CATCH3_EXPECT_CALL(inner)().Returns(4); + CATCH3_EXPECT_CALL(outer)().Does([&inner] { + return inner() + 1; + }); + CHECK(outer() == 5); +} + +TEST_CASE("Strings match by value, whatever their type", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(log, void(const char *)); + CATCH3_MOCK_FUNCTION(size, void(std::size_t)); + CATCH3_EXPECT_CALL(log)("text"); + CATCH3_EXPECT_CALL(log)(nullptr); + CATCH3_EXPECT_CALL(size)(3); + + const std::string text = "text"; + log(text.c_str()); + log(nullptr); + size(3); +} + +TEST_CASE("Calls from other threads are counted", "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, int()); + CATCH3_EXPECT_CALL(tick)().Times(400).Returns(1); + std::vector<std::thread> threads; + for (int thread = 0; thread < 4; ++thread) + { + threads.emplace_back([&tick] { + for (int call = 0; call < 100; ++call) + { + tick(); + } + }); + } + for (auto &thread : threads) + { + thread.join(); + } +} + +TEST_CASE("A mock stands in for a free function behind a seam", + "[progmasoft][mock]") +{ + CATCH3_MOCK_FUNCTION(clock, long()); + CATCH3_EXPECT_CALL(clock)().Returns(50L); + { + const ScopedSeam seam(clockSeam, clock.AsFunction()); + CHECK(SecondsSince(20) == 30); + } + CHECK(SecondsSince(0) == 0); + + { + const ScopedSeam seam( + pointerSeam, + +[] { + return 9L; + }); + CHECK(pointerSeam() == 9); + } + CHECK(pointerSeam == nullptr); +} + +TEST_CASE("Failures of mocks fail the test case that owns them", + "[progmasoft][mock]") +{ + const Scratch scratch; + const auto run = RunFixture(scratch, { "--no-history", "[mock]" }); + INFO(run.output); + CHECK(ExitCode(run) == 42); + const auto message = [&run](const std::string &name) { + const auto &test = run.Test(name); + CHECK(test["outcome"].AsString() == "failed"); + std::string text; + for (const auto &assertion : + test["attempts"].Items().front()["assertions"].Items()) + { + text += assertion["message"].AsString() + "\n"; + } + return text; + }; + + const auto unexpected = message("fixture mock unexpected call"); + CHECK_THAT(unexpected, + ContainsSubstring("unexpected call: Get(\"b\"), expected\n" + " Get(\"a\")")); + CHECK_THAT(message("fixture mock with no expectations"), + ContainsSubstring("nothing was expected of this mock")); + CHECK_THAT(message("fixture mock unmet expectation"), + ContainsSubstring("unmet expectation: tick() needs exactly 2 " + "calls, called once")); + CHECK_THAT(message("fixture mock too many calls"), + ContainsSubstring("too many calls: tick(), expected exactly 1 " + "call of tick()")); + CHECK_THAT(message("fixture mock out of order"), + ContainsSubstring("call out of order: second() came before " + "first()")); + CHECK_THAT(message("fixture mock past its step"), + ContainsSubstring("call out of order: first(), its step of the " + "sequence has passed")); + CHECK_THAT(message("fixture mock with nothing to return"), + ContainsSubstring("nothing to return from name")); + + const auto leaving = message("fixture mock left by a failure"); + CHECK_THAT(leaving, ContainsSubstring("the test's own failure")); + CHECK_FALSE(ContainsSubstring("unmet").match(leaving)); +} diff --git a/tests/Progmasoft/RunnerFixture.cpp b/tests/Progmasoft/RunnerFixture.cpp index 37d196a3..2796912f 100644 --- a/tests/Progmasoft/RunnerFixture.cpp +++ b/tests/Progmasoft/RunnerFixture.cpp @@ -4,6 +4,7 @@ // A test binary whose tests misbehave on purpose. The runner's integration // tests start it and check what the runner made of each misbehaviour. +#include <Progmasoft/Catch3/Mock.hpp> #include <Progmasoft/Catch3/Runner.hpp> #include <Progmasoft/Catch3/TestMacros.hpp> #include <chrono> @@ -184,6 +185,72 @@ TEST_CASE("fixture slow four", "[fixture][slow]") CHECK(true); } +TEST_CASE("fixture mock unexpected call", "[fixture][mock]") +{ + Progmasoft::Catch3::MockFunction<int(std::string)> get{ "Get" }; + CATCH3_EXPECT_CALL(get)("a").Returns(1); + CHECK(get("a") == 1); + CHECK(get("b") == 0); +} + +TEST_CASE("fixture mock with no expectations", "[fixture][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, void()); + tick(); +} + +TEST_CASE("fixture mock unmet expectation", "[fixture][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, void()); + CATCH3_EXPECT_CALL(tick)().Times(2); + tick(); +} + +TEST_CASE("fixture mock too many calls", "[fixture][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, void()); + CATCH3_EXPECT_CALL(tick)(); + tick(); + tick(); +} + +TEST_CASE("fixture mock out of order", "[fixture][mock]") +{ + Progmasoft::Catch3::Sequence order; + CATCH3_MOCK_FUNCTION(first, void()); + CATCH3_MOCK_FUNCTION(second, void()); + CATCH3_EXPECT_CALL(first)().InSequence(order); + CATCH3_EXPECT_CALL(second)().InSequence(order); + second(); +} + +TEST_CASE("fixture mock past its step", "[fixture][mock]") +{ + Progmasoft::Catch3::Sequence order; + CATCH3_MOCK_FUNCTION(first, void()); + CATCH3_MOCK_FUNCTION(second, void()); + CATCH3_EXPECT_CALL(first)().AtLeast(1).InSequence(order); + CATCH3_EXPECT_CALL(second)().InSequence(order); + first(); + second(); + first(); +} + +TEST_CASE("fixture mock with nothing to return", "[fixture][mock]") +{ + CATCH3_MOCK_FUNCTION(name, std::string & ()); + CATCH3_EXPECT_CALL(name)(); + name(); + FAIL("the call above leaves the test case"); +} + +TEST_CASE("fixture mock left by a failure", "[fixture][mock]") +{ + CATCH3_MOCK_FUNCTION(tick, void()); + CATCH3_EXPECT_CALL(tick)(); + FAIL("the test's own failure"); +} + TEST_CASE("fixture skips", "[fixture][skip]") { SKIP("not on this machine"); diff --git a/tests/meson.build b/tests/meson.build index 26ae225d..f6b26899 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -73,6 +73,7 @@ progmasoft_property_tests = executable( 'ProgmasoftPropertyTests', [ 'Progmasoft/Generators.tests.cpp', + 'Progmasoft/Mock.tests.cpp', 'Progmasoft/Property.tests.cpp', 'Progmasoft/ForAll.tests.cpp', 'Progmasoft/Results.tests.cpp', diff --git a/xmake.lua b/xmake.lua index 2e4a2b79..e7686aff 100644 --- a/xmake.lua +++ b/xmake.lua @@ -32,6 +32,7 @@ target("catch3") "src/Progmasoft/Results.cpp", "src/Progmasoft/XmlWriter.cpp", "src/Progmasoft/ChoiceShrinker.cpp", + "src/Progmasoft/Mock.cpp", "src/Progmasoft/StrategyContext.cpp", "src/Progmasoft/PropertyEngine.cpp", "src/Progmasoft/Snapshot.cpp"