diff options
| author | Leitwolf11 | 2026-10-23 23:25:41 +0300 |
|---|---|---|
| committer | Leitwolf11 | 2026-10-23 23:25:41 +0300 |
| commit | 93d4e7dbebab867e0a7f59fc25308460a7a76280 (patch) | |
| tree | 54c568a3063d49e21f039b6d38f40f51be7991f0 /extras/catch_amalgamated.cpp | |
| parent | 83c6fc3ac13abe83fa39f7b5f8559ccaf4412233 (diff) | |
| download | catch3-93d4e7dbebab867e0a7f59fc25308460a7a76280.tar.gz | |
Replace Catch2's runner with the Catch3 runner
Catch3 keeps Catch2's API for writing tests (test cases, sections,
assertions, matchers, generators, listeners, benchmarks) and replaces its
session, command line and reporters with a runner of its own.
The runner starts the test binary again as worker processes that take the
next test when they are free. A crash, an exit or a hang ends one test, not
the run. On top of that it adds:
- retries that tell deterministic, flaky and order-dependent failures
apart, and a bisection that names the test leaving state behind;
- a history of verdicts and durations, kept under a file lock, that drives
automatic quarantine, adaptive time limits, predictive ordering,
incremental runs, failure replay and analytics;
- selection by change, past failure or prediction, dependency closure,
and sharding by a stable hash or by recorded durations;
- tags for serial and isolated tests, shared resources, dependencies,
time limits and retries;
- a failure category, a signature, an analysis and a replay command for
every failure;
- console, JUnit XML (with flakyFailure and rerunFailure), JSON and Chrome
trace reports, and a versioned discovery listing (--list=json);
- a temporary directory per test (TMPDIR, TMP, TEMP and
TestTemporaryDirectory());
- Bazel's test environment: XML output file, test filter, sharding, seed,
premature exit file and undeclared outputs.
The command line is parsed by a hand-written, typed parser. Catch::Session
remains as a thin facade over the runner for custom main functions.
The result model in Progmasoft/Catch3/Results.hpp was rewritten around
tests and attempts, and the JUnit writer now writes it. The property
engine's example database is updated under a file lock with an atomic
rename, so parallel workers can write it at the same time.
Removed: Clara, the command-line and reporter-spec parsers, every reporter
and CATCH_REGISTER_REPORTER, listing, text flow, console colour and width,
the JSON and XML writers, sharding in the test registry, approval tests and
their baselines, Conan packaging, and the unused third_party/clara.hpp.
Listeners keep working; reporters' list and skip events are gone.
The Catch2 library is now compiled as C++20; code that includes only
Catch2's headers still needs C++14. Duplicate test cases are still refused
at start-up. Version 0.3.0.
ExtraTests now turn exceptions off for clang-cl with MSVC-style options,
which -fno-exceptions did not do.
Diffstat (limited to 'extras/catch_amalgamated.cpp')
| -rw-r--r-- | extras/catch_amalgamated.cpp | 17818 |
1 files changed, 11928 insertions, 5890 deletions
diff --git a/extras/catch_amalgamated.cpp b/extras/catch_amalgamated.cpp index 230849f1..506726dc 100644 --- a/extras/catch_amalgamated.cpp +++ b/extras/catch_amalgamated.cpp @@ -7,10 +7,14 @@ // SPDX-License-Identifier: BSL-1.0 // Catch v3.16.1 -// Generated: 2026-10-11 02:09:25.622664 +// Generated: 2026-10-23 23:12:01.455813 // ---------------------------------------------------------- // This file is an amalgamation of multiple different files. // You probably shouldn't edit it directly. +// +// The parts that come from src/Progmasoft, the Catch3 runner, keep +// their own copyright and license lines (MPL-2.0 WITH +// AdditionRef-Progmasoft-Exception-1.1). The runner needs C++20. // ---------------------------------------------------------- #include "catch_amalgamated.hpp" @@ -36,6 +40,2117 @@ #endif // defined(CATCH_PLATFORM_WINDOWS) #endif // CATCH_WINDOWS_H_PROXY_HPP_INCLUDED +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +#include <cassert> +#include <optional> +#include <string> +#include <utility> +#include <variant> + +namespace Progmasoft::Catch3::Runner +{ + /// A failure with a message written for the person who has to act on it. + class Error final + { + public: + explicit Error(std::string message) + : message_{std::move(message)} + { + } + + [[nodiscard]] const std::string & + Message() const noexcept + { + return message_; + } + + /// The same failure, with what was being done in front of it. + [[nodiscard]] Error + Context(const std::string &context) const + { + return Error{context + ": " + message_}; + } + + private: + std::string message_; + }; + + /// A value, or the error that prevented it. + template<typename Value> + class [[nodiscard]] Expected final + { + public: + Expected(Value value) + : state_{std::in_place_index<0>, std::move(value)} + { + } + + Expected(Error error) + : state_{std::in_place_index<1>, std::move(error)} + { + } + + [[nodiscard]] explicit + operator bool() const noexcept + { + return state_.index() == 0; + } + + [[nodiscard]] Value & + operator*() & + { + assert(*this); + return std::get<0>(state_); + } + + [[nodiscard]] const Value & + operator*() const & + { + assert(*this); + return std::get<0>(state_); + } + + [[nodiscard]] Value && + operator*() && + { + assert(*this); + return std::get<0>(std::move(state_)); + } + + [[nodiscard]] Value * + operator->() + { + assert(*this); + return &std::get<0>(state_); + } + + [[nodiscard]] const Value * + operator->() const + { + assert(*this); + return &std::get<0>(state_); + } + + [[nodiscard]] const Error & + GetError() const + { + assert(!*this); + return std::get<1>(state_); + } + + private: + std::variant<Value, Error> state_; + }; + + /// Success, or the error that happened instead. + class [[nodiscard]] Status final + { + public: + Status() = default; + + Status(Error error) + : error_{std::move(error)} + { + } + + [[nodiscard]] static Status + Ok() + { + return Status{}; + } + + [[nodiscard]] explicit + operator bool() const noexcept + { + return !error_.has_value(); + } + + [[nodiscard]] const Error & + GetError() const + { + assert(error_.has_value()); + return *error_; + } + + private: + std::optional<Error> error_; + }; +} // namespace Progmasoft::Catch3::Runner + +#include <filesystem> +#include <string> +#include <vector> + +namespace Progmasoft::Catch3::Runner +{ + /// What Git says about the checkout the tests come from. + struct Changes final + { + std::filesystem::path root; + std::string commit; + /// Changed since the revision, staged or not, and untracked files; + /// absolute paths. + std::vector<std::filesystem::path> files; + }; + + /// The changes since `revision` (HEAD when empty) in the checkout that + /// holds `directory`. An error when Git or the checkout is missing. + [[nodiscard]] Expected<Changes> + ReadChanges(const std::filesystem::path &directory, + const std::string &revision); + + /// The commit checked out in the checkout that holds `directory`, or + /// empty. + [[nodiscard]] std::string + CurrentCommit(const std::filesystem::path &directory); + + /// Whether two spellings name the same source file: equal after + /// normalisation, or one a relative path the other ends with. + [[nodiscard]] bool + SameSourceFile(const std::filesystem::path &left, + const std::filesystem::path &right); +} // namespace Progmasoft::Catch3::Runner +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + + +#include <cstdint> +#include <initializer_list> +#include <memory> +#include <string> +#include <string_view> +#include <utility> +#include <vector> + +namespace Progmasoft::Catch3::Runner +{ + class JsonValue; + + /// The members of an object, in the order they were added or read. + using JsonMembers = std::vector<std::pair<std::string, JsonValue>>; + + /// One JSON value: the messages between the runner and its workers, the + /// reports, the history and the coverage exports of LLVM are all JSON. + /// + /// Numbers are kept as `double`, which represents every integer the + /// runner writes (counts, microseconds, 32-bit seeds) exactly. + class JsonValue final + { + public: + enum class Kind : std::uint8_t + { + kNull, + kBoolean, + kNumber, + kString, + kArray, + kObject, + }; + + JsonValue() = default; + JsonValue(std::nullptr_t) {} + JsonValue(bool value); + JsonValue(double value); + JsonValue(int value); + JsonValue(unsigned value); + JsonValue(long value); + JsonValue(unsigned long value); + JsonValue(long long value); + JsonValue(unsigned long long value); + JsonValue(const char *value); + JsonValue(std::string value); + JsonValue(std::string_view value); + + JsonValue(const JsonValue &other); + JsonValue(JsonValue &&other) noexcept; + JsonValue & + operator=(const JsonValue &other); + JsonValue & + operator=(JsonValue &&other) noexcept; + ~JsonValue(); + + [[nodiscard]] static JsonValue + Array(std::vector<JsonValue> items = {}); + + [[nodiscard]] static JsonValue + Object(JsonMembers members = {}); + + [[nodiscard]] Kind + GetKind() const noexcept + { + return kind_; + } + + [[nodiscard]] bool + IsNull() const noexcept + { + return kind_ == Kind::kNull; + } + + [[nodiscard]] bool + IsObject() const noexcept + { + return kind_ == Kind::kObject; + } + + [[nodiscard]] bool + IsArray() const noexcept + { + return kind_ == Kind::kArray; + } + + [[nodiscard]] bool + IsString() const noexcept + { + return kind_ == Kind::kString; + } + + [[nodiscard]] bool + IsNumber() const noexcept + { + return kind_ == Kind::kNumber; + } + + /// The value as a Boolean; anything else reads as `fallback`. + [[nodiscard]] bool + AsBoolean(bool fallback = false) const noexcept; + + [[nodiscard]] double + AsNumber(double fallback = 0.0) const noexcept; + + [[nodiscard]] std::int64_t + AsInteger(std::int64_t fallback = 0) const noexcept; + + [[nodiscard]] std::uint64_t + AsUnsigned(std::uint64_t fallback = 0) const noexcept; + + /// The text of a string; anything else reads as the empty string. + [[nodiscard]] const std::string & + AsString() const noexcept; + + /// The items of an array; anything else has none. + [[nodiscard]] const std::vector<JsonValue> & + Items() const noexcept; + + [[nodiscard]] std::vector<JsonValue> & + MutableItems(); + + /// The members of an object; anything else has none. + [[nodiscard]] const JsonMembers & + Members() const noexcept; + + [[nodiscard]] JsonMembers & + MutableMembers(); + + /// The member named `key`, or null when there is none. + [[nodiscard]] const JsonValue & + operator[](std::string_view key) const noexcept; + + /// The member named `key` of an object, added when missing. + JsonValue & + Set(std::string_view key, JsonValue value); + + /// Appends to an array. + void + Append(JsonValue value); + + /// The compact JSON text of the value, on one line. + [[nodiscard]] std::string + ToString() const; + + /// The JSON text with two spaces of indentation per level. + [[nodiscard]] std::string + ToPrettyString() const; + + private: + void + Write(std::string &output, int indent, int depth) const; + + void + Destroy() noexcept; + + void + CopyFrom(const JsonValue &other); + + void + MoveFrom(JsonValue &&other) noexcept; + + Kind kind_ = Kind::kNull; + bool boolean_ = false; + double number_ = 0.0; + std::string string_; + std::unique_ptr<std::vector<JsonValue>> items_; + std::unique_ptr<JsonMembers> members_; + }; + + /// Reads one JSON document; whitespace may surround it. + [[nodiscard]] Expected<JsonValue> + ParseJson(std::string_view text); + + /// Appends `text` to `output` as a JSON string literal. + void + AppendJsonString(std::string &output, std::string_view text); +} // namespace Progmasoft::Catch3::Runner + +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +#include <cstddef> +#include <cstdint> +#include <optional> +#include <string> +#include <vector> + +namespace Progmasoft::Catch3 +{ + /// How one attempt at a test ended. + enum class Outcome : std::uint8_t + { + kPassed, + /// An assertion failed, or the test threw. + kFailed, + /// It failed, but `[!shouldfail]` or `[!mayfail]` allows that. + kFailedAsExpected, + /// `SKIP` was called. + kSkipped, + /// The process running it died: a signal, a structured exception or + /// an exit in the middle of the test. + kCrashed, + /// It ran past its time limit and was stopped. + kTimedOut, + /// A test it depends on did not pass, so it was not run. + kBlocked, + /// The run stopped before reaching it. + kNotRun, + /// It passed last time and nothing it depends on changed, so the + /// incremental run kept that result. + kUpToDate, + }; + + /// Why a failure happened, as far as the runner can tell. + enum class FailureCategory : std::uint8_t + { + kNone, + /// A REQUIRE or CHECK expression was false. + kAssertion, + /// An exception escaped the test or an assertion. + kUnexpectedException, + /// REQUIRE_THROWS and its kind did not see the expected exception. + kMissingException, + /// FAIL or FAIL_CHECK. + kExplicitFailure, + /// A signal or structured exception inside the test. + kCrash, + kTimeout, + /// The worker could not run the test at all, for example because it + /// failed to start. + kInfrastructure, + /// A test it depends on failed. + kDependency, + }; + + /// How a failure relates to earlier runs. + enum class FailureHistory : std::uint8_t + { + /// There is no history to compare with. + kUnknown, + /// The test passed in the previous run. + kNew, + /// The test failed in the previous run too. + kPersistent, + /// The test passes and fails with the same inputs. + kIntermittent, + }; + + /// One failed or reported assertion. + struct AssertionRecord final + { + std::string file; + std::size_t line = 0; + /// The macro, such as REQUIRE. + std::string macro; + /// The expression as written. + std::string expression; + /// The expression with its operands' values. + std::string expansion; + std::string message; + /// INFO, CAPTURE and UNSCOPED_INFO messages in force. + std::vector<std::string> context; + /// The sections the assertion was in, outermost first. + std::vector<std::string> sections; + FailureCategory category = FailureCategory::kAssertion; + bool passed = false; + /// The failure is allowed by `[!mayfail]` or `[!shouldfail]`. + bool allowed = false; + }; + + /// The statistics of one BENCHMARK. + struct BenchmarkRecord final + { + std::string name; + std::size_t samples = 0; + std::size_t iterations = 0; + double meanNanoseconds = 0.0; + double lowMeanNanoseconds = 0.0; + double highMeanNanoseconds = 0.0; + double standardDeviationNanoseconds = 0.0; + double outlierVariance = 0.0; + }; + + /// One section entered while a test ran, with its timing. + struct SectionRecord final + { + std::string name; + std::string file; + std::size_t line = 0; + /// Microseconds since the start of the attempt. + std::int64_t startMicroseconds = 0; + /// -1 for a section that never ended: the attempt crashed or timed + /// out inside it. + std::int64_t durationMicroseconds = 0; + /// 0 for a section directly in the test case. + std::size_t depth = 0; + }; + + /// One run of a test in a worker. A test may be tried several times: + /// retries, repeats and the probes that look for an order dependency. + struct Attempt final + { + Outcome outcome = Outcome::kNotRun; + FailureCategory category = FailureCategory::kNone; + std::size_t number = 1; + /// The worker that ran it, and the tests it ran before, in order. + std::size_t worker = 0; + std::vector<std::string> precededBy; + bool freshProcess = false; + std::uint32_t seed = 0; + std::int64_t startMicroseconds = 0; + std::int64_t durationMicroseconds = 0; + std::size_t assertionsPassed = 0; + std::size_t assertionsFailed = 0; + std::vector<AssertionRecord> assertions; + std::vector<BenchmarkRecord> benchmarks; + std::vector<SectionRecord> sections; + /// What the test wrote to std::cout and std::cerr, captured. + std::string standardOutput; + std::string standardError; + /// What reached the worker's own streams while the test ran, such as + /// printf output or a sanitizer report. + std::string processOutput; + /// The fatal condition the worker reported before it died. + std::string fatalMessage; + /// How the worker ended, for a crash. + std::string exitDescription; + /// The last assertion started before a crash or a timeout. + std::string lastLocation; + /// The directory the test had for its temporary files; kept when the + /// attempt failed, so its contents can be inspected. + std::string temporaryDirectory; + }; + + /// One test of the run and everything the runner learned about it. + struct TestResult final + { + /// Identifies the test across runs and builds: its name, with its + /// class and, when names repeat, its tags. History is kept by it. + std::string id; + std::string name; + std::string className; + std::string file; + std::size_t line = 0; + std::vector<std::string> tags; + /// The verdict over all attempts. + Outcome outcome = Outcome::kNotRun; + FailureCategory category = FailureCategory::kNone; + FailureHistory history = FailureHistory::kUnknown; + /// It failed and then passed with the same inputs. + bool flaky = false; + /// It is in quarantine, so its failure does not fail the run. + bool quarantined = false; + /// Why it was selected, when not every test runs: "failed last + /// run", "covers a changed file", "dependency of ...". + std::string selectionReason; + /// A stable identifier of the failure: its category, location and + /// message with numbers removed. Tests that share it probably share a + /// cause. + std::string signature; + /// What the runner found out about the failure, one finding per + /// entry, most useful first. + std::vector<std::string> analysis; + /// A command line that runs the failure again the way it happened. + std::string replayCommand; + std::vector<Attempt> attempts; + std::int64_t predictedDurationMicroseconds = 0; + double predictedFailureProbability = 0.0; + + /// The sum of the attempts' durations. + [[nodiscard]] std::int64_t + DurationMicroseconds() const noexcept; + + /// Whether the result fails the run. + [[nodiscard]] bool + FailsRun() const noexcept; + }; + + /// The totals of a run, by verdict. + struct RunSummary final + { + std::size_t tests = 0; + std::size_t passed = 0; + std::size_t failed = 0; + std::size_t failedAsExpected = 0; + std::size_t skipped = 0; + std::size_t crashed = 0; + std::size_t timedOut = 0; + std::size_t blocked = 0; + std::size_t notRun = 0; + std::size_t upToDate = 0; + std::size_t flaky = 0; + std::size_t quarantined = 0; + std::size_t attempts = 0; + std::size_t assertionsPassed = 0; + std::size_t assertionsFailed = 0; + /// Wall-clock time of the run, and the time its tests took added up; + /// their ratio is the speed-up of running in parallel. + std::int64_t wallMicroseconds = 0; + std::int64_t testMicroseconds = 0; + }; + + /// A whole run of one test binary. + struct RunResult final + { + std::string binary; + std::uint32_t seed = 0; + std::size_t workers = 0; + /// The shard, when the run was one of several. + unsigned shardIndex = 0; + unsigned shardCount = 1; + /// The tests that matched before selection narrowed them. + std::size_t candidates = 0; + std::vector<TestResult> tests; + /// Test specs that matched nothing. + std::vector<std::string> unmatchedSpecs; + /// Messages about the run itself, such as a history that could not + /// be read. + std::vector<std::string> notes; + std::int64_t startUnixMilliseconds = 0; + std::int64_t wallMicroseconds = 0; + }; + + [[nodiscard]] RunSummary + Summarize(const RunResult &run); + + /// The name an outcome has in reports: "passed", "timed-out", ... + [[nodiscard]] const char * + OutcomeName(Outcome outcome) noexcept; + + [[nodiscard]] std::optional<Outcome> + OutcomeFromName(const std::string &name) noexcept; + + [[nodiscard]] const char * + CategoryName(FailureCategory category) noexcept; + + [[nodiscard]] std::optional<FailureCategory> + CategoryFromName(const std::string &name) noexcept; + + [[nodiscard]] const char * + HistoryName(FailureHistory history) noexcept; + + /// The failure of an attempt as text: each failed assertion with its + /// expansion, messages and sections, or what is known about a crash or a + /// timeout. Empty for an attempt that did not fail. + [[nodiscard]] std::string + DescribeFailure(const Attempt &attempt); + + /// The first line of `DescribeFailure`, for a summary. + [[nodiscard]] std::string + SummarizeFailure(const Attempt &attempt); + + /// Whether an outcome counts as a failure of the attempt. + [[nodiscard]] bool + IsFailure(Outcome outcome) noexcept; +} // namespace Progmasoft::Catch3 + +#include <cstdint> +#include <filesystem> +#include <map> +#include <optional> +#include <string> +#include <vector> + +namespace Progmasoft::Catch3::Runner +{ + /// One result of one test in one run. + struct TestRecord final + { + std::int64_t run = 0; + Outcome outcome = Outcome::kNotRun; + FailureCategory category = FailureCategory::kNone; + std::int64_t durationMicroseconds = 0; + std::size_t attempts = 0; + bool flaky = false; + std::string signature; + /// What the test's result depended on; equal fingerprints mean the + /// same inputs. + std::string fingerprint; + }; + + /// How to run a failure again the way it happened. + struct ReplayRecord final + { + std::int64_t run = 0; + std::uint32_t seed = 0; + /// The tests the worker ran before, in order. + std::vector<std::string> precededBy; + bool freshProcess = false; + }; + + struct QuarantineRecord final + { + std::int64_t since = 0; + std::string reason; + }; + + /// Everything remembered about one test. + struct TestHistory final + { + /// The most recent last. + std::vector<TestRecord> records; + std::optional<QuarantineRecord> quarantine; + std::optional<ReplayRecord> lastFailure; + + [[nodiscard]] const TestRecord * + Last() const noexcept; + + /// A percentile of the durations of recent runs that finished on + /// their own; nothing without such runs. + [[nodiscard]] std::optional<std::int64_t> + DurationPercentile(double fraction) const; + + /// The probability that the next run fails: a failure rate in which + /// each older run counts less, starting from a prior for tests with + /// little history. + [[nodiscard]] double + FailureProbability() const; + + /// Flaky results, and failures that alternate with passes on the + /// same fingerprint, among the last `window` runs. + [[nodiscard]] std::size_t + FlakyEvents(std::size_t window) const; + + /// Passes in a row, counting back from the latest run. + [[nodiscard]] std::size_t + PassStreak() const; + }; + + /// One run, as the history remembers it. + struct RunRecord final + { + std::int64_t id = 0; + std::int64_t unixMilliseconds = 0; + std::string commit; + std::string binaryFingerprint; + std::uint32_t seed = 0; + std::size_t tests = 0; + std::size_t failed = 0; + std::int64_t wallMicroseconds = 0; + }; + + /// When to put a test into quarantine and let it out. + struct QuarantinePolicy final + { + std::size_t window = 20; + std::size_t flakyEventsToEnter = 2; + std::size_t passesToLeave = 10; + }; + + /// The results of earlier runs of one test binary. + /// + /// It lives in one JSON file. Several runs of the same binary may finish + /// at the same time, so `Commit` changes the file under a lock: it reads + /// it again, adds its run and replaces it in one rename. + class History final + { + public: + History() = default; + + /// Reads the file; a missing file is an empty history. + [[nodiscard]] static Expected<History> + Load(const std::filesystem::path &file); + + /// Adds `run` to the history in `file`, updates quarantine and + /// returns the history as written. + [[nodiscard]] static Expected<History> + Commit(const std::filesystem::path &file, + const RunResult &run, + const RunRecord &record, + const std::map<std::string, std::string> &fingerprints, + const QuarantinePolicy &policy); + + [[nodiscard]] const TestHistory * + Find(const std::string &key) const; + + [[nodiscard]] const std::map<std::string, TestHistory> & + Tests() const noexcept + { + return tests_; + } + + [[nodiscard]] const std::vector<RunRecord> & + Runs() const noexcept + { + return runs_; + } + + [[nodiscard]] JsonValue + ToJson() const; + + [[nodiscard]] static Expected<History> + FromJson(const JsonValue &value); + + /// The key a test result is filed under. + [[nodiscard]] static std::string + KeyOf(const TestResult &test); + + private: + void + Apply(const RunResult &run, + RunRecord record, + const std::map<std::string, std::string> &fingerprints, + const QuarantinePolicy &policy); + + std::int64_t nextRun_ = 1; + std::vector<RunRecord> runs_; + std::map<std::string, TestHistory> tests_; + }; + + /// The file the history of a binary is kept in under `directory`. + [[nodiscard]] std::filesystem::path + HistoryFile(const std::filesystem::path &directory, + const std::string &binaryName); +} // namespace Progmasoft::Catch3::Runner +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + + + +#include <chrono> +#include <cstddef> +#include <optional> +#include <string> +#include <vector> + +namespace Catch +{ + class IConfig; +} + +namespace Progmasoft::Catch3::Runner +{ + /// One registered test and what its tags tell the runner. + struct TestEntry final + { + /// The position in registration order, the same in every process + /// started from one binary. + std::size_t index = 0; + /// A name that identifies the test across runs and builds: the test + /// name, with its class when it has one, and its tags when another + /// test has the same name. + std::string key; + std::string name; + std::string className; + std::string file; + std::size_t line = 0; + std::vector<std::string> tags; + bool hidden = false; + bool benchmark = false; + bool mayFail = false; + bool shouldFail = false; + + // From the tags the runner reserves. + /// `[!serial]`: nothing else runs at the same time. + bool serial = false; + /// `[!isolated]`: runs in a process of its own. + bool isolated = false; + /// `[!quarantine]`: in quarantine whatever its history says. + bool quarantined = false; + /// `[!resource:NAME]`: tests sharing a resource never overlap. + std::vector<std::string> resources; + /// `[!depends:NAME]`: runs after these tests, and only if they pass. + std::vector<std::string> dependsOn; + /// `[!timeout:SECONDS]`. + std::optional<std::chrono::milliseconds> timeout; + /// `[!retries:N]`. + std::optional<unsigned> retries; + }; + + /// The tests registered in this binary. + class Catalog final + { + public: + /// Reads the test registry. Problems with a reserved tag, such as a + /// timeout that is not a number, are errors. + [[nodiscard]] static Expected<Catalog> + Load(); + + [[nodiscard]] const std::vector<TestEntry> & + Tests() const noexcept + { + return tests_; + } + + [[nodiscard]] const std::vector<Catch::TestCaseHandle> & + Handles() const noexcept + { + return handles_; + } + + /// The test of a key or of a name; null when there is none. + [[nodiscard]] const TestEntry * + Find(const std::string &keyOrName) const; + + /// A digest of every key in order. A worker reports it, so the runner + /// knows the worker sees the same tests. + [[nodiscard]] const std::string & + Identity() const noexcept + { + return identity_; + } + + /// The tests the specs of `config` select, in registration order, as + /// Catch2 selects them: hidden tests only when a spec names them. + /// Specs that matched nothing are added to `unmatched`. + [[nodiscard]] std::vector<std::size_t> + Match(const Catch::IConfig &config, + std::vector<std::string> &unmatched) const; + + private: + std::vector<TestEntry> tests_; + std::vector<Catch::TestCaseHandle> handles_; + std::string identity_; + }; +} // namespace Progmasoft::Catch3::Runner +// Copyright (c) 2026 Progmasoft. +// SPDX-License-Identifier: MPL-2.0 WITH AdditionRef-Progmasoft-Exception-1.1 + + +#include <chrono> +#include <cstdint> +#include <optional> +#include <string> +#include <vector> + +namespace Progmasoft::Catch3::Runner +{ + /// What the command line asks for. + enum class Mode : std::uint8_t + { + kRun, + kListTests, + kListTags, + kAnalytics, + kHelp, + kVersion, + /// Internal: run tests on behalf of a runner, over the protocol. + kWorker, + }; + + enum class ListFormat : std::uint8_t + { + |