diff options
| author | sp-dani-garcia | 2019-08-29 17:47:56 +0200 |
|---|---|---|
| committer | Martin Hořeňovský | 2019-10-27 10:02:41 +0100 |
| commit | 51b29ced1ac2bde83d977283f6e0b4f871898b8a (patch) | |
| tree | 901d0ce6a74961c3917b5b04638bca21a0406e4d | |
| parent | 9a558171d836cf7b292883b6f65bdd937c30cc76 (diff) | |
| download | catch3-51b29ced1ac2bde83d977283f6e0b4f871898b8a.tar.gz | |
Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2
Close #1738 (this is a cherry-pick and fixup of that PR)
| -rw-r--r-- | codecov.yml | 1 | ||||
| -rw-r--r-- | docs/ci-and-misc.md | 6 | ||||
| -rw-r--r-- | docs/release-process.md | 4 | ||||
| -rw-r--r-- | docs/reporters.md | 1 | ||||
| -rw-r--r-- | include/reporters/catch_reporter_sonarqube.hpp | 181 | ||||
| -rw-r--r-- | projects/CMakeLists.txt | 1 | ||||
| -rw-r--r-- | projects/SelfTest/Baselines/sonarqube.sw.approved.txt | 1730 | ||||
| -rw-r--r-- | projects/SelfTest/TestMain.cpp | 1 | ||||
| -rwxr-xr-x | scripts/approvalTests.py | 4 | ||||
| -rw-r--r-- | scripts/releaseCommon.py | 2 |
10 files changed, 1927 insertions, 4 deletions
diff --git a/codecov.yml b/codecov.yml index 94d88d83..75809ee2 100644 --- a/codecov.yml +++ b/codecov.yml @@ -14,6 +14,7 @@ coverage: - "**/catch_reporter_tap.hpp" - "**/catch_reporter_automake.hpp" - "**/catch_reporter_teamcity.hpp" + - "**/catch_reporter_sonarqube.hpp" - "**/external/clara.hpp" diff --git a/docs/ci-and-misc.md b/docs/ci-and-misc.md index 8c330872..40b7cec9 100644 --- a/docs/ci-and-misc.md +++ b/docs/ci-and-misc.md @@ -12,7 +12,7 @@ Build Systems may refer to low-level tools, like CMake, or larger systems that r ## Continuous Integration systems -Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (currently we also offer TeamCity, TAP and Automake reporters). +Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (currently we also offer TeamCity, TAP, Automake and SonarQube reporters). Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them. @@ -65,6 +65,10 @@ The Automake Reporter writes out the [meta tags](https://www.gnu.org/software/au Because of the incremental nature of Catch's test suites and ability to run specific tests, our implementation of TAP reporter writes out the number of tests in a suite last. +### SonarQube Reporter +```-r sonarqube``` +[SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format for tests metrics. + ## Low-level tools ### Precompiled headers (PCHs) diff --git a/docs/release-process.md b/docs/release-process.md index 7fa5f7e1..ca48da03 100644 --- a/docs/release-process.md +++ b/docs/release-process.md @@ -42,8 +42,8 @@ Tag version and release title should be same as the new version, description should contain the release notes for the current release. Single header version of `catch.hpp` *needs* to be attached as a binary, as that is where the official download link links to. Preferably -it should use linux line endings. All non-bundled reporters (Automake, -TAP, TeamCity) should also be attached as binaries, as they might be +it should use linux line endings. All non-bundled reporters (Automake, TAP, +TeamCity, SonarQube) should also be attached as binaries, as they might be dependent on a specific version of the single-include header. Since 2.5.0, the release tag and the "binaries" (headers) should be PGP diff --git a/docs/reporters.md b/docs/reporters.md index 32b3419b..a33e55bf 100644 --- a/docs/reporters.md +++ b/docs/reporters.md @@ -29,6 +29,7 @@ Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)). * `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format. * `automake` writes in a format that correspond to [automake .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files +* `sonarqube` writes the [SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format. You see what reporters are available from the command line by running with `--list-reporters`. diff --git a/include/reporters/catch_reporter_sonarqube.hpp b/include/reporters/catch_reporter_sonarqube.hpp new file mode 100644 index 00000000..b860293c --- /dev/null +++ b/include/reporters/catch_reporter_sonarqube.hpp @@ -0,0 +1,181 @@ +/* + * Created by Daniel Garcia on 2018-12-04. + * Copyright Social Point SL. All rights reserved. + * + * Distributed under the Boost Software License, Version 1.0. (See accompanying + * file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) + */ +#ifndef CATCH_REPORTER_SONARQUBE_HPP_INCLUDED +#define CATCH_REPORTER_SONARQUBE_HPP_INCLUDED + + +// Don't #include any Catch headers here - we can assume they are already +// included before this header. +// This is not good practice in general but is necessary in this case so this +// file can be distributed as a single header that works with the main +// Catch single header. + +#include <map> + +namespace Catch { + + struct SonarQubeReporter : CumulativeReporterBase<SonarQubeReporter> { + + SonarQubeReporter(ReporterConfig const& config) + : CumulativeReporterBase(config) + , xml(config.stream()) { + m_reporterPrefs.shouldRedirectStdOut = true; + m_reporterPrefs.shouldReportAllAssertions = true; + } + + ~SonarQubeReporter() override; + + static std::string getDescription() { + return "Reports test results in the Generic Test Data SonarQube XML format"; + } + + static std::set<Verbosity> getSupportedVerbosities() { + return { Verbosity::Normal }; + } + + void noMatchingTestCases(std::string const& /*spec*/) override {} + + void testRunStarting(TestRunInfo const& testRunInfo) override { + CumulativeReporterBase::testRunStarting(testRunInfo); + xml.startElement("testExecutions"); + xml.writeAttribute("version", "1"); + } + + void testGroupEnded(TestGroupStats const& testGroupStats) override { + CumulativeReporterBase::testGroupEnded(testGroupStats); + writeGroup(*m_testGroups.back()); + } + + void testRunEndedCumulative() override { + xml.endElement(); + } + + void writeGroup(TestGroupNode const& groupNode) { + std::map<std::string, TestGroupNode::ChildNodes> testsPerFile; + for(auto const& child : groupNode.children) + testsPerFile[child->value.testInfo.lineInfo.file].push_back(child); + + for(auto const& kv : testsPerFile) + writeTestFile(kv.first.c_str(), kv.second); + } + + void writeTestFile(const char* filename, TestGroupNode::ChildNodes const& testCaseNodes) { + XmlWriter::ScopedElement e = xml.scopedElement("file"); + xml.writeAttribute("path", filename); + + for(auto const& child : testCaseNodes) + writeTestCase(*child); + } + + void writeTestCase(TestCaseNode const& testCaseNode) { + // All test cases have exactly one section - which represents the + // test case itself. That section may have 0-n nested sections + assert(testCaseNode.children.size() == 1); + SectionNode const& rootSection = *testCaseNode.children.front(); + writeSection("", rootSection, testCaseNode.value.testInfo.okToFail()); + } + + void writeSection(std::string const& rootName, SectionNode const& sectionNode, bool okToFail) { + std::string name = trim(sectionNode.stats.sectionInfo.name); + if(!rootName.empty()) + name = rootName + '/' + name; + + if(!sectionNode.assertions.empty() || !sectionNode.stdOut.empty() || !sectionNode.stdErr.empty()) { + XmlWriter::ScopedElement e = xml.scopedElement("testCase"); + xml.writeAttribute("name", name); + xml.writeAttribute("duration", static_cast<long>(sectionNode.stats.durationInSeconds * 1000)); + + writeAssertions(sectionNode, okToFail); + } + + for(auto const& childNode : sectionNode.childSections) + writeSection(name, *childNode, okToFail); + } + + void writeAssertions(SectionNode const& sectionNode, bool okToFail) { + for(auto const& assertion : sectionNode.assertions) + writeAssertion( assertion, okToFail); + } + + void writeAssertion(AssertionStats const& stats, bool okToFail) { + AssertionResult const& result = stats.assertionResult; + if(!result.isOk()) { + std::string elementName; + if(okToFail) { + elementName = "skipped"; + } + else { + switch(result.getResultType()) { + case ResultWas::ThrewException: + case ResultWas::FatalErrorCondition: + elementName = "error"; + break; + case ResultWas::ExplicitFailure: + elementName = "failure"; + break; + case ResultWas::ExpressionFailed: + elementName = "failure"; + break; + case ResultWas::DidntThrowException: + elementName = "failure"; + break; + + // We should never see these here: + case ResultWas::Info: + case ResultWas::Warning: + case ResultWas::Ok: + case ResultWas::Unknown: + case ResultWas::FailureBit: + case ResultWas::Exception: + elementName = "internalError"; + break; + } + } + + XmlWriter::ScopedElement e = xml.scopedElement(elementName); + + ReusableStringStream messageRss; + messageRss << result.getTestMacroName() << "(" << result.getExpression() << ")"; + xml.writeAttribute("message", messageRss.str()); + + ReusableStringStream textRss; + if (stats.totals.assertions.total() > 0) { + textRss << "FAILED:\n"; + if (result.hasExpression()) { + textRss << "\t" << result.getExpressionInMacro() << "\n"; + } + if (result.hasExpandedExpression()) { + textRss << "with expansion:\n\t" << result.getExpandedExpression() << "\n"; + } + } + + if(!result.getMessage().empty()) + textRss << result.getMessage() << "\n"; + + for(auto const& msg : stats.infoMessages) + if(msg.type == ResultWas::Info) + textRss << msg.message << "\n"; + + textRss << "at " << result.getSourceInfo(); + xml.writeText(textRss.str(), false); + } + } + + private: + XmlWriter xml; + }; + +#ifdef CATCH_IMPL + SonarQubeReporter::~SonarQubeReporter() {} +#endif + + CATCH_REGISTER_REPORTER( "sonarqube", SonarQubeReporter ) + +} // end namespace Catch + +#endif // CATCH_REPORTER_SONARQUBE_HPP_INCLUDED
\ No newline at end of file diff --git a/projects/CMakeLists.txt b/projects/CMakeLists.txt index cbe21fcf..0aafc178 100644 --- a/projects/CMakeLists.txt +++ b/projects/CMakeLists.txt @@ -282,6 +282,7 @@ set(REPORTER_HEADERS ${HEADER_DIR}/reporters/catch_reporter_tap.hpp ${HEADER_DIR}/reporters/catch_reporter_teamcity.hpp ${HEADER_DIR}/reporters/catch_reporter_xml.h + ${HEADER_DIR}/reporters/catch_reporter_sonarqube.hpp ) set(REPORTER_SOURCES ${HEADER_DIR}/reporters/catch_reporter_bases.cpp diff --git a/projects/SelfTest/Baselines/sonarqube.sw.approved.txt b/projects/SelfTest/Baselines/sonarqube.sw.approved.txt new file mode 100644 index 00000000..65f4655a --- /dev/null +++ b/projects/SelfTest/Baselines/sonarqube.sw.approved.txt @@ -0,0 +1,1730 @@ +<?xml version="1.0" encoding="UTF-8"?> +<testExecutions version="1"loose text artifact +> + <file path="projects/<exe-name>/IntrospectiveTests/CmdLine.tests.cpp"> + <testCase name="Parse test names and tags/Empty test spec should have no filters" duration="{duration}"/> + <testCase name="Parse test names and tags/Test spec from empty string should have no filters" duration="{duration}"/> + <testCase name="Parse test names and tags/Test spec from just a comma should have no filters" duration="{duration}"/> + <testCase name="Parse test names and tags/Test spec from name should have one filter" duration="{duration}"/> + <testCase name="Parse test names and tags/Test spec from quoted name should have one filter" duration="{duration}"/> + <testCase name="Parse test names and tags/Test spec from name should have one filter" duration="{duration}"/> + <testCase name="Parse test names and tags/Wildcard at the start" duration="{duration}"/> + <testCase name="Parse test names and tags/Wildcard at the end" duration="{duration}"/> + <testCase name="Parse test names and tags/Wildcard at both ends" duration="{duration}"/> + <testCase name="Parse test names and tags/Redundant wildcard at the start" duration="{duration}"/> + <testCase name="Parse test names and tags/Redundant wildcard at the end" duration="{duration}"/> + <testCase name="Parse test names and tags/Redundant wildcard at both ends" duration="{duration}"/> + <testCase name="Parse test names and tags/Wildcard at both ends, redundant at start" duration="{duration}"/> + <testCase name="Parse test names and tags/Just wildcard" duration="{duration}"/> + <testCase name="Parse test names and tags/Single tag" duration="{duration}"/> + <testCase name="Parse test names and tags/Single tag, two matches" duration="{duration}"/> + <testCase name="Parse test names and tags/Two tags" duration="{duration}"/> + <testCase name="Parse test names and tags/Two tags, spare separated" duration="{duration}"/> + <testCase name="Parse test names and tags/Wildcarded name and tag" duration="{duration}"/> + <testCase name="Parse test names and tags/Single tag exclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/One tag exclusion and one tag inclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/One tag exclusion and one wldcarded name inclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/One tag exclusion, using exclude:, and one wldcarded name inclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/name exclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/wildcarded name exclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/wildcarded name exclusion with tag inclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/wildcarded name exclusion, using exclude:, with tag inclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/two wildcarded names" duration="{duration}"/> + <testCase name="Parse test names and tags/empty tag" duration="{duration}"/> + <testCase name="Parse test names and tags/empty quoted name" duration="{duration}"/> + <testCase name="Parse test names and tags/quoted string followed by tag exclusion" duration="{duration}"/> + <testCase name="Parse test names and tags/Leading and trailing spaces in test spec" duration="{duration}"/> + <testCase name="Parse test names and tags/Leading and trailing spaces in test name" duration="{duration}"/> + <testCase name="Process can be configured on command line/empty args don't cause a crash" duration="{duration}"/> + <testCase name="Process can be configured on command line/default - no arguments" duration="{duration}"/> + <testCase name="Process can be configured on command line/test lists/Specify one test case using" duration="{duration}"/> + <testCase name="Process can be configured on command line/test lists/Specify one test case exclusion using exclude:" duration="{duration}"/> + <testCase name="Process can be configured on command line/test lists/Specify one test case exclusion using ~" duration="{duration}"/> + <testCase name="Process can be configured on command line/reporter/-r/console" duration="{duration}"/> + <testCase name="Process can be configured on command line/reporter/-r/xml" duration="{duration}"/> + <testCase name="Process can be configured on command line/reporter/--reporter/junit" duration="{duration}"/> + <testCase name="Process can be configured on command line/reporter/Only one reporter is accepted" duration="{duration}"/> + <testCase name="Process can be configured on command line/reporter/must match one of the available ones" duration="{duration}"/> + <testCase name="Process can be configured on command line/debugger/-b" duration="{duration}"/> + <testCase name="Process can be configured on command line/debugger/--break" duration="{duration}"/> + <testCase name="Process can be configured on command line/abort/-a aborts after first failure" duration="{duration}"/> + <testCase name="Process can be configured on command line/abort/-x 2 aborts after two failures" duration="{duration}"/> + <testCase name="Process can be configured on command line/abort/-x must be numeric" duration="{duration}"/> + <testCase name="Process can be configured on command line/nothrow/-e" duration="{duration}"/> + <testCase name="Process can be configured on command line/nothrow/--nothrow" duration="{duration}"/> + <testCase name="Process can be configured on command line/output filename/-o filename" duration="{duration}"/> + <testCase name="Process can be configured on command line/output filename/--out" duration="{duration}"/> + <testCase name="Process can be configured on command line/combinations/Single character flags can be combined" duration="{duration}"/> + <testCase name="Process can be configured on command line/use-colour/without option" duration="{duration}"/> + <testCase name="Process can be configured on command line/use-colour/auto" duration="{duration}"/> + <testCase name="Process can be configured on command line/use-colour/yes" duration="{duration}"/> + <testCase name="Process can be configured on command line/use-colour/no" duration="{duration}"/> + <testCase name="Process can be configured on command line/use-colour/error" duration="{duration}"/> + <testCase name="Process can be configured on command line/Benchmark options/samples" duration="{duration}"/> + <testCase name="Process can be configured on command line/Benchmark options/resamples" duration="{duration}"/> + <testCase name="Process can be configured on command line/Benchmark options/resamples" duration="{duration}"/> + <testCase name="Process can be configured on command line/Benchmark options/resamples" duration="{duration}"/> + <testCase name="Test with special, characters "in name" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/GeneratorsImpl.tests.cpp"> + <testCase name="Generators internals/Single value" duration="{duration}"/> + <testCase name="Generators internals/Preset values" duration="{duration}"/> + <testCase name="Generators internals/Generator combinator" duration="{duration}"/> + <testCase name="Generators internals/Explicitly typed generator sequence" duration="{duration}"/> + <testCase name="Generators internals/Filter generator" duration="{duration}"/> + <testCase name="Generators internals/Take generator/Take less" duration="{duration}"/> + <testCase name="Generators internals/Take generator/Take more" duration="{duration}"/> + <testCase name="Generators internals/Map with explicit return type" duration="{duration}"/> + <testCase name="Generators internals/Map with deduced return type" duration="{duration}"/> + <testCase name="Generators internals/Repeat/Singular repeat" duration="{duration}"/> + <testCase name="Generators internals/Repeat/Actual repeat" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive auto step/Integer" duration="{duration}"/> + <testCase name="Generators internals/Range/Negative auto step/Integer" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Integer/Exact" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Integer/Slightly over end" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Integer/Slightly under end" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Floating Point/Exact" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Floating Point/Slightly over end" duration="{duration}"/> + <testCase name="Generators internals/Range/Positive manual step/Floating Point/Slightly under end" duration="{duration}"/> + <testCase name="Generators internals/Range/Negative manual step/Integer/Exact" duration="{duration}"/> + <testCase name="Generators internals/Range/Negative manual step/Integer/Slightly over end" duration="{duration}"/> + <testCase name="Generators internals/Range/Negative manual step/Integer/Slightly under end" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/PartTracker.tests.cpp"> + <testCase name="Tracker" duration="{duration}"/> + <testCase name="Tracker/successfully close one section" duration="{duration}"/> + <testCase name="Tracker/fail one section" duration="{duration}"/> + <testCase name="Tracker/fail one section/re-enter after failed section" duration="{duration}"/> + <testCase name="Tracker/fail one section/re-enter after failed section and find next section" duration="{duration}"/> + <testCase name="Tracker/successfully close one section, then find another" duration="{duration}"/> + <testCase name="Tracker/successfully close one section, then find another/Re-enter - skips S1 and enters S2" duration="{duration}"/> + <testCase name="Tracker/successfully close one section, then find another/Re-enter - skips S1 and enters S2/Successfully close S2" duration="{duration}"/> + <testCase name="Tracker/successfully close one section, then find another/Re-enter - skips S1 and enters S2/fail S2" duration="{duration}"/> + <testCase name="Tracker/open a nested section" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp"> + <testCase name="Comparison ops" duration="{duration}"/> + <testCase name="Our PCG implementation provides expected results for known seeds/Default seeded" duration="{duration}"/> + <testCase name="Our PCG implementation provides expected results for known seeds/Specific seed" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/String.tests.cpp"> + <testCase name="StringRef/Empty string" duration="{duration}"/> + <testCase name="StringRef/From string literal" duration="{duration}"/> + <testCase name="StringRef/From sub-string" duration="{duration}"/> + <testCase name="StringRef/Substrings/zero-based substring" duration="{duration}"/> + <testCase name="StringRef/Substrings/non-zero-based substring" duration="{duration}"/> + <testCase name="StringRef/Substrings/Pointer values of full refs should match" duration="{duration}"/> + <testCase name="StringRef/Substrings/Pointer values of substring refs should also match" duration="{duration}"/> + <testCase name="StringRef/Substrings/Past the end substring" duration="{duration}"/> + <testCase name="StringRef/Substrings/Substring off the end are trimmed" duration="{duration}"/> + <testCase name="StringRef/Comparisons are deep" duration="{duration}"/> + <testCase name="StringRef/from std::string/implicitly constructed" duration="{duration}"/> + <testCase name="StringRef/from std::string/explicitly constructed" duration="{duration}"/> + <testCase name="StringRef/from std::string/assigned" duration="{duration}"/> + <testCase name="StringRef/to std::string/explicitly constructed" duration="{duration}"/> + <testCase name="StringRef/to std::string/assigned" duration="{duration}"/> + <testCase name="StringRef at compilation time/Simple constructors" duration="{duration}"/> + <testCase name="StringRef at compilation time/UDL construction" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/StringManip.tests.cpp"> + <testCase name="Trim strings" duration="{duration}"/> + <testCase name="replaceInPlace/replace single char" duration="{duration}"/> + <testCase name="replaceInPlace/replace two chars" duration="{duration}"/> + <testCase name="replaceInPlace/replace first char" duration="{duration}"/> + <testCase name="replaceInPlace/replace last char" duration="{duration}"/> + <testCase name="replaceInPlace/replace all chars" duration="{duration}"/> + <testCase name="replaceInPlace/replace no chars" duration="{duration}"/> + <testCase name="replaceInPlace/escape '" duration="{duration}"/> + <testCase name="splitString" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/Tag.tests.cpp"> + <testCase name="Tag alias can be registered against tag patterns/The same tag alias can only be registered once" duration="{duration}"/> + <testCase name="Tag alias can be registered against tag patterns/Tag aliases must be of the form [@name]" duration="{duration}"/> + <testCase name="shortened hide tags are split apart" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/ToString.tests.cpp"> + <testCase name="Directly creating an EnumInfo" duration="{duration}"/> + <testCase name="parseEnums/No enums" duration="{duration}"/> + <testCase name="parseEnums/One enum value" duration="{duration}"/> + <testCase name="parseEnums/Multiple enum values" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/IntrospectiveTests/Xml.tests.cpp"> + <testCase name="XmlEncode/normal string" duration="{duration}"/> + <testCase name="XmlEncode/empty string" duration="{duration}"/> + <testCase name="XmlEncode/string with ampersand" duration="{duration}"/> + <testCase name="XmlEncode/string with less-than" duration="{duration}"/> + <testCase name="XmlEncode/string with greater-than" duration="{duration}"/> + <testCase name="XmlEncode/string with quotes" duration="{duration}"/> + <testCase name="XmlEncode/string with control char (1)" duration="{duration}"/> + <testCase name="XmlEncode/string with control char (x7F)" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Valid utf-8 strings" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Invalid utf-8 strings/Various broken strings" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Invalid utf-8 strings/Overlong encodings" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Invalid utf-8 strings/Surrogate pairs" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Invalid utf-8 strings/Invalid start byte" duration="{duration}"/> + <testCase name="XmlEncode: UTF-8/Invalid utf-8 strings/Missing continuation byte(s)" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/UsageTests/Approx.tests.cpp"> + <testCase name="A comparison that uses literals instead of the normal constructor" duration="{duration}"/> + <testCase name="Absolute margin" duration="{duration}"/> + <testCase name="Approx setters validate their arguments" duration="{duration}"/> + <testCase name="Approx with exactly-representable margin" duration="{duration}"/> + <testCase name="Approximate PI" duration="{duration}"/> + <testCase name="Approximate comparisons with different epsilons" duration="{duration}"/> + <testCase name="Approximate comparisons with floats" duration="{duration}"/> + <testCase name="Approximate comparisons with ints" duration="{duration}"/> + <testCase name="Approximate comparisons with mixed numeric types" duration="{duration}"/> + <testCase name="Comparison with explicitly convertible types" duration="{duration}"/> + <testCase name="Default scale is invisible to comparison" duration="{duration}"/> + <testCase name="Epsilon only applies to Approx's value" duration="{duration}"/> + <testCase name="Greater-than inequalities with different epsilons" duration="{duration}"/> + <testCase name="Less-than inequalities with different epsilons" duration="{duration}"/> + <testCase name="Some simple comparisons between doubles" duration="{duration}"/> + <testCase name="Use a custom approx" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/UsageTests/BDD.tests.cpp"> + <testCase name="Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods/Given: No operations precede me" duration="{duration}"/> + <testCase name="Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods/Given: No operations precede me/When: We get the count/Then: Subsequently values are higher" duration="{duration}"/> + <testCase name="Scenario: Do that thing with the thing/Given: This stuff exists/And given: And some assumption/When: I do this/Then: it should do this" duration="{duration}"/> + <testCase name="Scenario: Do that thing with the thing/Given: This stuff exists/And given: And some assumption/When: I do this/Then: it should do this/And: do that" duration="{duration}"/> + <testCase name="Scenario: This is a really long scenario name to see how the list command deals with wrapping/Given: A section name that is so long that it cannot fit in a single console width/When: The test headers are printed as part of the normal running of the scenario/Then: The, deliberately very long and overly verbose (you see what I did there?) section names must wrap, along with an indent" duration="{duration}"/> + <testCase name="Scenario: Vector resizing affects size and capacity/Given: an empty vector" duration="{duration}"/> + <testCase name="Scenario: Vector resizing affects size and capacity/Given: an empty vector/When: it is made larger/Then: the size and capacity go up" duration="{duration}"/> + <testCase name="Scenario: Vector resizing affects size and capacity/Given: an empty vector/When: it is made larger/Then: the size and capacity go up/And when: it is made smaller again/Then: the size goes down but the capacity stays the same" duration="{duration}"/> + <testCase name="Scenario: Vector resizing affects size and capacity/Given: an empty vector/When: we reserve more space/Then: The capacity is increased but the size remains the same" duration="{duration}"/> + </file> + <file path="projects/<exe-name>/UsageTests/Class.tests.cpp"> + <testCase name="A METHOD_AS_TEST_CASE based test run that fails" duration="{duration}"> + <failure message="REQUIRE(s == "world")"> +FAILED: + REQUIRE( s == "world" ) +with expansion: + "hello" == "world" +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A METHOD_AS_TEST_CASE based test run that succeeds" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>::m_a.size() == 1)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 ) +with expansion: + 0 == 1 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>::m_a.size() == 1)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 ) +with expansion: + 0 == 1 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>::m_a.size() == 1)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 ) +with expansion: + 0 == 1 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>::m_a.size() == 1)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 ) +with expansion: + 0 == 1 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() < 2)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 ) +with expansion: + 6 < 2 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() < 2)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 ) +with expansion: + 2 < 2 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() < 2)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 ) +with expansion: + 6 < 2 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>" duration="{duration}"> + <failure message="REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() < 2)"> +FAILED: + REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 ) +with expansion: + 2 < 2 +Class.tests.cpp:<line number> + </failure> + </testCase> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>" duration="{duration}"/> + <testCase name="A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>" duration="{duration}"/> + <testCase name="A TEMP |