diff options
| author | Martin Hořeňovský | 2021-12-27 10:04:48 +0100 |
|---|---|---|
| committer | Martin Hořeňovský | 2021-12-28 00:22:53 +0100 |
| commit | f00b6e2019a9e5bdbe37dcc0be5756322363909d (patch) | |
| tree | 6d90c64777f096034150be1386a6a38939a8f3e6 | |
| parent | 45577a1f4c84eca059453908220cbdc7e1d34f93 (diff) | |
| download | catch3-f00b6e2019a9e5bdbe37dcc0be5756322363909d.tar.gz | |
Add more tests for various less common tag-related cases
| -rw-r--r-- | docs/test-cases-and-sections.md | 10 | ||||
| -rw-r--r-- | tests/CMakeLists.txt | 1 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/automake.sw.approved.txt | 5 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/compact.sw.approved.txt | 14 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/console.std.approved.txt | 4 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/console.sw.approved.txt | 120 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/junit.sw.approved.txt | 7 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/sonarqube.sw.approved.txt | 7 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/tap.sw.approved.txt | 30 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/teamcity.sw.approved.txt | 10 | ||||
| -rw-r--r-- | tests/SelfTest/Baselines/xml.sw.approved.txt | 149 | ||||
| -rw-r--r-- | tests/SelfTest/IntrospectiveTests/Tag.tests.cpp | 35 | ||||
| -rw-r--r-- | tests/SelfTest/IntrospectiveTests/TestSpecParser.tests.cpp | 55 |
13 files changed, 437 insertions, 10 deletions
diff --git a/docs/test-cases-and-sections.md b/docs/test-cases-and-sections.md index 06d2f47f..8cfb282b 100644 --- a/docs/test-cases-and-sections.md +++ b/docs/test-cases-and-sections.md @@ -44,7 +44,15 @@ The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run) -Tag names are not case sensitive and can contain any ASCII characters. This means that tags `[tag with spaces]` and `[I said "good day"]` are both allowed tags and can be filtered on. Escapes are not supported however and `[\]]` is not a valid tag. +Tag names are not case sensitive and can contain any ASCII characters. +This means that tags `[tag with spaces]` and `[I said "good day"]` +are both allowed tags and can be filtered on. However, escapes are not +supported however and `[\]]` is not a valid tag. + +The same tag can be specified multiple times for a single test case, +but only one of the instances of identical tags will be kept. Which one +is kept is functionally random. + ### Special Tags diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 6857f1bd..02e02747 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -87,6 +87,7 @@ set(TEST_SOURCES ${SELF_TEST_DIR}/IntrospectiveTests/RandomNumberGeneration.tests.cpp ${SELF_TEST_DIR}/IntrospectiveTests/Reporters.tests.cpp ${SELF_TEST_DIR}/IntrospectiveTests/Tag.tests.cpp + ${SELF_TEST_DIR}/IntrospectiveTests/TestSpecParser.tests.cpp ${SELF_TEST_DIR}/IntrospectiveTests/TextFlow.tests.cpp ${SELF_TEST_DIR}/IntrospectiveTests/Sharding.tests.cpp ${SELF_TEST_DIR}/IntrospectiveTests/String.tests.cpp diff --git a/tests/SelfTest/Baselines/automake.sw.approved.txt b/tests/SelfTest/Baselines/automake.sw.approved.txt index ea0e0ce2..05a96c7a 100644 --- a/tests/SelfTest/Baselines/automake.sw.approved.txt +++ b/tests/SelfTest/Baselines/automake.sw.approved.txt @@ -128,6 +128,7 @@ Nor would this :test-result: FAIL Custom std-exceptions can be custom translated :test-result: PASS Default scale is invisible to comparison :test-result: PASS Directly creating an EnumInfo +:test-result: PASS Empty tag is not allowed :test-result: FAIL EndsWith string matcher :test-result: PASS Enums can quickly have stringification enabled using REGISTER_ENUM :test-result: PASS Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM @@ -180,7 +181,9 @@ Nor would this :test-result: FAIL Output from all sections is reported :test-result: PASS Overloaded comma or address-of operators are not used :test-result: PASS Parse test names and tags +:test-result: PASS Parsed tags are matched case insensitive :test-result: PASS Parsing sharding-related cli flags +:test-result: PASS Parsing tags with non-alphabetical characters is pass-through :test-result: PASS Parsing warnings :test-result: PASS Pointers can be compared to null :test-result: PASS Precision of floating point stringification can be set @@ -224,6 +227,7 @@ Message from section two :test-result: PASS Stringifying std::chrono::time_point<system_clock> :test-result: FAIL Tabs and newlines show in output :test-result: PASS Tag alias can be registered against tag patterns +:test-result: PASS Tags with spaces and non-alphanumerical characters are accepted :test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 0 :test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 1 :test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 2 @@ -242,6 +246,7 @@ Message from section two :test-result: PASS TemplateTestSig: vectors can be sized and resized - float,4 :test-result: PASS TemplateTestSig: vectors can be sized and resized - int,5 :test-result: PASS TemplateTestSig: vectors can be sized and resized - std::string,15 +:test-result: PASS Test case with identical tags keeps just one :test-result: PASS Test case with one argument :test-result: PASS Test enum bit values :test-result: PASS Test with special, characters "in name diff --git a/tests/SelfTest/Baselines/compact.sw.approved.txt b/tests/SelfTest/Baselines/compact.sw.approved.txt index f58324f4..fab13bf2 100644 --- a/tests/SelfTest/Baselines/compact.sw.approved.txt +++ b/tests/SelfTest/Baselines/compact.sw.approved.txt @@ -491,6 +491,7 @@ ToString.tests.cpp:<line number>: passed: enumInfo->lookup(1) == "Value2" for: V ToString.tests.cpp:<line number>: passed: enumInfo->lookup(3) == "{** unexpected enum value **}" for: {** unexpected enum value **} == "{** unexpected enum value **}" +Tag.tests.cpp:<line number>: passed: Catch::TestCaseInfo( "", { "fake test name", "[]" }, dummySourceLineInfo ) Matchers.tests.cpp:<line number>: failed: testStringForMatching(), EndsWith( "Substring" ) for: "this string contains 'abc' as a substring" ends with: "Substring" Matchers.tests.cpp:<line number>: failed: testStringForMatching(), EndsWith( "this", Catch::CaseSensitive::No ) for: "this string contains 'abc' as a substring" ends with: "this" (case insensitive) EnumToString.tests.cpp:<line number>: passed: stringify( EnumClass3::Value1 ) == "Value1" for: "Value1" == "Value1" @@ -1203,6 +1204,9 @@ CmdLine.tests.cpp:<line number>: passed: !(spec.matches(*fakeTestCase("hidden an CmdLine.tests.cpp:<line number>: passed: !(spec.matches(*fakeTestCase("only foo", "[foo]"))) for: !false CmdLine.tests.cpp:<line number>: passed: !(spec.matches(*fakeTestCase("only hidden", "[.]"))) for: !false CmdLine.tests.cpp:<line number>: passed: spec.matches(*fakeTestCase("neither foo nor hidden", "[bar]")) for: true +TestSpecParser.tests.cpp:<line number>: passed: spec.hasFilters() for: true +TestSpecParser.tests.cpp:<line number>: passed: spec.getInvalidSpecs().empty() for: true +TestSpecParser.tests.cpp:<line number>: passed: spec.matches( testCase ) for: true CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "--shard-count=8" }) for: {?} CmdLine.tests.cpp:<line number>: passed: config.shardCount == 8 for: 8 == 8 CmdLine.tests.cpp:<line number>: passed: !(result) for: !{?} @@ -1215,6 +1219,12 @@ CmdLine.tests.cpp:<line number>: passed: !(result) for: !{?} CmdLine.tests.cpp:<line number>: passed: result.errorMessage(), ContainsSubstring("Shard index must be a non-negative number") for: "Shard index must be a non-negative number" contains: "Shard index must be a non-negative number" CmdLine.tests.cpp:<line number>: passed: cli.parse({ "test", "--shard-index=0" }) for: {?} CmdLine.tests.cpp:<line number>: passed: config.shardIndex == 0 for: 0 == 0 +TestSpecParser.tests.cpp:<line number>: passed: spec.hasFilters() for: true with 1 message: 'tagString := "[tag with spaces]"' +TestSpecParser.tests.cpp:<line number>: passed: spec.getInvalidSpecs().empty() for: true with 1 message: 'tagString := "[tag with spaces]"' +TestSpecParser.tests.cpp:<line number>: passed: spec.matches( testCase ) for: true with 1 message: 'tagString := "[tag with spaces]"' +TestSpecParser.tests.cpp:<line number>: passed: spec.hasFilters() for: true with 1 message: 'tagString := "[I said "good day" sir!]"' +TestSpecParser.tests.cpp:<line number>: passed: spec.getInvalidSpecs().empty() for: true with 1 message: 'tagString := "[I said "good day" sir!]"' +TestSpecParser.tests.cpp:<line number>: passed: spec.matches( testCase ) for: true with 1 message: 'tagString := "[I said "good day" sir!]"' CmdLine.tests.cpp:<line number>: passed: cli.parse( { "test", "-w", "NoAssertions" } ) for: {?} CmdLine.tests.cpp:<line number>: passed: config.warnings == WarnAbout::NoAssertions for: 1 == 1 CmdLine.tests.cpp:<line number>: passed: !(cli.parse( { "test", "-w", "NoTests" } )) for: !{?} @@ -1627,6 +1637,8 @@ Tag.tests.cpp:<line number>: passed: registry.add( "[no ampersat]", "", Catch::S Tag.tests.cpp:<line number>: passed: registry.add( "[the @ is not at the start]", "", Catch::SourceLineInfo( "file", 3 ) ) Tag.tests.cpp:<line number>: passed: registry.add( "@no square bracket at start]", "", Catch::SourceLineInfo( "file", 3 ) ) Tag.tests.cpp:<line number>: passed: registry.add( "[@no square bracket at end", "", Catch::SourceLineInfo( "file", 3 ) ) +Tag.tests.cpp:<line number>: passed: testCase.tags.size() == 2 for: 2 == 2 +Tag.tests.cpp:<line number>: passed: testCase.tags, VectorContains( Tag( "tag with spaces" ) ) && VectorContains( Tag( "I said \"good day\" sir!"_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} ) Class.tests.cpp:<line number>: passed: Template_Fixture<TestType>::m_a == 1 for: 1 == 1 Class.tests.cpp:<line number>: passed: Template_Fixture<TestType>::m_a == 1 for: 1 == 1 Class.tests.cpp:<line number>: passed: Template_Fixture<TestType>::m_a == 1 for: 1.0 == 1 @@ -1773,6 +1785,8 @@ Misc.tests.cpp:<line number>: passed: v.size() == V for: 15 == 15 Misc.tests.cpp:<line number>: passed: v.capacity() >= V for: 15 >= 15 Misc.tests.cpp:<line number>: passed: v.size() == V for: 15 == 15 Misc.tests.cpp:<line number>: passed: v.capacity() >= V for: 15 >= 15 +Tag.tests.cpp:<line number>: passed: testCase.tags.size() == 1 for: 1 == 1 +Tag.tests.cpp:<line number>: passed: testCase.tags[0] == Tag( "tag1" ) for: {?} == {?} VariadicMacros.tests.cpp:<line number>: passed: with 1 message: 'no assertions' Tricky.tests.cpp:<line number>: passed: 0x<hex digits> == bit30and31 for: 3221225472 (0x<hex digits>) == 3221225472 CmdLine.tests.cpp:<line number>: passed: diff --git a/tests/SelfTest/Baselines/console.std.approved.txt b/tests/SelfTest/Baselines/console.std.approved.txt index be4f8e78..92b8d9d9 100644 --- a/tests/SelfTest/Baselines/console.std.approved.txt +++ b/tests/SelfTest/Baselines/console.std.approved.txt @@ -1426,6 +1426,6 @@ due to unexpected exception with message: Why would you throw a std::string? =============================================================================== -test cases: 379 | 302 passed | 70 failed | 7 failed as expected -assertions: 2169 | 2013 passed | 129 failed | 27 failed as expected +test cases: 384 | 307 passed | 70 failed | 7 failed as expected +assertions: 2183 | 2027 passed | 129 failed | 27 failed as expected diff --git a/tests/SelfTest/Baselines/console.sw.approved.txt b/tests/SelfTest/Baselines/console.sw.approved.txt index 25a85bce..ac9f7eee 100644 --- a/tests/SelfTest/Baselines/console.sw.approved.txt +++ b/tests/SelfTest/Baselines/console.sw.approved.txt @@ -3894,6 +3894,15 @@ with expansion: "{** unexpected enum value **}" ------------------------------------------------------------------------------- +Empty tag is not allowed +------------------------------------------------------------------------------- +Tag.tests.cpp:<line number> +............................................................................... + +Tag.tests.cpp:<line number>: PASSED: + REQUIRE_THROWS( Catch::TestCaseInfo( "", { "fake test name", "[]" }, dummySourceLineInfo ) ) + +------------------------------------------------------------------------------- EndsWith string matcher ------------------------------------------------------------------------------- Matchers.tests.cpp:<line number> @@ -8783,6 +8792,27 @@ with expansion: true ------------------------------------------------------------------------------- +Parsed tags are matched case insensitive +------------------------------------------------------------------------------- +TestSpecParser.tests.cpp:<line number> +............................................................................... + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.hasFilters() ) +with expansion: + true + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.getInvalidSpecs().empty() ) +with expansion: + true + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.matches( testCase ) ) +with expansion: + true + +------------------------------------------------------------------------------- Parsing sharding-related cli flags shard-count ------------------------------------------------------------------------------- @@ -8888,6 +8918,60 @@ with expansion: 0 == 0 ------------------------------------------------------------------------------- +Parsing tags with non-alphabetical characters is pass-through +------------------------------------------------------------------------------- +TestSpecParser.tests.cpp:<line number> +............................................................................... + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.hasFilters() ) +with expansion: + true +with message: + tagString := "[tag with spaces]" + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.getInvalidSpecs().empty() ) +with expansion: + true +with message: + tagString := "[tag with spaces]" + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.matches( testCase ) ) +with expansion: + true +with message: + tagString := "[tag with spaces]" + +------------------------------------------------------------------------------- +Parsing tags with non-alphabetical characters is pass-through +------------------------------------------------------------------------------- +TestSpecParser.tests.cpp:<line number> +............................................................................... + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.hasFilters() ) +with expansion: + true +with message: + tagString := "[I said "good day" sir!]" + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.getInvalidSpecs().empty() ) +with expansion: + true +with message: + tagString := "[I said "good day" sir!]" + +TestSpecParser.tests.cpp:<line number>: PASSED: + REQUIRE( spec.matches( testCase ) ) +with expansion: + true +with message: + tagString := "[I said "good day" sir!]" + +------------------------------------------------------------------------------- Parsing warnings NoAssertions ------------------------------------------------------------------------------- @@ -11498,6 +11582,22 @@ Tag.tests.cpp:<line number>: PASSED: CHECK_THROWS( registry.add( "[@no square bracket at end", "", Catch::SourceLineInfo( "file", 3 ) ) ) ------------------------------------------------------------------------------- +Tags with spaces and non-alphanumerical characters are accepted +------------------------------------------------------------------------------- +Tag.tests.cpp:<line number> +............................................................................... + +Tag.tests.cpp:<line number>: PASSED: + REQUIRE( testCase.tags.size() == 2 ) +with expansion: + 2 == 2 + +Tag.tests.cpp:<line number>: PASSED: + REQUIRE_THAT( testCase.tags, VectorContains( Tag( "tag with spaces" ) ) && VectorContains( Tag( "I said \"good day\" sir!"_catch_sr ) ) ) +with expansion: + { {?}, {?} } ( Contains: {?} and Contains: {?} ) + +------------------------------------------------------------------------------- Template test case method with test types specified inside std::tuple - MyTypes - 0 ------------------------------------------------------------------------------- @@ -12775,6 +12875,22 @@ with expansion: 15 >= 15 ------------------------------------------------------------------------------- +Test case with identical tags keeps just one +------------------------------------------------------------------------------- +Tag.tests.cpp:<line number> +............................................................................... + +Tag.tests.cpp:<line number>: PASSED: + REQUIRE( testCase.tags.size() == 1 ) +with expansion: + 1 == 1 + +Tag.tests.cpp:<line number>: PASSED: + REQUIRE( testCase.tags[0] == Tag( "tag1" ) ) +with expansion: + {?} == {?} + +------------------------------------------------------------------------------- Test case with one argument ------------------------------------------------------------------------------- VariadicMacros.tests.cpp:<line number> @@ -17424,6 +17540,6 @@ Misc.tests.cpp:<line number> Misc.tests.cpp:<line number>: PASSED: =============================================================================== -test cases: 379 | 286 passed | 86 failed | 7 failed as expected -assertions: 2186 | 2013 passed | 146 failed | 27 failed as expected +test cases: 384 | 291 passed | 86 failed | 7 failed as expected +assertions: 2200 | 2027 passed | 146 failed | 27 failed as expected diff --git a/tests/SelfTest/Baselines/junit.sw.approved.txt b/tests/SelfTest/Baselines/junit.sw.approved.txt index 3181ebbd..a82176bd 100644 --- a/tests/SelfTest/Baselines/junit.sw.approved.txt +++ b/tests/SelfTest/Baselines/junit.sw.approved.txt @@ -1,7 +1,7 @@ <?xml version="1.0" encoding="UTF-8"?> <testsuitesloose text artifact > - <testsuite name="<exe-name>" errors="17" failures="129" tests="2186" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}"> + <testsuite name="<exe-name>" errors="17" failures="129" tests="2200" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}"> <properties> <property name="random-seed" value="1"/> <property name="filters" value="~[!nonportable]~[!benchmark]~[approvals] *"/> @@ -451,6 +451,7 @@ Exception.tests.cpp:<line number> </testcase> <testcase classname="<exe-name>.global" name="Default scale is invisible to comparison" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Directly creating an EnumInfo" time="{duration}" status="run"/> + <testcase classname="<exe-name>.global" name="Empty tag is not allowed" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="EndsWith string matcher" time="{duration}" status="run"> <failure message="testStringForMatching(), EndsWith( "Substring" )" type="CHECK_THAT"> FAILED: @@ -1068,12 +1069,14 @@ Message.tests.cpp:<line number> <testcase classname="<exe-name>.global" name="Parse test names and tags/Leading and trailing spaces in test name" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parse test names and tags/Shortened hide tags are split apart when parsing" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parse test names and tags/Shortened hide tags also properly handle exclusion" time="{duration}" status="run"/> + <testcase classname="<exe-name>.global" name="Parsed tags are matched case insensitive" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/shard-count" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/Negative shard count reports error" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/Zero shard count reports error" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/shard-index" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/Negative shard index reports error" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing sharding-related cli flags/Shard index 0 is accepted" time="{duration}" status="run"/> + <testcase classname="<exe-name>.global" name="Parsing tags with non-alphabetical characters is pass-through" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing warnings/NoAssertions" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing warnings/NoTests is no longer supported" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Parsing warnings/Combining multiple warnings" time="{duration}" status="run"/> @@ -1273,6 +1276,7 @@ Misc.tests.cpp:<line number> </testcase> <testcase classname="<exe-name>.global" name="Tag alias can be registered against tag patterns/The same tag alias can only be registered once" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Tag alias can be registered against tag patterns/Tag aliases must be of the form [@name]" time="{duration}" status="run"/> + <testcase classname="<exe-name>.global" name="Tags with spaces and non-alphanumerical characters are accepted" time="{duration}" status="run"/> <testcase classname="<exe-name>.Template_Fixture" name="Template test case method with test types specified inside std::tuple - MyTypes - 0" time="{duration}" status="run"/> <testcase classname="<exe-name>.Template_Fixture" name="Template test case method with test types specified inside std::tuple - MyTypes - 1" time="{duration}" status="run"/> <testcase classname="<exe-name>.Template_Fixture" name="Template test case method with test types specified inside std::tuple - MyTypes - 2" time="{duration}" status="run"/> @@ -1331,6 +1335,7 @@ Misc.tests.cpp:<line number> <testcase classname="<exe-name>.global" name="TemplateTestSig: vectors can be sized and resized - std::string,15/resizing smaller changes size but not capacity/We can use the 'swap trick' to reset the capacity" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="TemplateTestSig: vectors can be sized and resized - std::string,15/reserving bigger changes capacity but not size" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="TemplateTestSig: vectors can be sized and resized - std::string,15/reserving smaller does not change size or capacity" time="{duration}" status="run"/> + <testcase classname="<exe-name>.global" name="Test case with identical tags keeps just one" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Test case with one argument" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Test enum bit values" time="{duration}" status="run"/> <testcase classname="<exe-name>.global" name="Test with special, characters "in name" time="{duration}" status="run"/> diff --git a/tests/SelfTest/Baselines/sonarqube.sw.approved.txt b/tests/SelfTest/Baselines/sonarqube.sw.approved.txt index 0762507d..72c2e692 100644 --- a/tests/SelfTest/Baselines/sonarqube.sw.approved.txt +++ b/tests/SelfTest/Baselines/sonarqube.sw.approved.txt @@ -241,12 +241,19 @@ <testCase name="startsWith" duration="{duration}"/> </file> <file path="tests/<exe-name>/IntrospectiveTests/Tag.tests.cpp"> + <testCase name="Empty tag is not allowed" duration="{duration}"/> <testCase name="Tag alias can be registered against tag patterns/The same tag alias can only be registered once" duration="{duration}"/> <testCase name="Tag alias can be registered against tag patterns/Tag aliases must be of the form [@name]" duration="{duration}"/> + <testCase name="Tags with spaces and non-alphanumerical characters are accepted" duration="{duration}"/> + <testCase name="Test case with identical tags keeps just one" duration="{duration}"/> <testCase name="empty tags are not allowed" duration="{duration}"/> <testCase name="shortened hide tags are split apart" duration="{duration}"/> <testCase name="tags with dots in later positions are not parsed as hidden" duration="{duration}"/> </file> + <file path="tests/<exe-name>/IntrospectiveTests/TestSpecParser.tests.cpp"> + <testCase name="Parsed tags are matched case insensitive" duration="{duration}"/> + <testCase name="Parsing tags with non-alphabetical characters is pass-through" duration="{duration}"/> + </file> <file path="tests/<exe-name>/IntrospectiveTests/ToString.tests.cpp"> <testCase name="Directly creating an EnumInfo" duration="{duration}"/> <testCase name="Range type with sentinel" duration="{duration}"/> diff --git a/tests/SelfTest/Baselines/tap.sw.approved.txt b/tests/SelfTest/Baselines/tap.sw.approved.txt index 746f2599..e3d75dd8 100644 --- a/tests/SelfTest/Baselines/tap.sw.approved.txt +++ b/tests/SelfTest/Baselines/tap.sw.approved.txt @@ -976,6 +976,8 @@ ok {test-number} - enumInfo->lookup(0) == "Value1" for: Value1 == "Value1" ok {test-number} - enumInfo->lookup(1) == "Value2" for: Value2 == "Value2" # Directly creating an EnumInfo ok {test-number} - enumInfo->lookup(3) == "{** unexpected enum value **}" for: {** unexpected enum value **} == "{** unexpected enum value **}" +# Empty tag is not allowed +ok {test-number} - Catch::TestCaseInfo( "", { "fake test name", "[]" }, dummySourceLineInfo ) # EndsWith string matcher not ok {test-number} - testStringForMatching(), EndsWith( "Substring" ) for: "this string contains 'abc' as a substring" ends with: "Substring" # EndsWith string matcher @@ -2328,6 +2330,12 @@ ok {test-number} - !(spec.matches(*fakeTestCase("only foo", "[foo]"))) for: !fal ok {test-number} - !(spec.matches(*fakeTestCase("only hidden", "[.]"))) for: !false # Parse test names and tags ok {test-number} - spec.matches(*fakeTestCase("neither foo nor hidden", "[bar]")) for: true +# Parsed tags are matched case insensitive +ok {test-number} - spec.hasFilters() for: true +# Parsed tags are matched case insensitive +ok {test-number} - spec.getInvalidSpecs().empty() for: true +# Parsed tags are matched case insensitive +ok {test-number} - spec.matches( testCase ) for: true # Parsing sharding-related cli flags ok {test-number} - cli.parse({ "test", "--shard-count=8" }) for: {?} # Parsing sharding-related cli flags @@ -2352,6 +2360,18 @@ ok {test-number} - result.errorMessage(), ContainsSubstring("Shard index must be ok {test-number} - cli.parse({ "test", "--shard-index=0" }) for: {?} # Parsing sharding-related cli flags ok {test-number} - config.shardIndex == 0 for: 0 == 0 +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.hasFilters() for: true with 1 message: 'tagString := "[tag with spaces]"' +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.getInvalidSpecs().empty() for: true with 1 message: 'tagString := "[tag with spaces]"' +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.matches( testCase ) for: true with 1 message: 'tagString := "[tag with spaces]"' +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.hasFilters() for: true with 1 message: 'tagString := "[I said "good day" sir!]"' +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.getInvalidSpecs().empty() for: true with 1 message: 'tagString := "[I said "good day" sir!]"' +# Parsing tags with non-alphabetical characters is pass-through +ok {test-number} - spec.matches( testCase ) for: true with 1 message: 'tagString := "[I said "good day" sir!]"' # Parsing warnings ok {test-number} - cli.parse( { "test", "-w", "NoAssertions" } ) for: {?} # Parsing warnings @@ -2899,6 +2919,10 @@ ok {test-number} - registry.add( "[the @ is not at the start]", "", Catch::Sourc ok {test-number} - registry.add( "@no square bracket at start]", "", Catch::SourceLineInfo( "file", 3 ) ) # Tag alias can be registered against tag patterns ok {test-number} - registry.add( "[@no square bracket at end", "", Catch::SourceLineInfo( "file", 3 ) ) +# Tags with spaces and non-alphanumerical characters are accepted +ok {test-number} - testCase.tags.size() == 2 for: 2 == 2 +# Tags with spaces and non-alphanumerical characters are accepted +ok {test-number} - testCase.tags, VectorContains( Tag( "tag with spaces" ) ) && VectorContains( Tag( "I said \"good day\" sir!"_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} ) # Template test case method with test types specified inside std::tuple - MyTypes - 0 ok {test-number} - Template_Fixture<TestType>::m_a == 1 for: 1 == 1 # Template test case method with test types specified inside std::tuple - MyTypes - 1 @@ -3191,6 +3215,10 @@ ok {test-number} - v.capacity() >= V for: 15 >= 15 ok {test-number} - v.size() == V for: 15 == 15 # TemplateTestSig: vectors can be sized and resized - std::string,15 ok {test-number} - v.capacity() >= V for: 15 >= 15 +# Test case with identical tags keeps just one +ok {test-number} - testCase.tags.size() == 1 for: 1 == 1 +# Test case with identical tags keeps just one +ok {test-number} - testCase.tags[0] == Tag( "tag1" ) for: {?} == {?} # Test case with one argument ok {test-number} - with 1 message: 'no assertions' # Test enum bit values @@ -4374,5 +4402,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0 ok {test-number} - # xmlentitycheck ok {test-number} - -1..2186 +1..2200 diff --git a/tests/SelfTest/Baselines/teamcity.sw.approved.txt b/tests/SelfTest/Baselines/teamcity.sw.approved.txt index cadec0c4..4f2193fd 100644 --- a/tests/SelfTest/Baselines/teamcity.sw.approved.txt +++ b/tests/SelfTest/Baselines/teamcity.sw.approved.txt @@ -294,6 +294,8 @@ Exception.tests.cpp:<line number>|nunexpected exception with message:|n "custom ##teamcity[testFinished name='Default scale is invisible to comparison' duration="{duration}"] ##teamcity[testStarted name='Directly creating an EnumInfo'] ##teamcity[testFinished name='Directly creating an EnumInfo' duration="{duration}"] +##teamcity[testStarted name='Empty tag is not allowed'] +##teamcity[testFinished name='Empty tag is not allowed' duration="{duration}"] ##teamcity[testStarted name='EndsWith string matcher'] Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "Substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "Substring"|n'] Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "this", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "this" (case insensitive)|n'] @@ -465,8 +467,12 @@ Message.tests.cpp:<line number>|nexplicit failure with message:|n "Message from ##teamcity[testFinished name='Overloaded comma or address-of operators are not used' duration="{duration}"] ##teamcity[testStarted name='Parse test names and tags'] ##teamcity[testFinished name='Parse test names and tags' duration="{duration}"] +##teamcity[testStarted name='Parsed tags are matched case insensitive'] +##teamcity[testFinished name='Parsed tags are matched case insensitive' duration="{duration}"] ##teamcity[testStarted name='Parsing sharding-related cli flags'] ##teamcity[testFinished name='Parsing sharding-related cli flags' duration="{duration}"] +##teamcity[testStarted name='Parsing tags with non-alphabetical characters is pass-through'] +##teamcity[testFinished name='Parsing tags with non-alphabetical characters is pass-through' duration="{duration}"] ##teamcity[testStarted name='Parsing warnings'] ##teamcity[testFinished name='Parsing warnings' duration="{duration}"] ##teamcity[testStarted name='Pointers can be compared to null'] @@ -553,6 +559,8 @@ Misc.tests.cpp:<line number>|nexpression failed|n CHECK( s1 == s2 )|nwith expan ##teamcity[testFinished name='Tabs and newlines show in output' duration="{duration}"] ##teamcity[testStarted name='Tag alias can be registered against tag patterns'] ##teamcity[testFinished name='Tag alias can be registered against tag patterns' duration="{duration}"] +##teamcity[testStarted name='Tags with spaces and non-alphanumerical characters are accepted'] +##teamcity[testFinished name='Tags with spaces and non-alphanumerical characters are accepted' duration="{duration}"] ##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 0'] ##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"] ##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 1'] @@ -589,6 +597,8 @@ Misc.tests.cpp:<line number>|nexpression failed|n CHECK( s1 == s2 )|nwith expan ##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - int,5' duration="{duration}"] ##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - std::string,15'] ##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - std::string,15' duration="{duration}"] +##teamcity[testStarted name='Test case with identical tags keeps just one'] +##teamcity[testFinished name='Test case with identical tags keeps just one' duration="{duration}"] ##teamcity[testStarted name='Test case with one argument'] ##teamcity[testFinished name='Test case with one argument' duration="{duration}"] ##teamcity[testStarted name='Test enum bit values'] diff --git a/tests/SelfTest/Baselines/xml.sw.approved.txt b/tests/SelfTest/Baselines/xml.sw.approved.txt index b9bb761e..2551e03b 100644 --- a/tests/SelfTest/Baselines/xml.sw.approved.txt +++ b/tests/SelfTest/Baselines/xml.sw.approved.txt @@ -4306,6 +4306,17 @@ Nor would this </Expression> <OverallResult success="true"/> </TestCase> + <TestCase name="Empty tag is not allowed" filename="tests/<exe-name>/IntrospectiveTests/Tag.tests.cpp" > + <Expression success="true" type="REQUIRE_THROWS" filename="tests/<exe-name>/IntrospectiveTests/Tag.tests.cpp" > + <Original> + Catch::TestCaseInfo( "", { "fake test name", "[]" }, dummySourceLineInfo ) + </Original> + <Expanded> + Catch::TestCaseInfo( "", { "fake test name", "[]" }, dummySourceLineInfo ) + </Expanded> + </Expression> + <OverallResult success="true"/> + </TestCase> <TestCase name="EndsWith string matcher" tags="[.][failing][matchers]" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" > <Expression success="false" type="CHECK_THAT" filename="tests/<exe-name>/UsageTests/Matchers.tests.cpp" > <Original> @@ -10700,6 +10711,33 @@ Nor would this </Section> <OverallResult success="true"/> </TestCase> + <TestCase name="Parsed tags are matched case insensitive" tags="[test-spec][test-spec-parser]" filename="tests/<exe-name>/IntrospectiveTests/TestSpecParser.tests.cpp" > + <Expressi |