Add new SKIP macro for skipping tests at runtime (#2360)

* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>

22 files changed, 3485 insertions, 2118 deletions

diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index b40fbceb..023f478b 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -116,6 +116,7 @@ set(TEST_SOURCES
         ${SELF_TEST_DIR}/UsageTests/Generators.tests.cpp
         ${SELF_TEST_DIR}/UsageTests/Message.tests.cpp
         ${SELF_TEST_DIR}/UsageTests/Misc.tests.cpp
+        ${SELF_TEST_DIR}/UsageTests/Skip.tests.cpp
         ${SELF_TEST_DIR}/UsageTests/ToStringByte.tests.cpp
         ${SELF_TEST_DIR}/UsageTests/ToStringChrono.tests.cpp
         ${SELF_TEST_DIR}/UsageTests/ToStringGeneral.tests.cpp
@@ -272,6 +273,10 @@ add_test(NAME TestSpecs::OverrideFailureWithNoMatchedTests
   COMMAND $<TARGET_FILE:SelfTest> "___nonexistent_test___" --allow-running-no-tests
 )
 
+add_test(NAME TestSpecs::OverrideAllSkipFailure
+  COMMAND $<TARGET_FILE:SelfTest> "tests can be skipped dynamically at runtime" --allow-running-no-tests
+)
+
 add_test(NAME TestSpecs::NonMatchingTestSpecIsRoundTrippable
     COMMAND $<TARGET_FILE:SelfTest> Tracker, "this test does not exist" "[nor does this tag]"
 )
diff --git a/tests/ExtraTests/CMakeLists.txt b/tests/ExtraTests/CMakeLists.txt
index d5b27fba..4172d7a0 100644
--- a/tests/ExtraTests/CMakeLists.txt
+++ b/tests/ExtraTests/CMakeLists.txt
@@ -488,15 +488,32 @@ set_tests_properties(TestSpecs::EmptySpecWithNoTestsFails
   PROPERTIES
     WILL_FAIL ON
 )
+
 add_test(
   NAME TestSpecs::OverrideFailureWithEmptySpec
   COMMAND $<TARGET_FILE:NoTests> --allow-running-no-tests
 )
+
 add_test(
   NAME List::Listeners::WorksWithoutRegisteredListeners
   COMMAND $<TARGET_FILE:NoTests> --list-listeners
 )
+
+
+add_executable(AllSkipped ${TESTS_DIR}/X93-AllSkipped.cpp)
+target_link_libraries(AllSkipped PRIVATE Catch2::Catch2WithMain)
+
+add_test(
+  NAME TestSpecs::SkippingAllTestsFails
+  COMMAND $<TARGET_FILE:AllSkipped>
+)
+set_tests_properties(TestSpecs::SkippingAllTestsFails
+  PROPERTIES
+    WILL_FAIL ON
+)
+
 set( EXTRA_TEST_BINARIES
+    AllSkipped
     PrefixedMacros
     DisabledMacros
     DisabledExceptions-DefaultHandler
diff --git a/tests/ExtraTests/X93-AllSkipped.cpp b/tests/ExtraTests/X93-AllSkipped.cpp
new file mode 100644
index 00000000..8e7d0afe
--- /dev/null
+++ b/tests/ExtraTests/X93-AllSkipped.cpp
@@ -0,0 +1,16 @@
+
+//              Copyright Catch2 Authors
+// Distributed under the Boost Software License, Version 1.0.
+//   (See accompanying file LICENSE.txt or copy at
+//        https://www.boost.org/LICENSE_1_0.txt)
+
+// SPDX-License-Identifier: BSL-1.0
+
+#include <catch2/catch_test_macros.hpp>
+
+TEST_CASE( "this test case is being skipped" ) { SKIP(); }
+
+TEST_CASE( "all sections in this test case are being skipped" ) {
+    SECTION( "A" ) { SKIP(); }
+    SECTION( "B" ) { SKIP(); }
+}
diff --git a/tests/SelfTest/Baselines/automake.sw.approved.txt b/tests/SelfTest/Baselines/automake.sw.approved.txt
index 332439a7..64e92396 100644
--- a/tests/SelfTest/Baselines/automake.sw.approved.txt
+++ b/tests/SelfTest/Baselines/automake.sw.approved.txt
@@ -297,6 +297,7 @@ Message from section two
 :test-result: PASS X/level/1/b
 :test-result: PASS XmlEncode
 :test-result: PASS XmlWriter writes boolean attributes as true/false
+:test-result: SKIP a succeeding test can still be skipped
 :test-result: PASS analyse no analysis
 :test-result: PASS array<int, N> -> toString
 :test-result: PASS benchmark function call
@@ -309,10 +310,14 @@ Message from section two
 :test-result: PASS comparisons between const int variables
 :test-result: PASS comparisons between int variables
 :test-result: PASS convertToBits
+:test-result: SKIP dynamic skipping works with generators
 :test-result: PASS empty tags are not allowed
 :test-result: PASS erfc_inv
 :test-result: PASS estimate_clock_resolution
 :test-result: PASS even more nested SECTION tests
+:test-result: XFAIL failed assertions before SKIP cause test case to fail
+:test-result: XFAIL failing for some generator values causes entire test case to fail
+:test-result: XFAIL failing in some unskipped sections causes entire test case to fail
 :test-result: FAIL first tag
 loose text artifact
 :test-result: FAIL has printf
@@ -331,6 +336,10 @@ loose text artifact
 :test-result: FAIL mix info, unscoped info and warning
 :test-result: FAIL more nested SECTION tests
 :test-result: PASS nested SECTION tests
+a!
+b1!
+!
+:test-result: FAIL nested sections can be skipped dynamically at runtime
 :test-result: PASS non streamable - with conv. op
 :test-result: PASS non-copyable objects
 :test-result: PASS normal_cdf
@@ -352,9 +361,11 @@ loose text artifact
 :test-result: PASS run_for_at_least, chronometer
 :test-result: PASS run_for_at_least, int
 :test-result: FAIL second tag
+:test-result: SKIP sections can be skipped dynamically at runtime
 :test-result: FAIL send a single char to INFO
 :test-result: FAIL sends information to INFO
 :test-result: PASS shortened hide tags are split apart
+:test-result: SKIP skipped tests can optionally provide a reason
 :test-result: PASS splitString
 :test-result: FAIL stacks unscoped info in loops
 :test-result: PASS startsWith
@@ -376,6 +387,7 @@ loose text artifact
 :test-result: PASS strlen3
 :test-result: PASS tables
 :test-result: PASS tags with dots in later positions are not parsed as hidden
+:test-result: SKIP tests can be skipped dynamically at runtime
 :test-result: FAIL thrown std::strings are translated
 :test-result: PASS toString on const wchar_t const pointer returns the string contents
 :test-result: PASS toString on const wchar_t pointer returns the string contents
diff --git a/tests/SelfTest/Baselines/automake.sw.multi.approved.txt b/tests/SelfTest/Baselines/automake.sw.multi.approved.txt
index a7461ae5..d6f5ebe9 100644
--- a/tests/SelfTest/Baselines/automake.sw.multi.approved.txt
+++ b/tests/SelfTest/Baselines/automake.sw.multi.approved.txt
@@ -290,6 +290,7 @@
 :test-result: PASS X/level/1/b
 :test-result: PASS XmlEncode
 :test-result: PASS XmlWriter writes boolean attributes as true/false
+:test-result: SKIP a succeeding test can still be skipped
 :test-result: PASS analyse no analysis
 :test-result: PASS array<int, N> -> toString
 :test-result: PASS benchmark function call
@@ -302,10 +303,14 @@
 :test-result: PASS comparisons between const int variables
 :test-result: PASS comparisons between int variables
 :test-result: PASS convertToBits
+:test-result: SKIP dynamic skipping works with generators
 :test-result: PASS empty tags are not allowed
 :test-result: PASS erfc_inv
 :test-result: PASS estimate_clock_resolution
 :test-result: PASS even more nested SECTION tests
+:test-result: XFAIL failed assertions before SKIP cause test case to fail
+:test-result: XFAIL failing for some generator values causes entire test case to fail
+:test-result: XFAIL failing in some unskipped sections causes entire test case to fail
 :test-result: FAIL first tag
 :test-result: FAIL has printf
 :test-result: PASS is_unary_function
@@ -323,6 +328,7 @@
 :test-result: FAIL mix info, unscoped info and warning
 :test-result: FAIL more nested SECTION tests
 :test-result: PASS nested SECTION tests
+:test-result: FAIL nested sections can be skipped dynamically at runtime
 :test-result: PASS non streamable - with conv. op
 :test-result: PASS non-copyable objects
 :test-result: PASS normal_cdf
@@ -344,9 +350,11 @@
 :test-result: PASS run_for_at_least, chronometer
 :test-result: PASS run_for_at_least, int
 :test-result: FAIL second tag
+:test-result: SKIP sections can be skipped dynamically at runtime
 :test-result: FAIL send a single char to INFO
 :test-result: FAIL sends information to INFO
 :test-result: PASS shortened hide tags are split apart
+:test-result: SKIP skipped tests can optionally provide a reason
 :test-result: PASS splitString
 :test-result: FAIL stacks unscoped info in loops
 :test-result: PASS startsWith
@@ -368,6 +376,7 @@
 :test-result: PASS strlen3
 :test-result: PASS tables
 :test-result: PASS tags with dots in later positions are not parsed as hidden
+:test-result: SKIP tests can be skipped dynamically at runtime
 :test-result: FAIL thrown std::strings are translated
 :test-result: PASS toString on const wchar_t const pointer returns the string contents
 :test-result: PASS toString on const wchar_t pointer returns the string contents
diff --git a/tests/SelfTest/Baselines/compact.sw.approved.txt b/tests/SelfTest/Baselines/compact.sw.approved.txt
index ff6b2875..1d8ea9a8 100644
--- a/tests/SelfTest/Baselines/compact.sw.approved.txt
+++ b/tests/SelfTest/Baselines/compact.sw.approved.txt
@@ -2060,6 +2060,8 @@ Xml.tests.cpp:<line number>: passed: encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F
 Xml.tests.cpp:<line number>: passed: stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?>
 <Element1 attr1="true" attr2="false"/>
 " ( contains: "attr1="true"" and contains: "attr2="false"" )
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped:
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.point.count() == 23 for: 23.0 == 23
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.lower_bound.count() == 23 for: 23.0 == 23
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.upper_bound.count() == 23 for: 23.0 == 23
@@ -2149,6 +2151,9 @@ FloatingPoint.tests.cpp:<line number>: passed: convertToBits( -0. ) == ( 1ULL <<
 9223372036854775808 (0x<hex digits>)
 FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
 FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 41'
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
 Tag.tests.cpp:<line number>: passed: Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
 InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.103560) == Approx(-0.09203687623843015) for: -0.0920368762 == Approx( -0.0920368762 )
 InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.067400) == Approx(-0.05980291115763361) for: -0.0598029112 == Approx( -0.0598029112 )
@@ -2158,6 +2163,14 @@ InternalBenchmark.tests.cpp:<line number>: passed: res.outliers.total() == 0 for
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: failed: 3 == 4
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
 loose text artifact
 Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
 Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary2)>::value'
@@ -2215,6 +2228,10 @@ Misc.tests.cpp:<line number>: passed: a < b for: 1 < 2
 Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
 Misc.tests.cpp:<line number>: passed: b != a for: 2 != 1
 Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
+a!
+b1!
+Skip.tests.cpp:<line number>: skipped:
+!
 Tricky.tests.cpp:<line number>: passed: s == "7" for: "7" == "7"
 Tricky.tests.cpp:<line number>: passed: ti == typeid(int) for: {?} == {?}
 InternalBenchmark.tests.cpp:<line number>: passed: normal_cdf(0.000000) == Approx(0.50000000000000000) for: 0.5 == Approx( 0.5 )
@@ -2299,9 +2316,13 @@ InternalBenchmark.tests.cpp:<line number>: passed: x >= old_x for: 128 >= 64
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.elapsed >= time for: 128 ns >= 100 ns
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.result == Timing.iterations + 17 for: 145 == 145
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.iterations >= time.count() for: 128 >= 100
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: failed: false with 1 message: '3'
 Message.tests.cpp:<line number>: failed: false with 2 messages: 'hi' and 'i := 7'
 Tag.tests.cpp:<line number>: passed: testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
 StringManip.tests.cpp:<line number>: passed: splitStringRef("", ','), Equals(std::vector<StringRef>()) for: {  } Equals: {  }
 StringManip.tests.cpp:<line number>: passed: splitStringRef("abc", ','), Equals(std::vector<StringRef>{"abc"}) for: { abc } Equals: { abc }
 StringManip.tests.cpp:<line number>: passed: splitStringRef("abc,def", ','), Equals(std::vector<StringRef>{"abc", "def"}) for: { abc, def } Equals: { abc, def }
@@ -2367,6 +2388,7 @@ Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_
 Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(data)) for: 6 == 6
 Tag.tests.cpp:<line number>: passed: testcase.tags.size() == 1 for: 1 == 1
 Tag.tests.cpp:<line number>: passed: testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
+Skip.tests.cpp:<line number>: skipped:
 Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'Why would you throw a std::string?'
 Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
 Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
@@ -2463,7 +2485,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
 InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
-test cases:  395 |  305 passed |  83 failed |  7 failed as expected
-assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
+test cases:  404 |  305 passed |  84 failed | 5 skipped | 10 failed as expected
+assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
 
 
diff --git a/tests/SelfTest/Baselines/compact.sw.multi.approved.txt b/tests/SelfTest/Baselines/compact.sw.multi.approved.txt
index d11bdbae..1b450553 100644
--- a/tests/SelfTest/Baselines/compact.sw.multi.approved.txt
+++ b/tests/SelfTest/Baselines/compact.sw.multi.approved.txt
@@ -2053,6 +2053,8 @@ Xml.tests.cpp:<line number>: passed: encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F
 Xml.tests.cpp:<line number>: passed: stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?>
 <Element1 attr1="true" attr2="false"/>
 " ( contains: "attr1="true"" and contains: "attr2="false"" )
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped:
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.point.count() == 23 for: 23.0 == 23
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.lower_bound.count() == 23 for: 23.0 == 23
 InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.upper_bound.count() == 23 for: 23.0 == 23
@@ -2142,6 +2144,9 @@ FloatingPoint.tests.cpp:<line number>: passed: convertToBits( -0. ) == ( 1ULL <<
 9223372036854775808 (0x<hex digits>)
 FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
 FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 41'
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
 Tag.tests.cpp:<line number>: passed: Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
 InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.103560) == Approx(-0.09203687623843015) for: -0.0920368762 == Approx( -0.0920368762 )
 InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.067400) == Approx(-0.05980291115763361) for: -0.0598029112 == Approx( -0.0598029112 )
@@ -2151,6 +2156,14 @@ InternalBenchmark.tests.cpp:<line number>: passed: res.outliers.total() == 0 for
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: failed: 3 == 4
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: failed: explicitly
 Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
 Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary2)>::value'
 Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary3)>::value'
@@ -2207,6 +2220,7 @@ Misc.tests.cpp:<line number>: passed: a < b for: 1 < 2
 Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
 Misc.tests.cpp:<line number>: passed: b != a for: 2 != 1
 Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
+Skip.tests.cpp:<line number>: skipped:
 Tricky.tests.cpp:<line number>: passed: s == "7" for: "7" == "7"
 Tricky.tests.cpp:<line number>: passed: ti == typeid(int) for: {?} == {?}
 InternalBenchmark.tests.cpp:<line number>: passed: normal_cdf(0.000000) == Approx(0.50000000000000000) for: 0.5 == Approx( 0.5 )
@@ -2291,9 +2305,13 @@ InternalBenchmark.tests.cpp:<line number>: passed: x >= old_x for: 128 >= 64
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.elapsed >= time for: 128 ns >= 100 ns
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.result == Timing.iterations + 17 for: 145 == 145
 InternalBenchmark.tests.cpp:<line number>: passed: Timing.iterations >= time.count() for: 128 >= 100
+Skip.tests.cpp:<line number>: passed:
+Skip.tests.cpp:<line number>: skipped:
+Skip.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: failed: false with 1 message: '3'
 Message.tests.cpp:<line number>: failed: false with 2 messages: 'hi' and 'i := 7'
 Tag.tests.cpp:<line number>: passed: testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
+Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
 StringManip.tests.cpp:<line number>: passed: splitStringRef("", ','), Equals(std::vector<StringRef>()) for: {  } Equals: {  }
 StringManip.tests.cpp:<line number>: passed: splitStringRef("abc", ','), Equals(std::vector<StringRef>{"abc"}) for: { abc } Equals: { abc }
 StringManip.tests.cpp:<line number>: passed: splitStringRef("abc,def", ','), Equals(std::vector<StringRef>{"abc", "def"}) for: { abc, def } Equals: { abc, def }
@@ -2359,6 +2377,7 @@ Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_
 Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(data)) for: 6 == 6
 Tag.tests.cpp:<line number>: passed: testcase.tags.size() == 1 for: 1 == 1
 Tag.tests.cpp:<line number>: passed: testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
+Skip.tests.cpp:<line number>: skipped:
 Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'Why would you throw a std::string?'
 Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
 Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
@@ -2455,7 +2474,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
 InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
-test cases:  395 |  305 passed |  83 failed |  7 failed as expected
-assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
+test cases:  404 |  305 passed |  84 failed | 5 skipped | 10 failed as expected
+assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
 
 
diff --git a/tests/SelfTest/Baselines/console.std.approved.txt b/tests/SelfTest/Baselines/console.std.approved.txt
index 39fd845b..dd3199f0 100644
--- a/tests/SelfTest/Baselines/console.std.approved.txt
+++ b/tests/SelfTest/Baselines/console.std.approved.txt
@@ -1165,6 +1165,14 @@ due to unexpected exception with message:
   unexpected exception
 
 -------------------------------------------------------------------------------
+a succeeding test can still be skipped
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
 checkedElse, failing
 -------------------------------------------------------------------------------
 Misc.tests.cpp:<line number>
@@ -1186,6 +1194,87 @@ Misc.tests.cpp:<line number>: FAILED:
 with expansion:
   false
 
+-------------------------------------------------------------------------------
+dynamic skipping works with generators
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+explicitly with message:
+  skipping because answer = 41
+
+-------------------------------------------------------------------------------
+dynamic skipping works with generators
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+explicitly with message:
+  skipping because answer = 43
+
+-------------------------------------------------------------------------------
+failed assertions before SKIP cause test case to fail
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: FAILED:
+  CHECK( 3 == 4 )
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
+failing for some generator values causes entire test case to fail
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: FAILED:
+
+-------------------------------------------------------------------------------
+failing for some generator values causes entire test case to fail
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
+failing for some generator values causes entire test case to fail
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: FAILED:
+
+-------------------------------------------------------------------------------
+failing for some generator values causes entire test case to fail
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
+failing in some unskipped sections causes entire test case to fail
+  skipped
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
+failing in some unskipped sections causes entire test case to fail
+  not skipped
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: FAILED:
+
 loose text artifact
 -------------------------------------------------------------------------------
 just failure
@@ -1304,6 +1393,19 @@ Misc.tests.cpp:<line number>: FAILED:
 with expansion:
   1 == 2
 
+a!
+b1!
+-------------------------------------------------------------------------------
+nested sections can be skipped dynamically at runtime
+  B
+  B2
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+!
 -------------------------------------------------------------------------------
 not prints unscoped info from previous failures
 -------------------------------------------------------------------------------
@@ -1339,6 +1441,15 @@ with message:
   this SHOULD be seen only ONCE
 
 -------------------------------------------------------------------------------
+sections can be skipped dynamically at runtime
+  skipped
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
 send a single char to INFO
 -------------------------------------------------------------------------------
 Misc.tests.cpp:<line number>
@@ -1362,6 +1473,16 @@ with messages:
   i := 7
 
 -------------------------------------------------------------------------------
+skipped tests can optionally provide a reason
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+explicitly with message:
+  skipping because answer = 43
+
+-------------------------------------------------------------------------------
 stacks unscoped info in loops
 -------------------------------------------------------------------------------
 Message.tests.cpp:<line number>
@@ -1384,6 +1505,14 @@ with messages:
   6
 
 -------------------------------------------------------------------------------
+tests can be skipped dynamically at runtime
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................
+
+Skip.tests.cpp:<line number>: SKIPPED:
+
+-------------------------------------------------------------------------------
 thrown std::strings are translated
 -------------------------------------------------------------------------------
 Exception.tests.cpp:<line number>
@@ -1394,6 +1523,6 @@ due to unexpected exception with message:
   Why would you throw a std::string?
 
 ===============================================================================
-test cases:  395 |  319 passed |  69 failed |  7 failed as expected
-assertions: 2148 | 1993 passed | 128 failed | 27 failed as expected
+test cases:  404 |  319 passed |  69 failed | 6 skipped | 10 failed as expected
+assertions: 2156 | 1997 passed | 128 failed | 31 failed as expected
 
diff --git a/tests/SelfTest/Baselines/console.sw.approved.txt b/tests/SelfTest/Baselines/console.sw.approved.txt
index 57fa6239..48609d97 100644
--- a/tests/SelfTest/Baselines/console.sw.approved.txt
+++ b/tests/SelfTest/Baselines/console.sw.approved.txt
@@ -14660,6 +14660,16 @@ with expansion:
   " ( contains: "attr1="true"" and contains: "attr2="false"" )
 
 -------------------------------------------------------------------------------
+a succeeding test can still be skipped
+-------------------------------------------------------------------------------
+Skip.tests.cpp:<line number>
+...............................................................................